#1
  1. No Profile Picture
    Contributing User
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Oct 2000
    Location
    Back in the real world.
    Posts
    5,966
    Rep Power
    190

    Human eye/brain perception, how to simulate this in 3D space


    Ok, hereīs the challenge:
    In other words: "How to make a 3D panorama shoot?"
    I did search the whole web, but what they call "3D panorama" is IMO only 2D because they donīt have a depth axis....

    Details:
    I am programming on this 3D project for quite some time now. For a new feature, we want to have a surround feeling. So we connected three monitors now. This was no problem so far. They are arranged like this:
    Code:
    / --- \
    1  2  3
    You can see, the left and right monitors are not the sideviews but they extend the front view.

    Problem:
    1) the picture is now stretched because it is three times as wide. I solved this by modifying the view frustum. But now:
    2) I have a fish-eye lense optic. I modify the FOV so the effect is minimal on the center screen. But now:
    3) Objects on the side screens have a strong skew + scale. Natural if you were using a lens, but I need to make it look like "real", i.e. like on the side of your eyes (*1).

    Now I tried to build the world like a panorama shoot, i.e. I am turning the camera 30° and then I render the whole scene again. Looks exactly how I wanted it... BUT NOW:
    4) When I look up or down, everything is totally fīd up.
    i.e. a horizontal line will be horizontal on the middle screen, but diagonal on the side screens. Not even the start/end points are touching anymore.

    ...So this was wrong too.

    I talked to a photographer, this seems to be the same problem for them too. (Is there only horizontal panorama shots? I never noticed that ...)

    Anybody got another idea how to approach this?

    Tia,
    M.

    *1) Is it my brain that makes me recognize shapes correctly even though they are distorted by the eye lens? I guess so. But how can I trick the eye so it accepts the picture on the monitors to be a 3d object and that we need the same anti-distortion too?
  2. #2
  3. Me likey breadsticks...
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jan 2003
    Location
    Los Angeles
    Posts
    1,189
    Rep Power
    18
    Wow sounds like really interesting stuff. Not sure if this is for game programming or not, but you might get better responses over at the flipcode.com forums gamedev.net forums, or even OpenGL.org forums as I hardly see many graphics related questions here.

    BTW is this for work or hobby? Sounds like fun stuff that I'd be interested in taking a look at if it's not proprietary info and you're willing to share. Unfortunately I can't help out with your question as I haven't had free time in recent months to work with graphics programming.

    HTH,
    -b
    PostgreSQL, it's what's for dinner...
  4. #3
  5. No Profile Picture
    .
    Devshed Newbie (0 - 499 posts)

    Join Date
    Dec 2002
    Posts
    296
    Rep Power
    12
    this might be a useful direction to look into: quicktime virtual reality authoring software. it has the capability to take photographs taken from the same point but different angles (camera rotated on a tripod) and meld them together seemlessly. that's actually stunningly clever because just making a standard amature panoramic (david hockney style) picture the angles go haywire and appear completely disjointed because of different focal points and perspectives. but qt vr authoring software has the capability to remove that disjointedness. god knows how, but it does.
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Oct 2000
    Location
    Back in the real world.
    Posts
    5,966
    Rep Power
    190
    To bcyde:
    Iīve been to flipcode and gamedev (hint to others: GREAT SITES!), no success (yet). But I didnīt visit their forums yet, Iīll give that a try tomorrow. The OpenGL site, FAQ and redbook were great for starting, but now...
    I guess Iīll have to find a biology/physics professor or something like that.

    I feel the solution is similar to the algorithm to map a texture on a sphere, but understanding / porting this algorithm is beyond my scope
    At least at the moment I donīt feel like getting that deep into maths. It was hard enough to brush up my (little) knowledge about vectors and matrices already. I am not the youngest anymore LOL...

    Itīs not a game though. But everything that applies to game programming is useful here too.
    It is a commercial project that I am doing at work. Iīm working on it for nearly a year already. First presentation was on the "Paris Air Show" in June for over one week straight. And it didnīt break a single time

    Sorry, I can not share the sources or much more details. The code is our companyīs property and the project contains highly confidential data from the customer. But if you want to talk about 3D in general, PM me...

    For the last presentation we had rented a stereo VR helmet with an attached IMU device (borrowed because this damn thing costs about $40,000 - way too much for a "non-direct-revenue" project...). First everybody was very sceptic (including me), but after the first day, the customer loved it. The visitors too.
    Next time we thus need to make the experience even better...

    That was kinda cool. Too bad I did not get to play with the HMD for longer (even rental was expensive as hell).
    Most of the time, the job actually *is* great fun. But the working hours are horrible (youīd probably laugh at me if you knew...). And the other half of my job is making B2B PHP web apps...

    to balance:
    I just read your post while writing this. Thanks for the hint. I did make QTVRs for other projects. But it is not really what I need here. This is all real-time OpenGL rendering, only the textures are pre-rendered. The objects move, can change shape and interact with each other... Maybe I should have made this more clear.
    If I could get the QTVR algorithms used, this would be different though - I can tell which hardware we need, so it would be eg. a quad-xeon, 4GB ram, rendering the necessary panorama pictures in realtime or something similar...

    generally:
    Think of a game like Q3A or Unreal. How would they solve the above monitor arrangement? (without side view)

    ...
    M.
    Last edited by M.Hirsch; July 24th, 2003 at 02:53 PM.
  8. #5
  9. Me likey breadsticks...
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jan 2003
    Location
    Los Angeles
    Posts
    1,189
    Rep Power
    18
    Well I posted a link to your thread on a gaming forum that I know a few game developers view. Only 1 person I know of (Skrag on that forum) that is a professional game developer responded here's
    his response. and he follows up shortly. Not much more info, but I have my feelers out there helping you (since you've been able to help me and so many people in the past )

    -b
    PostgreSQL, it's what's for dinner...
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Oct 2000
    Location
    Back in the real world.
    Posts
    5,966
    Rep Power
    190
    Thank you. I really appreciate this

    The first guy that replied, called "john carmack", do you know if it actually is him?
    John Carmack, the god of 3D gaming? Maybe I can get a flame from him LOL (those who donīt understand the joke, search the lounge for "flamed by famous people")

    n8,
    M.
  12. #7
  13. Me likey breadsticks...
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jan 2003
    Location
    Los Angeles
    Posts
    1,189
    Rep Power
    18
    Hah, no that's not good ol' JC. To catch JC's attention you'll probably have to post at /. or opengl.org.

    -b
    PostgreSQL, it's what's for dinner...
  14. #8
  15. Doggie
    Devshed Novice (500 - 999 posts)

    Join Date
    Jul 2003
    Location
    Seattle, WA
    Posts
    751
    Rep Power
    12
    I can visualize the solution in my head, but I'm not sure if I can explain it properly. Is your engine software rendering, or hardware? I'm not sure if you could do this with hardware rendering, but here's my idea:

    Normally, you figure out your point of view and how it projects through the screen, like so:


    But you need to render it as a single, large image wrapping around 180 degrees, then take the front 60 degrees, or whatever and make that your front view.


    So your calculated view may be 1920x480, with the width divided up into three, 640 pixel wide views.

    I don't think you can use a "one size fits all" engine to do this with though, since your fov would be different for your x coord, then your y coord. Most engines probably only work on the 4:3 aspect of a normal screen.
  16. #9
  17. No Profile Picture
    Contributing User
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Oct 2000
    Location
    Back in the real world.
    Posts
    5,966
    Rep Power
    190
    This is hardware accelerated OpenGL rendering. And the good thing is: The engine will never leave the platform it was developed for. At least not in the near future.
    Processing power is not a problem at the moment. Itīs a P4/2800, 1GB ram, Parhelia-512/256MB. 3072x768 pixels for the whole display.

    The problem is not hard- or software, but the algorithm (thus I chose this forum )

    MS Paint session? Ok, but please donīt tell me that I canīt draw - I know

    This is how you define the viewport and FOV in opengl.


    And this is the effect I get rendering the scene with modifed FOV for eg. -30°, 0° and +30° of camera rotation using a horizontal view.
    The outer monitors show the strong distortion I get. This is three rectangular objects the same size.
    The eye / brain does not like the lines. The red lines are part of a single horizontal line. The dotted lines is what I "see but canīt see", i.e. what my brain makes of it and leads to the confusion.

    Now the worst part:

    When you are looking up, the panorama is destroyed. Depending on the angle, you see the lines go nearly vertical on the side views, but exactly horizontal on the main screen.
    I cannot just turn the camera 30° eg and re-render. Iīll have to modify something else, but I canīt find out what. Is it the "roll" axis maybe? But how?
    At the moment I am using simple "heading / tilt" coordinates for the camera, but modifying to h/p/b would not be a problem.
  18. #10
  19. Doggie
    Devshed Novice (500 - 999 posts)

    Join Date
    Jul 2003
    Location
    Seattle, WA
    Posts
    751
    Rep Power
    12
    I don't seem to be able to connect to your server for some reason, so I can't see your images.

    If it's a rotation problem, try a YXY rotation.

    ie: pan to the angle you want to view (Y coord), tilt up to the proper angle (giving you the center view, X coord) then pan left and right for you other two views. (Y coord again)
  20. #11
  21. No Profile Picture
    Junior Member
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2003
    Posts
    1
    Rep Power
    0
    Originally posted by M.Hirsch

    And this is the effect I get rendering the scene with modifed FOV for eg. -30°, 0° and +30° of camera rotation using a horizontal view.
    The outer monitors show the strong distortion I get. This is three rectangular objects the same size.
    The eye / brain does not like the lines. The red lines are part of a single horizontal line. The dotted lines is what I "see but canīt see", i.e. what my brain makes of it and leads to the confusion.

    Now the worst part:

    When you are looking up, the panorama is destroyed. Depending on the angle, you see the lines go nearly vertical on the side views, but exactly horizontal on the main screen.
    I cannot just turn the camera 30° eg and re-render. Iīll have to modify something else, but I canīt find out what. Is it the "roll" axis maybe? But how?
    At the moment I am using simple "heading / tilt" coordinates for the camera, but modifying to h/p/b would not be a problem. [/B]
    After seeing your very nicely hand-drawn pictures

    If you've got a horizontal line in space that goes across the center of the center screen, it should NOT go across the center of the side screens. It should cross the side screens at something like a 30 degree angle (or maybe not exactly 30, but definately at an angle), from the center of the edge closest to the middle screeen moving up towards the top.

    If this is not true, then your camera rotation is off somehow, and although it could be chalked up to imperfect drawing, your pictures make it look like the line goes straight across the center of the side monitors. As a simple proof that the line should cross at a diagonal, place your thumbs and forefingers in an L shape, framing a horizontal line, then turn your whole body to the right, and note how the line crosses the "frame" of your hands.

    As I mentioned elsewhere, the monitor positioning would be very important as well. It would probably be worth figuring out actual eye distance versus screen width to figure out the actual field of view to use rather than using traditional values, otherwise you'll get nasty discontinuities at the screen "edges". I'm just not sure how much you can get away with compressing the field of view into the space of one screen, and if you don't treat each screen as a perfect window into the virtual world, you're pretty much guaranteed to have problems at the edges that the brain might have trouble dealing with.

    The problem you're seeing when you tilt up is almost certain to be in part due to whatever is causing your line to cross the center of the side screens in combination with a possibly incorrect rotation order. You'll want to apply pitch first, then roll, then yaw.

    If the pictures are just hard to read correctly, and the line does actually cut a diagonal towards the top of the side screens like it should, then you probably just need to get the screen positioning right, and figure out why the tilt goes wonky.
  22. #12
  23. No Profile Picture
    Contributing User
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Oct 2000
    Location
    Back in the real world.
    Posts
    5,966
    Rep Power
    190
    Thank you!

    The angles could be wrong because I drew that from my mind. Iīll have to look it up next week again. If the problem persists, Iīll post some screenshots. Iīll check my rotation order on monday too. I did have troubles with that earlier, maybe it is still wrong.

    Something else I thought of: The human eyes are not in the middle of your neck, so if you turn your head, your eyes actually donīt turn around their middle axis (like my camera does), but they "walk" on a circle line. Do you think this should be taken in consideration too?

    Eye-Screen distance is too variable. It was planned to work from a close distance (~50cm) for a single viewer, plus from about 4-5m distance for the audience walking by. Slowly I doubt the whole idea was good...

    M.

    PS: Welcome to DevShed Forums!

IMN logo majestic logo threadwatch logo seochat tools logo