Tele-Immersion is an immensely cool idea, that has potentially limitless applications. It ties in elements of
Virtual Reality and
Augmented Reality. At the time of writing, the main stumbling block is that it might actually by theoretically impossible to shift the volume of data around that would be required for a convincing realtime stereoscopic projection. Some elements of existing and hypothetical prototype systems under development by the US government are :
Firstly, full 3D imaging and scanning : every surface within the source area has to be scanned at a very fine level of detail. One of the tools used is bouncing points of light around faster than the eye can see and/or outside of the visual spectrum. As well as the shape, colour and texture of surfaces, many other factors have to be recorded and reconstructed in realtime. As people perceive subtle movements and details of the
human face and hands much more acutely than say, in an
inanimate carbon rod, specialised subsystems have to be developed to concentrate on making these areas accurate and consistent.
Once all the data is captured, it has to be encoded and piped to the destination, requiring rather more bandwidth than is currently available. Then the projection has to be generated, using a rather large number of projectors and mirrors, while scanning the observer's eye movements (all the observers that is ... some kind of lightweight glasses are required to house the sensors practically). The framerate can be chopped up enough to cater for everone's POV, and shutters in the glasses are synced to that particular timeslice. As an added party trick, the system can remove the model of the glasses from the projection that is sent (in fact, many trivial manipulations to the model can be incorporated ... very holodeck).
The basic model for a T-I room would be that one half of the physical room is taken up by the system, and when it is in operation, the virtual room created is effectively the two "real" rooms coupled together. AR objects can be passed between the two parties, employing haptic feedback.