The World of 3D: Real or Virtual?

What is 3D? The term signifies a three-dimensional space with length, width and depth. We see images in three dimensions because that's the way our eyes function.

Let's do an interesting little experiment. Hold your forefinger 10 centimeters in front of your eyes, centered between the two. In turn, close the left eye and then the right. You will notice that in fact, the view from left eye is significantly different to the view from the right.

Investigating this phenomenon, 19th-century scientists and imaging experimenters discovered that if they could limit the audience's left eye to an image viewed from the left, and right eye to one viewed from the right, the brain and visual cortex would automatically synthesize the two images into 3D. This created an effect that was very different from normal 2D film, which simply provided the two eyes with identical signals.

Taiwan's first live-action 3D drama, Clownfish, was shot by Charlie Chu. A love story between the male character Wu Guoyu, who has had an inferiority complex since his childhood, and a blind girl Xuerou, the content is inspirational and warm, and very touching. But unfortunately inadequate marketing meant that it failed at the box office. The picture shows e-book stills of Clownfish (anaglyph 3D technique).

In order to achieve the trick of providing different signals, shot from different angles, for left and right eyes simultaneously, 3D has to be shot with some type of dual-lensed imaging apparatus. The offset (distance between centers) and angle between the two lenses have to be accurately aligned, so three-dimensional images can be accurately simulated. If everything is correctly aligned, they will in fact be just like the images we see with our eyes.

Movie director Charlie Chu launched Taiwan's first live-action 3D digital movie (shooting real people rather than using computer animation), and is currently immersed in mastering 3D technology. He explains that because the average distance between the pupils of an adult's eyes is about six to seven centimeters (for Africans it can be as wide as 8 cm), while for children it is about 5.3 cm, the distance between the two camera lenses must be based on these figures. If the offset is too great, the angular difference between the images will exceed the scope of the audience's visual tolerance. "That's going to make audiences feel dizzy and nauseated pretty quickly."

The two camera units needed for 3D can be arranged either vertically (one above the other) or horizontally (one beside the other). In the vertical arrangement, the cameras are mounted at 90 degrees to each other on a specially designed stand, and a mirror reflects the image of the scene into the lens of the camera that is not pointing directly at the scene.

The advantage of using this configuration is that the offset between the two lenses can be minimized, or even reduced to zero (so that both lenses shoot identical images), making the system particularly suitable for close-ups. The disadvantage is that the cameras and associated equipment weigh around 70 to 80 kilograms, and can usually only shoot from a fixed point as they are extremely difficult to maneuver. Such equipment is also very expensive, about NT$18-20 million, within the reach of only a few domestic production houses.

The horizontal mode is relatively simple in comparison. As long as the two lenses are fixed side by side to imitate human sight, and recording simultaneously, basic 3D images can be produced quite easily. Because everything is fixed, neither the angle nor the offset of the lenses can be adjusted, unlike in vertical mode. The advantage, however, is that the weight of the equipment can be kept within 10-20 kg, so it is relatively maneuverable and can be used on booms or tracks. This makes it particularly suited to long shots and wide scenes where the lens offset is less critical.

"A newbie 3D filmmaker with a lower production budget could get started with horizontal mode. Even if you only use two simple home DVs, a good 3D effect can be produced," Charlie Chu enthuses. He used two Canon EOS 5D digital cameras for a successful short film of fireworks in 3D!

(below) The 3D images of the tree-trunk do not display any angular differences because the cinematographer has set the zero point (screen position) on the trunk. Chu's punching actions, on the other hand, show clear differences since he is standing in front of the tree. If an audience were watching through 3D glasses, they would have the impression of having punches thrown at them.

Once the cameras are placed appropriately, attention turns to the other key technique for 3D: synchronic shooting-getting the two cameras to work accurately in synchrony. The technique uses the same principle as human sight. Our eyes have slightly different perspectives on an object, but the images are immediately processed through our left and right eyes, our brains synthesizing the two images to create one. If the images are not perceived simultaneously, the synthesis will result in meaningless garbage, something that could quickly drive a person crazy.

Setting up synchronic filming was a nightmare in the days of mechanical film, but in today's digital era, things are much simpler. Post-production editing, however, can be rather more troublesome. Matching the timing of the left and right image segments becomes critical. Because it is extremely difficult for an editor to distinguish slight differences between the two segments, great care has to be taken to avoid small errors. The takes and time codes of each lens placing have to be recorded very precisely during filming to become the reference for post-production work.

In addition to these special techniques, the aesthetic considerations of 3D are very different to those of regular 2D filming. The sense of space in a shot assumes a new relevance, and has to be taken into account for every scene.

Chu explains. For example, the director hopes to create a scene with an aesthetic of deep perspective. It's a wide scene with buildings in the background, and trees at center front. The trees can be set up at the "zero" point (the "projection plane," equivalent to the surface of the screen in a cinema), and the actors placed between the trees and buildings. Thus, the shot will have a negative parallax image with an entirely concave effect. The cinema audience will experience a vision of breadth and depth, as though they are actually a part of the scene.

If, however, the script calls for a fistfight, the director may design a scene wherein the actors throw punches at each other with their fists stretching out just in front of the trees. The audience, meanwhile, sitting in front of the cinema screen, will see the "positive parallax" image of actors' fists stretching out towards them. "They may even feel as though the punches are aimed at them, and try to duck or escape," says Chu.

Because 3D creates such a unique sense of space, these principles have to be kept in mind whether when writing the plot or when planning the day's shooting. The techniques learned in two dimensions just don't work in this new medium.

Chu, who is currently working as technical advisor to Mayday 3DNA, a 3D film of a live concert by Taiwan pop-rock band Mayday, cites an example. When the film crew were filming the live concert in Beijing, some of the cameramen ignored suggestions and insisted on doing things in the traditional MTV style. A lot of their shots were done with unstable cameras, and they also zoomed rapidly in and out, switching quickly between close-ups and wide shots. "When we examined the production after the shooting, everyone felt dizzy and nauseated after only a few minutes of watching. The whole production had to be thrown away."

Chu explains that this is because the rapidly overlapping positive and negative parallax images make the audience's visual perception try to respond to a rapid succession of large movements. At the same time, their senses of hearing and touch have not been receiving the expected information at the same intensity. Their brain will sense these extreme inconsistencies, and create a strong feeling of vertigo to signal its distress.

Lighting for 3D photography also needs special consideration. Because both polarized and liquid-crystal shutter glasses for 3D viewing (see "3D TV Explained," p. 32) are similar to sunglasses in that they reduce the intensity of light, 3D needs to be shot in a well-lit environment. Sometimes a producer will deliberately try to brighten the screen, so the audience wearing their 3D glasses will not see a screen bathed in darkness. Some genres of film, however, pose special difficulties in this area. Hou Hsiao-hsien's film Flowers of Shanghai, for example, which has a dim, almost gloomy atmosphere, has special requirements for light and shadow which must be carefully considered when 3D techniques are being employed. This is a major challenge for cinematographers who are used to traditional methods.

Taiwan's first live-action 3D drama, Clownfish, was shot by Charlie Chu. A love story between the male character Wu Guoyu, who has had an inferiority complex since his childhood, and a blind girl Xuerou, the content is inspirational and warm, and very touching. But unfortunately inadequate marketing meant that it failed at the box office. The picture shows e-book stills of Clownfish (anaglyph 3D technique).

Besides the real-life images of live-action filming, animation is another major aspect of 3D filmmaking.

Compared to the excitement and challenges of live shooting, animated 3D, which is done mainly on computers, is relatively simple, whether in terms of adjusting depth of field, changing virtual camera positions or adjusting camera offset. Scenes using multiple angles of view are generally easily created, usually by setting up appropriate computer scenes and simultaneously "filming" on two or more virtual cameras. Then, the three-dimensional animation effects can be accomplished by superimposing the images using computer software.

However, because 3D animation is fundamentally a regular process that everyone is used to, a lot of extra effort is required on making the end product more interesting. Vibrant "actors," vivid scenes and a rich subject matter are essential in order to attract audiences. From this perspective, production requires a lot of careful thought and hard work.

With huge global box-office sales, Avatar is a stunning example of the possibilities of 3D. Planet Pandora and the human world are separate, the former being animated and the latter "real," with a scene ratio of about 4:1 between the two. In order to minimize the gap in vividness between the two worlds, director James Cameron used a group of specialist actors. Their performance of the expressions and movements of Navi creatures were shot using the most advanced "motion capture" techniques, and then these were used for the animated characters.

Reuben Langdon, who was the actor for the main character of Jake in Avatar, participated in a forum entitled "Digital 3D, Amazing Taipei" at this year's Taipei Film Festival and explained motion capture. The entire bodies of the human actors were wired with hundreds of electronic sensors-movements of the face, in particular, were precisely measured in order to catch changes in emotion which require the most delicate adjustments to expression. When the actors made any facial expression or movement, the sensors transmitted this information to a computer, and through software conversion, all their movements were applied to the animated characters.

"It's hard work acting around a pile of wires and sensors. Once I was almost decapitated by wires connected to my head because I moved rather violently. I damaged some of those very expensive sensors too!" Langdon remarks with a wry smile that he was almost put on the production crew's blacklist because of that accident!

Only Avatar, with total investment of US$300 million (about NT$9.6 billion) has had the money to spend on motion capture techniques to enhance the reality of their animated images. Taiwan's moviemakers, in contrast, have much less capital to play with than Hollywood, so they have resorted to science to reconstruct reality.

Digimax is one of Taiwan's leading animation production companies. In 2007 they obtained exclusive licensing from the US National Aeronautics and Space Administration (NASA) to produce a film based on the Cassini-Huygens space mission to Saturn and Titan. The project was first submitted to NASA in 1982 by a group of scientists including Ip Wing-huen, a professor at the Graduate Institute of Astronomy, National Central University in Taiwan, and had the support of 19 countries including the US and a number of European countries.

The 3D animation Quantum Quest: A Cassini Space Odyssey took two years to make and cost nearly US$60 million (about NT$1.92 billion) to complete, which is regarded as an enormous investment for Taiwan's domestic film industry. Since the film used a great deal of information and images provided by NASA, many spectacular space scenes which had never been seen before were able to be used in the film. To date, the broadcast rights for the film have been sold to a number of countries including Taiwan, Japan and China, and museums in the US, South Korea and India are negotiating to buy the rights. Jeff T.J. Yang, a vice president at Digimax, points out that it is not easy to convert cold scientific data into spectacular images. The production team spent huge amounts of time consulting NASA researchers, and then used the most advanced computer animation software to continually reproduce and modify the images. Eventually, they created simulated space scenes which amazed even the scientists themselves.

Experiencing the filmmakers' work, the audience feel as if they are traveling through space or across the surface of Titan (one of Saturn's moons), and exploring the depths of the Solar System's largest canyon on Mars. Even for many NASA researchers, this was their first experience of the virtual reality of outer space.

Whether it is 3D shot live or used in animation, the process poses unique technical problems. If audiences were more aware of the multifarious techniques involved in creating such amazing images when they're watching 3D, they would perhaps be more appreciative of the wondrous experience it can provide.

Taiwan's first live-action 3D drama, Clownfish, was shot by Charlie Chu. A love story between the male character Wu Guoyu, who has had an inferiority complex since his childhood, and a blind girl Xuerou, the content is inspirational and warm, and very touching. But unfortunately inadequate marketing meant that it failed at the box office. The picture shows e-book stills of Clownfish (anaglyph 3D technique).

Taiwan's first live-action 3D drama, Clownfish, was shot by Charlie Chu. A love story between the male character Wu Guoyu, who has had an inferiority complex since his childhood, and a blind girl Xuerou, the content is inspirational and warm, and very touching. But unfortunately inadequate marketing meant that it failed at the box office. The picture shows e-book stills of Clownfish (anaglyph 3D technique).

(right) The vertical mode camera equipment is expensive and cumbersome, but it can shoot exquisite close-ups. The picture shows Charlie Chu shooting a live-action scene with such a camera rig.