Diverse Applications for 3D

On March 19, 2010, an international film festival with a focus on health, the ninth ImagéSanté Festival, was held in Liège, the capital of the French-speaking part of Belgium. The organizers showed a three-hour-long real-time program at the Sauvenière Cinema in the city center. It was full of extremely provocative and bloody scenes, but it wasn't a horror movie. The presentation was actually real-life cerebral neurosurgery taking place in a hospital, the Centre Hospitalier Universitaire, 16 kilometers from the cinema.

Using 3D images to mark tumors (the yellow block on the left picture and the pink dots at the bottom of the right picture) can not only help doctors to confirm the location of lesions and plan in advance how best to approach the operation, but can also help to determine the risks involved, and make clear the pros and cons of surgery to patients and their families.

This world's first live 3D operation was performed by leading Belgian neurosurgeon Professor Didier Martin. Organizing such a broadcast was not easy. All the video equipment had to be rigorously sterilized, and the film crew were suited out in sterile clothing, the same as that normally worn by doctors and nurses. As a further precaution, they had to keep at least 10 meters away from the medical team while shooting to avoid the risk of people running into each other, and the danger of disrupting the surgery.

Since 3D real-time imaging creates an enormous amount of data (a normal broadcast uses a transmission rate of 500 megabytes per second), the film team sought the support of two major Belgian computer companies in setting up the networks. The broadcast was a complete success, winning accolades especially from the medical professionals watching the film in the cinema, who were able to interact with the surgical team using a two-way audio hookup.

Bloody scenes that result from procedures such as neurosurgery can be disturbing for many people. However, for medical professionals, 3D imaging with its spectacular sense of visual depth providing the strong sense of "being there" has brought the ability to "see" the precise location of a lesion, to provide remote but realistic experience of surgical procedures, and to demonstrate emergency procedures if a massive hemorrhage or other emergency situation is encountered during an operation.

After the film, National Belgian Radio and Television interviewed the departing audience. According to one spectator, "This was better than Avatar!"

Converting satellite images into 3D allows Taiwan's decision makers to more clearly grasp the changing topography of the island. The pictures show Xiaolin Village before and after the Typhoon Morakot disaster. The village, once a place of rich and fertile farmland nestled among rivers and mountains, was completely swallowed by landslides.

In fact, three-dimensional imaging has been a part of medical procedures for years. For example, since 2001 the National Center for High-Performance Computing (NCHC) in the National Applied Research Laboratories at Hsinchu Science Park has cooperated with Chang Gung Memorial Hospital and National Taiwan University Hospital (NTUH). They have used special software and high-speed computing to convert examination data such as X-ray, computed tomography and magnetic resonance images provided by hospitals into 3D format, which can then be rotated and examined from different sides.

When doctors wear specially made 3D glasses, they can see patients' nerve fibers in minute detail and observe the spread of tumors. The application of 3D can help avoid errors in complex surgery and make instruction more efficient, and can also be of assistance in the preoperative planning stage of surgery.

Charlie H. Chang, associate researcher in the Software Technology Group at NCHC, points out that 3D imaging also helps to improve relationships between doctors and patients or patients' families. He cites an example when the NTUH created 3D images for a brain tumor patient. This patient's tumor was active in an area of the brain which controls language and logical thought. With the assistance provided by the 3D images, the family was able to clearly understand the reasons for the patient's short-term memory loss. The nerve fibers around the tumor were being severely deformed because of sustained pressure.

As a nerve injury is generally irreversible, patients cannot resume normal functions even if the tumor is removed. They may even lose the ability to speak, but without surgery, the patient's life may be threatened if the tumors continue to grow.

"It is a very difficult decision for patients and their families whether to have an operation or not. However, at least with the assistance of 3D imaging, doctors can use simpler and more understandable techniques to explain the advantages and the disadvantages of an operation to patients' families, and unwarranted medical disputes after an operation can be reduced significantly," says Chang.

Taiwan has also made significant progress in 3D transmission of real-time surgical procedures. For example, in June this year the Industrial Technology Research Institute (ITRI) announced the launch of a two-year cooperative program with the Asian Institute of Tele Surgery (AITS) in Show Chwan Memorial Hospital. In its initial stage, the program will use current image transmission systems of minimally invasive surgery to capture 2D signals on an endoscope, which will be simultaneously converted into 3D images displayed on a special monitor, so surgeons will be able to view real-time 3D video during a surgical procedure.

In the second phase, they will cooperate to develop real-time 3D imaging and a system that provides a perspective of depth in localized surgical procedures. During minimally invasive surgery, the surgeon will be able to conduct the procedure via the normal 2D screen on an endoscope. But images of the site will be simultaneously available on the 3D screen, which should give the surgeon a more accurate grasp of the relative positions when suturing or cutting. This new technique is expected to significantly enhance the quality and efficiency of current 2D endoscopic technology.

Converting satellite images into 3D allows Taiwan's decision makers to more clearly grasp the changing topography of the island. The pictures show Xiaolin Village before and after the Typhoon Morakot disaster. The village, once a place of rich and fertile farmland nestled among rivers and mountains, was completely swallowed by landslides.

Besides its diverse applications in medicine, three-dimensional imaging is also widely used in scientific research and the reconstruction of geographic data. For example, the NCHC has a long-term cooperative agreement with Chiang Ann-shyn, director of the Brain Research Center, College of Life Sciences at National Tsing Hua University. Together they have established the world's first 3D imaging database of the Drosophila (fruit fly) cerebral neural network. Chiang uses the database as the basis of research, and has published a number of internationally recognized articles, such as "The Olfactory Neural Network Map in Drosophila Brain."

Every year, the Center for Space and Remote Sensing Research at National Central University regularly receives four sets of the latest satellite images of Taiwan from FORMOSAT-2. The NCHC then adds elevation data for the entire island to the images, converting them to 3D.

Wearing 3D glasses and using a mouse and keyboard to operate 3D imaging equipment, researchers and government officials can experience the effect of sitting in a helicopter overlooking all Taiwan. They can also easily compare changes to the island's topography after a typhoon or earthquake.

Typhoon Morakot, for example, devastated much of Southern Taiwan last year. Through the NCHC's enhanced 3D pre- and post-disaster images, it can be seen that the Xiaolin Village region was originally surrounded by green mountains, but during the typhoon, the whole village was engulfed and carried away by a landslide. Mt. Xiandu, which was suspected of being the chief culprit for the disaster, was laid bare after the typhoon as though flattened by a massive explosion. The images were all the more shocking being displayed in 3D. While we may sometimes feel in awe of the power of Nature, we cannot help but worry that if extreme climate events gradually become the norm, will Taiwan be able to cope? Even now, the nation is confronting a variety of severe environmental problems.

Three-dimensional geographic images, however, certainly provide a valuable reference for any changes to the national topography. In addition, they assist in soil and water conservation and disaster prevention.

Will 3D movies become the mainstream? When will 3D television take off, if ever? For scientists, these "soft" issues are not particularly relevant. They are asking how more diversified approaches to science and technology can be developed utilizing the amazing sense of visual depth and space brought by 3D images. For science, this is the critical challenge for the future.

Using 3D images to mark tumors (the yellow block on the left picture and the pink dots at the bottom of the right picture) can not only help doctors to confirm the location of lesions and plan in advance how best to approach the operation, but can also help to determine the risks involved, and make clear the pros and cons of surgery to patients and their families.