Sensory attraction

Lending itself to immersive and interactive modes of appreciation, new media art lures the viewer’s soul by appealing to the senses.

Following up on the “Taste Your Soil” theme of Austria’s Ars Electronica 2021, Taiwanese new media artist Huang Hsin-chien has invited chef André ­Chiang and dancer Billy Chang to collaborate on We Are What We Eat, a performance art project where the hosts will rethink the relationship between hu­man­ity and the earth from their own professional perspect­ives. An innovative feast of art, it promises to bring together visual images, sounds, and flavors.

Huang, the project’s virtual reality (VR) director, has won numerous awards in Taiwan as well as overseas. Working to distill the best of Sinophone culture, he is adept at tapping into international insights to give digital technology a cultural relevance.

In fact, new media artworks resemble mini-films. They invite viewers to share in a VR experi­ence where imagin­ary scenes appear to be happening in the real world. New media artists are like film dir­ectors who design their own plots and arrange the settings of their stories. The Starry Sand Beach, a VR project directed by Huang and coproduced by the ­Oready Innovation Lab and the French National Museum of Natural History, was funded by Taiwan’s Creative Content Agency as an international coproduction. Selected in 2021 for the Venice VR Expanded competition, it gives artistic expression to science, broadens our awareness of Earth’s fragility, and contemplates both the origins of life and our current situation. At the same time, it reveals urgent environ­­mental issues, giving art a social role to play.

Huang Hsin-chien is an eminent Taiwanese new media artist who has won numerous awards in Taiwan and abroad.

Huang Hsin-chien’s Bloodline explores the implications of blood relationships.

Shaping virtual worlds

Huang Hsin-chien’s experience of vision loss makes him cherish his sensory perceptions. “One of my corneas was donated by a deceased person from abroad.” When he was a child, Huang was almost completely blind in his right eye. Grateful for his regained sight, he devotes himself to the pursuit of visual art. After obtaining a degree in mechan­ical engineering, he went to the United States to study design. In a place that attracted talented young people from all over the world, he was able to explore new media art to his heart’s content.

While studying at the IIT Institute of Design in Chicago, Huang was noticed by the American new media artist Laurie Anderson, who at the time was a jury member for a film festival. Huang spent nearly two years collaborating with this eminent interdisciplinary artist on La Camera Insab­biata, a project that probed human nature. The work gained international acclaim, winning Best VR Experience at the 2017 Venice International Film Festival.

Huang, who served as an art director at Sega and Sony, has a flair for exploring human nature through game design. His projects capture the imagination of his audience, including award juries. In 2019 his Bodyless was nominated for the Interactive competition at the Venice International Film Fest­ival, and in 2020 it received an honorary mention at the Prix Ars Electronica in Austria and won the Golden Mask award at the NewImages Festival in Paris. In 2021 Huang’s Samsara—a sci-fi project that examines the meaning of life by reference to the Buddhist concepts of samsara, trans­migration, and transcendence—was again selected for the Venice VR Expanded competition, as well as winning the Virtual Cinema competition at the SXSW Film Festival in Austin, Texas, and the Best VR Story award at Cannes XR. This project, which took a year to complete, draws on the concept of “embodied cognition” through VR and inter­action. Within the short span of 30 minutes, it offers the audi­ence an immersive experi­ence and much food for thought.

VR projects are time-consuming. Every second of virtual experience requires a lot of teamwork and dedication.

A graduate student in the Department of Design at National Taiwan Normal University wears motion capture devices to experience the process of creating a VR work.

Into the pictures

Wouldn’t it be amazing if we could walk into a painting or a photograph? Directed by Huang Hsin-chien and Oready CEO Benjamin Chang, Kuo Hsueh-Hu: Three States of Home Gazing was one of the Best Immersive works in the National Winners category of the 2020 Asian Academy Creative Awards; in 2021, it was nominated for the iF Design Award and won the Entertainment, Content Creation, and Streaming Media Design category of the A’ Design Award and Competition.

“We used three of Kuo’s classic paintings—The Setting Sun by Fort Provintia, Colorful Boats on the Tan Chiang River, and Festival on South Street—to create three types of VR interactions, focusing on space, time, and physical ex­peri­ence, which correspond to three themes: states of mind, gazes, and longings,” Chang says. Wearing VR equipment, viewers are able to experience what it was like to be at a festival in Taipei’s Dadaocheng in 1930. The project brings together contrarieties: home and abroad, past and present, the paintings per se and viewers’ perceptions.

VR practitioners are particularly proficient at creating works that respond to existing paintings. For example, in 2019 the Kaohsiung Museum of Fine Arts exhibited The Universe of Liu Kuo-sung, which not only included a traditional display of Liu’s Chinese paintings, but also invited viewers to “look at,” “play with,” and “enter into” Liu’s universe of art from three different thematic angles: “Genesis,” “Chessboard,” and “Earth.” This immersive approach—which anim­ated originally static paintings—situated viewers in the midst of Liu’s subtle and robust brushwork, enabling them to enjoy the fresh immediacy of the interactive experience.

In 2017, the National Taiwan Museum of Fine Arts collaborated with Axis 3D Technology on a VR exhibition based on Lee Ming-tse’s Lotus Pond in Zuoying, a seemingly unsophisticated naïve painting packed with theatrical details. The exhibition blended imaginary storylines inspired by Taiwan’s folk culture with the artist’s nostalgia for the past spirit of his hometown.

Garden Taipei/Formosa: Taiwan Grand Tour, an exhibition curated by Huang Hsin-chien in 2021, included a VR artwork portraying Hakka scenes: Floating Childhoods. A collaborative project between Oready and the Hakka Public Communication Foundation, it displays the simple, unadorned lifestyle of old Taiwan, and was widely praised. Based on precious old photographs of Hakka communities in the Liudui area of Pingtung and Kao­hsiung, it vividly conjures up the historic area of Wu­gou­shui in Pingtung’s Wanluan Township.

Digital utopias

“The development of ‘digital twin’ technology has contributed substantially to the evolution of new media art.” The ability to create a virtual counterpart of a real-­world entity signals an infinite variety of new poss­ibil­ities. Axis 3D Technology general manager Tom Lai, who has been involved in several of Huang’s projects, provides unfailing support for Taiwan’s new media artists.

Vendors are always looking for the world’s latest software and hardware in order to provide Taiwan’s new media artists with a complete set of 3D and VR solutions. They help artists produce astonishing work with facility, to a technical standard that rivals the West.

For example, the 2022 edition of the Taiwan Lantern Festival in Kaohsiung will feature a “Dance of Light” which uses digital technology to craft exquisite and gorgeous sights and sounds. Multiple audience members at a time will be invited to interact with the performance remotely and synchronously. It represents a giant step forward for new media art in Taiwan.

In Bodyless, Huang Hsin-chien taps into his own childhood memories to recreate the social atmosphere of Taiwan under martial law.(courtesy of Huang Hsin-chien)

In Samsara, Huang Hsin-chien takes the audience on a sci-fi journey where they experience transmigration and transcendence in a virtual world..(courtesy of Huang Hsin-chien)

A multilateral endeavor

With government support, the training of world-class high-tech experts ensures that Taiwan’s VR industry will continue to develop and flourish through the integration of software and hardware.

Taipei National University of the Arts is the first educational institution in Taiwan to promote the ­application of modern technology to art. In its wake, National Taiwan Normal University has set up a program devoted to “integrating digital technology into the arts and human­ities,” which is divided into three streams: anima­tion, virtual reality, and multimedia. National Tsing Hua University’s College of Arts also regards the integ­ration of technology and art as one of its main goals.

In addition to the input of vendors, government support is a crucial driving force. “Many of the wearable VR devices we use now are produced locally,” Benjamin Chang says. It was Chang’s idea to commission the Industrial Technology Research Institute and the Institute for Information Industry’s IoT Service Hub to develop VR gloves, as part of the Industrial Development Bureau’s IoT Integrated Service project.

“These can be used in very many ways,” Chang says. “Wearing a VR headset and VR gloves and engaging in interactive activities can make physiotherapy much more lively and interesting.” Third-generation VR gloves are also applied to professional training in the handling of precision instruments and heavy machinery. They help significantly reduce the dangers involved in training workers for high-risk occupations.

In La Camera Insabbiata, which is composed of eight interactive spaces, the audience can experience the sensation of flying in a huge virtual room.(courtesy of Huang Hsin-chien)

Three classic paintings by Taiwanese artist Kuo Hsueh-hu (1908–2012) were used to produce an interactive VR project focusing on space, time, and physical experience.(courtesy of Huang Hsin-chien)

The metaverse

“As yet we don’t see any limits to new media art,” says Huang Hsin-chien, setting his sights far in the future. In a world where virtual reality is gradually expand­ing its territory, the digitization of art has become an irresistible trend. For example, through the performance art project “We Are What We Eat,” custom­ers are able to buy “non-fungible tokens” (NFTs) via blockchain technology and thus own digital memor­ies of co-­creating “super­sensory” meals with the three artists—Huang Hsin-chien, André ­Chiang, and Billy Chang. Stored in the virtual world, these memories will never vanish, and the project will generate new value for art.

At Art Taipei 2021, exhibitors not only began to sell physical artworks through the NFT model for the first time but also brought out an online gallery that combined NFTs with VR. Hong Kong’s M+, a new museum of visual culture that opened its doors in November 2021, shows that the concept of “being more than just a building or a collection” is an irreversible trend for museums, and that virtual exhibitions will become a reality. The displaying and collecting of art have entered an entirely new age.

“The metaverse is attracting every­one’s attention,” says Huang Hsin-chien, who has long been immersed in VR art. With major international corporations pouring capital into upgrading VR technology, the age of the metaverse no longer seems far off. By using a virtual reality headset, augmented reality smartglasses, a mobile phone, a personal computer, or a video game console, we can easily travel between reality and virtuality.

From ARK·TPE to the Taiwan Grand Tour, Taiwan’s new media artists continue to achieve new mile­stones. The supersensory world built upon extended reality technology is not merely for entertain­ment, but inevitably will become a space where we live and work. The combination of technology and art signals new opportunities for each, giving birth to a boundless metaverse with neither a starting point nor an end. Growing at an astonishing speed, new media art is ­quietly integrating itself into our daily lives.

Many people came to experience new media art at the 2021 Taiwan Creative Content Fest, which had “the Metaverse” as its theme.

Held at Taipei’s Songshan Cultural and Creative Park in 2021, the second Taiwan Creative Content Fest invited the public to experience the infinite possibilities of new media art.