Promotional material for animation To Dear Me Photo: Courtesy of Tong Hua

A team of 12 crew members has created what could be a historic project in the Chinese animation industry - To Dear Me, an AI-generated animation that won Best Short Film at the International Film Festival hosted in Venice in September. Along with global recognition, it also received the Best Film Award in the AIGC (AI-Generated Content) short film unit at the 14th Beijing International Film Festival this past April.

"I'm not very good at drawing, but AI has given me the power of a 'magic brush,' enabling me to create animation and tell a story about self-healing," Tong Hua, the director of the animation, told the Global Times on Thursday.

As the application of AI technology continues to gain momentum in the film industry, its impact on animation is becoming increasingly apparent. Unlike traditional Chinese animations, which often draw inspiration from mythology, the film tells an emotional story about "the self" on an international stage.

Promotional material for animation To Dear Me  Photo: Courtesy of Tong Hua

'To Dear Me'

The film, inspired by the animated short Liebestraum by Chinese artist Yan Xiaoxuan, is a poignant ­exploration of self-discovery and healing. The female protagonist faces abandonment by her father, ­betrayal by her friends, and the loss of her lover in various stages of her life. Ultimately, she embraces herself, as well as the child she once was.

Unlike traditional animation, the film was created using an AI-assisted process that transformed live-action characters into animated ones. The team used tools such as ComfyUI and ControlNet to achieve exquisite visual effects. They also employed multiple shades of blue, including the iconic hues of ­Picasso and Chinese artist Wei Qimei, blending oil painting techniques with pointillism. This attention to color helped define the film's unique style.

From a business perspective, the collaborative production team of ­AinimateLab and the Communication University of China took an unconventional approach to distribution. The film started as a sketch, with actors being filmed in live action before AI was used to transform the footage into animation. This innovative blend of AI and traditional live-action film-making marked a bold experiment.

"If we had used traditional methods, the production time would have been double or triple what it was. AI greatly accelerated our workflow," Tong said. "For something as intricate as the dandelion fluff, drawing it manually would have been very time-consuming, and the results might not have been as good."

It's worth noting that the entry of AI tools into the film industry, including animation, has sparked considerable debate. The 2023 Hollywood strikes, which initially focused on workers' benefits, also raised concerns about AI potentially replacing human roles, with issues around compensation and job security taking center stage.

"I've never believed that AI poses a threat to human creativity," Tong said. 

She shared that works from across the globe left a lasting impression with their innovative techniques and style, and she and her team thought there was no way they'd win first place after watching all the films made with skilled AI use.

"When we asked the judges why we won, they said AI is just a tool - it shouldn't be about showing off technology. Our film had the strongest emotional resonance," Tong said.

That's where Tong believes that AI will never replace humans. She noted that while great art is always the result of collaboration and the exchange of creative minds, something AI cannot replicate, current AI technology feels like a blind box, you never know how what kind of image it will generate, and it's impossible to achieve a perfectly matched art style every time.

At the end of To Dear Me, the protagonist embraces her childhood self, smiling with fine crow's feet around her eyes. The team used ­Runway for facial expression recognition but ultimately relied on manual retouching to bring out the character's subtle emotions.

"Sometimes, human emotions are conveyed through something as simple as the wrinkles around the eyes," Tong remarked.

Chinese director Tong Hua (sixth from left) poses for photos with others in Venice, Italy. Photo: Courtesy of Tong Hua

AI or no AI

The adoption of AI tools by the film industry has, in fact, become an undeniable trend. Domestically, productions like the short narrative series Sanxingdui: Future Apocalypse and the fully AI-produced microfilm Chinese Mythology highlight how AI is driving advancements in the domestic film sector.

However, Liu Shuliang, an animation teacher at the Communication University of China, said that the animation industry is currently in the process of taming AI. "If you don't carefully adjust the parameters or train the model, simply feeding live-action footage to the AI can result in shaky visuals or bizarre anomalies," Liu said.

Internationally, October saw the release of Where the Robots Grow, the first feature-length animated film made entirely with AI tools. Forbes hailed it as "a milestone," noting that cinematic AI is closer than ever, with animation serving as its first full-length expression.

Tian Miaozi, a core member of the Metaverse Working Committee of the China Association of ­Communication Enterprises, told the Global Times that there remains a gap between domestic and international applications of AI technology. However, some domestic teams are closing that gap, with their work ­approaching international standards.  "Some films still rely on traditional software and methods, while others are more willing to experiment with AI, though it hasn't fully replaced conventional workflows," Tian said.

Tong also noted that at the festival, a Mexican team mentioned that it is developing frame interpolation technology, which it plans to make an open-source product. "For now, tools like Magnificent AI, which can modify image details, require a $9,800 annual fee. Many AI tools still only generate content but can't modify it. But we're seeing a lot of new, open-source technologies being developed, and there will undoubtedly be more innovations in the future," she noted.