Madison Genthry 
(Che Bin, Soul’s Vice President and Product Lead)
The year 2024 stands for a significant shift in the digital landscape as artificial intelligence (AI) increasingly intertwines with social networking platforms, revolutionizing how people connect, communicate, and interact. The CEO of Soul, Zhang Lu is at the helm of a social app that started experimenting with the groundbreaking technology since AI came onto the scene.
A pioneering Chinese social platform, Soul is known for the remarkable appeal it holds for Gen Z. The astonishing part is that, unlike a lot of other social apps that get off to a promising start but lose steam along the way, Soul Zhang Lu’s creation has been able to consistently increase its young user base for a good long 8 years. What’s more, the social platform has been kept in a perpetual state of flux with new and innovative features being introduced regularly. This has ensured that the users keep using for more.
And if that is not enough, the team of Soul Zhang Lu puts significant efforts into both market research and in-house technical development. For instance, Just So Soul, the research division of the platform recently conducted a survey to understand the views of Gen Z on AIGC technology. The responses revealed that more than 60% of the young participants were more than keen on befriending an AI character.
This finding provided the impetus that Soul Zhang Lu and her team needed to start working on artificial intelligence with human-like qualities; more specifically, a large language model capable of interpreting human emotions.
Soul’s Vice President and Product Lead, Che Bin expressed the platform’s aims succinctly at the recent ChinaJoy AIGC conference when he emphasized the importance of emotional intelligence in AI. He pointed out that in the realm of social networking, AI possessing “EQ” is just as important as “IQ”.
Moreover, Che Bin revealed that Soul Zhang Lu’s vision for the future is to create a social platform where AI can truly understand and respond to human emotions. While this may seem like a lofty goal at the moment, the performance of the Soul team at MER24 clearly suggests that the company has the technical expertise and the spirit of innovation needed to develop AI-powered features that can offer personalized companionship, facilitate empathetic interactions, and enhance overall user well-being.
MER24 is a top-tier AI-innovation-specific competition that attracts participants from the best universities as well as leading tech companies. Soul Zhang Lu’s team was a participant in the most competitive category of the event, Semi-Supervised Learning (SEMI). To lift the proverbial cup, the team had to win against almost 100 other participating groups.
So the hard-won victory was quite an accomplishment but what made it particularly noteworthy was the fact that the implication of their presentation is a massive step in the direction of making machines emotionally intelligent.
It should be mentioned here that the developers working at Soul App are no strangers to innovatively using AIGC technology to transform platform features. The company already has independently developed large language and large voice models under its belt. The abilities of these models for multimodal emotional recognition have allowed Soul App to introduce features such as:
- AI Goudan, which is an AI assistant capable of offering users a more fulfilling and emotionally resonant digital socialization experience
- Werewolf Awakening, which is a gaming application that allows players to team up with or against AI gamebots that are capable of humanlike interaction, strategizing, and decision-making
- Echoverse, which allows users to create personalized AI characters that can interact with humanlike acumen.
These features are just some of the latest additions to the already impressive suite of AI-powered applications that the team has created for the platform’s users. That said, at this time, the company’s focus is squarely on creating an emotionally intelligent model, and that is what their MER24 submission was all about.
Soul Zhang Lu’s developers took two unique approaches to work through the deficits that take away from the ability of LLMs to interpret human emotions. First, the team fine-tuned the performance of their EmoVCLIP model capable of video emotion recognition. Improvements in the model’s performance were made by using a self-training strategy that involved iteratively labeling unlabeled data with pseudo-labels. This innovative approach helped to train the model to gauge human emotions more accurately and improved its generalization performance.
Then, Soul Zhang Lu’s team used a technique called Modality Dropout, a first in the field of multimodal emotion recognition. This addressed the competitive effects between different modalities within the model, which, in turn, significantly improved the model’s emotion recognition accuracy. The net effect was high accuracy in human emotion recognition and greatly enhanced generalization performance. And, that is how the Soul Team won the competition.
While the win was noteworthy, what will be truly exciting is the practical application of these techniques. Che Bin clarified that Soul Zhang Lu and her team view the integration of AI into platform applications in three dimensions, the users, the technology, and the overall environment. Soul’s MER24 submission scores on all three fronts.
Because it is capable of accurately interpreting human emotions, when applied to platform or standalone features, it will be able to hold empathetic, pertinent, and emotionally valid conversations, thus offering users what they are looking for. The multimodal ability of this model will enable the application to have a high EQ to go along with its high IQ. And because it will strike emotional chords, it will be able to power applications that give users “AHA moments”, which, according to Che Bin, occur when users realize the true value of an application in terms of the emotional fulfillment that it offers.
