TikTok announced Symphony Avatars last month as part of its “Creative AI Suite,” aimed at helping create personalized videos without the need to hire real actors. The feature has been rolling out for the past week to TikTok Ads Manager accounts, or it was supposed to be limited to those accounts.
CNN producer and digital content creator Jon Sarlin gained access to the AI tool using his personal account and found that there are no safeguards when it comes to sensitive content.
Generative AI is in extensive use across the tech world, and TikTok is no exception. The platform first announced Symphony Avatars last month as part of its “Creative AI Suite,” enabling businesses, brands, and creators to create fully customized ads using generative AI and the likenesses of paid actors (avatars).
This feature was then rolled out earlier this week, but only for people with a TikTok Ads Manager account. However, it seems like this limitation was briefly not in place, with a CNN reporter gaining access to one of the Symphony AI tools using their personal account. This led to the reporter finding practically no guardrails or safeguards in this AI tool.
Uh-oh! TikTok is in trouble once again!
In a post on X (formerly Twitter), Sarlin showed some examples of videos, created with Symphony Assistant, including avatars, reciting the aforementioned problematic scripts. What’s even worse is that there were no watermarks or any indications on the videos to show that they were AI-generated.
Unsurprisingly, CNN reached out to TikTok for comment and received the following statement: “A technical error, which has now been resolved, allowed an extremely small number of users to create content using an internal testing version of the tool for a few days. If CNN had attempted to upload the harmful content it created, this content would have been rejected for violating our policies. TikTok is an industry leader in responsible AIGC creation, and we will continue to test and build in the safety mitigations we apply to all TikTok products before public launch.”
It’s unclear whether TikTok accidentally rolled out an “internal testing version” of the AI tool or just missed placing the appropriate filters and safeguard mechanisms, but now the issue has been remedied, according to TikTok (we understand that Symphony Assistant has been pulled off, at least for the moment).
AI isn’t going anywhere anytime soon, so expect more stories like this one to pop up. What do you think about it? AI is just a tool, and it’s us humans who ultimately use it unethically. But there should be mechanisms in place to prevent such mishaps, right?
