Does your smart speaker—or conversational AI interface—find it hard to understand you? You’re not alone, especially if your voice is not the standard issue your device’s dataset was trained on (i.e. American-English-Perceived-Male).
Speech recognition and natural language processing are crucial for the future of the voice-activated internet services, but most of the datasets are owned by a handful of mega-tech companies (the usual suspects) and need much broader inputs in order to understand the rest of the people who share the planet. That’s where Mozilla and its Mozilla Common Voice come in.
“Mozilla Common Voice is an ambitious, open-source initiative aimed at democratizing and diversifying voice technology,” says Mark Surman, Executive Director at Mozilla. “Common Voice enables people to donate their voices to a free and publicly available database that startups, researchers, and developers can then use to train voice-enabled apps, products, and services.”
The project is part of the Mozilla Foundation’s larger mission to ensure the internet is a global public resource, open and accessible to all. For the past four years the team has been documenting and validating human soundwaves, using acoustic modeling, mapping between audio frames and applying a semantic layer, to build out a massive open-source voice database, that anyone can use to innovate in voice-first apps.
Much of its early work was grassroots and donation-based, but two recent investments—$1.5 million from Nvidia and a further $3.4 million from other partnerships to do groundbreaking work with native African languages—are taking the project to the next level.
“Common Voice is now the world’s largest multi-language public domain voice data set, with more than 12,000 hours of voice data and 75 different languages—from Kiswahili to Mandarin to Welsh,” says Chenai Chair, Special Advisor for Africa Innovation at Mozilla. “Around 400,000 people worldwide have contributed to the project so far. Each entry in the dataset consists of a unique MP3 and corresponding text file. Many of the 9,000 recorded hours in the dataset also include demographic metadata like age, sex, and accent that can help train the accuracy of speech recognition engines.”
Importantly, all data is accessible and interoperable, rather than locked up inside for-profit siloes. “Anyone, anywhere can download and use the Common Voice corpus,” says Chair. “It’s CC-0, meaning all data is in the public domain. Community members have been building all kinds of exciting projects using Common Voice, from models that can work for indigenous languages with smaller training sets, to pronunciation training software for second-language learners.”
“Common Voice has already done a good deal to democratize voice tech, and our Nvidia partnership allows us to grow this work even further,” adds Surman. “With Nvidia’s $1.5 million investment, we will accelerate the growth of Common Voice’s data set, engage more communities and volunteers in the project, and support the hiring of new staff.”
While 66.7% of US adults have had at least one vaccination, according to CDC data, the COVID-19 virus continues to devastate the emerging world. The World Health Organization (WHO) confirms that the number of cases across Africa has increased by almost 40% in the past week. The further $3.4 million in financial support for Mozilla Common Voice, from sources including The Bill & Melinda Gates Foundation and the UK’s Foreign Commonwealth & Development Office, will go toward developing native African languages support to create COVID voice-based assistance.
“As voice becomes more and more central to accessing the internet, people across Africa are being left behind. Currently, neither Amazon’s Alexa, Apple’s Siri, nor Google [Assistant] support a single native African language,” Chair says. “As a result, much of the internet remains inaccessible—and African innovation and opportunity are stifled. Common Voice’s Africa work aims to close this gap and unlock that missed opportunity.”
Chair’s team is already collaborating with African companies, start-ups, and universities to develop locally suitable, voice-enabled technology solutions, including a COVID-19 bot for Rwanda. In 2019, Mozilla and Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) hosted a voice tech hackathon in Kigali, led by Mozilla Fellow Remy Muhire. The goal was to bring together local technologists with big, bright ideas for local voice-recognition applications.
“This hackathon gave way to Digital Umuganda, a Rwandan startup that proposed a new take on an old idea,” Chair explains. “In Rwandan culture, ‘Umuganda’ is a concept of self-help and cooperation. Every last Saturday of the month, people gather together to build physical infrastructures such as roads, schools, and more. Digital Umuganda brought this concept online, with the goal of building digital infrastructure such as voice data. This project quickly gained momentum, and it led to Mbaza, the AI-backed COVID-19 chatbot with speech-to-text and text-to-speech functionality, providing critical COVID-19 information in the Kinyarwanda language.”
For Executive Director Mark Surman, these projects in Africa, unlocking opportunities for the 1.2 billion people on that continent, are a powerful example of what Mozilla Common Voice set out to do. But it’s also part of a much larger vision to take back the internet from the mega-corps, especially around voice and AI.
“Common Voice is also a template for a bigger project: giving people more control over their data in the era of artificial intelligence,” says Surman. “AI has an outsized influence on our lives and societies—it dictates what we read, what we watch, and how we get around. But the data powering this AI—much like voice data—is controlled by a select few corporations. Mozilla’s goal is to shift this paradigm—to make the building blocks of AI something that everyone can benefit from, through, as an example, Mozilla’s new Data Futures Lab.”
If you want to help the democratization of the web using your voice, speak here.
