Why African Whisper?
I recently got asked this question and I have decided to answer it publicly as I did.
If you’ve tried the speech-to-text on Google Cloud, Azure and etc, you might have noticed that it’s fairly good on widely spoken African languages such as Swahili, Amharic, Afrikaans but very poor and sometimes not supported for less-widely spoken languages.
Solution: build our own models that support speech recognition of these languages.
Ofcourse, I’m not the first person to come up with such a solution, startups such as Lelapa AI and Vambo AI are doing that already.
Then why?
With Whisper being open-sourced by OpenAI, it meant you could use the model (trained on 680k hours of multi-lingual data on translation and transcription tasks) commercially and own it.
African Whisper Description:
***
Framework for seamless fine-tuning and deploying Whisper Model developed to advance Automatic Speech Recognition (ASR): translation and transcription capabilities for African languages.
***
With the African Whisper framework you can fine-tune any version of the model on any multi-lingual dataset an API to production not limited to African language datasets.
The framework is also built with enhancements from research papers in this field, you can read about them here.
All you need is compute.
Validation
I also used the framework during a recent Build with AI hackathon by Google Developer Groups: Nairobi and won first prize.
We built an AI Speech therapist for kids with special needs using the framework to deploy a speech-to-text model(finetuned on Swahili) and Gemini as the LLM API provider.
Demos
Using a fine-tuned model to generate word-level translation on Swahili clips, complete with Speaker labelling and Voice Activity Detection done on Google Colab.
A usage demo (v0.2.5)
A deployment and monitoring demo.
If you’re interested, check it out here and don’t forget to leave a star.
Email: keviinkibe@gmail.com
LinkedIn: Kevin Kibe
