Multi-Modal Systems with the OpenAI API
Transcribe speech, generate spoken replies and moderate content with OpenAI’s audio and moderation models, then build a multilingual voice support chatbot.
Applications that accept only typed text leave out people who would rather speak, and they can be misused by anyone who sends harmful input. The lessons work with OpenAI’s audio models to turn recorded speech into transcripts, including speech in languages other than English, and to turn written replies into speech. Moderation models then flag inappropriate messages written by users or generated by a model, and the pieces come together in a voice-driven support assistant.
What You Will Achieve
- Transcribe recorded audio such as a podcast and translate non-English speech into English text
- Turn written text into spoken audio in English and other languages with the text-to-speech models
- Send moderation requests and read the category scores to judge whether a message is inappropriate
- Combine transcription, translation, moderation and speech output into a chatbot that answers customer voice messages in their own language
Program Overview
Speech-to-Text, Text-to-Speech and Moderation Models
Three kinds of OpenAI models beyond text generation, each tried on its own before they are combined.
- Speech-to-text transcription and translation
- Text-to-speech voices in other languages
- Why text moderation models matter
- Moderation category scores
Case Study: Customer Support Chatbot
A voice-based chatbot is built step by step, taking a customer’s call through transcription, translation, moderation and a spoken reply.
- Transcribing calls and deriving the customer’s language
- Translating transcripts and fixing translation mistakes
- Moderating the call and the model response
- Preparing a translated spoken response
Certificate Awarded (Paid)
Job Outlook
These skills apply to building voice features, multilingual support tools and content safety checks in applications that call the OpenAI API, such as customer service chatbots.
Good for: Developers who want to add voice input, spoken replies and content checks to a chatbot or support app built on OpenAI models.
Keep in mind: The multi-modal scope here means speech and text: the lessons cover transcription, spoken audio and moderation, and the second half is a single chatbot case study.
Bottom line: A focused look at OpenAI’s audio and moderation models, tied together in one multilingual voice support bot.
Frequently Asked Questions
Which languages does the course work with?
English is the working language, but the lessons also transcribe and translate non-English audio, with Portuguese as one example. The text-to-speech lessons produce audio in other languages, and the chatbot detects the caller's language and answers in it.
What content does the moderation part check?
Moderation is applied twice. The lessons check messages coming in from users and also the replies the model generates, using the category scores the moderation models return to spot inappropriate content before it reaches anyone.
How do the two chapters relate to each other?
The first chapter tries the speech-to-text, text-to-speech and moderation models one at a time. The second chapter is a case study that chains all three into a single support chatbot handling voice messages.


There are no reviews yet.