Master speech-to-text transcription with Whisper API, text-to-speech generation, audio processing, and voice-enabled application development. Build intelligent voice interfaces and audio features.
Speech and audio AI capabilities are transforming how users interact with applications - from voice assistants and transcription services to podcast summarization and accessibility features. With APIs like Whisper for speech-to-text and modern text-to-speech services, you can add professional voice capabilities to any application.
Whether you're building a transcription service, voice-controlled interface, podcast platform, accessibility tool, or customer support system, audio AI unlocks new interaction patterns and user experiences.
In this lesson, you'll master speech-to-text with Whisper API, text-to-speech generation, audio file processing, multilingual support, and building production-ready voice features.
Whisper API provides state-of-the-art speech recognition with multilingual support - perfect for transcription, dictation, voice commands, and accessibility.
Getting Started with Whisper:
1import OpenAI from 'openai';
2import fs from 'fs';
3
4const openai = new OpenAI({
5 apiKey: process.env.OPENAI_API_KEY
6});
7
8// Transcribe audio file
9async function transcribeAudio(audioFilePath) {
10 const response = await openai.audio.transcriptions.create({
11 file: fs.createReadStream(audioFilePath),
12 model: "whisper-1",
13 language: "en", // Optional: specify language
14 response_format: "json" // json, text, srt, vtt, verbose_json
15 });
16
17 return response.text;
18}
19
20// Usage
21const transcript = await transcribeAudio("./meeting.mp3");
22console.log(transcript);Supported Formats:
Response Formats:
1// JSON (default) - just the text
2const jsonResponse = await openai.audio.transcriptions.create({
3 file: fs.createReadStream("audio.mp3"),
4 model: "whisper-1",
5 response_format: "json"
6});
7console.log(jsonResponse.text);
8
9// Verbose JSON - with timestamps
10const verboseResponse = await openai.audio.transcriptions.create({
11 file: fs.createReadStream("audio.mp3"),
12 model: "whisper-1",
13 response_format: "verbose_json"
14});
15console.log(verboseResponse.segments); // Array of segments with timestamps
16
17// SRT (subtitles)
18const srtResponse = await openai.audio.transcriptions.create({
19 file: fs.createReadStream("audio.mp3"),
20 model: "whisper-1",
21 response_format: "srt"
22});
23console.log(srtResponse); // SRT formatted subtitlesAudio Translation:
1// Translate audio to English
2async function translateAudio(audioFilePath) {
3 const response = await openai.audio.translations.create({
4 file: fs.createReadStream(audioFilePath),
5 model: "whisper-1",
6 response_format: "json"
7 });
8
9 return response.text;
10}
11
12// Translates any language to English
13const translation = await translateAudio("./spanish-audio.mp3");Convert text to natural-sounding speech with OpenAI TTS API - perfect for audiobooks, voice assistants, accessibility, and content creation.
Basic Text-to-Speech:
1// Generate speech from text
2async function textToSpeech(text, options = {}) {
3 const response = await openai.audio.speech.create({
4 model: "tts-1", // or "tts-1-hd" for higher quality
5 voice: options.voice || "alloy", // alloy, echo, fable, onyx, nova, shimmer
6 input: text,
7 response_format: options.format || "mp3", // mp3, opus, aac, flac
8 speed: options.speed || 1.0 // 0.25 to 4.0
9 });
10
11 // Save to file
12 const buffer = Buffer.from(await response.arrayBuffer());
13 fs.writeFileSync("speech.mp3", buffer);
14
15 return "speech.mp3";
16}
17
18// Usage
19await textToSpeech(
20 "Hello! Welcome to our AI-powered application.",
21 { voice: "nova", speed: 1.0 }
22);Voice Options:
1const voices = {
2 alloy: "Neutral and balanced",
3 echo: "Male voice",
4 fable: "British accent",
5 onyx: "Deep male voice",
6 nova: "Female voice",
7 shimmer: "Soft female voice"
8};
9
10// Test all voices
11async function compareVoices(text) {
12 for (const [voice, description] of Object.entries(voices)) {
13 console.log(`Generating with ${voice} (${description})...`);
14 await textToSpeech(text, { voice });
15 }
16}Complete voice application with speech-to-text, text-to-speech, and voice commands.
1// Voice AI Application
2import OpenAI from 'openai';
3
4const openai = new OpenAI({
5 apiKey: process.env.OPENAI_API_KEY
6});
7
8// Speech-to-text
9async function transcribe(audioFile) {
10 const response = await openai.audio.transcriptions.create({
11 file: fs.createReadStream(audioFile),
12 model: "whisper-1"
13 });
14 return response.text;
15}
16
17// Text-to-speech
18async function speak(text) {
19 const response = await openai.audio.speech.create({
20 model: "tts-1",
21 voice: "nova",
22 input: text
23 });
24
25 const buffer = Buffer.from(await response.arrayBuffer());
26 fs.writeFileSync("output.mp3", buffer);
27}Complete voice application demonstrating transcription, synthesis, and voice commands.
Great work with speech and audio AI! Next: "Hugging Face and Open-Source Models" - learn to use thousands of open-source AI models.
Your progress is saved in this browser, on this device only.