Air translate

Text to SpeechSpeech to TextVoices
PreviousNext

Live translation using WebSockets, inspired by the new feature of Apple AirPods Pro 3, which doesn’t require special hardware—just your web app.

CY
Cristian Yosafat Hernandez RuizI’m a full-stack developer and founder of yafacture.com, with a focus on building scalable, real-time applications. My core stack includes TypeScript, React/Next.js, React Native, and NestJS, supported by databases like MongoDB, Redis, and PostgreSQL.
Air translate
DemoCode

Air Translate

Overview

With Air Translate, powered by ElevenLabs, you can listen to people speaking in different languages and instantly hear real-time translations similar like AirPods Pro 3. The platform captures speech, transcribes it, translates it into your preferred language, and plays it back in a natural, expressive voice — all with minimal delay. This makes conversations across different languages feel as natural and fluid as speaking face-to-face.

Key Features

  • Real-time Speech-to-Text (STT): Captures and transcribes speech instantly using OpenAI Realtime and ElevenLabs STT.
  • Automatic Translation: Converts transcripts into multiple target languages seamlessly.
  • Natural Voice TTS Streaming: Generates lifelike, low-latency speech using ElevenLabs TTS API.
  • Group Conversations: Broadcasts translated speech and captions to all participants, excluding the speaker, via Socket.IO.
  • Cross-Platform Support: Works on web and mobile

How It Works

The microphone stream is captured and sent to the backend via WebRTC/WebSockets. STT processes the audio via ElevenLabs STT). The transcript is translated into the target language(s) using OpenAI. The translated text is sent to ElevenLabs TTS streaming, which returns an audio stream in a natural-sounding voice. The audio and captions are broadcasted in real time to all connected users. This architecture solves the challenge of keeping latency low while chaining together STT → Translation → TTS across multiple users.

Technologies Used

  • ElevenLabs API (Text-to-Speech, Speech-to-Speech, Custom Voices)
  • OpenAI API: Realtime translation
  • NestJS + Socket.IO: Signaling, audio/text routing
  • Next.js / React / React Native: Cross-platform client applications
  • WebRTC: Real-time audio streaming between peers

Getting Started

# Clone app project
git clone https://github.com/BlakePro/air-translate-app
cd air-translate-app
pnpm i
pnpm dev
open http://localhost:3000
 
# Clone api/socket project
git clone https://github.com/BlakePro/air-translate-api
cd air-translate-api
pnpm i
pnpm dev

Demo

Include screenshots, GIFs, or embed your demo video here.

Future Plans

Reduce latency using socket in TTS

Acknowledgments

Inspired by Apple's AirPods Pro features.