AI-TRANSLATOR

AI-TRANSLATOR

AI-TRANSLATOR is a robust synchronous speech and text translation tool that excels in parallel delivery to various output formats. It captures speech or text, translates it into a specified target language, and then provides simultaneous outputs to multiple independent destinations. These outputs can be in the form of text sinks like captions or on-screen displays, as well as voice sinks that route translated speech to virtual audio devices or audio streams.

Key Features

The standout feature of AI-TRANSLATOR is its ability to operate independently of output sources. The core translation engine generates streams of translation events without needing to know the pathways to which these outputs will be directed. This unique architecture enables a single translation to be displayed simultaneously on a stage screen, broadcasted as captions, and heard in a voice channel.

Current Status

The project has reached Stage 1, where a Minimum Viable Product (MVP) has been implemented and is rigorously tested. The following components are functional:

• Event bus and sink subsystem
• Console and WebSocket interfaces
• Transcript and Telegram sinks
• Integration with audio sources for real-time processing.

Supported Platforms

• Linux: The primary development platform utilizing Nix flake for the development shell, with audio processing via PipeWire and ALSA.
• macOS: The project is compatible for building and running with the support of Xcode Command Line Tools; utilizes CoreAudio for audio handling.
• Windows: Support for x86_64 architecture with audio handling done through WASAPI. It's designed for easy integration with audio management tools, although specific features for Windows arm64 are not supported.

Architectural Overview

AI-TRANSLATOR employs a modular architecture that includes:

Sources
  microphone / system audio            text input
        │                                   │
        ▼                                   ▼
Translation core  ──  transono::TranslationLine<P> + Provider
  speech recognition → translation → translated text + translated speech
        │  SessionEvent
        ▼
Translation event bus
  normalization into TranslationEvent, monotonic seq, timestamps
  fan-out to subscribers with isolation and an overflow policy
        │
        ├──────────────► Text sinks   (any number, in parallel)
        └──────────────► Voice sinks  (any number, in parallel)

In this structure, the core translation engine remains unaware of the specific sink implementations, allowing for maximum modularity and flexibility. Each sink operates in isolation to prevent delays caused by slower sinks, ensuring a seamless workflow.

Supported Sinks

Currently implemented sinks include:

• Console Sink: Terminal output for monitoring and debugging.
• WebSocket Sink: Provides a local HTTP server with a full-screen display for presentations.
• Transcript Sink: Captures translations in formats like TXT, SRT, VTT, or JSONL.
• Telegram Sink: Allows sharing translations through a Telegram channel.

Future enhancements will introduce webhook support for platforms like Discord and Slack, and virtual audio devices will enable enhanced voice channel services.

Live Broadcasting Feature

AI-TRANSLATOR supports live audio broadcasting into Telegram voice chats. This feature operates as a user account rather than a bot, leveraging the native media engine to facilitate real-time translation delivery within voice conversations.

Configuration

The configuration model is built upon a simple TOML structure which allows custom settings for core translation parameters and sink specifications. Secrets are managed through environment variables to maintain security and privacy.

Example configuration for translating to Russian from various input sources:

[core]
target_lang = "ru"

Requirements

• Nix with flakes for development on Linux.
• Xcode Command Line Tools for macOS.
• MSVC toolchain for Windows.
• An account with an AI provider for translation services.

Documentation and Examples

Comprehensive documentation is included within the repository, detailing code structure, build processes, and example configurations for various use cases.

For further detail, reference the documentation.

AI-TRANSLATOR is built on the TRANSONO library, integrating advanced audio processing, asynchronous runtime support, and effective logging techniques to provide a comprehensive translation solution.