React AI Voice Avatar

React AI Voice Avatar

An Open-Source Alternative to Real-Time Avatar APIs
react-ai-voice-avatar is an innovative solution that delivers a cutting-edge, talking 3D avatar experience directly in the browser. Unlike proprietary services like HeyGen Interactive Avatar and Tavus, this open-source project operates entirely on the user's GPU, eliminating the need for video streaming, per-minute charges, and external integrations.

Key Features

• Local Rendering: The 3D avatar is fully rendered on the user's device, ensuring privacy and minimal latency.
• State-of-the-art Interaction: Leverage a lip-synced avatar that converses in real time with users, powered by various language models including OpenAI and Anthropic.
• Voice Mode: For applications that do not require a visual component, utilize the headless React hook for a streamlined audio interface, similar to ChatGPT and Gemini.

Features Breakdown

1. Highly Interactive: The package handles all the complexities of voice interaction, including microphone capture, speech recognition, and turn-taking mechanics, ensuring seamless user experiences.
2. Efficient Resource Management: The system maintains performance by downloading necessary models only once, allowing subsequent visits to utilize cached resources for faster engagement.
3. Customizable and Extensible: Integrate with your own backend models seamlessly or use pre-defined configurations for easy implementation. Select from different voice engines or deploy a fully local setup for offline environments.

Getting Started

Integration into your app is straightforward. The library provides two entry points:

• Full 3D Avatar: For applications requiring the complete visual experience, import the standard AiVoiceAvatar component, enabling a rich, interactive user interface.
• Headless Voice Mode: For a more minimalistic approach, use the headless hook by importing useAiVoiceAvatar from react-ai-voice-avatar/headless. This version is lightweight and focuses solely on audio functionality.

import { useAiVoiceAvatar } from 'react-ai-voice-avatar/headless';

function VoiceMode() {
    const voice = useAiVoiceAvatar({
        onSubmit: text => fetch('/api/chat', { method: 'POST', body: text }).then(r => r.body),
    });
    // Additional configurations here
}

Performance and Engagement

Designed for high performance, the avatar operates efficiently, managing substantial model files while engaging users quickly. With lazy loading capabilities and architectural best practices, it optimizes resource usage, making it ideal for applications requiring fast load times.

Compatibility

This package is built with TypeScript and plays nicely with modern React environments, ensuring developers can easily adopt it in existing projects.

Try the Live Demo

Experience the potential of this technology by visiting the live demo, which showcases both the 3D avatar and the headless mode in action: Live Demo - Engage in conversations, observe lip-syncing, and interact seamlessly.

For comprehensive guidance, access the detailed README within the repository to explore advanced configurations and integration options.