Parkiet - Create realistic Dutch speech with a powerful TTS model.

Parkiet

Create realistic Dutch speech with a powerful TTS model.

Pitch

Parkiet is an advanced Dutch text-to-speech model with 1.6B parameters. Based on the Parakeet architecture, it enables highly realistic voice generation from text, including emotional tone and nonverbal sounds. The model supports voice cloning and multiple speakers, providing versatility for creative audio applications.

Description

Parkiet is an advanced Dutch text-to-speech (TTS) model equipped with 1.6 billion parameters, designed for high-quality and realistic voice synthesis. This model is built upon the innovative Parakeet architecture and has been adapted from Dia to JAX, enabling scalable training capabilities.

Key Features

Realistic Voice Generation: Parkiet excels at producing lifelike voice outputs from text inputs, making it suitable for a variety of applications.
Emotion and Tone Control: Users can adjust the emotional tone of the generated speech, allowing for expressive audio outputs.
Support for Multiple Speakers: The model can generate speech with up to four distinct speakers within a single prompt, enhancing versatility in dialogue and storytelling.
Voice Cloning: Parkiet allows for the cloning of voices with minimal audio input, achieving impressive results with just a small amount of sample data.

Here is a sample of the capabilities:

Text	Audio Sample
[S1] denk je dat je een open source model kan trainen met weinig geld en middelen? [S2] ja ik denk het wel. [S1] oh ja, hoe dan? [S2] nou kijk maar in de repo op Git Hub of Hugging Face.
[S1] hoeveel stemmen worden er ondersteund? [S2] nou, uhm, ik denk toch wel meer dan twee. [S3] ja, ja, d dat is het mooie aan dit model. [S4] ja klopt, het ondersteund tot vier verschillende stemmen per prompt.
[S1] h h et is dus ook mogelijk, om eh ... uhm, heel veel t te st stotteren in een prompt.
[S1] (laughs) luister, ik heb een mop, wat uhm, drinkt een webdesigner het liefst? [S2] nou ... ? [S1] Earl Grey (laughs) . [S2] (laughs) heel goed.
[S1] je hebt maar weinig audio nodig om een stem te clonen de rest van deze tekst is uitgesproken door een computer. [S2] wauw, dat klinkt wel erg goed. [S1] ja, ik hoop dat je er wat aan hebt.

Usage Guidelines

When generating text with multiple speakers, utilize [S1], [S2], [S3], and [S4] to denote speakers, ensuring a coherent dialogue flow. Prompts should be crafted using lower case letters with proper punctuation, and appropriate stuttering or laughter can be included to enhance realism. For optimal performance, it is advised to adhere to the output style exemplified by WhisperD-NL.

Model Performance

The Parkiet model showcases superior quality when run on JAX, although it may require more setup than its PyTorch counterpart. In comparison, the PyTorch implementation tends to generate more artifacts due to minor differences in the attention kernel.

Research and Development

Parkiet is intended for research and educational purposes. Ethical guidelines prohibit misuse, including identity misrepresentation, creation of misleading content, and any illegal activities. Responsible use is essential in maintaining the integrity and ethical standards of this technology.

For comprehensive details on training the model for various languages, including data preparation and TPU setup, refer to the TRAINING.md document.

Acknowledgments

Attributions are extended to Darefsky for his insightful work on Parakeet, the Dia team for their open-source contributions, and Bas Büller for ongoing discussions throughout the development process.

0 comments

No comments yet.

New comment