NeuroBrix

NeuroBrix serves as an innovative open-source inference engine that facilitates the running of a myriad of AI models, including language models, vision-language models, speech recognition, speech synthesis, image generation, and video processing. Designed for ease of use, it allows developers to operate these models from a single container format and command-line tool without the need to create or manage separate pipelines for each model type. NeuroBrix exclusively executes pre-published models packaged as .nbx containers, which can be downloaded from the NeuroBrix Hub.

Key Features

• Unified Engine and Container Format: Operate various AI models using one engine and container type. Models packaged as .nbx hold their computation graphs, weights, and connections. The NeuroBrix engine efficiently processes these models across different modalities, such as language generation, speech processing, and image outputs, through common execution flows.

• Dual Compute Branches: NeuroBrix supports two compute branches that utilize the same container format. The default PyTorch branch executes models using NVIDIA GPU capabilities, while the Triton branch (--triton) employs custom Triton kernels, providing versatility in operational execution.

• Intelligent Device Placement: The engine automatically assesses the available hardware and memory, planning optimal component placements across available devices. Users can preview the chosen execution plan and alternatives without loading the model using the command neurobrix run --explain-plan.

• Predictive Resource Handling: When a request cannot be handled due to hardware limitations, NeuroBrix provides clear feedback specifying the resources needed against those available.

Supported Hardware

Currently, NeuroBrix operates on NVIDIA GPUs and CPUs, with support for Apple Silicon in development. Note that AMD GPUs and Windows environments are not officially supported.

Model Hub

NeuroBrix offers access to a diverse range of models categorized into various domains such as language processing, speech understanding, image and video generation, and more. Each model can be easily accessed and imported from the hub, enabling developers to quickly utilize AI capabilities with minimal setup.

Example Commands

NeuroBrix simplifies workflow with straightforward commands. Below are examples of how to run different models:

# Example for a language model  
neurobrix run --model TinyLlama-1.1B-Chat-v1.0 --prompt "Explain recursion." --max-tokens 512

# Example for speech-to-text  
neurobrix run --model whisper-large-v3-turbo --audio recording.wav

# Example for image generation  
neurobrix run --model Flex.1-alpha --prompt "a lighthouse at dusk" --steps 20 --seed 42 --output lighthouse.png

Contributing and Support

The NeuroBrix project welcomes contributors for bug reports, model requests, and code enhancements. Detailed documentation is available for developers to help streamline their contributions. Access the documentation and participate in the project's growth.