create-llm - Effortlessly train your own custom Language Model.

Projects Leaderboard

Pitch

Description

Comments

create-llm

chubby_indigo_leta

Effortlessly train your own custom Language Model.

Visit project

Pitch

create-llm simplifies the complex task of training language models by providing production-ready templates and tools in a single command. Support for various architectures and optimization for different use cases ensure rapid development, making it possible to go from concept to deployment seamlessly.

Description

create-llm
The fastest way to start training your own Language Model

Create production-ready LLM training projects in seconds. This tool functions like create-next-app, but for training custom language models, simplifying the complex process of setting up an LLM training environment.

📦 npm Package • 📖 Documentation • 🐛 Report Bug • 💡 Request Feature

npx @theanikrtgiri/create-llm my-awesome-llm
cd my-awesome-llm
pip install -r requirements.txt
python training/train.py

**That's it! You're training an LLM.** ✨

Why choose create-llm?

Training a language model from scratch can be a daunting task, requiring knowledge of several components, including:

✅ Model architecture (like GPT, BERT, T5)
✅ Data preprocessing pipeline
✅ Tokenizer training
✅ Training loop with callbacks
✅ Checkpoint management
✅ Evaluation metrics
✅ Text generation capabilities
✅ Deployment tools

create-llm streamlines this by providing all necessary tools in one command.

Key Features

🎯 Right-Sized Templates

Choose from four templates optimized for various use cases:

NANO (1M parameters) - Learn in 2 minutes on any laptop
TINY (6M parameters) - Prototype in 15 minutes on CPU
SMALL (100M parameters) - Production-grade models in hours
BASE (1B parameters) - Research-ready models in days

🔧 Comprehensive Toolkit

Includes everything needed out of the box:

PyTorch training infrastructure
Data preprocessing pipeline
Tokenizer training (BPE, WordPiece, Unigram)
Checkpoint management with auto-save
TensorBoard integration
Live training dashboard
Interactive chat interface
Model comparison tools
Deployment scripts

📊 Smart Defaults

Features intelligent configuration that:

Auto-detects vocabulary size from tokenizer
Warns about model/data size mismatches
Detects overfitting during training
Suggests optimal hyperparameters
Handles cross-platform paths

🎨 Plugin System

Optional integrations include:

WandB - Experiment tracking
HuggingFace - Model sharing
SynthexAI - Synthetic data generation

Quick Start

🚀 One-Command Setup

# Using npx (recommended - no installation needed)
npx @theanikrtgiri/create-llm my-llm

# Or install globally
npm install -g @theanikrtgiri/create-llm
create-llm my-llm

🎯 Interactive Setup

npx @theanikrtgiri/create-llm

You will be prompted for:

📝 Project name
🎯 Template (NANO, TINY, SMALL, BASE)
🔤 Tokenizer type (BPE, WordPiece, Unigram)
🔌 Optional plugins (WandB, HuggingFace, SynthexAI)

⚡ Quick Mode

# Specify everything upfront
npx create-llm my-llm --template tiny --tokenizer bpe --skip-install

Project Structure

my-llm/
├── 📁 data/
│   ├── raw/              # Your training data goes here
│   ├── processed/        # Tokenized data (auto-generated)
│   ├── dataset.py        # PyTorch dataset classes
│   └── prepare.py        # Data preprocessing script
│
├── 📁 models/
│   ├── architectures/    # Model implementations
│   ├── __init__.py
│   └── config.py        # Configuration loader
│
├── 📁 tokenizer/
│   ├── train.py         # Tokenizer training script
│   └── tokenizer.json   # Trained tokenizer (auto-generated)
│
├── 📁 training/
│   ├── train.py         # Main training script
│   ├── trainer.py       # Trainer class
│   └── dashboard/       # Live training dashboard
│
├── 📁 evaluation/
│   ├── evaluate.py      # Model evaluation
│   └── generate.py      # Text generation
│
├── 📁 plugins/          # Optional integrations
│
├── 📁 checkpoints/      # Saved models (auto-generated)
│
├── 📄 llm.config.js    # Main configuration file
├── 📄 requirements.txt # Python dependencies
├── 📄 chat.py         # Interactive chat interface
├── 📄 deploy.py       # Deployment script
└── 📄 README.md       # Project documentation

Complete Workflow

1️⃣ Create Your Project

npx @theanikrtgiri/create-llm my-llm --template tiny --tokenizer bpe
cd my-llm

2️⃣ Install Dependencies

pip install -r requirements.txt

3️⃣ Add Your Data
Place your text files in data/raw/:

# Example: Download Shakespeare
curl https://www.gutenberg.org/files/100/100-0.txt > data/raw/shakespeare.txt

# Or add your own files
cp /path/to/your/data.txt data/raw/

**💡 Pro Tip:** Start with at least 1MB of text for meaningful results

4️⃣ Train Tokenizer

python tokenizer/train.py --data data/raw/

**🔤 This creates a vocabulary from your data**

5️⃣ Prepare Dataset

python data/prepare.py

**📊 This tokenizes and prepares your data for training**

6️⃣ Start Training

# Basic training
python training/train.py

# With live dashboard (recommended!)
python training/train.py --dashboard
# Then open http://localhost:5000

# Resume from checkpoint
python training/train.py --resume checkpoints/checkpoint-1000.pt

**📈 Watch your model learn in real-time!**

7️⃣ Evaluate Your Model

python evaluation/evaluate.py --checkpoint checkpoints/checkpoint-best.pt

8️⃣ Generate Text

python evaluation/generate.py \
  --checkpoint checkpoints/checkpoint-best.pt \
  --prompt "Once upon a time" \
  --temperature 0.8

**✨ See your model's creativity in action!**

9️⃣ Interactive Chat

python chat.py --checkpoint checkpoints/checkpoint-best.pt

**💬 Chat with your trained model!**

🔟 Deploy

# To Hugging Face
python deploy.py --to huggingface --repo-id username/my-model

# To Replicate
python deploy.py --to replicate --model-name my-model

**🚀 Share your model with the world!**

Advanced Features

Live Training Dashboard

Monitor training in real-time with a web interface, providing insights into loss curves, learning rate schedules, and resource usage.
Start the dashboard using:

python training/train.py --dashboard

Model Comparison

Quickly compare multiple trained models for performance and metrics:

python compare.py checkpoints/model-v1/ checkpoints/model-v2/

Checkpoint Management

Utilizes automatic checkpoint management to save the best model based on validation loss, maintain configurable checkpoints, and resume training from any checkpoint.

Custom Plugin Development

Create custom plugins to extend functionality, enhancing the training experience or integrating with other services.

Best Practices

Data Preparation

Data Quantity:
- NANO: 100+ examples
- TINY: 1,000+ examples
- SMALL: 10,000+ examples
- BASE: 100,000+ examples
Data Quality:
- Use clean, well-formatted text with utmost consistency in encoding (UTF-8).

Training Tips

Avoid Overfitting: Monitor perplexity during training and adjust model complexity as needed.
Optimize Training: Use smaller models for smaller datasets to attain reliable results.

Troubleshooting

Common issues may arise, such as vocabulary size mismatches or memory errors. Refer to detailed guidance in the documentation for resolution strategies.

Getting Help

Roadmap

Future releases aim to include additional model architectures, distributed training support, model quantization tools, and a web UI for easier project management, along with more deployment targets.

Acknowledgments

Utilizes contributions from numerous open-source tools and the LLM community, enhancing usability and functionality.