Yet Another ENV Script

A streamlined environment setup script, Yet Another ENV Script (YAES), is designed for efficient AI development on RunPod and other Linux-based platforms. Specifically tailored for PyTorch templates, this script facilitates a quick and seamless setup process by leveraging uv for superior dependency management and integrating high-performance features from Hugging Face.

Key Features

• Ultra-fast Installation: Utilizes uv instead of conventional pip, leading to significantly faster dependency resolution and installation times.
• Isolated Environments: Constructs two separate virtual environments to mitigate dependency conflicts between quantization and inference tasks.
• High-performance Transfer: Automatically activates the features HF_HUB_ENABLE_HF_TRANSFER and HF_XET_HIGH_PERFORMANCE, optimizing model download speeds.
• CUDA Optimized: Preconfigured for PyTorch with support for CUDA 13.0, ensuring hardware acceleration is enabled right from the start.

Usage Instructions

After executing the setup script, two activation scripts are generated:

• Activate Quantization Environment
  Use the following command:

  source activate_quant.sh  
  

  This environment includes essential packages such as llmcompressor, transformers, accelerate, and more.

• Activate Inference Environment (vLLM)
  To activate, run:

  source activate_vllm.sh  
  

  This environment provides vllm==0.18.0 and relevant Hugging Face components.

• Exit Environment
  To deactivate any active environment, use:

  deactivate  
  

The activation scripts also automatically configure key environment variables to enhance performance, enabling efficient model handling and operation.

Note: Ensure NVIDIA drivers are installed and monitor disk space, as storage requirements may increase with AI models and virtual environments.