MAPLE: Manifold-Activated Pruning of Latent Experts

MAPLE: Manifold-Activated Pruning of Latent Experts

MAPLE is a state-of-the-art framework designed for the structural compression of Mixture-of-Experts (MoE) models, enabling significant optimizations that allow powerful language models, such as Gemma 4 26B-A4B, to operate efficiently within the constraints of consumer laptops.

Key Features

• Efficient Compression: Reduces the model size from 9.92 GB to 4.75 GB, achieving 1.48 effective bits/parameter. This enables 100% in-VRAM GPU execution on 6 GB laptops, achieving a throughput of 45.4 Tokens/Second.
• Targeted Model Exhaustive Profiling: Utilizing advanced profiling techniques, MAPLE prunes dormant multilingual and general experts to retain only those essential for technical reasoning and code synthesis, improving the effectiveness of the model drastically.

Domain Specialization

MAPLE is finely tuned for tasks involving Python programming, algorithm synthesis, data structures, and formal STEM reasoning. The resultant 4.75 GB checkpoint is optimized to contain only the most relevant aspects needed for effective technical discourse.

Custom Domain Pruning Pipeline

MAPLE's pruning pipeline is flexible and can be adapted to various specialized fields, such as medical, financial, legal, or creative domains. The process involves three core steps:

1. Calibration: Profile routing frequency and activation energy using a custom text corpus.
2. Selection: Experts are scored based on a composite energy metric, leading to a refined selection.
3. Container Assembly: Construct the optimized GGUF model for seamless in-VRAM execution.

Model Execution

The optimized 4.75 GB GGUF model can be executed directly with llama.cli:

llama-cli \
  -hf AmSach/Gemma-4-26B-A4B-MAPLE-Sub5GB-GGUF \
  -ngl 99 \
  -c 2048 \
  -n 512 \
  --temp 0.4 \
  --top-p 0.9 \
  --repeat-penalty 1.1 \
  -r "<turn|>" \
  -p "<|turn>user\nExplain why the sky is blue using Rayleigh scattering.\n<turn|>\n<|turn>model\n"

Performance Metrics

Benchmarking reveals a 2.09x size reduction and a 6.57x speedup in generation speed compared to the original model. MAPLE achieves 100% fidelity in tasks such as Fibonacci calculation and physics reasoning, providing robustness in its domain of specialization.

Overcoming Challenges

MAPLE addresses common pitfalls in expert activation estimates through innovative profiling methods and adjustments to ensure model integrity during execution. This makes it a reliable choice for technical applications requiring precision and performance.

Repository Structure

The repository includes necessary scripts for calibration, expert selection, and benchmark testing, facilitating easy integration and usage.

Model Distribution

The optimized model is available for download on Hugging Face, providing easy access to the compressed GGUF model container:

• Model Repository
• Direct Download

MAPLE is an innovative solution for enhancing the capabilities of language models while minimizing resource use, fostering more efficient and powerful applications across various technical domains.