LeanPass

LeanPass is a lightweight and transparent library designed for neural network experiments, built using NumPy. This library is especially well-suited for small-scale projects and educational purposes, offering a clear and readable codebase that makes it easy to understand the underlying mechanics of automatic differentiation.

Key Features

• Minimal Dependencies: LeanPass relies solely on NumPy, ensuring a minimal footprint for easy installation and usage.
• Readable Implementation: Rather than opting for heavy abstraction, LeanPass provides a clear structure that simplifies comprehension and experimentation.
• Core Building Blocks: The library includes essential components like tensors, layers, and optimizers, facilitating efficient model building and experimentation.

Functionality

LeanPass offers various capabilities:

• Tensor Objects with Autodiff: Supports reverse-mode automatic differentiation for gradient calculations.
• Activation Functions: Implements popular functions including ReLU, sigmoid, softmax, tanh, LeakyReLU, and GELU.
• Layering Options: Contains linear layers and multilayer perceptrons for complex model architectures.
• Loss Functions: Provides utilities for common loss calculations such as mean squared error, cross-entropy, and binary cross-entropy.
• Optimizers: Includes Stochastic Gradient Descent (SGD) and Adam optimizers, both of which support additional parameters like momentum and L2 weight decay.
• Dropout Layer: Facilitates dropout during training for improved regularization.
• Gradient Inspection: Easily inspect gradients after backpropagation for debugging and understanding model behavior.
• Computation Graph Visualization: Capable of exporting computation graphs for clarity on data flow and operations within the model.

Quick Start Example

To get started with LeanPass, a simple example can be utilized as follows:

from leanpass import Tensor, nn

x = Tensor([[1.0, 2.0]], requires_grad=False)
model = nn.MLP([2, 16, 3])
logits = model(x)
print(logits)

Training Loop Example

A demonstration of a complete training loop is provided, which includes synthetic data and visible loss metrics:

import numpy as np
from leanpass import Tensor, nn, optim

np.random.seed(1)
x_data = np.random.randn(150, 2)
y_data = np.eye(3)[np.random.randint(0, 3, 150)]

model = nn.MLP([2, 16, 3])
optimizer = optim.Adam(model.parameters(), lr=0.01)

for step in range(100):
    x = Tensor(x_data, requires_grad=False)
    y = Tensor(y_data, requires_grad=False)
    
    logits = model(x)
    loss = nn.cross_entropy_loss(logits, y)

    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if step % 20 == 0:
        print(f"Step {step:03d} — Loss: {loss.data:.6f}")

Documentation

Comprehensive documentation, including API references and hands-on guides for various tasks, is available at LeanPass Documentation.

LeanPass seeks to empower users to explore neural networks efficiently while minimizing complexity, making it an excellent resource for educators, researchers, and hobbyists alike.