picodl

picodl is a lightweight deep learning library created from the ground up in Python, featuring a comprehensive autograd engine, modular layers, loss functions, optimizers, and a training loop. This unique framework, built entirely on top of plain NumPy, provides developers with a transparent and educational environment that eliminates the reliance on more complex frameworks like PyTorch or TensorFlow.

Key Features:

• Autograd Engine: Effortlessly compute gradients with a hand-crafted implementation that provides clear insights into the backpropagation process.
• Custom Neural Network Modules: Build neural networks using fundamental components, such as layers and activation functions, allowing for complete control over architecture.
• Optimizers: Incorporate popular optimization algorithms to enhance model training.
• No Hidden Frameworks: Every component operates independently, promoting an understanding of deep learning mechanics without the abstractions present in larger libraries.

Documentation

For detailed usage guidelines and API references, visit the documentation.

Example Usage

Here’s a simple example of how to define and train a neural network using picodl:

from picodl.nn import NeuralNet
from picodl.layers import Linear, GELU
from picodl.loss import CrossEntropyLoss
from picodl.optim import AdamW
from picodl.train import train

net = NeuralNet([
    Linear(784, 128),
    GELU(),
    Linear(128, 10),
])

train(net, x_train, y_train,
      num_epochs=10,
      loss=CrossEntropyLoss(),
      optimizer=AdamW(lr=0.001, weight_decay=0.01))

net.save("model.picodl")

Future Enhancements

• GPU Acceleration: Plans are in place to implement GPU support for faster training times.
• Custom Math Backend: Development of a specialized math module, named picomath, written in C, to enhance performance further.