dolphin

Dolphin

Dolphin serves as an experimental transformer stack and symbolic computation engine, crafted entirely from first principles using pure Python. This framework stands out by omitting external libraries, such as NumPy and PyTorch, ensuring that all operations—ranging from tensor arithmetic to backpropagation and attention mechanisms—are executed through handwritten, minimal Python logic.

Key Features

• Symbolic Autodiff Engine: Dolphin includes a fully custom Tensor class supporting 1D, 2D, and 3D tensors, capable of tracking computation graphs and gradients, while implementing backpropagation with minimal overhead.

• Transformer Stack: A pure Python implementation of a transformer architecture that encompasses various essential components:

  • Multi-head self-attention mechanism
  • Layer normalization
  • GELU activation functions
  • Feedforward layers with residual paths

• Zero-Dependency System: Dolphin operates without the need for any external libraries. It is fully compatible with vanilla Python, eliminating dependencies like NumPy or PyTorch.

Project Structure

Dolphin's codebase is organized into several modules, each serving a distinct purpose:

• tensor.py: Contains the core symbolic engine along with operations for tensor manipulation, backward propagation, and gradient tracking.
• transformers.py: Implements the complete transformer encoder stack, including multi-head self-attention and feedforward layers.
• activations.py: Provides activation functions and loss calculations without relying on external libraries.
• layers.py: Includes utilities and normalization layers for constructing neural architectures.
• optimizers.py: Features optimization algorithms for updating model parameters, including implementations of SGD, Adam, and Momentum.

Example Pipeline

DolphinTest01.py demonstrates a complete pipeline showcasing Dolphin's capabilities, including:

• Tokenizing and preprocessing text from the Brown corpus using NLTK
• Building a vocabulary and transforming text into integer sequences
• Initializing embedding weights and feeding tokens into a multi-layer transformer stack
• Training with backpropagation and using an Adam optimizer to refine the model
• Generating sequences through a greedy sampling loop

Purpose and Value

Dolphin aims to explore the inner workings of transformers and automatic differentiation, providing transparency and flexibility for researchers, students, and developers. It is not designed as a high-performance framework; rather, it serves as a comprehensive educational tool, ideal for:

• Learning fundamental concepts of deep learning architectures
• Experimenting with symbolic tensor operations
• Understanding the mechanics behind library-free machine learning implementations

Dolphin provides an insightful look into the principles of modern machine learning, making it suitable for those interested in the foundations of deep learning without the abstractions commonly found in other libraries.