Dolphin redefines deep learning by enabling custom transformer architectures and symbolic computation without the need for external libraries. This pure Python implementation allows a clear understanding of tensor operations and backpropagation, making it an ideal tool for researchers and developers eager to explore the fundamentals of machine learning.
Dolphin serves as an experimental transformer stack and symbolic computation engine, crafted entirely from first principles using pure Python. This framework stands out by omitting external libraries, such as NumPy and PyTorch, ensuring that all operations—ranging from tensor arithmetic to backpropagation and attention mechanisms—are executed through handwritten, minimal Python logic.
Symbolic Autodiff Engine: Dolphin includes a fully custom Tensor class supporting 1D, 2D, and 3D tensors, capable of tracking computation graphs and gradients, while implementing backpropagation with minimal overhead.
Transformer Stack: A pure Python implementation of a transformer architecture that encompasses various essential components:
Zero-Dependency System: Dolphin operates without the need for any external libraries. It is fully compatible with vanilla Python, eliminating dependencies like NumPy or PyTorch.
Dolphin's codebase is organized into several modules, each serving a distinct purpose:
tensor.py: Contains the core symbolic engine along with operations for tensor manipulation, backward propagation, and gradient tracking.transformers.py: Implements the complete transformer encoder stack, including multi-head self-attention and feedforward layers.activations.py: Provides activation functions and loss calculations without relying on external libraries.layers.py: Includes utilities and normalization layers for constructing neural architectures.optimizers.py: Features optimization algorithms for updating model parameters, including implementations of SGD, Adam, and Momentum.DolphinTest01.py demonstrates a complete pipeline showcasing Dolphin's capabilities, including:
Dolphin aims to explore the inner workings of transformers and automatic differentiation, providing transparency and flexibility for researchers, students, and developers. It is not designed as a high-performance framework; rather, it serves as a comprehensive educational tool, ideal for:
Dolphin provides an insightful look into the principles of modern machine learning, making it suitable for those interested in the foundations of deep learning without the abstractions commonly found in other libraries.
No comments yet.
Sign in to be the first to comment.