PRKL Artificial Neural Network Framework

The PRKL-ANN is an educational framework designed specifically for those looking to learn and experiment with Artificial Neural Networks (ANNs) using C++. The framework is built with an intuitive API, allowing users to easily navigate through its features.

Key Features

• Multi-Layer Perceptrons (MLPs): Supports fully connected layers that facilitate the construction of neural network models.
• Stochastic Gradient Descent (SGD): Implements backpropagation-based learning using SGD, a crucial algorithm in training neural networks.
• Adaptive Learning Rate (ALR): Dynamically adjusts learning rates to enhance convergence stability during model training.
• Swish Activation Function: Utilizes the Swish activation function and its derivative for neuron activations, promoting better learning outcomes.
• Configurable Training Process: Offers a highly configurable training environment with stable defaults that yield quick convergence, exemplified on the popular MNIST dataset.
• Pre-Trained MNIST Models: Includes models that achieve an impressive 97.59% accuracy with a loss factor of 0.0466, streamlining the learning and experimentation process.
• Custom Dataset Formats: Features custom binary formats for datasets (.prklset) and trained models (.prklmodel), providing flexibility in data handling.
• Command-Line Interface (CLI) Tools: Equips users with CLI tools for training and evaluating models, accommodating various datasets and supervised learning setups.

Limitations

It is important to note what PRKL-ANN is not currently optimized for:

• Performance optimization (lazy memory management and lack of focus on cache efficiency).
• Convolutional Neural Networks (CNNs) and Large Language Models (LLMs) or Transformers support.

Example Usage

Training and evaluating models with PRKL-ANN can be accomplished through simple CLI commands. Here’s how you might train a model using a dataset and evaluate it afterwards:

# Train a model with a dataset  
prkl-train -t dataset.prklset -o model.prklmodel -p 50 -c 784,128,64,64,10  

# Evaluate the trained model  
prkl-evaluate -e evaluation.prklset -m model.prklmodel  

To conduct inference on models, create the model in C++ via ann_model, utilize ann_model::forward_propagate(), and retrieve activations from the output layer. A sample program demonstrating this is available in mnist-digits.cpp.

Importing Datasets

Importing datasets into PRKL-ANN is straightforward despite limited documentation. For instance, a Python script can be utilized to import the original MNIST dataset:

from mlxtend.data import loadlocal_mnist  
import numpy as np  
import struct  
  
D, L = loadlocal_mnist(  
        images_path='train-images.idx3-ubyte',   
        labels_path='train-labels.idx1-ubyte')  
  
with open("mnnist-digits-training.prklset", "wb") as f:  
    f.write(struct.pack("!QQQ", 28*28, 10, D.shape[0]))  
    for i in range(D.shape[0]):  
        for d in D[i]:  
            f.write(struct.pack("!1f", float(d) / 255.0))  
        l = np.zeros(10, dtype=np.float32)  
        l[L[i]] = 1.0  
        f.write(struct.pack("!10f", *l))  

Roadmap

Future enhancements include:

• Implementing dropout layers for regularization.
• Adding support for the Adam optimizer.
• Introducing Convolutional Neural Networks (CNNs).
• Exploring recurrent networks for sequence-based tasks.