ParallelFinder

ParallelFinder is a lightweight utility designed to streamline the training of multiple Keras models concurrently, offering an efficient way to compare their performance based on final loss and last-epoch duration. This tool is particularly useful for those looking to optimize their model selection process without the overhead of sequential training.

Key Features

• Concurrent Training: Supports the simultaneous training of multiple models by creating a separate process for each model using multiprocessing. This allows for faster experimentation and evaluation of diverse model architectures.
• Performance Tracking: Implements a shared, process-safe logging mechanism to track per-model and global best metrics, enhancing the understanding of model performance.

Components

ParallelFinder Class

• Initialization: Accepts a list of functions that produce compiled Keras models, setting the stage for a streamlined training process.

FinderCallback

• Monitors the duration of each training epoch and logs the final loss and time taken on completion of the last epoch, also updating global best metrics if new records are achieved.

Usage Example

To utilize ParallelFinder, define model constructors that return compiled Keras models:

from tensorflow import keras

def build_dense():
    model = keras.Sequential([
        keras.Input(shape=(784,)),
        keras.layers.Dense(128, activation='relu'),
        keras.layers.Dense(10, activation='softmax'),
    ])
    model.compile(optimizer='adam', loss='categorical_crossentropy')
    return model

def build_cnn():
    model = keras.Sequential([
        keras.Input(shape=(784,)),
        keras.layers.Reshape((28, 28, 1)),
        keras.layers.Conv2D(16, (3, 3), activation='relu'),
        keras.layers.Flatten(),
        keras.layers.Dense(10, activation='softmax'),
    ])
    model.compile(optimizer='adam', loss='categorical_crossentropy')
    return model

After defining models, instantiate the ParallelFinder and initiate training:

from parallel_finder import ParallelFinder
from tensorflow.keras.datasets import mnist
from tensorflow.keras.utils import to_categorical

# Load dataset
(x_train, y_train), _ = mnist.load_data()
x_train = x_train.reshape(-1, 784).astype('float32') / 255
 y_train = to_categorical(y_train, 10)

# List of model-building functions
model_constructors = [build_dense, build_cnn]

# Create finder instance
finder = ParallelFinder(model_constructors)

# Train models
finder.find(
    train_data=x_train,
    train_labels=y_train,
    epochs=5,
    batch_size=64
)

# Display results
for key, val in finder.logs.items():
    print(f'{key}: {val}')

Understanding the Results

The finder.logs provides detailed insights into the training outcomes, including:

• Individual model losses and epoch times.
• Global metrics like the best final loss and the shortest epoch time across all models.

Performance Notes

• GPU Management: In multi-GPU setups, configure Keras models appropriately to manage memory growth.
• Data Handling: For larger datasets, consider using lighter datasets or shared memory approaches to mitigate data transfer overhead between processes.
• Customization: Modify the FinderCallback to include additional metrics or validations as needed.