RBFleX-NAS: Training-Free Neural Architecture Search Leveraging Radial Basis Function Kernels

RBFleX-NAS introduces an innovative solution for Neural Architecture Search (NAS) by utilizing Radial Basis Function (RBF) kernels to enhance efficiency in the evaluation of neural networks without the need for extensive training. Traditional NAS techniques typically involve comprehensive training phases that can be both time-consuming and computationally demanding. RBFleX-NAS addresses these hurdles by offering a training-free framework that optimally distinguishes between well-performing and poorly-performing architectures, significantly improving performance accuracy with reduced evaluation time.

Key Features:

• Superior Performance: RBFleX-NAS surpasses existing training-free NAS methodologies, providing enhanced top-1 accuracy while keeping the search time short, as evidenced in benchmarks such as NAS-Bench-201 and NAS-Bench-SSS.
• Optimal Hyperparameter Detection: Incorporating an advanced detection algorithm, RBFleX-NAS effectively identifies the best hyperparameters utilizing the outputs from activation functions and last-layer input features.
• Expanded Activation Function Exploration: The framework extends activation function designs through NAFBee, a new benchmark that allows for diverse exploration of activation functions, significantly benefiting the search for the best-performing networks.

Environmental Requirements:

RBFleX-NAS operates in a Python 3 environment and requires the following libraries:

• numpy
• torchvision
• torch
• scipy
• nats_bench

Supported NAS Benchmarks:

RBFleX-NAS is designed to work seamlessly with leading NAS benchmarks such as:

• NAS-Bench-201
• NATS-Bench-SSS
• Network Design Space
• TransNAS-Bench-101

Download Datasets:

RBFleX-NAS is compatible with a variety of image datasets including CIFAR-10, CIFAR-100, ImageNet, Taskonomy, and SST-2, facilitating extensive training and testing.

Usage Example:

To utilize RBFleX-NAS, users can configure parameters for their experiments, such as setting batch sizes, specifying datasets, and determining the number of trials as shown in the Python example below:

# Example configuration for experimentation  
batch_size_NE = 3  
dataset = 'cifar10'  
maxtrials = 10  
Num_Networks = 1000  

RBFleX-NAS is an essential tool for researchers and practitioners aiming to optimize neural network architectures efficiently. With its training-free approach, it not only saves time and resources but also enhances the ability to accurately predict optimal architecture performance.

Paper (IEEE TNNLS)

https://ieeexplore.ieee.org/document/10959729