A.P.E.X. - Adaptive Parallel Extremal Dispatch

A.P.E.X. is a cutting-edge Java sorting framework meticulously designed for high-performance processing of large fixed-width 64-bit key/value record datasets. It leverages several advanced techniques, including descriptor-driven radix planning, parallel scatter, per-bucket dispatch, tuple projection, tiny-sort fallbacks, and local refinement. This allows A.P.E.X. to efficiently sort unsigned 64-bit keys while also offering an optional mode for signed-key ordering, thereby minimizing unnecessary passes over bits that have already been resolved.

Key Features

• Adaptive Execution: A.P.E.X. adapts its execution strategy to the observed structure of the input data, optimizing the radix geometry and bucket routing dynamically, rather than relying on fixed pre-defined strategies.
• Parallel Processing: Designed with multi-core hardware in mind, the framework supports concurrent histogramming, scatter operations, and bucket refinement, ensuring maximum efficiency.
• Extremal Descriptor Usage: By employing per-bucket extremal descriptors using bitwise operations, A.P.E.X. intelligently identifies which key bits remain variable, thus optimizing subsequent processing steps.

Innovative Dispatching Model

The dispatching mechanism of A.P.E.X. routes buckets through the most efficient available paths, such as:

• Global and bucket reverse techniques
• Tiny sort operations
• Projection of tuples
• LSD (Least Significant Digit) and refinement cycles

Interactive Visualizer

A.P.E.X. comes equipped with an interactive browser-based visualizer that provides a comprehensive explanation of the execution plan. This includes details such as the source array layout, MSD scatter paths, tiny sort routes, tuple routing, LSD refinement mechanisms, and the final sorted arrangement. The visualizer can be accessed here and supports various input control configurations, allowing users to experiment with different record counts and data types.

Documentation

Comprehensive documentation is available, including:

• Data and Descriptors: Insights into record layout, data topologies, and descriptors.
• Usage Guide: Detailed commands, runtime options, and common scenarios for use.
• Comparison Guide: Benchmarking tools and baselines for evaluation.
• Operation Model: In-depth analysis of execution flow and dispatch rules.

Requirements

To run A.P.E.X., the following are recommended:

• JDK 25 or newer with jdk.incubator.vector
• Adequate memory allocation for handling large datasets

Build Instructions

To compile A.P.E.X., run the provided commands tailored for PowerShell or Bash shell environments. Follow the build instructions detailed in the README for successful setup.

Usage

Execute the main sorting program with configurable parameters, adjusting for record count, mode, and thread count:

java --enable-preview --enable-native-access=ALL-UNNAMED --add-modules jdk.incubator.vector -Xmx16G -XX:MaxDirectMemorySize=80g -cp out main.Apex mode=RANDOM records=10m threads=16

Data Modes

A.P.E.X. supports various data modes including RANDOM, SORTED, DUPLICATES, and several others, providing flexibility for various testing scenarios.

A.P.E.X. is intrinsically designed to facilitate algorithm experimentation, performance analytics, and visual exploration of adaptive radix dispatch behaviors, making it a valuable asset for researchers and practitioners in data science and software engineering.