AdaptiveFlow

AdaptiveFlow is a lightweight Java 21 library designed to execute task directed acyclic graphs (DAGs) using Virtual Threads without any runtime dependencies. This library enables efficient workflow orchestration by providing a fluent API that allows users to construct, validate, and execute workflows seamlessly with minimal overhead.

Key Features

• Simple and Concise: With fewer than 15 public types, AdaptiveFlow leverages Java 21's features—like virtual threads, records, and CompletableFuture—to deliver a straightforward and efficient orchestration solution. No bloated frameworks or configurations are necessary.
• Task Execution with Virtual Threads: Each task runs in its own virtual thread, maximizing resource utilization and minimizing management overhead. This approach guarantees that various branches of tasks can execute concurrently, mirroring the machine's capabilities.
• Fluent Workflow Definition: Build workflows in a single pass and let the engine handle validation and execution. Tasks can be defined to run in parallel or sequentially, enhancing performance and clarity.

Example Usage

Here's a brief example illustrating how to define and execute a workflow using AdaptiveFlow:

import io.github.varun51.adaptiveflow.WorkflowBuilder;
import io.github.varun51.adaptiveflow.WorkflowResult;
import io.github.varun51.adaptiveflow.RetryPolicy;
import java.time.Duration;

WorkflowResult result = WorkflowBuilder.builder("etl")
        .task("extract", ctx -> fetchRows())
        .task("validate", ctx -> validate(ctx.<Rows>result("extract")))
        .parallel(
                WorkflowBuilder.TaskRef.of("summary", ctx -> summarize(ctx.<Rows>result("extract"))),
                WorkflowBuilder.TaskRef.of("archives", ctx -> archive(ctx.<Rows>result("extract")))
        )
        .task("load", ctx -> load(ctx.<Summary>result("summary"), ctx.<Archive>result("archives")))
        .retry(RetryPolicy.exponentialBackoff(4, Duration.ofMillis(100)))
        .execute();

if (result.isSuccess()) {
    Summary summary = result.<Summary>result("summary");
}

• Robust Error Handling: The library provides sophisticated retry policies for task failures, allowing users to define behaviors for task retries, including fixed delays or exponential backoffs with jitter to prevent synchronized failures.
• Built for Performance: AdaptiveFlow minimizes overhead and maximizes efficiency through end-to-end optimized operations, validated by benchmarking results showing thousands of workflows processed per second.

Future Directions

Post 1.0, AdaptiveFlow aims to introduce adaptive scheduling capabilities that are resource-aware, with features that evolve based on live execution patterns.

This Java library is ideal for developers seeking an efficient way to orchestrate complex workflows within Java applications using the latest advancements in the Java ecosystem.