autodrive

Autonomous 2D Parking Agent with Deep Q-Learning

The Autodrive project is an advanced implementation of an autonomous 2D parking agent, expertly constructed from scratch using Rust and the Bevy Engine. This agent harnesses the power of a multilayer perceptron (MLP) and a custom Deep Q-Network (DQN) training loop to master the challenge of parking in dynamically generated virtual environments.

Key Features

• Custom Neural Network: A fully implemented MLP neural network designed in Rust.
• Deep Q-Network Training: Utilizes experience replay and a target network for improved learning efficiency.
• Lidar Sensing: Employs ray-based lidar for enhanced environment perception.
• Car Dynamics: Features unique 2D car dynamics and collision detection mechanisms.
• Randomized Environments: Generates procedurally randomized maps and parking challenges to mimic real-world variability.
• Reward Shaping: Implements custom reward systems tailored for effective autonomous parking.
• Model Management: Offers model serialization and real-time greedy inference from previously saved models.
• Visualization: Integrates real-time simulation and visual feedback through Bevy ECS.

Getting Started

To initiate a training session with a randomly initialized model, the following command can be utilized:

cargo run

To continue training from a saved model, use:

cargo run -- --model=path/to/model.bin --train

For inference with a previously saved model, execute:

cargo run -- --model=path/to/model.bin

Controls

In inference mode, pressing Space allows users to abandon the current episode, register it as a crash, generate a new map, and reset the vehicle.

Technical Architecture

This implementation features a robust architectural design that efficiently manages the model and training processes:

• Model Architecture: Detailed descriptions of the MLP topology and state encoding can be found in the Model Architecture documentation.
• Training Dynamics: Insights into the DQN training methodology, including experience replay and optimization techniques, are available in DQN Training Architecture.
• Environmental Setup: The application features Bevy entities representing the car, obstacles, sensors, and goals where each session utilizes randomized maps.
• Observations and Actions: The agent calculates a 9-value state vector based on lidar distances, speed, distance to the goal, and relative angles, which it utilizes to determine actions based on the Q-values produced by the MLP.

Results and Analysis

Benchmark results derived from the model_V1.bin checkpoint include simulation metrics such as episode counts, success rates, crashes, and average steps. Documentation on the reward baselines and empirical observations is provided in the Reward Analysis V1. Future versions will include updates on reward calculation methodologies.

Technology Stack

• Rust 2024: Foundation for the application, agent logic, and simulation processes.
• Bevy 0.19: Manages ECS scheduling, windowing, and rendering, enhancing the application loop.
• rand 0.10: Facilitates procedural generation of maps and goals.

This project serves as an innovative exploration of reinforcement learning in the realm of autonomous vehicle simulation, providing insights into the intricacies of deep learning methodologies while ensuring a practical demonstration of capabilities.