Note_rl

notedance

A library for training RL agents in Keras and PyTorch.

1likes0comments

Visit project 1

Project details

About this project

This library simplifies the process of training reinforcement learning agents using Keras and PyTorch. By inheriting from the provided classes, training agents becomes straightforward, allowing users to focus on developing cutting-edge AI without the typical complexities of algorithm implementation.

Note_rl is a powerful reinforcement learning library designed specifically for use with Keras and PyTorch, allowing users to effectively train agents with minimal effort. By simply inheriting from the RL or RL_pytorch classes, developers can streamline the training process for their models built within these frameworks.

Key Features

Ease of Use: The library simplifies the implementation of reinforcement learning algorithms, making it accessible for both beginners and experienced practitioners.
Flexible Agent Training: Example training configurations for various methods, including DQN, PPO, and DDPG, are provided for both Keras and PyTorch frameworks.

Example Usage

Below are some code snippets illustrating how to utilize Note_rl for training agents in both Keras and PyTorch:

Training with Keras

DQN Example

import tensorflow as tf
from Note_rl.policy import EpsGreedyQPolicy
from Note_rl.examples.keras.DQN import DQN

model = DQN(4, 128, 2)
model.set(policy=EpsGreedyQPolicy(0.01), pool_size=10000, batch=64, update_steps=10)
optimizer = tf.keras.optimizers.Adam()
train_loss = tf.keras.metrics.Mean(name='train_loss')
model.train(train_loss, optimizer, 100)

PPO Example

import tensorflow as tf
from Note_rl.policy import SoftmaxPolicy
from Note_rl.examples.keras.PPO import PPO

model = PPO(4, 128, 2, 0.7, 0.7)
model.set(policy=SoftmaxPolicy(), pool_size=10000, batch=64, update_steps=1000, PPO=True)
optimizer = [tf.keras.optimizers.Adam(1e-4), tf.keras.optimizers.Adam(5e-3)]
train_loss = tf.keras.metrics.Mean(name='train_loss')
model.train(train_loss, optimizer, 100)

Training with PyTorch

DQN Example

import torch
from Note_rl.policy import EpsGreedyQPolicy
from Note_rl.examples.pytorch.DQN import DQN

model = DQN(4, 128, 2)
model.set(policy=EpsGreedyQPolicy(0.01), pool_size=10000, batch=64, update_steps=10)
optimizer = torch.optim.Adam(model.param)
model.train(optimizer, 100)

Multi-Agent Example

import torch
from Note_rl.policy import SoftmaxPolicy
from Note_rl.examples.pytorch.MADDPG import DDPG

model = DDPG(128, 0.1, 0.98, 0.005)
model.set(policy=SoftmaxPolicy(), pool_size=3000, batch=32, trial_count=10, MARL=True)
optimizer = [torch.optim.Adam(model.param[0]), torch.optim.Adam(model.param[1])]
model.train(optimizer, 100)

Distributed Training

The library supports distributed training techniques like MirroredStrategy and MultiWorkerMirroredStrategy, enabling efficient scaling of the training process across multiple GPUs or machines.

Example for MirroredStrategy

import tensorflow as tf
from Note_rl.policy import EpsGreedyQPolicy
from Note_rl.examples.keras.DQN import DQN

strategy = tf.distribute.MirroredStrategy()
BATCH_SIZE_PER_REPLICA = 64
GLOBAL_BATCH_SIZE = BATCH_SIZE_PER_REPLICA * strategy.num_replicas_in_sync

with strategy.scope():
  model = DQN(4, 128, 2)
  optimizer = tf.keras.optimizers.Adam()
model.set(policy=EpsGreedyQPolicy(0.01), pool_size=10000, batch=GLOBAL_BATCH_SIZE, update_steps=10)
model.distributed_training(optimizer, strategy, 100)

Note_rl enables users to dive into reinforcement learning with ease, offering comprehensive examples and a clear, structured approach. For further details and additional examples, explore the documentation.

Comments

Your draft will be saved

Start the conversation

Share the first comment.