Welcome to UtilsRL’s documentation!

UtilsRL is a Python package designed for agile RL algorithm development. At first it was not intended for providing competitve implementations of baseline RL algorithms, but for fast integration into existing RL code bases and experimenting ideas at least cost. It provides:

• Basic RL modules

• Experiment management utilities

• Tensorboard loggers

• Tensorboard parsing and visualization utilities

With UtilsRL, you can implement the full version of PPO2 (think about the tricks!) within 200 lines of python code. We also tried integrating UtilsRL into other RL frameworks / code bases, and the cost of migration turned out to be surprisingly low.

Currently UtilsRL is maintained by researchers from Lamda-RL Group, any bug reports / feature requests are welcome and will be dealt with ASAP.

Installation

UtilsRL is currently hosted on PyPI. It requires Python >= 3.6. You can simply install UtilsRL from PyPI with:

$ pip install UtilsRL

After Installation, try with:

import UtilsRL
print(UtilsRL.__version__)

If no error occurs, you have installed UtilsRL successfully.

Tutorials:

• A Minimal Example of PPO with Continuous Control
  • 1. Set up logger and arguments
  • 2. Add environment info to args
  • 3. Define the structures of networks
  • 4. Define the actor udpate logic, action selection logic and training loops
  • 5. Record the results
• Benchmarks and Baseline algorithms

Module Design:

• UtilsRL.exp
  • Argument Parsing
  • Experiment Setup
  • Some other features?
• UtilsRL.rl
  • RL Actors
  • RL Critics
  • Normalizers
• UtilsRL.plot
  • log2pd
  • logs2pd
• UtilsRL.logger
  • UtilsRL.logger.DummyLogger
  • UtilsRL.logger.ColoredLogger
  • UtilsRL.logger.TensorboardLogger
• UtilsRL.monitor
  • UtilsRL.monitor.Monitor

Indices and tables

• Index

• Module Index

• Search Page