Merge pull request #52 from OpenRL-Lab/main

update

Author: huangshiyu13

.dockerignore

@@ -0,0 +1,11 @@
+__pycache__/
+logs
+.pytest_cache/
+.coverage
+.coverage.*
+.idea/
+logs/
+.pytype/
+htmlcov/
+.vscode/
+.git/

.github/workflows/style_check.yml

@@ -12,7 +12,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.8, 3.9]
+        python-version: [3.8]
 
     steps:
       - uses: actions/checkout@v3

.github/workflows/unit_test.yml

@@ -1,6 +1,6 @@
 name: unit_test
 
-on: [ push, pull_request ]
+on: [ pull_request ]
 
 jobs:
   test_unittest:
@@ -9,7 +9,7 @@ jobs:
     if: "! contains(toJSON(github.event.commits.*.message), '[ci skip]')"
     strategy:
       matrix:
-        python-version: [ 3.8, 3.9 ]
+        python-version: [ 3.8 ]
 
     steps:
       - uses: actions/checkout@v3

Makefile

@@ -22,4 +22,10 @@ format:
 	# Reformat using black
 	black ${PYTHON_FILES} --preview
 
-commit-checks: format lint
+commit-checks: format lint
+
+docker-cpu:
+	./scripts/build_docker.sh
+
+docker-gpu:
+	USE_GPU=True ./scripts/build_docker.sh

README.md

@@ -28,6 +28,8 @@
 
 OpenRL-v0.0.8 is updated on May 4, 2023 
 
+The main branch is the latest version of OpenRL, which is under active development. If you just want to have a try with OpenRL, you can switch to the stable branch.
+
 ## 欢迎来到OpenRL
 
 [English](./README_en.md) | [中文文档](https://openrl-docs.readthedocs.io/zh/latest/) | [Documentation](https://openrl-docs.readthedocs.io/en/latest/)
@@ -58,12 +60,14 @@ OpenRL-Lab将持续维护和更新OpenRL，欢迎大家加入我们的[开源社
 - [欢迎来到OpenRL](#欢迎来到openrl)
 - [目录](#目录)
 - [安装](#安装)
+- [使用Docker](#使用docker)
 - [快速上手](#快速上手)
 - [Gallery](#Gallery)
 - [使用OpenRL的项目](#使用OpenRL的项目)
 - [反馈和贡献](#反馈和贡献)
 - [维护人员](#维护人员)
 - [支持者](#支持者)
+  - [↳ Contributors](#-contributors)  
   - [↳ Stargazers](#-stargazers)
   - [↳ Forkers](#-forkers)
 - [Citing OpenRL](#citing-openrl)
@@ -82,9 +86,9 @@ pip install openrl
 conda install -c openrl openrl
 ```
 
-用户也可以从源码安装OpenRL:
+想要修改源码的用户也可以从源码安装OpenRL:
 ```bash
-git clone https://github.com/OpenRL-Lab/openrl && cd openrl
+git clone https://github.com/OpenRL-Lab/openrl.git && cd openrl
 pip install -e .
 ```
 
@@ -93,6 +97,36 @@ pip install -e .
 openrl --version
 ```
 
+## 使用Docker
+
+OpenRL目前也提供了包含显卡支持和非显卡支持的Docker镜像。
+如果用户的电脑上没有英伟达显卡，则可以通过以下命令获取不包含显卡插件的镜像：
+```bash
+sudo docker pull openrllab/openrl-cpu
+```
+
+如果用户想要通过显卡加速训练，则可以通过以下命令获取：
+```bash
+sudo docker pull openrllab/openrl
+```
+
+镜像拉取成功后，用户可以通过以下命令运行OpenRL的Docker镜像：
+```bash
+# 不带显卡加速
+sudo docker run -it openrllab/openrl-cpu
+# 带显卡加速
+sudo docker run -it --gpus all --net host openrllab/openrl
+```
+
+进入Docker镜像后，用户可以通过以下命令查看OpenRL的版本然后运行测例：
+```bash
+# 查看Docker镜像中OpenRL的版本
+openrl --version
+# 运行测例
+openrl --mode train --env CartPole-v1
+```
+
+
 ## 快速上手
 
 OpenRL为强化学习入门用户提供了简单易用的接口，
@@ -189,6 +223,12 @@ OpenRL框架目前还在持续开发和文档建设，欢迎加入我们让该
 
 ## 支持者
 
+### ↳ Contributors
+
+<a href="https://github.com/OpenRL-Lab/openrl/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=OpenRL-Lab/openrl" />
+</a>
+
 ### &#8627; Stargazers
 
 [![Stargazers repo roster for @OpenRL-Lab/openrl](https://reporoster.com/stars/OpenRL-Lab/openrl)](https://github.com/OpenRL-Lab/openrl/stargazers)

README_en.md

@@ -28,6 +28,8 @@
 
 OpenRL-v0.0.8 is updated on May 4, 2023
 
+The main branch is the latest version of OpenRL, which is under active development. If you just want to have a try with OpenRL, you can switch to the stable branch.
+
 ## Welcome to OpenRL
 
 [中文介绍](./README.md) | [Documentation](https://openrl-docs.readthedocs.io/en/latest/) | [中文文档](https://openrl-docs.readthedocs.io/zh/latest/)
@@ -77,12 +79,14 @@ For more information about OpenRL, please refer to the [documentation](https://o
 - [Welcome to OpenRL](#welcome-to-openrl)
 - [Outline](#outline)
 - [Installation](#installation)
+- [Use Docker](#use-docker)
 - [Quick Start](#quick-start)
 - [Gallery](#gallery)
 - [Projects Using OpenRL](#projects-using-openrl)
 - [Feedback and Contribution](#feedback-and-contribution)
 - [Maintainers](#maintainers)
 - [Supporters](#supporters)
+  - [↳ Contributors](#-contributors) 
   - [↳ Stargazers](#-stargazers)
   - [↳ Forkers](#-forkers)
 - [Citing OpenRL](#citing-openrl)
@@ -103,10 +107,10 @@ If users are using Anaconda or Miniconda, they can also install OpenRL via conda
 conda install -c openrl openrl
 ```
 
-Users can also install OpenRL from source code:
+Users who want to modify the source code can also install OpenRL from the source code:
 
 ```bash
-git clone https://github.com/OpenRL-Lab/openrl && cd openrl
+git clone https://github.com/OpenRL-Lab/openrl.git && cd openrl
 pip install -e .
 ```
 
@@ -116,6 +120,35 @@ After installation, users can check the version of OpenRL through command line:
 openrl --version
 ```
 
+## Use Docker
+
+OpenRL currently provides Docker images with and without GPU support. 
+If the user's computer does not have an NVIDIA GPU, they can obtain an image without the GPU plugin using the following command:
+```bash
+sudo docker pull openrllab/openrl-cpu
+```
+
+If the user wants to accelerate training with a GPU, they can obtain it using the following command:
+```bash
+sudo docker pull openrllab/openrl
+```
+
+After successfully pulling the image, users can run OpenRL's Docker image using the following commands:
+```bash
+# Without GPU acceleration
+sudo docker run -it openrllab/openrl-cpu
+# With GPU acceleration 
+sudo docker run -it --gpus all --net host openrllab/openrl
+```
+
+Once inside the Docker container, users can check OpenRL's version and then run test cases using these commands: 
+```bash 
+# Check OpenRL version in Docker container  
+openrl --version  
+# Run test case  
+openrl --mode train --env CartPole-v1  
+```
+
 ## Quick Start
 
 OpenRL provides a simple and easy-to-use interface for beginners in reinforcement learning. 
@@ -213,6 +246,12 @@ to apply for joining the OpenRL team).
 
 ## Supporters
 
+### ↳ Contributors
+
+<a href="https://github.com/OpenRL-Lab/openrl/graphs/contributors">
+  <img src="https://contrib.rocks/image?repo=OpenRL-Lab/openrl" />
+</a>
+
 ### &#8627; Stargazers
 
 [![Stargazers repo roster for @OpenRL-Lab/openrl](https://reporoster.com/stars/OpenRL-Lab/openrl)](https://github.com/OpenRL-Lab/openrl/stargazers)

docker/Dockerfile

@@ -0,0 +1,23 @@
+ARG PARENT_IMAGE
+FROM $PARENT_IMAGE
+
+WORKDIR /openrl
+
+ADD setup.py setup.py
+ADD openrl openrl
+ADD README.md README.md
+
+ENV VENV /root/venv
+
+RUN \
+    pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
+
+RUN \
+    python3 -m pip install --upgrade pip --no-cache-dir && \
+    python3 -m pip install --no-cache-dir . && \
+    python3 -m pip install --no-cache-dir ".[nlp]" && \
+    rm -rf $HOME/.cache/pip
+
+ENV PATH=$VENV/bin:$PATH
+
+CMD /bin/bash

examples/cartpole/README.md

@@ -0,0 +1,7 @@
+## How to Use
+
+Users can train CartPole via:
+
+```shell
+python train_ppo.py
+```

examples/cartpole/train_ppo.py

@@ -19,7 +19,7 @@ def train():
     return agent
 
 
-def test(agent):
+def evaluation(agent):
     # 开始测试环境
     env = make("CartPole-v1", render_mode="group_human", env_num=9, asynchronous=True)
     agent.set_env(env)
@@ -37,4 +37,4 @@ def test(agent):
 
 if __name__ == "__main__":
     agent = train()
-    test(agent)
+    evaluation(agent)

examples/mpe/README.md

@@ -0,0 +1,7 @@
+## How to Use
+
+Users can train MPE via:
+
+```shell
+python train_ppo.py --config mpe_ppo.yaml
+```