12. Skip to content

12. ACT Model#

12.1 Introduction#

Make sure your computer has NVIDIA graphics card (memory less than 16G may not be able to train most of the models) and the nvidia-smi command is ready (driver installed).

It is recommended to use anaconda to manage python environments. You can download and install it by running the following commands(if download very slowly, you can click here to download manually):

wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/miniconda/Miniconda3-py38_4.9.2-Linux-x86_64.sh
chmod u+x Miniconda3-py38_4.9.2-Linux-x86_64.sh && ./Miniconda3-py38_4.9.2-Linux-x86_64.sh

Restart your terminal and you can now use conda: 

conda config --set auto_activate_base false && conda deactivate

12.2 Installation#

It is recommended to use a conda python environment. If you do not have one, create and activate it by using the following commands:

conda create -n act python=3.10 && conda activate act

Clone the reposity or contact after-sales service to obtain the package:

git clone --depth 1 https://github.com/OpenGHz/Imitate-All.git && cd Imitate-All

Install the necessary packages by running the following commands:

pip install -r requirements/train_eval.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

What's more, for policy evaluation, make sure you have set up the robot control environment for both software and hardware, such as AIRBOT Play, TOK2, MMK2 and so on. And the airbot-data-collection package is also required for evaluation which can be obtained through our after-sales service.

12.3 Pre-trained Weights#

When you run the training/evaluating command for the first time, it automatically downloads the pre-trained weight file. If your terminal doesn't have a proxy, the download might fail. You can manually download the file using this link (copy and paste it into your browser): https://download.pytorch.org/models/resnet18-f37072fd.pth. Alternatively, you can contact our after-sales service to obtain the file and then move it to the specified directory by executing the following commands:

mkdir -p ~/.cache/torch/hub/checkpoints/ && mv resnet18-f37072fd.pth ~/.cache/torch/hub/checkpoints/

12.4 Parameter Configuration#

Before training or inference, parameter configuration is necessary. Create a Python file in the ./configurations/task_configs directory with the same name as the task ( not recommended to modify or rename the example_task.py file directly) to configure the task. This configuration mainly involves modifying various paths (using the replace_task_name function to use default paths or manually specifying paths), camera names (camera_names), state_dim, action_dim, and so on. Below is an example from example_task.py, which demonstrates how to modify configs based on the default configuration in template.py without rewriting everything (for more adjustable configurations, refer to ./configurations/task_configs/template.py):

The above file can be used as a reference for single-arm task training and inference. For different AIRBOT robots, please select the corresponding default configuration file in the configurations/task_configs/airbots directory. For example, for single AIRBOT Play task, please refer to airbot_play_example.py, and for dual AIRBOT Play task, please refer to airbot_ptk_example.py.

The camera_names parameter can be commonly used camera ID numbers, or the usb port numbers to which the cameras are connected. Please refer to the data collection document for how to obtain the camera usb port number.

12.5 Data Preparation#

When training with default paths, place the .mcap data files in the ./data/mcap/<task_name> folder. You can create the directory with the following command:

mkdir -p data/mcap

You can then copy the data manually or using a command like this (remember to modify the paths in the command):

cp -r /path/to/your/task/mcap_folder data/mcap/<task_name>

12.6 Policy Training#

Please complete 12.2 Installation and 12.4 Parameter Configuration first (training with at least 2 data instances is required; otherwise, an error will occur due to the inability to split the training and validation sets).

Navigate to the repo folder and activate the Conda environment:

conda activate act

Then run the training command:

python3 policy_train.py -tn example_task

The above commands, with just -tn args, will use the configurations from the .py config files in the configurations/task_configs folder corresponding to the given task name.

After training, by default, you can find two folders in ./my_ckpt/<task_name>/<time_stamp> directory. The ckpt folder contains all weight files (referred to as the process folder), while the folder with the same name as <task_name> (called the core folder) contains the following files:

  • Final weights and optimal weights: policy_best.ckpt and policy_last.ckpt respectively.
  • Statistical data: dataset_stats.pkl.
  • Crucial training information (including initial joint angles, training parameter configurations, etc.): key_info.pkl.
  • The training loss curves: train_val_kl_seed_0.png, train_val_l1_seed_0.png and train_val_loss_seed_0.png.
  • The simple description of the training result: description.txt, such as Best ckpt: val loss 0.174929 @ epoch9499 with seed 0.

For ease of use in the future, it's recommended to store the core folder in the specified disk's Imitate-All/my_ckpt folder.

12.7 Policy Evaluating#

Make sure you have installed the required dependencies for controlling your robots in simulation or reality. The following example shows how to use a AIRBOT Play robotic arm to evaluate a policy. If you use realsense, please install it first: pip install pyrealsense2 || pip install pyrealsense2-beta.

12.7.1 Environment Setup#

  • Connect the robotic arms.
  • Connect the cameras in the same order / usb port as that of data collection and so if you haven't unplugged them since data collection, you can skip this step.
  • Run the robotic arm servers: 
    airbot_server -i can_left -p 50051
airbot_server -i can_right -p 50053
    For single arm task, just run one of the commands and modify the -i argument to the actual value, e.g. can_follow.

12.7.2 Check Task Configurations#

Policy inference requires the same configuration file as used for training, and the parameters of the inference part need to be adjusted. This mainly includes modify the environment_maker function and the TASK_CONFIG_DEFAULT["eval"] dictionary.

12.7.3 Executing Commands#

Navigate to the reposity folder and activate the conda environment:

conda activate act

Evaluation command and parameters(Please make sure that no other programs occupy the camera, such as data collection related programs, etc.):

python3 policy_evaluate.py -tn example_task -cf configurations/basic_configs/example/environment/airbot_ptk.yaml -ts 20250608-213703 -show -mts 250
  • -ts: Timestamp corresponding to the task (check the path where policy training results are saved, e.g., my_ckpt/example_task/20240325-153007).
  • -show: Display live images during inference via OpenCV windows.
  • -cf: The path of the configuration file. The config file path in the above command is for dual arm task. For single arm task, refer to airbot_play_with_usbcams.yaml in the same directory. You should modify the configurations in the file according to your usage, such as the camera indexes, the arm ports, etc. For realsense camera use, please refer to airbot_play_with_realsenses.yaml.
  • -mts: The maximum timesteps for each rollout of the evaluation, usually equal to or slightly larger than the episode length.

After the robotic arm starts and moves to the initial pose defined by the task, you can see the instructions in the terminal: Press Enter to start evaluation or z and Enter to exit... (no -show, click the terminal and press the key) or Press Enter to start evaluation or ESC to exit... (use -show, click any of the image windows and press the key).

After each evaluation, you can find evaluation-related files (including process videos) in the corresponding timestamp folder inside the eval_results folder in the current directory.

12.8 Information Viewing#

After policy training, key information and dataset stats will be stored in the key_info.pkl file and dataset_stats.pkl, which can be viewed using the following steps.

Navigate to the repo folder and activate the conda environment:

conda activate act

Then, use the following command to view information for a specified timestamp:

python3 show_info.py -tn example_task -ts 20240420-214215 -in key_info

You will see key information related to that task in the terminal, including:

This includes the absolute path to the mcap data used during training, training parameter configurations, initial joint values of the first episode for inference, and other information.

This information ensures experiment reproducibility. If the camera is rigidly attached to the robotic arm, replicating the robotic arm's behavior is relatively straightforward. Object placement can be determined through retraining data replication.

For dataset stats, just set -in stats in the above command.