Isaac GR00T N1.7

Very First Experiment

We trained an Isaac GR00T N1.7 model on our red cube dataset for 30K training steps. As shown in Fig 1, the model successfully learned to transfer the cube, and it also generalized to other cube colors. This is our best result. Initially, the robot shook violently, but after some trial and error, we were able to smooth out the robot using a combination of EMA low-pass filtering, an increase in the action chunk length to 32, and robot action clipping. See the next section for implementation details. Note that our ACT models also required action clipping to smooth out their trajectories, while our openpi models did not seem to need any smoothing or clipping.

• EMA low-pass filtering: At run time, actions at are filtered using a't = α*at + (1-α)*a't-1, where a' is the filtered action that is sent to the robot. After some trial and error, the best value was α=0.25.
• Action chunk size: The best model was trained with the action chunk size set to 32 in the config file, trossen_ai_config_h32.py. Also, the open_loop_horizon option was set to 32 when running the robot using main_gr00t_trossen.py. The longer action chunks seem to be less noisy and also more reliable.
• Action clipping: The Trossen robot parameter, max_relative_target, was set to 0.1 which keeps the maximum change in any action — joint value — below 0.1 radians. At this value, clipping happens only once in a while, but when it does, it prevents sudden large movements.
• Real-time chunking: Introduced by Physical Intelligence, Real-Time Action Chunking (RTC) is designed to interpolate successive action chunks to create a smooth, consistent trajectory. GR00T has the RTC machinery, but to use it requires some mods to gr00t/policy/gr00t_policy.py and additions to examples/trossen_ai/main_gr00t_trossen.py. We tried RTC to solve the robot shaking problem but it did not help. We believe this is because intra-chunk noise was a bigger problem than inter-chunk discontinuities.

High Dexterity Experiments

We trained an Isaac GR00T N1.7 model on our 50 episode bead-on-a-string dataset for 30K training steps. The resulting model did not succeed at placing the bead on the string, although it did get close. We then performed two iterations of policy improvement using human interventions, adding 25 episodes per iteration. At each iteration, the model was trained for an additional 20K steps. As shown in Fig 2, the model successfully learned to place a bead on a string, although the success rate is only about 20% and is not yet reliably repeatable. This is therefore a preliminary result, but we are excited to see that GR00T is capable of performing this high dexterity task and of doing so smoothly!

• Parameters: We found that exactly the same EMA smoothing, action clipping, and chunk size parameters that worked best for the cube transfer, above, worked best for bead-on-a-string too!
• Causal confusion: Our model, even after policy improvement, gave us results which were extremely inconsistent. Sometimes, the robot would even get stuck in a micro-loop of states very close to each other, effectively a freeze. After eliminating all other causes it turned out that the model was using the low camera to rely on the horizontal position of the string as it hangs vertically while it is held by the left gripper before bead placement. In the original dataset this was a good signal, but in testing the string was not always vertical and this confused the model. This is a classic case of causal confusion (see paper) which arises because GR00T, unlike pi0.5, is able to include the low camera (at the center bottom of the Fig 2 video) as input. One would expect this additional sensory data to improve model performance, but what it does is add a strongly correlated and powerful signal -- the horizontal position of the string -- which is present in our original and augmented datasets. However, during testing the string was sometimes pulled off to the side and did not hang vertically. This messed up the horizontal position signal. So although this signal is correlated with actions in the training data, it is not actually a causal driver of the task. We hope to solve this problem by further augmenting the dataset with more human intervention episodes, but this time purposely adding natural variations in string position.

Implementation Details

The code here uses our fork of Isaac GR00T N1.7. At the moment, the latest code is in the develop branch. The commands below are not an exhaustive set of instructions. You first need to install our develop branch or install the original repository directly from GR00T, and then cut and paste our mods in if you want to use them. We did not add any dependencies not in the original GR00T repository.