add-il-docker-stack-and-updated-isaaclab-in-it

.github/templates/docker_context/docker_context.sh

@@ -21,17 +21,17 @@ fi
 
 # Loop through each module
 while read -r module; do
+    module_out=$(basename "$(echo "$module" | sed -n 's/modules\/docker-compose\.\(.*\)\.yaml/\1/p')")
 
-    # Retrieve docker compose service names
-    services=$(docker compose -f "$module" --profile deploy --profile develop config --services)
-    module_out=$(basename $(echo "$module" | sed -n 's/modules\/docker-compose\.\(.*\)\.yaml/\1/p'))
-
-    # Skip simulation module
+    # Skip Isaac Sim modules (need GPU/NGC/X11; no multi-stage CI Dockerfile)
     # TODO: Add custom handling for embedded testing
-    if [[ 'simulation' = $module_out || 'embedded' = $module_out ]]; then
+    if [[ 'simulation' = $module_out || 'simulation_il' = $module_out || 'embedded' = $module_out ]]; then
         continue
     fi
 
+    # Retrieve docker compose service names
+    services=$(docker compose -f "$module" --profile deploy --profile develop config --services)
+
     # Only work with modules that are modified
     if [[ $MODIFIED_MODULES != *$module_out* && $TEST_ALL = "false"  ]]; then
         continue

.github/workflows/build_and_unitest.yml

@@ -30,6 +30,7 @@ jobs:
 
       - name: Setup Watod Environment
         run: |
+          MONO_DIR="$GITHUB_WORKSPACE"
           MODULES_DIR="$GITHUB_WORKSPACE/modules"
           . ./watod_scripts/watod-setup-docker-env.sh
         shell: bash

.github/workflows/build_base_images.yml

@@ -18,6 +18,7 @@ jobs:
 
       - name: Setup Watod Environment
         run: |
+          MONO_DIR="$GITHUB_WORKSPACE"
           MODULES_DIR="$GITHUB_WORKSPACE/modules"
           . ./watod_scripts/watod-setup-docker-env.sh
         shell: bash

assets/lerobot/README.md

@@ -30,4 +30,9 @@ git add assets/lerobot/
 
 Source: [isaac-sim/Sim-to-Real-SO-101-Workshop](https://github.com/isaac-sim/Sim-to-Real-SO-101-Workshop) (Apache-2.0).
 
-Used by `autonomy/simulation/Teleop/so101-leader teleoperation/`.
+Used by:
+- `simulation_il` watod Docker — [`docker/simulation/isaac_il/QUICKSTART.md`](../../docker/simulation/isaac_il/QUICKSTART.md)
+- `autonomy/simulation/so101_vial_task/`
+- `autonomy/simulation/Teleop/so101-leader teleoperation/`
+
+Run `./assets/lerobot/sync_so101_vial_assets.sh --full` before first sim IL eval if you see `[WARNING] No textures found`.

assets/lerobot/so101_vial_task/hdri/yaw_mapping.yaml

@@ -0,0 +1,22 @@
+abandoned_garage_1k.exr: null
+afrikaans_church_interior_1k.exr: [110, -40]
+blaubeuren_church_square_1k.exr: null
+blaubeuren_night_1k.exr: null
+blue_photo_studio_1k.exr: null
+boma_1k.exr: [110, -40]
+brown_photostudio_01_1k.exr: [120, -40]
+burnt_warehouse_1k.exr: [120, -50]
+fireplace_1k.exr: [0, -170]
+hospital_room_2_1k.exr: null
+metro_noord_1k.exr: null
+monkstown_castle_1k.exr: null
+moon_lab_1k.exr: [120, -30]
+moonlit_golf_1k.exr: [90, -60]
+outdoor_chapel_1k.exr: [120, -50]
+poly_haven_studio_1k.exr: null
+rogland_moonlit_night_1k.exr: [120, -50]
+satara_night_1k.exr: [100, -45]
+studio_small_03_1k.exr: [350, 160]
+studio_small_09_1k.exr: [120, -30]
+university_workshop_1k.exr: [120, -30]
+winter_evening_1k.exr: [60, -60]

assets/lerobot/sync_so101_vial_assets.sh

@@ -15,7 +15,7 @@ set -euo pipefail
 REPO="isaac-sim/Sim-to-Real-SO-101-Workshop"
 BRANCH="main"
 USD_BASE="https://media.githubusercontent.com/media/${REPO}/${BRANCH}/source/sim_to_real_so101/assets/usd"
-HDRI_BASE="https://media.githubusercontent.com/media/${REPO}/${BRANCH}/source/sim_to_real_so101/assets/hdri"
+HDRI_BASE="https://github.com/${REPO}/raw/${BRANCH}/source/sim_to_real_so101/assets/hdri"
 ROOT="$(cd "$(dirname "$0")" && pwd)"
 DEST="${ROOT}/so101_vial_task/usd"
 TEX_DEST="${DEST}/tex"

autonomy/il/README.md

@@ -140,7 +140,11 @@ PYTHONPATH=$(pwd) /home/hy/IsaacLab/isaaclab.sh -p so101_leader_teleop.py --reco
 
 Output: `datasets/record_so101_sim_vision/001/`.
 
-## Train (LeRobot, external)
+## Train (LeRobot)
+
+**SO101 vial sim IL (recommended):** inside `simulation_il` Docker — [`docker/simulation/isaac_il/QUICKSTART.md`](../../docker/simulation/isaac_il/QUICKSTART.md) (`il-train`, `--policy.push_to_hub=false`, `--steps=...`).
+
+**Generic / host** (outside Isaac docker):
 
 ```bash
 lerobot-train \

autonomy/simulation/Teleop/so101-leader teleoperation/README.md

@@ -91,7 +91,9 @@ Optional HDRI sky randomization: copy workshop `assets/hdri/` to `assets/lerobot
 
 For workshop-parity **Gym envs**, automatic success detection, policy eval, and depth/seg MP4 sidecars, use:
 
-`autonomy/simulation/so101_vial_task/` — see its README (`lerobot_agent.py`, `lerobot_eval.py`).
+**[`docker/simulation/isaac_il/QUICKSTART.md`](../../../../docker/simulation/isaac_il/QUICKSTART.md)** — watod `simulation_il` Docker (recommended).
+
+Task reference: `autonomy/simulation/so101_vial_task/README.md` (`lerobot_agent.py`, `lerobot_eval.py`).
 
 The InteractiveScene teleop above stays the lightweight RGB collection path via `humanoid_il`.
 

autonomy/simulation/so101_vial_task/README.md

@@ -2,16 +2,22 @@
 
 Workshop-parity stack for the SO101 **vial → rack** task: registered Gym envs, success detection, policy eval, and rich LeRobot recording with depth/segmentation MP4 export.
 
-Assets live under `assets/lerobot/` (sync with `./assets/lerobot/sync_so101_vial_assets.sh --full`).
+## Run this (recommended)
 
-## Install
+Use the **`simulation_il`** watod Docker image (Isaac Lab 2.3.2 + LeRobot). Do **not** use host `env_isaaclab` for IL — wrong Python/stack version.
+
+| Doc | Contents |
+|-----|----------|
+| **[`docker/simulation/isaac_il/QUICKSTART.md`](../../../docker/simulation/isaac_il/QUICKSTART.md)** | Exact copy-paste: build, train ACT, eval |
+| **[`docker/simulation/isaac_il/README.md`](../../../docker/simulation/isaac_il/README.md)** | Full setup, troubleshooting, workflows |
 
 ```bash
-cd autonomy/simulation/so101_vial_task
-pip install -e .
+# host
+ACTIVE_MODULES="simulation_il"   # in watod-config.local.sh
+./watod up -d && ./watod -t simulation_il_dev
 ```
 
-Requires Isaac Lab, LeRobot, and synced USD/HDRI assets.
+Assets: `./assets/lerobot/sync_so101_vial_assets.sh --full` (see [`assets/lerobot/README.md`](../../../assets/lerobot/README.md)).
 
 ## Registered Gym tasks
 
@@ -24,52 +30,62 @@ Requires Isaac Lab, LeRobot, and synced USD/HDRI assets.
 
 Observation groups: `policy` (joints), `visual` (RGB/depth/seg), `subtask` (grasp/placed flags). Success uses gripper contact sensor + rack slot geometry (`humanoid_so101_vial_task/mdp/terms.py`).
 
-## 1. Leader teleop + rich recording (Gym)
+Cameras: `ego` (gripper), `external_D455` (lightbox) — matches [CursedRock17/so101_teleop_vials_sim_and_real](https://huggingface.co/datasets/CursedRock17/so101_teleop_vials_sim_and_real).
 
-Physical SO101 Leader drives sim; **S** start/stop, **R** reset world, **C** cancel episode while recording.
+## 1. Train ACT (inside `simulation_il` container)
 
 ```bash
-cd autonomy/simulation/so101_vial_task
-PYTHONPATH=$(pwd) /home/hy/IsaacLab/isaaclab.sh -p scripts/lerobot_agent.py \
-  --task Lerobot-So101-Teleop-Vials-To-Rack-DR \
-  --port /dev/ttyACM0 \
-  --repo_id humanoid/so101_vial_dr \
-  --repo_root ../../datasets/record_so101_gym/001 \
-  --task_name "vial to rack" \
-  --save_mp4 --depth --instance_id_seg
+il-train \
+  --dataset.repo_id=CursedRock17/so101_teleop_vials_sim_and_real \
+  --policy.type=act \
+  --policy.push_to_hub=false \
+  --output_dir=/workspace/humanoid/outputs/train/so101_hf_act \
+  --policy.device=cuda \
+  --steps=10000 \
+  --batch_size=8 \
+  --save_freq=5000 \
+  --job_name=so101_hf_smoke
 ```
 
-Optional MP4 sidecars: `--save_mp4`, `--depth`, `--instance_id_seg` (via `LeRobotRecorder`).
-
-## 2. Policy eval in sim
+See [QUICKSTART.md](../../../docker/simulation/isaac_il/QUICKSTART.md) for full train/eval commands and pitfalls.
 
-Local LeRobot checkpoint (ACT, etc.):
+## 2. Policy eval in sim (inside container)
 
 ```bash
-PYTHONPATH=$(pwd) /home/hy/IsaacLab/isaaclab.sh -p scripts/lerobot_eval.py \
+cd /workspace/humanoid/autonomy/simulation/so101_vial_task
+
+PYTHONPATH=$(pwd) $ISAACLAB/isaaclab.sh -p scripts/lerobot_eval.py \
   --task Lerobot-So101-Teleop-Vials-To-Rack-DR-Eval \
   --policy_type lerobot \
-  --policy_path outputs/train/so101_act_v1/checkpoints/last/pretrained_model \
-  --num_episodes 10 \
-  --rename_map '{"ego":"observation.images.ego","external_D455":"observation.images.external_D455"}'
+  --policy_path /workspace/humanoid/outputs/train/so101_hf_act/checkpoints/last/pretrained_model \
+  --num_episodes 5
 ```
 
-Remote GR00T server:
+- **Local ACT:** omit `--rename_map` (sim cameras already `ego` / `external_D455`).
+- **GR00T:** `--policy_type groot` and `--rename_map '{"external_D455": "front", "ego": "wrist"}'`.
+
+Reports success rate when `success` termination fires (vial placed in rack slot).
+
+## 3. Leader teleop + rich recording (needs USB leader)
+
+Physical SO101 Leader drives sim; **S** start/stop, **R** reset world, **C** cancel episode while recording.
 
 ```bash
-PYTHONPATH=$(pwd) /home/hy/IsaacLab/isaaclab.sh -p scripts/lerobot_eval.py \
-  --policy_type groot --policy_host localhost --policy_port 5555
+cd $TASK_ROOT
+PYTHONPATH=$(pwd) $ISAACLAB/isaaclab.sh -p scripts/lerobot_agent.py \
+  --task Lerobot-So101-Teleop-Vials-To-Rack-DR \
+  --port /dev/ttyACM0 \
+  --repo_root /workspace/humanoid/datasets/record_so101_gym/001 \
+  --save_mp4 --depth --instance_id_seg
 ```
 
-Reports success rate when `success` termination fires (vial placed in rack slot).
-
-## 3. Lightweight teleop (InteractiveScene)
+## 4. Lightweight teleop (InteractiveScene)
 
-For keyboard/leader collection without Gym, use the existing path:
+RGB-only path without Gym:
 
 `autonomy/simulation/Teleop/so101-leader teleoperation/` + `autonomy/il/` (`--record --cameras --domain_rand`).
 
-That path is RGB-only in LeRobot; use **this** package for depth/seg MP4 sidecars and automatic success scoring.
+Use **this** package for depth/seg MP4 sidecars and automatic success scoring.
 
 ## Layout
 
@@ -84,3 +100,14 @@ so101_vial_task/
     ├── lerobot_agent.py   # teleop + rich record
     └── lerobot_eval.py    # policy rollout + success rate
 ```
+
+## Host install (legacy — not for IL docker workflow)
+
+Only if developing outside Docker on a matching Isaac Lab 2.3.2 + Python 3.11 stack:
+
+```bash
+cd autonomy/simulation/so101_vial_task
+pip install -e .
+```
+
+Requires Isaac Lab, LeRobot, and synced USD/HDRI assets.