├── Assets ├── bongo_board │ ├── meshes │ │ └── BongoBoard_Roller.stl │ └── roller.urdf └── robinion_meshes │ ├── collision │ ├── head_pitch_collision.stl │ ├── head_yaw_collision.stl │ ├── l_ankle_roll_pitch_collision.stl │ ├── l_arm_1_collision.stl │ ├── l_arm_2_collision.stl │ ├── l_arm_3_collision.stl │ ├── l_foot_collision.stl │ ├── l_gripper_collision.stl │ ├── l_hip_roll_pitch_collision.stl │ ├── l_hip_yaw_collision.stl │ ├── l_knee_collision.stl │ ├── l_shank_back_collision.stl │ ├── l_shank_front_collision.stl │ ├── l_shoulder_collision.stl │ ├── l_thigh_back_collision.stl │ ├── l_thigh_front_collision.stl │ ├── lower_body_collision.stl │ ├── r_ankle_roll_pitch_collision.stl │ ├── r_arm_1_collision.stl │ ├── r_arm_2_collision.stl │ ├── r_arm_3_collision.stl │ ├── r_foot_collision.stl │ ├── r_gripper_collision.stl │ ├── r_hip_roll_pitch_collision.stl │ ├── r_hip_yaw_collision.stl │ ├── r_knee_collision.stl │ ├── r_shank_back_collision.stl │ ├── r_shank_front_collision.stl │ ├── r_shoulder_collision.stl │ ├── r_thigh_back_collision.stl │ ├── r_thigh_front_collision.stl │ ├── robinion2.urdf │ └── upper_body_collision.stl │ ├── fixed_robinion_torso_center.urdf │ └── visual │ ├── head_pitch_visual.stl │ ├── head_yaw_visual.stl │ ├── l_ankle_roll_pitch_visual.stl │ ├── l_arm_1_visual.stl │ ├── l_arm_2_visual.stl │ ├── l_arm_3_visual.stl │ ├── l_foot_visual.stl │ ├── l_gripper_closed_visual.stl │ ├── l_gripper_visual.stl │ ├── l_hip_roll_pitch_visual.stl │ ├── l_hip_yaw_visual.stl │ ├── l_knee_visual.stl │ ├── l_shank_back_visual.stl │ ├── l_shank_front_visual.stl │ ├── l_shoulder_visual.stl │ ├── l_thigh_back_visual.stl │ ├── l_thigh_front_visual.stl │ ├── lower_body_visual.stl │ ├── r_ankle_roll_pitch_visual.stl │ ├── r_arm_1_visual.stl │ ├── r_arm_2_visual.stl │ ├── r_arm_3_visual.stl │ ├── r_foot_visual.stl │ ├── r_gripper_closed_visual.stl │ ├── r_gripper_visual.stl │ ├── r_hip_roll_pitch_visual.stl │ ├── r_hip_yaw_visual.stl │ ├── r_knee_visual.stl │ ├── r_shank_back_visual.stl │ ├── r_shank_front_visual.stl │ ├── r_shoulder_visual.stl │ ├── r_thigh_back_visual.stl │ ├── r_thigh_front_visual.stl │ ├── standard │ ├── head_pitch_visual.stl │ ├── head_yaw_visual.stl │ ├── left_ankle_roll_pitch_visual.stl │ ├── left_foot_visual.stl │ ├── left_gripper_visual.stl │ ├── left_hip_pitch_visual.stl │ ├── left_hip_roll_pitch_visual.stl │ ├── left_leg_pitch_knee_ankle_visual.stl │ ├── left_shoulder_visual.stl │ ├── right_ankle_roll_pitch_visual.stl │ ├── right_foot_visual.stl │ ├── right_gripper_visual.stl │ ├── right_hip_pitch_visual.stl │ ├── right_hip_roll_pitch_visual.stl │ ├── right_leg_pitch_knee_ankle_visual.stl │ └── right_shoulder_visual.stl │ └── upper_body_visual.stl ├── Code ├── cfg │ ├── task │ │ └── FixedRobinionBalanceBoardSim.yaml │ └── train │ │ └── FixedRobinionBalanceBoardSimPPO.yaml └── tasks │ └── fixed_robinion_balance_board_sim.py ├── Docs ├── Demo.gif └── Reinforcement Learning and Action Space Shaping for a Humanoid Agent in a Highly Dynamic Environment.pdf └── README.md /Assets/bongo_board/meshes/BongoBoard_Roller.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/bongo_board/meshes/BongoBoard_Roller.stl -------------------------------------------------------------------------------- /Assets/bongo_board/roller.urdf: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/head_pitch_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/head_pitch_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/head_yaw_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/head_yaw_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_ankle_roll_pitch_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_ankle_roll_pitch_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_arm_1_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_arm_1_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_arm_2_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_arm_2_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_arm_3_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_arm_3_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_foot_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_foot_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_gripper_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_gripper_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_hip_roll_pitch_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_hip_roll_pitch_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_hip_yaw_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_hip_yaw_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_knee_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_knee_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_shank_back_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_shank_back_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_shank_front_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_shank_front_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_shoulder_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_shoulder_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_thigh_back_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_thigh_back_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/l_thigh_front_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/l_thigh_front_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/lower_body_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/lower_body_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_ankle_roll_pitch_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_ankle_roll_pitch_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_arm_1_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_arm_1_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_arm_2_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_arm_2_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_arm_3_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_arm_3_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_foot_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_foot_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_gripper_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_gripper_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_hip_roll_pitch_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_hip_roll_pitch_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_hip_yaw_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_hip_yaw_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_knee_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_knee_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_shank_back_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_shank_back_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_shank_front_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_shank_front_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_shoulder_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_shoulder_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_thigh_back_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_thigh_back_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/r_thigh_front_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/r_thigh_front_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/robinion2.urdf: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | /robinion 11 | gazebo_ros_control/DefaultRobotHWSim 12 | 13 | 14 | 15 | 16 | 0.2 17 | 0.2 18 | Gazebo/White 19 | false 20 | 21 | 22 | 0.2 23 | 0.2 24 | Gazebo/White 25 | false 26 | 27 | 28 | 29 | 0.2 30 | 0.2 31 | Gazebo/White 32 | false 33 | 34 | 35 | 0.2 36 | 0.2 37 | Gazebo/White 38 | false 39 | 40 | 41 | 42 | 0.2 43 | 0.2 44 | Gazebo/White 45 | false 46 | 47 | 48 | 0.2 49 | 0.2 50 | Gazebo/White 51 | false 52 | 53 | 54 | 0.2 55 | 0.2 56 | Gazebo/White 57 | false 58 | 59 | 60 | 0.2 61 | 0.2 62 | Gazebo/White 63 | false 64 | 65 | 66 | 67 | 0.2 68 | 0.2 69 | Gazebo/White 70 | false 71 | 72 | 73 | 0.2 74 | 0.2 75 | Gazebo/White 76 | false 77 | 78 | 79 | 0.2 80 | 0.2 81 | Gazebo/White 82 | false 83 | 84 | 85 | 0.2 86 | 0.2 87 | Gazebo/White 88 | false 89 | 90 | 91 | 92 | 0.2 93 | 0.2 94 | Gazebo/White 95 | false 96 | 97 | 98 | 0.2 99 | 0.2 100 | Gazebo/White 101 | false 102 | 103 | 104 | 0.2 105 | 0.2 106 | Gazebo/White 107 | false 108 | 109 | 110 | 0.2 111 | 0.2 112 | Gazebo/White 113 | false 114 | 115 | 116 | 0.2 117 | 0.2 118 | Gazebo/White 119 | false 120 | 121 | 122 | 0.2 123 | 0.2 124 | Gazebo/White 125 | false 126 | 127 | 128 | 0.2 129 | 0.2 130 | Gazebo/White 131 | false 132 | 133 | 134 | 0.2 135 | 0.2 136 | Gazebo/White 137 | false 138 | 139 | 140 | 100000.0 141 | 100.0 142 | 1.5 143 | 1.5 144 | 1 0 0 145 | 1.0 146 | 0.001 147 | Gazebo/White 148 | false 149 | 150 | 151 | 152 | 0.2 153 | 0.2 154 | Gazebo/White 155 | false 156 | 157 | 158 | 0.2 159 | 0.2 160 | Gazebo/White 161 | false 162 | 163 | 164 | 0.2 165 | 0.2 166 | Gazebo/White 167 | false 168 | 169 | 170 | 0.2 171 | 0.2 172 | Gazebo/White 173 | false 174 | 175 | 176 | 0.2 177 | 0.2 178 | Gazebo/White 179 | false 180 | 181 | 182 | 0.2 183 | 0.2 184 | Gazebo/White 185 | false 186 | 187 | 188 | 0.2 189 | 0.2 190 | Gazebo/White 191 | false 192 | 193 | 194 | 0.2 195 | 0.2 196 | Gazebo/White 197 | false 198 | 199 | 200 | 100000.0 201 | 100.0 202 | 1.5 203 | 1.5 204 | 1 0 0 205 | 1.0 206 | 0.001 207 | Gazebo/White 208 | false 209 | 210 | 211 | 212 | 213 | 1.012300 214 | 215 | 320 216 | 240 217 | 218 | 219 | 0.001000 220 | 100.000000 221 | 222 | 223 | 1 224 | 30.000000 225 | 1 226 | 227 | true 228 | 30 229 | robinion/camera 230 | image_raw 231 | camera_info 232 | cam_link 233 | 0.07 234 | 0.0 235 | 0.0 236 | 0.0 237 | 0.0 238 | 0.0 239 | 240 | 241 | 242 | 243 | true 244 | 245 | true 246 | 125 247 | true 248 | __default_topic__ 249 | 250 | robinion/imu 251 | imu_link 252 | 125.0 253 | 0.0 254 | 0 0 0 255 | 0 0 0 256 | imu_link 257 | 258 | 259 | 260 | 261 | 262 | 263 | 264 | 265 | 266 | 267 | 268 | 269 | 270 | 271 | 272 | 273 | 274 | 275 | 276 | 277 | 278 | 279 | 280 | 281 | 282 | 283 | 284 | 285 | 286 | 287 | 288 | 289 | 290 | 291 | 292 | 293 | 294 | 295 | 296 | True 297 | 298 | 299 | transmission_interface/SimpleTransmission 300 | 301 | hardware_interface/EffortJointInterface 302 | 303 | 304 | hardware_interface/EffortJointInterface 305 | 1 306 | 307 | 308 | 309 | 310 | 311 | 312 | 313 | 314 | 315 | 316 | 317 | 318 | 319 | 320 | 321 | 322 | 323 | 324 | 325 | 326 | 327 | 328 | 329 | 330 | 331 | 332 | 333 | 334 | 335 | 336 | 337 | 338 | 339 | 340 | 341 | 342 | True 343 | 344 | 345 | transmission_interface/SimpleTransmission 346 | 347 | hardware_interface/EffortJointInterface 348 | 349 | 350 | hardware_interface/EffortJointInterface 351 | 1 352 | 353 | 354 | 355 | 356 | 357 | 358 | 359 | 360 | 361 | 362 | 363 | 364 | 365 | 366 | 367 | 368 | 369 | 370 | 371 | 372 | 373 | 374 | 375 | 376 | 377 | 378 | 379 | 380 | 381 | True 382 | 383 | 384 | transmission_interface/SimpleTransmission 385 | 386 | hardware_interface/EffortJointInterface 387 | 388 | 389 | hardware_interface/EffortJointInterface 390 | 1 391 | 392 | 393 | 394 | 395 | 396 | 397 | 398 | 399 | 400 | 401 | 402 | 403 | 404 | 405 | 406 | 407 | 408 | 409 | 410 | 411 | 412 | 413 | 414 | 415 | 416 | 417 | 418 | 419 | 420 | 421 | 422 | 423 | 424 | 425 | 426 | 427 | 428 | 429 | 430 | 431 | 432 | 433 | 434 | True 435 | 436 | 437 | transmission_interface/SimpleTransmission 438 | 439 | hardware_interface/EffortJointInterface 440 | 441 | 442 | hardware_interface/EffortJointInterface 443 | 1 444 | 445 | 446 | 447 | 448 | 449 | 450 | 451 | 452 | 453 | 454 | 455 | 456 | 457 | 458 | 459 | 460 | 461 | 462 | 463 | 464 | 465 | 466 | 467 | 468 | 469 | 470 | 471 | 472 | 473 | True 474 | 475 | 476 | transmission_interface/SimpleTransmission 477 | 478 | hardware_interface/EffortJointInterface 479 | 480 | 481 | hardware_interface/EffortJointInterface 482 | 1 483 | 484 | 485 | 486 | 487 | 488 | 489 | 490 | 491 | 492 | 493 | 494 | 495 | 496 | 497 | 498 | 499 | 500 | 501 | 502 | 503 | 504 | 505 | 506 | 507 | 508 | 509 | 510 | 511 | 512 | True 513 | 514 | 515 | transmission_interface/SimpleTransmission 516 | 517 | hardware_interface/EffortJointInterface 518 | 519 | 520 | hardware_interface/EffortJointInterface 521 | 1 522 | 523 | 524 | 525 | 526 | 527 | 528 | 529 | 530 | 531 | 532 | 533 | 534 | 535 | 536 | 537 | 538 | 539 | 540 | 541 | 542 | 543 | 544 | 545 | 546 | 547 | 548 | 549 | 550 | 551 | True 552 | 553 | 554 | transmission_interface/SimpleTransmission 555 | 556 | hardware_interface/EffortJointInterface 557 | 558 | 559 | hardware_interface/EffortJointInterface 560 | 1 561 | 562 | 563 | 564 | 565 | 566 | 567 | 568 | 569 | 570 | 571 | 572 | 573 | 574 | 575 | 576 | 577 | 578 | 579 | 580 | 581 | 582 | 590 | 591 | 592 | 593 | 594 | 595 | 596 | 597 | 598 | 599 | 600 | 601 | 602 | 603 | 604 | 605 | 606 | 607 | 608 | 609 | 610 | 611 | 612 | 613 | 614 | 615 | 616 | 617 | 618 | 619 | 620 | 621 | 622 | True 623 | 624 | 625 | transmission_interface/SimpleTransmission 626 | 627 | hardware_interface/EffortJointInterface 628 | 629 | 630 | hardware_interface/EffortJointInterface 631 | 1 632 | 633 | 634 | 635 | 636 | 637 | 638 | 639 | 640 | 641 | 642 | 643 | 644 | 645 | 646 | 647 | 648 | 649 | 650 | 651 | 652 | 653 | 654 | 655 | 656 | 657 | 658 | 659 | 660 | 661 | True 662 | 663 | 664 | transmission_interface/SimpleTransmission 665 | 666 | hardware_interface/EffortJointInterface 667 | 668 | 669 | hardware_interface/EffortJointInterface 670 | 1 671 | 672 | 673 | 674 | 675 | 676 | 677 | 678 | 679 | 680 | 681 | 682 | 683 | 684 | 685 | 686 | 687 | 688 | 689 | 690 | 691 | 692 | 693 | 694 | 695 | 696 | 697 | 698 | 699 | 700 | True 701 | 702 | 703 | transmission_interface/SimpleTransmission 704 | 705 | hardware_interface/EffortJointInterface 706 | 707 | 708 | hardware_interface/EffortJointInterface 709 | 1 710 | 711 | 712 | 713 | 714 | 715 | 716 | 717 | 718 | 719 | 720 | 721 | 722 | 723 | 724 | 725 | 726 | 727 | 728 | 729 | 730 | 731 | 732 | 733 | 734 | 735 | 736 | 737 | 738 | 739 | True 740 | 741 | 742 | transmission_interface/SimpleTransmission 743 | 744 | hardware_interface/EffortJointInterface 745 | 746 | 747 | hardware_interface/EffortJointInterface 748 | 1 749 | 750 | 751 | 752 | 753 | 754 | 755 | 756 | 757 | 758 | 759 | 760 | 761 | 762 | 763 | 764 | 765 | 766 | 767 | 768 | 769 | 770 | 778 | 779 | 780 | 781 | 782 | 783 | 784 | 785 | 786 | 787 | 788 | 789 | 790 | 791 | 792 | 793 | 794 | 795 | 796 | 797 | 798 | 799 | 800 | 801 | 802 | 803 | 804 | 805 | 806 | 807 | 808 | 809 | 810 | True 811 | 812 | 813 | transmission_interface/SimpleTransmission 814 | 815 | hardware_interface/EffortJointInterface 816 | 817 | 818 | hardware_interface/EffortJointInterface 819 | 1 820 | 821 | 822 | 823 | 824 | 825 | 826 | 827 | 828 | 829 | 830 | 831 | 832 | 833 | 834 | 835 | 836 | 837 | 838 | 839 | 840 | 841 | 842 | 843 | 844 | 845 | 846 | 847 | 848 | 849 | True 850 | 851 | 852 | transmission_interface/SimpleTransmission 853 | 854 | hardware_interface/EffortJointInterface 855 | 856 | 857 | hardware_interface/EffortJointInterface 858 | 1 859 | 860 | 861 | 862 | 863 | 864 | 865 | 866 | 867 | 868 | 869 | 870 | 871 | 872 | 873 | 874 | 875 | 876 | 877 | 878 | 879 | 880 | 881 | 882 | 883 | 884 | 885 | 886 | 887 | 888 | True 889 | 890 | 891 | transmission_interface/SimpleTransmission 892 | 893 | hardware_interface/EffortJointInterface 894 | 895 | 896 | hardware_interface/EffortJointInterface 897 | 1 898 | 899 | 900 | 901 | 907 | 908 | 909 | 910 | 911 | 912 | 913 | 914 | 915 | 916 | 917 | 918 | 919 | 920 | 921 | 922 | 923 | 924 | 925 | 926 | 927 | True 928 | 929 | 930 | transmission_interface/SimpleTransmission 931 | 932 | hardware_interface/EffortJointInterface 933 | 934 | 935 | hardware_interface/EffortJointInterface 936 | 1 937 | 938 | 939 | 940 | 946 | 947 | 948 | 949 | 950 | 951 | 952 | 953 | 954 | 955 | 956 | 957 | 958 | 959 | 960 | 961 | 962 | 963 | 964 | 965 | 966 | True 967 | 968 | 969 | transmission_interface/SimpleTransmission 970 | 971 | hardware_interface/EffortJointInterface 972 | 973 | 974 | hardware_interface/EffortJointInterface 975 | 1 976 | 977 | 978 | 979 | 985 | 986 | 987 | 988 | 989 | 990 | 991 | 992 | 993 | 994 | 995 | 996 | 997 | 998 | 999 | 1000 | 1001 | 1002 | 1003 | 1004 | 1005 | True 1006 | 1007 | 1008 | transmission_interface/SimpleTransmission 1009 | 1010 | hardware_interface/EffortJointInterface 1011 | 1012 | 1013 | hardware_interface/EffortJointInterface 1014 | 1 1015 | 1016 | 1017 | 1018 | 1019 | 1020 | 1021 | 1022 | 1023 | 1024 | 1025 | 1026 | 1027 | 1028 | 1029 | 1030 | 1031 | 1032 | 1033 | 1034 | 1035 | 1036 | 1037 | 1038 | 1039 | 1040 | 1041 | 1042 | 1043 | 1044 | True 1045 | 1046 | 1047 | transmission_interface/SimpleTransmission 1048 | 1049 | hardware_interface/EffortJointInterface 1050 | 1051 | 1052 | hardware_interface/EffortJointInterface 1053 | 1 1054 | 1055 | 1056 | 1057 | 1058 | 1059 | 1060 | 1061 | 1062 | 1063 | 1064 | 1065 | 1066 | 1067 | 1068 | 1069 | 1070 | 1071 | 1072 | 1073 | 1074 | 1075 | 1076 | 1077 | 1078 | 1079 | 1080 | 1081 | 1082 | 1083 | True 1084 | 1085 | 1086 | transmission_interface/SimpleTransmission 1087 | 1088 | hardware_interface/EffortJointInterface 1089 | 1090 | 1091 | hardware_interface/EffortJointInterface 1092 | 1 1093 | 1094 | 1095 | 1096 | 1097 | 1098 | 1099 | 1100 | 1101 | 1102 | 1103 | 1104 | 1105 | 1106 | 1107 | 1108 | 1109 | 1110 | 1111 | 1112 | 1113 | 1114 | 1115 | 1116 | 1117 | 1118 | 1119 | 1120 | 1121 | 1122 | True 1123 | 1124 | 1125 | transmission_interface/SimpleTransmission 1126 | 1127 | hardware_interface/EffortJointInterface 1128 | 1129 | 1130 | hardware_interface/EffortJointInterface 1131 | 1 1132 | 1133 | 1134 | 1135 | 1141 | 1142 | 1143 | 1144 | 1145 | 1146 | 1147 | 1148 | 1149 | 1150 | 1151 | 1152 | 1153 | 1154 | 1155 | 1156 | 1157 | 1158 | 1159 | 1160 | 1161 | True 1162 | 1163 | 1164 | transmission_interface/SimpleTransmission 1165 | 1166 | hardware_interface/EffortJointInterface 1167 | 1168 | 1169 | hardware_interface/EffortJointInterface 1170 | 1 1171 | 1172 | 1173 | 1174 | 1180 | 1181 | 1182 | 1183 | 1184 | 1185 | 1186 | 1187 | 1188 | 1189 | 1190 | 1191 | 1192 | 1193 | 1194 | 1195 | 1196 | 1197 | 1198 | 1199 | 1200 | True 1201 | 1202 | 1203 | transmission_interface/SimpleTransmission 1204 | 1205 | hardware_interface/EffortJointInterface 1206 | 1207 | 1208 | hardware_interface/EffortJointInterface 1209 | 1 1210 | 1211 | 1212 | 1213 | 1219 | 1220 | 1221 | 1222 | 1223 | 1224 | 1225 | 1226 | 1227 | 1228 | 1229 | 1230 | 1231 | 1232 | 1233 | 1234 | 1235 | 1236 | 1237 | 1238 | 1239 | True 1240 | 1241 | 1242 | transmission_interface/SimpleTransmission 1243 | 1244 | hardware_interface/EffortJointInterface 1245 | 1246 | 1247 | hardware_interface/EffortJointInterface 1248 | 1 1249 | 1250 | 1251 | 1252 | 1253 | 1254 | 1255 | 1256 | 1257 | 1258 | 1259 | 1260 | 1261 | 1262 | 1263 | 1264 | 1265 | 1266 | 1267 | 1268 | 1269 | 1270 | 1272 | 1273 | 1274 | -------------------------------------------------------------------------------- /Assets/robinion_meshes/collision/upper_body_collision.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/collision/upper_body_collision.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/fixed_robinion_torso_center.urdf: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | /robinion 11 | gazebo_ros_control/DefaultRobotHWSim 12 | 13 | 14 | 15 | 16 | 0.2 17 | 0.2 18 | Gazebo/White 19 | false 20 | 21 | 22 | 0.2 23 | 0.2 24 | Gazebo/White 25 | false 26 | 27 | 28 | 29 | 0.2 30 | 0.2 31 | Gazebo/White 32 | false 33 | 34 | 35 | 0.2 36 | 0.2 37 | Gazebo/White 38 | false 39 | 40 | 41 | 42 | 0.2 43 | 0.2 44 | Gazebo/White 45 | false 46 | 47 | 48 | 0.2 49 | 0.2 50 | Gazebo/White 51 | false 52 | 53 | 54 | 0.2 55 | 0.2 56 | Gazebo/White 57 | false 58 | 59 | 60 | 0.2 61 | 0.2 62 | Gazebo/White 63 | false 64 | 65 | 66 | 67 | 0.2 68 | 0.2 69 | Gazebo/White 70 | false 71 | 72 | 73 | 0.2 74 | 0.2 75 | Gazebo/White 76 | false 77 | 78 | 79 | 0.2 80 | 0.2 81 | Gazebo/White 82 | false 83 | 84 | 85 | 0.2 86 | 0.2 87 | Gazebo/White 88 | false 89 | 90 | 91 | 92 | 0.2 93 | 0.2 94 | Gazebo/White 95 | false 96 | 97 | 98 | 0.2 99 | 0.2 100 | Gazebo/White 101 | false 102 | 103 | 104 | 0.2 105 | 0.2 106 | Gazebo/White 107 | false 108 | 109 | 110 | 0.2 111 | 0.2 112 | Gazebo/White 113 | false 114 | 115 | 116 | 0.2 117 | 0.2 118 | Gazebo/White 119 | false 120 | 121 | 122 | 0.2 123 | 0.2 124 | Gazebo/White 125 | false 126 | 127 | 128 | 0.2 129 | 0.2 130 | Gazebo/White 131 | false 132 | 133 | 134 | 0.2 135 | 0.2 136 | Gazebo/White 137 | false 138 | 139 | 140 | 100000.0 141 | 100.0 142 | 1.5 143 | 1.5 144 | 1 0 0 145 | 1.0 146 | 0.001 147 | Gazebo/White 148 | false 149 | 150 | 151 | 152 | 0.2 153 | 0.2 154 | Gazebo/White 155 | false 156 | 157 | 158 | 0.2 159 | 0.2 160 | Gazebo/White 161 | false 162 | 163 | 164 | 0.2 165 | 0.2 166 | Gazebo/White 167 | false 168 | 169 | 170 | 0.2 171 | 0.2 172 | Gazebo/White 173 | false 174 | 175 | 176 | 0.2 177 | 0.2 178 | Gazebo/White 179 | false 180 | 181 | 182 | 0.2 183 | 0.2 184 | Gazebo/White 185 | false 186 | 187 | 188 | 0.2 189 | 0.2 190 | Gazebo/White 191 | false 192 | 193 | 194 | 0.2 195 | 0.2 196 | Gazebo/White 197 | false 198 | 199 | 200 | 100000.0 201 | 100.0 202 | 1.5 203 | 1.5 204 | 1 0 0 205 | 1.0 206 | 0.001 207 | Gazebo/White 208 | false 209 | 210 | 211 | 212 | 213 | 1.012300 214 | 215 | 320 216 | 240 217 | 218 | 219 | 0.001000 220 | 100.000000 221 | 222 | 223 | 1 224 | 30.000000 225 | 1 226 | 227 | true 228 | 30 229 | robinion/camera 230 | image_raw 231 | camera_info 232 | cam_link 233 | 0.07 234 | 0.0 235 | 0.0 236 | 0.0 237 | 0.0 238 | 0.0 239 | 240 | 241 | 242 | 243 | true 244 | 245 | true 246 | 125 247 | true 248 | __default_topic__ 249 | 250 | robinion/imu 251 | imu_link 252 | 125.0 253 | 0.0 254 | 0 0 0 255 | 0 0 0 256 | imu_link 257 | 258 | 259 | 260 | 261 | 262 | 269 | 270 | 271 | 272 | 273 | 274 | 275 | 276 | 277 | 278 | 279 | 280 | 281 | 282 | 283 | 284 | 285 | 286 | 287 | 288 | 289 | 290 | 291 | 292 | 293 | 294 | 295 | 296 | True 297 | 298 | 299 | transmission_interface/SimpleTransmission 300 | 301 | hardware_interface/EffortJointInterface 302 | 303 | 304 | hardware_interface/EffortJointInterface 305 | 1 306 | 307 | 308 | 309 | 310 | 311 | 312 | 313 | 314 | 315 | 316 | 317 | 318 | 319 | 320 | 321 | 322 | 323 | 324 | 325 | 326 | 327 | 334 | 335 | 336 | 337 | 338 | 339 | 340 | 341 | 342 | True 343 | 344 | 345 | transmission_interface/SimpleTransmission 346 | 347 | hardware_interface/EffortJointInterface 348 | 349 | 350 | hardware_interface/EffortJointInterface 351 | 1 352 | 353 | 354 | 355 | 356 | 357 | 358 | 359 | 360 | 361 | 362 | 363 | 364 | 365 | 366 | 367 | 368 | 369 | 370 | 371 | 372 | 373 | 374 | 375 | 376 | 377 | 378 | 379 | 380 | 381 | True 382 | 383 | 384 | transmission_interface/SimpleTransmission 385 | 386 | hardware_interface/EffortJointInterface 387 | 388 | 389 | hardware_interface/EffortJointInterface 390 | 1 391 | 392 | 393 | 394 | 395 | 396 | 397 | 398 | 399 | 400 | 401 | 402 | 403 | 404 | 405 | 406 | 407 | 408 | 409 | 410 | 411 | 412 | 419 | 426 | 427 | 428 | 429 | 430 | 431 | 432 | 433 | 434 | True 435 | 436 | 437 | transmission_interface/SimpleTransmission 438 | 439 | hardware_interface/EffortJointInterface 440 | 441 | 442 | hardware_interface/EffortJointInterface 443 | 1 444 | 445 | 446 | 447 | 448 | 449 | 450 | 451 | 452 | 453 | 454 | 455 | 456 | 457 | 458 | 459 | 460 | 461 | 462 | 463 | 464 | 465 | 466 | 467 | 468 | 469 | 470 | 471 | 472 | 473 | True 474 | 475 | 476 | transmission_interface/SimpleTransmission 477 | 478 | hardware_interface/EffortJointInterface 479 | 480 | 481 | hardware_interface/EffortJointInterface 482 | 1 483 | 484 | 485 | 486 | 487 | 488 | 489 | 490 | 491 | 492 | 493 | 494 | 495 | 496 | 497 | 498 | 499 | 500 | 501 | 502 | 503 | 504 | 505 | 506 | 507 | 508 | 509 | 510 | 511 | 512 | True 513 | 514 | 515 | transmission_interface/SimpleTransmission 516 | 517 | hardware_interface/EffortJointInterface 518 | 519 | 520 | hardware_interface/EffortJointInterface 521 | 1 522 | 523 | 524 | 525 | 526 | 527 | 528 | 529 | 530 | 531 | 532 | 533 | 534 | 535 | 536 | 537 | 538 | 539 | 540 | 541 | 542 | 543 | 544 | 545 | 546 | 547 | 548 | 549 | 550 | 551 | True 552 | 553 | 554 | transmission_interface/SimpleTransmission 555 | 556 | hardware_interface/EffortJointInterface 557 | 558 | 559 | hardware_interface/EffortJointInterface 560 | 1 561 | 562 | 563 | 564 | 565 | 566 | 567 | 568 | 569 | 570 | 571 | 572 | 573 | 574 | 575 | 576 | 577 | 578 | 579 | 580 | 581 | 582 | 590 | 591 | 592 | 593 | 594 | 595 | 596 | 597 | 598 | 599 | 600 | 601 | 602 | 603 | 604 | 605 | 606 | 607 | 608 | 609 | 610 | 611 | 612 | 613 | 614 | 615 | 616 | 617 | 618 | 619 | 620 | 621 | 622 | True 623 | 624 | 625 | transmission_interface/SimpleTransmission 626 | 627 | hardware_interface/EffortJointInterface 628 | 629 | 630 | hardware_interface/EffortJointInterface 631 | 1 632 | 633 | 634 | 635 | 636 | 637 | 638 | 639 | 640 | 641 | 642 | 643 | 644 | 645 | 646 | 647 | 648 | 649 | 650 | 651 | 652 | 653 | 654 | 655 | 656 | 657 | 658 | 659 | 660 | 661 | True 662 | 663 | 664 | transmission_interface/SimpleTransmission 665 | 666 | hardware_interface/EffortJointInterface 667 | 668 | 669 | hardware_interface/EffortJointInterface 670 | 1 671 | 672 | 673 | 674 | 675 | 676 | 677 | 678 | 679 | 680 | 681 | 682 | 683 | 684 | 685 | 686 | 687 | 688 | 689 | 690 | 691 | 692 | 693 | 694 | 695 | 696 | 697 | 698 | 699 | 700 | True 701 | 702 | 703 | transmission_interface/SimpleTransmission 704 | 705 | hardware_interface/EffortJointInterface 706 | 707 | 708 | hardware_interface/EffortJointInterface 709 | 1 710 | 711 | 712 | 713 | 714 | 715 | 716 | 717 | 718 | 719 | 720 | 721 | 722 | 723 | 724 | 725 | 726 | 727 | 728 | 729 | 730 | 731 | 732 | 733 | 734 | 735 | 736 | 737 | 738 | 739 | True 740 | 741 | 742 | transmission_interface/SimpleTransmission 743 | 744 | hardware_interface/EffortJointInterface 745 | 746 | 747 | hardware_interface/EffortJointInterface 748 | 1 749 | 750 | 751 | 752 | 753 | 754 | 755 | 756 | 757 | 758 | 759 | 760 | 761 | 762 | 763 | 764 | 765 | 766 | 767 | 768 | 769 | 770 | 778 | 779 | 780 | 781 | 782 | 783 | 784 | 785 | 786 | 787 | 788 | 789 | 790 | 791 | 792 | 793 | 794 | 795 | 796 | 797 | 798 | 799 | 800 | 801 | 802 | 803 | 804 | 805 | 806 | 807 | 808 | 809 | 810 | True 811 | 812 | 813 | transmission_interface/SimpleTransmission 814 | 815 | hardware_interface/EffortJointInterface 816 | 817 | 818 | hardware_interface/EffortJointInterface 819 | 1 820 | 821 | 822 | 823 | 824 | 825 | 826 | 827 | 828 | 829 | 830 | 831 | 832 | 833 | 834 | 835 | 836 | 837 | 838 | 839 | 840 | 841 | 842 | 843 | 844 | 845 | 846 | 847 | 848 | 849 | True 850 | 851 | 852 | transmission_interface/SimpleTransmission 853 | 854 | hardware_interface/EffortJointInterface 855 | 856 | 857 | hardware_interface/EffortJointInterface 858 | 1 859 | 860 | 861 | 862 | 863 | 864 | 865 | 866 | 867 | 868 | 869 | 870 | 871 | 872 | 873 | 874 | 875 | 876 | 877 | 878 | 879 | 880 | 881 | 882 | 883 | 884 | 885 | 886 | 887 | 888 | True 889 | 890 | 891 | transmission_interface/SimpleTransmission 892 | 893 | hardware_interface/EffortJointInterface 894 | 895 | 896 | hardware_interface/EffortJointInterface 897 | 1 898 | 899 | 900 | 901 | 907 | 908 | 909 | 910 | 911 | 912 | 913 | 914 | 915 | 916 | 917 | 918 | 919 | 920 | 921 | 922 | 923 | 924 | 925 | 926 | 927 | True 928 | 929 | 930 | transmission_interface/SimpleTransmission 931 | 932 | hardware_interface/EffortJointInterface 933 | 934 | 935 | hardware_interface/EffortJointInterface 936 | 1 937 | 938 | 939 | 940 | 946 | 947 | 948 | 949 | 950 | 951 | 952 | 953 | 954 | 955 | 956 | 957 | 958 | 959 | 960 | 961 | 962 | 963 | 964 | 965 | 966 | True 967 | 968 | 969 | transmission_interface/SimpleTransmission 970 | 971 | hardware_interface/EffortJointInterface 972 | 973 | 974 | hardware_interface/EffortJointInterface 975 | 1 976 | 977 | 978 | 979 | 985 | 986 | 987 | 988 | 989 | 990 | 991 | 992 | 993 | 994 | 995 | 996 | 997 | 998 | 999 | 1000 | 1001 | 1002 | 1003 | 1004 | 1005 | True 1006 | 1007 | 1008 | transmission_interface/SimpleTransmission 1009 | 1010 | hardware_interface/EffortJointInterface 1011 | 1012 | 1013 | hardware_interface/EffortJointInterface 1014 | 1 1015 | 1016 | 1017 | 1018 | 1019 | 1020 | 1021 | 1022 | 1023 | 1024 | 1025 | 1026 | 1027 | 1028 | 1029 | 1030 | 1031 | 1032 | 1033 | 1034 | 1035 | 1036 | 1037 | 1038 | 1039 | 1040 | 1041 | 1042 | 1043 | 1044 | True 1045 | 1046 | 1047 | transmission_interface/SimpleTransmission 1048 | 1049 | hardware_interface/EffortJointInterface 1050 | 1051 | 1052 | hardware_interface/EffortJointInterface 1053 | 1 1054 | 1055 | 1056 | 1057 | 1058 | 1059 | 1060 | 1061 | 1062 | 1063 | 1064 | 1065 | 1066 | 1067 | 1068 | 1069 | 1070 | 1071 | 1072 | 1073 | 1074 | 1075 | 1076 | 1077 | 1078 | 1079 | 1080 | 1081 | 1082 | 1083 | True 1084 | 1085 | 1086 | transmission_interface/SimpleTransmission 1087 | 1088 | hardware_interface/EffortJointInterface 1089 | 1090 | 1091 | hardware_interface/EffortJointInterface 1092 | 1 1093 | 1094 | 1095 | 1096 | 1097 | 1098 | 1099 | 1100 | 1101 | 1102 | 1103 | 1104 | 1105 | 1106 | 1107 | 1108 | 1109 | 1110 | 1111 | 1112 | 1113 | 1114 | 1115 | 1116 | 1117 | 1118 | 1119 | 1120 | 1121 | 1122 | True 1123 | 1124 | 1125 | transmission_interface/SimpleTransmission 1126 | 1127 | hardware_interface/EffortJointInterface 1128 | 1129 | 1130 | hardware_interface/EffortJointInterface 1131 | 1 1132 | 1133 | 1134 | 1135 | 1141 | 1142 | 1143 | 1144 | 1145 | 1146 | 1147 | 1148 | 1149 | 1150 | 1151 | 1152 | 1153 | 1154 | 1155 | 1156 | 1157 | 1158 | 1159 | 1160 | 1161 | True 1162 | 1163 | 1164 | transmission_interface/SimpleTransmission 1165 | 1166 | hardware_interface/EffortJointInterface 1167 | 1168 | 1169 | hardware_interface/EffortJointInterface 1170 | 1 1171 | 1172 | 1173 | 1174 | 1180 | 1181 | 1182 | 1183 | 1184 | 1185 | 1186 | 1187 | 1188 | 1189 | 1190 | 1191 | 1192 | 1193 | 1194 | 1195 | 1196 | 1197 | 1198 | 1199 | 1200 | True 1201 | 1202 | 1203 | transmission_interface/SimpleTransmission 1204 | 1205 | hardware_interface/EffortJointInterface 1206 | 1207 | 1208 | hardware_interface/EffortJointInterface 1209 | 1 1210 | 1211 | 1212 | 1213 | 1219 | 1220 | 1221 | 1222 | 1223 | 1224 | 1225 | 1226 | 1227 | 1228 | 1229 | 1230 | 1231 | 1232 | 1233 | 1234 | 1235 | 1236 | 1237 | 1238 | 1239 | True 1240 | 1241 | 1242 | transmission_interface/SimpleTransmission 1243 | 1244 | hardware_interface/EffortJointInterface 1245 | 1246 | 1247 | hardware_interface/EffortJointInterface 1248 | 1 1249 | 1250 | 1251 | 1252 | 1253 | 1254 | 1255 | 1256 | 1257 | 1258 | 1259 | 1260 | 1261 | 1262 | 1263 | 1264 | 1265 | 1266 | 1267 | 1268 | 1269 | 1270 | 1271 | 1272 | 1273 | 1274 | 1275 | 1276 | 1277 | 1278 | 1279 | 1280 | 1281 | 1282 | 1283 | 1284 | 1290 | 1291 | 1292 | 1293 | 1294 | 1295 | 1296 | 1297 | 1298 | 1299 | 1300 | 1301 | 1302 | 1303 | 1304 | 1305 | 1306 | 1307 | 1308 | 1309 | 1310 | 1311 | 1312 | 1313 | 1314 | 1315 | 1316 | 1317 | 1323 | 1324 | 1325 | 1326 | 1327 | 1328 | 1329 | 1330 | 1331 | 1332 | 1333 | 1334 | 1335 | 1336 | 1337 | 1338 | 1339 | 1340 | 1341 | 1342 | 1343 | 1344 | 1350 | 1351 | 1352 | 1353 | 1354 | 1355 | 1356 | 1357 | 1358 | 1359 | 1360 | 1361 | 1362 | 1363 | 1364 | 1365 | 1366 | 1367 | 1368 | 1369 | 1370 | 1371 | 1372 | 1373 | 1374 | 1375 | 1376 | 1377 | -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/head_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/head_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/head_yaw_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/head_yaw_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_ankle_roll_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_ankle_roll_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_arm_1_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_arm_1_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_arm_2_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_arm_2_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_arm_3_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_arm_3_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_foot_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_foot_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_gripper_closed_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_gripper_closed_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_gripper_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_gripper_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_hip_roll_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_hip_roll_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_hip_yaw_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_hip_yaw_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_knee_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_knee_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_shank_back_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_shank_back_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_shank_front_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_shank_front_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_shoulder_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_shoulder_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_thigh_back_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_thigh_back_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/l_thigh_front_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/l_thigh_front_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/lower_body_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/lower_body_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_ankle_roll_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_ankle_roll_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_arm_1_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_arm_1_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_arm_2_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_arm_2_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_arm_3_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_arm_3_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_foot_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_foot_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_gripper_closed_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_gripper_closed_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_gripper_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_gripper_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_hip_roll_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_hip_roll_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_hip_yaw_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_hip_yaw_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_knee_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_knee_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_shank_back_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_shank_back_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_shank_front_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_shank_front_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_shoulder_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_shoulder_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_thigh_back_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_thigh_back_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/r_thigh_front_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/r_thigh_front_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/head_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/head_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/head_yaw_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/head_yaw_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/left_ankle_roll_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/left_ankle_roll_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/left_foot_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/left_foot_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/left_gripper_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/left_gripper_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/left_hip_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/left_hip_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/left_hip_roll_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/left_hip_roll_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/left_leg_pitch_knee_ankle_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/left_leg_pitch_knee_ankle_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/left_shoulder_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/left_shoulder_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/right_ankle_roll_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/right_ankle_roll_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/right_foot_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/right_foot_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/right_gripper_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/right_gripper_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/right_hip_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/right_hip_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/right_hip_roll_pitch_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/right_hip_roll_pitch_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/right_leg_pitch_knee_ankle_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/right_leg_pitch_knee_ankle_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/standard/right_shoulder_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/standard/right_shoulder_visual.stl -------------------------------------------------------------------------------- /Assets/robinion_meshes/visual/upper_body_visual.stl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Assets/robinion_meshes/visual/upper_body_visual.stl -------------------------------------------------------------------------------- /Code/cfg/task/FixedRobinionBalanceBoardSim.yaml: -------------------------------------------------------------------------------- 1 | # used to create the object 2 | name: FixedRobinionBalanceBoardSim 3 | 4 | physics_engine: ${..physics_engine} 5 | 6 | debug_mode: False 7 | tolerance_disturbance_mode: False 8 | board_pitch_mode: False 9 | # if given, will override the device setting in gym. 10 | env: 11 | # numEnvs: ${...num_envs} 12 | actor: robinion 13 | numEnvs: ${resolve_default:512,${...num_envs}} 14 | envSpacing: 1.25 15 | maxEpisodeLength: 1000 16 | 17 | reset: 18 | min_height: 0.06 19 | max_height: 0.06 20 | robot_roller_reset_offset: 0.56 21 | 22 | clipActions: 1.0 23 | 24 | plane: 25 | staticFriction: 1.0 26 | dynamicFriction: 1.0 27 | restitution: 0.0 28 | 29 | # set to True if you use camera sensors in the environment 30 | enableCameraSensors: False 31 | 32 | reward_weight: 20.0 33 | reset_height_threshold: 0.35 34 | 35 | robot: 36 | control_mode: parallel_pitch 37 | controlled_joints: legs_shoulder_roll 38 | joints_list: 39 | free: 40 | all_joints: ["head_pitch_joint", "head_yaw_joint", "l_ankle_pitch_joint", "l_ankle_roll_joint", "l_elbow_pitch_joint", "l_elbow_yaw_joint", 41 | "l_hip_pitch_joint", "l_hip_roll_joint", "l_hip_yaw_joint", "l_knee_pitch_joint", "l_shoulder_pitch_joint", "l_shoulder_roll_joint", 42 | "r_ankle_pitch_joint", "r_ankle_roll_joint", "r_elbow_pitch_joint", "r_elbow_yaw_joint", "r_hip_pitch_joint", "r_hip_roll_joint", 43 | "r_hip_yaw_joint", "r_knee_pitch_joint", "r_shoulder_pitch_joint", "r_shoulder_roll_joint", "torso_pitch_joint"] 44 | 45 | legs_only: ["l_hip_roll_joint", "l_hip_pitch_joint", "l_knee_pitch_joint", "l_ankle_pitch_joint","r_hip_roll_joint", "r_hip_pitch_joint", 46 | "r_knee_pitch_joint", "r_ankle_pitch_joint", "l_ankle_roll_joint", "r_ankle_roll_joint"] 47 | 48 | parallel_pitch: 49 | all_joints: ["head_pitch_joint", "head_yaw_joint", "l_ankle_pitch_joint", "l_ankle_roll_joint", "l_elbow_pitch_joint", "l_elbow_yaw_joint", 50 | "l_hip_pitch_joint", "l_hip_roll_joint", "l_hip_yaw_joint", "l_knee_pitch_joint", "l_shoulder_pitch_joint", "l_shoulder_roll_joint", 51 | "r_ankle_pitch_joint", "r_ankle_roll_joint", "r_elbow_pitch_joint", "r_elbow_yaw_joint", "r_hip_pitch_joint", "r_hip_roll_joint", 52 | "r_hip_yaw_joint", "r_knee_pitch_joint", "r_shoulder_pitch_joint", "r_shoulder_roll_joint", "torso_pitch_joint"] 53 | 54 | legs_only: ["l_hip_roll_joint", "l_hip_pitch_joint", "l_knee_pitch_joint", "l_ankle_pitch_joint","r_hip_roll_joint", "r_hip_pitch_joint", 55 | "r_knee_pitch_joint", "r_ankle_pitch_joint", "l_ankle_roll_joint", "r_ankle_roll_joint"] 56 | 57 | legs_shoulder_roll: ["l_hip_roll_joint", "l_hip_pitch_joint", "l_knee_pitch_joint", "l_ankle_pitch_joint", "l_shoulder_roll_joint", 58 | "r_shoulder_roll_joint", "r_hip_roll_joint", "r_hip_pitch_joint", "r_knee_pitch_joint", "r_ankle_pitch_joint", 59 | "l_ankle_roll_joint", "r_ankle_roll_joint"] 60 | 61 | legs_shoulder_roll_pitch: ["l_hip_roll_joint", "l_hip_pitch_joint", "l_knee_pitch_joint", "l_ankle_pitch_joint", "l_shoulder_roll_joint", 62 | "r_shoulder_roll_joint", "r_hip_roll_joint", "r_hip_pitch_joint", "r_knee_pitch_joint", "r_ankle_pitch_joint", 63 | "l_ankle_roll_joint", "r_ankle_roll_joint", "l_shoulder_pitch_joint", r_shoulder_pitch_joint] 64 | 65 | joint_property: 66 | controlled_joints: 67 | stiffness: 80.0 68 | damping: 1.0 69 | 70 | fixed_joints: 71 | stiffness: 5000.0 72 | damping: 100.0 73 | 74 | actor_param: 75 | op3_param: 76 | min_height: 0.5 77 | max_height: 0.7 78 | 79 | robinion_param: 80 | min_height: 0.6 81 | max_height: 0.9 82 | 83 | sim: 84 | dt: 0.0166 # 1/60 s 85 | substeps: 2 86 | up_axis: "z" 87 | use_gpu_pipeline: ${eq:${...pipeline},"gpu"} 88 | gravity: [0.0, 0.0, -9.81] 89 | physx: 90 | num_threads: ${....num_threads} 91 | solver_type: ${....solver_type} 92 | use_gpu: ${contains:"cuda",${....sim_device}} # set to False to run on CPU 93 | num_position_iterations: 4 94 | num_velocity_iterations: 0 95 | contact_offset: 0.02 96 | rest_offset: 0.0 97 | bounce_threshold_velocity: 0.2 98 | max_depenetration_velocity: 10.0 99 | default_buffer_size_multiplier: 5.0 100 | max_gpu_contact_pairs: 8388608 # 8*1024*1024 101 | num_subscenes: ${....num_subscenes} 102 | contact_collection: 1 # 0: CC_NEVER (don't collect contact info), 1: CC_LAST_SUBSTEP (collect only contacts on last substep), 2: CC_ALL_SUBSTEPS (default - all contacts) 103 | 104 | task: 105 | randomize: False 106 | randomization_params: 107 | # specify which attributes to randomize for each actor type and property 108 | frequency: 600 # Define how many environment steps between generating new randomizations 109 | actions: 110 | range: [0.0, 0.003] 111 | operation: "additive" 112 | distribution: "gaussian" 113 | actor_params: 114 | actor: 115 | color: True 116 | rigid_body_properties: 117 | mass: 118 | range: [0.85, 1.15] 119 | operation: "scaling" 120 | distribution: "uniform" 121 | setup_only: True # Property will only be randomized once before simulation is started. See Domain Randomization Documentation for more info. 122 | dof_properties: 123 | damping: 124 | range: [0.9, 1.1] 125 | operation: "scaling" 126 | distribution: "uniform" 127 | stiffness: 128 | range: [0.75, 1.25] 129 | operation: "scaling" 130 | distribution: "uniform" 131 | -------------------------------------------------------------------------------- /Code/cfg/train/FixedRobinionBalanceBoardSimPPO.yaml: -------------------------------------------------------------------------------- 1 | params: 2 | seed: ${...seed} 3 | 4 | algo: 5 | name: a2c_continuous 6 | 7 | model: 8 | name: continuous_a2c_logstd 9 | 10 | network: 11 | name: actor_critic 12 | separate: False 13 | space: 14 | continuous: 15 | mu_activation: None 16 | sigma_activation: None 17 | 18 | mu_init: 19 | name: default 20 | sigma_init: 21 | name: const_initializer 22 | val: 0 23 | fixed_sigma: True 24 | mlp: 25 | units: [512, 256, 128] 26 | activation: elu 27 | d2rl: False 28 | 29 | initializer: 30 | name: default 31 | regularizer: 32 | name: None 33 | 34 | load_checkpoint: ${if:${...checkpoint},True,False} # flag which sets whether to load the checkpoint 35 | load_path: ${...checkpoint} # path to the checkpoint to load 36 | 37 | config: 38 | name: ${resolve_default:RobinionBalanceBoard,${....experiment}} 39 | full_experiment_name: ${.name} 40 | env_name: rlgpu 41 | multi_gpu: False 42 | ppo: True 43 | mixed_precision: True 44 | normalize_input: True 45 | normalize_value: True 46 | value_bootstrap: True 47 | num_actors: ${....task.env.numEnvs} 48 | reward_shaper: 49 | scale_value: 0.01 50 | normalize_advantage: True 51 | gamma: 0.99 52 | tau: 0.95 53 | learning_rate: 3e-5 54 | lr_schedule: constant 55 | schedule_type: legacy 56 | kl_threshold: 0.008 57 | score_to_win: 20000 58 | max_epochs: ${resolve_default:2250,${....max_iterations}} 59 | save_best_after: 50 60 | save_frequency: 100.0 61 | grad_norm: 1.0 62 | entropy_coef: 0.0 63 | truncate_grads: False 64 | e_clip: 0.2 65 | horizon_length: 32 66 | minibatch_size: 32768 67 | mini_epochs: 4 68 | critic_coef: 2 69 | clip_value: True 70 | seq_len: 4 71 | bounds_loss_coef: 0.0001 72 | -------------------------------------------------------------------------------- /Code/tasks/fixed_robinion_balance_board_sim.py: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2018-2021, NVIDIA Corporation 2 | # All rights reserved. 3 | # 4 | # Redistribution and use in source and binary forms, with or without 5 | # modification, are permitted provided that the following conditions are met: 6 | # 7 | # 1. Redistributions of source code must retain the above copyright notice, this 8 | # list of conditions and the following disclaimer. 9 | # 10 | # 2. Redistributions in binary form must reproduce the above copyright notice, 11 | # this list of conditions and the following disclaimer in the documentation 12 | # and/or other materials provided with the distribution. 13 | # 14 | # 3. Neither the name of the copyright holder nor the names of its 15 | # contributors may be used to endorse or promote products derived from 16 | # this software without specific prior written permission. 17 | # 18 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 19 | # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 20 | # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 21 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 22 | # FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 23 | # DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 24 | # SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 25 | # CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 26 | # OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27 | # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 | 29 | from distutils.log import error 30 | from logging import root 31 | from operator import index 32 | import gym 33 | import numpy as np 34 | import os 35 | import torch 36 | import math 37 | import sys 38 | import seaborn as sns 39 | import matplotlib.pyplot as plt 40 | 41 | from isaacgym import gymtorch 42 | from isaacgym import gymapi, gymutil 43 | from isaacgym.gymtorch import * 44 | 45 | from isaacgymenvs.utils.torch_jit_utils import * 46 | from .base.vec_task import VecTask 47 | 48 | from isaacgymenvs.utils import balance_board_utils 49 | from enum import IntEnum, auto 50 | 51 | class ControlMode(IntEnum): 52 | free = auto() 53 | parallel_pitch = auto() 54 | inverse_kinematics = auto() 55 | 56 | class ControlledJoints(IntEnum): 57 | all_joints = auto() 58 | legs_only = auto() 59 | legs_shoulder_roll = auto() 60 | legs_shoulder_roll_pitch = auto() 61 | 62 | class FixedRobinionBalanceBoardSim(VecTask): 63 | 64 | def __init__(self, cfg, sim_device, graphics_device_id, headless): 65 | self.cfg = cfg 66 | self.debug_mode = self.cfg["debug_mode"] 67 | self.tolerance_disturbance_mode = self.cfg["tolerance_disturbance_mode"] 68 | self.board_pitch_mode = self.cfg["board_pitch_mode"] 69 | 70 | self.load_cfg_params() 71 | 72 | _ctrl_mode = self.cfg["robot"]["control_mode"] 73 | _controlled_joints = self.cfg["robot"]["controlled_joints"] 74 | 75 | if _ctrl_mode == "free" and _controlled_joints == "all_joints": 76 | self.p_control_mode = ControlMode.free 77 | self.p_controlled_joints = ControlledJoints.all_joints 78 | 79 | elif _ctrl_mode == "free" and _controlled_joints == "legs_only": 80 | self.p_control_mode = ControlMode.free 81 | self.p_controlled_joints = ControlledJoints.legs_only 82 | 83 | elif _ctrl_mode == "parallel_pitch" and _controlled_joints == "all_joints": 84 | self.p_control_mode = ControlMode.parallel_pitch 85 | self.p_controlled_joints = ControlledJoints.all_joints 86 | 87 | elif _ctrl_mode == "parallel_pitch" and _controlled_joints == "legs_only": 88 | self.p_control_mode = ControlMode.parallel_pitch 89 | self.p_controlled_joints = ControlledJoints.legs_only 90 | 91 | elif _ctrl_mode == "parallel_pitch" and _controlled_joints == "legs_shoulder_roll": 92 | self.p_control_mode = ControlMode.parallel_pitch 93 | self.p_controlled_joints = ControlledJoints.legs_shoulder_roll 94 | 95 | elif _ctrl_mode == "parallel_pitch" and _controlled_joints == "legs_shoulder_roll_pitch": 96 | self.p_control_mode = ControlMode.parallel_pitch 97 | self.p_controlled_joints = ControlledJoints.legs_shoulder_roll_pitch 98 | 99 | elif _ctrl_mode == "inverse_kinematics": 100 | raise NotImplementedError 101 | self.p_control_mode = ControlMode.parallel_pitch 102 | 103 | self.mode_joint_name = self.p_control_mode.name + "_" + self.p_controlled_joints.name 104 | self.cfg["env"]["numObservations"] = 3 * len(self.mode_joints_dict[self.mode_joint_name + "_dof"]) + 24 105 | self.cfg["env"]["numActions"] = len(self.mode_joints_dict[self.mode_joint_name + "_dof"]) 106 | if self.p_control_mode == ControlMode.parallel_pitch: 107 | self.cfg["env"]["numActions"] -= 4 108 | 109 | super().__init__(config=self.cfg, sim_device=sim_device, graphics_device_id=graphics_device_id, headless=headless) 110 | self.extras = {} 111 | torch_zeros = lambda : torch.zeros(self.num_envs, dtype=torch.float, device=self.device, requires_grad=False) 112 | self.episode_sums = {"bongo_board": torch_zeros(), "upright": torch_zeros(), "bongo_vel" : torch_zeros(), "l_foot_position" : torch_zeros()} 113 | 114 | self.allocate_util_tensors() 115 | 116 | if self.viewer != None: 117 | cam_pos = gymapi.Vec3(5.0, 5.5, 0.85) 118 | cam_target = gymapi.Vec3(10.0, 10.0, 0.35) 119 | self.gym.viewer_camera_look_at(self.viewer, None, cam_pos, cam_target) 120 | 121 | def load_cfg_params(self): 122 | self.randomize = self.cfg["task"]["randomize"] 123 | self.randomization_params = self.cfg["task"]["randomization_params"] 124 | self.p_dt = self.cfg["sim"]["dt"] 125 | self.max_episode_length = self.cfg["env"]["maxEpisodeLength"] 126 | self.p_robot_reset_height_threshold = self.cfg["env"]["reset_height_threshold"] 127 | 128 | self.p_min_roller_height = self.cfg["env"]["reset"]["min_height"] 129 | self.p_max_roller_height = self.cfg["env"]["reset"]["max_height"] 130 | self.p_reset_range_height = self.p_max_roller_height - self.p_min_roller_height 131 | self.p_robot_roller_reset_offset_height = self.cfg["env"]["reset"]["robot_roller_reset_offset"] 132 | 133 | self.p_controlled_joints_stiffness = self.cfg["robot"]["joint_property"]["controlled_joints"]["stiffness"] 134 | self.p_controlled_joints_damping = self.cfg["robot"]["joint_property"]["controlled_joints"]["damping"] 135 | self.p_fixed_joints_stiffness = self.cfg["robot"]["joint_property"]["fixed_joints"]["stiffness"] 136 | self.p_fixed_joints_damping = self.cfg["robot"]["joint_property"]["fixed_joints"]["damping"] 137 | 138 | self.free_all_joints_dof = self.cfg["robot"]["joints_list"]["free"]["all_joints"] 139 | self.free_legs_only_dof = self.cfg["robot"]["joints_list"]["free"]["legs_only"] 140 | self.parallel_pitch_all_joints_dof = self.cfg["robot"]["joints_list"]["parallel_pitch"]["all_joints"] 141 | self.parallel_pitch_legs_only_dof = self.cfg["robot"]["joints_list"]["parallel_pitch"]["legs_only"] 142 | self.parallel_pitch_legs_shoulder_roll_dof = self.cfg["robot"]["joints_list"]["parallel_pitch"]["legs_shoulder_roll"] 143 | self.parallel_pitch_legs_shoulder_roll_pitch_dof = self.cfg["robot"]["joints_list"]["parallel_pitch"]["legs_shoulder_roll_pitch"] 144 | self.mode_joints_dict = {"free_all_joints_dof": self.free_all_joints_dof, 145 | "free_legs_only_dof": self.free_legs_only_dof, 146 | "parallel_pitch_all_joints_dof": self.parallel_pitch_all_joints_dof, 147 | "parallel_pitch_legs_only_dof": self.parallel_pitch_legs_only_dof, 148 | "parallel_pitch_legs_shoulder_roll_dof": self.parallel_pitch_legs_shoulder_roll_dof, 149 | "parallel_pitch_legs_shoulder_roll_pitch_dof": self.parallel_pitch_legs_shoulder_roll_pitch_dof} 150 | 151 | self.p_bongo_board_reward_weight = self.cfg["env"]["reward_weight"] 152 | 153 | def allocate_util_tensors(self): 154 | self.roller_id = torch.tensor([2*x for x in range(self.num_envs)], 155 | device=self.device, dtype=torch.long) 156 | self.robot_id = self.roller_id + 1 157 | 158 | self._root_tensor = self.gym.acquire_actor_root_state_tensor(self.sim) # Simulation format. shape : num_actors, 13 159 | self.root_tensor = gymtorch.wrap_tensor(self._root_tensor) # PyTorch, Torch Tensor. 160 | self.root_positions = self.root_tensor[:, 0:3] 161 | self.root_orientations = self.root_tensor[:, 3:7] 162 | self.root_linear_vels = self.root_tensor[:, 7:10] 163 | self.root_angular_vels = self.root_tensor[:, 10:13] 164 | 165 | self._dof_state_tensor = self.gym.acquire_dof_state_tensor(self.sim) # shape : num_envs * num_dofs, 2 166 | self.dof_state_tensor = gymtorch.wrap_tensor(self._dof_state_tensor) 167 | self.dof_pos = self.dof_state_tensor.view(self.num_envs, self.num_dof, 2)[..., 0] 168 | self.dof_vel = self.dof_state_tensor.view(self.num_envs, self.num_dof, 2)[..., 1] 169 | 170 | self._rigid_body_tensor = self.gym.acquire_rigid_body_state_tensor(self.sim) # shape: num_envs, num_rigid_bodies, 13 171 | self.rigid_body_state = gymtorch.wrap_tensor(self._rigid_body_tensor).view(self.num_envs, -1, 13) 172 | self.num_bodies = self.rigid_body_state.shape[1] 173 | 174 | self.gym.refresh_dof_state_tensor(self.sim) 175 | self.gym.refresh_actor_root_state_tensor(self.sim) 176 | self.gym.refresh_rigid_body_state_tensor(self.sim) 177 | 178 | self.reset_root_tensor = self.root_tensor.clone() 179 | self.reset_root_tensor[:, 7:13] = 0.0 180 | self.reset_dof_pos = self.dof_pos.clone() 181 | self.reset_dof_vel = self.dof_vel.clone() 182 | 183 | self.pd_targets_tensor = torch.zeros((self.num_envs, self.num_dof), device=self.device, dtype=torch.float32) 184 | self.default_pose_tensor = torch.zeros_like(self.pd_targets_tensor) 185 | 186 | if self.p_control_mode == ControlMode.parallel_pitch: 187 | self.parallel_pitch_weight_tensor = torch.ones(len(self.controlled_joints_index), device=self.device, dtype=torch.float32) 188 | self.parallel_pitch_weight_tensor[[self.l_knee_pitch_index_index, self.r_knee_pitch_index_index]] = -2.0 #knee angle will be twice than the hip angle 189 | 190 | # Unit vector pointing up, along the Z axis. 191 | self.z_basis_vec = torch.zeros((self.num_envs, 3), device=self.device, dtype=torch.float32) 192 | self.z_basis_vec[:, 2] = 1.0 193 | 194 | self.log_reward_tensor = torch.zeros((4, self.num_envs), device=self.device, dtype=torch.float32) 195 | 196 | #Compute the average disturbance the robot can tolerate 197 | if self.tolerance_disturbance_mode: 198 | self.forces = torch.zeros((self.num_envs, self.num_bodies, 3), device=self.device, dtype=torch.float) 199 | self.tolerance_count = 0 200 | self.force_magnitude = torch.zeros(self.num_envs, device=self.device, dtype=torch.float) 201 | self.total_force = 0 202 | self.fall_time = 0 203 | 204 | #Compute the avarage board angle 205 | if self.board_pitch_mode: 206 | self.board_pitch_ang_vel_sum = torch.zeros(1, device = self.device, dtype=torch.float32) 207 | self.average_board_pitch_ang_vel = torch.zeros(1, device = self.device, dtype=torch.float32) 208 | self.step_count = torch.zeros(1, device = self.device, dtype=torch.float32) 209 | 210 | def create_sim(self): 211 | self.up_axis_idx = self.set_sim_params_up_axis(self.sim_params, 'z') 212 | self.sim = super().create_sim(self.device_id, self.graphics_device_id, self.physics_engine, self.sim_params) 213 | self._create_ground_plane() 214 | print(f'num envs {self.num_envs} env spacing {self.cfg["env"]["envSpacing"]}') 215 | self._create_envs(self.num_envs, self.cfg["env"]['envSpacing'], int(np.sqrt(self.num_envs))) 216 | 217 | # If randomizing, apply once immediately on startup before the first sim step 218 | if self.randomize: 219 | self.apply_randomizations(self.randomization_params) 220 | 221 | def _create_ground_plane(self): 222 | plane_params = gymapi.PlaneParams() 223 | plane_params.normal = gymapi.Vec3(0.0, 0.0, 1.0) 224 | plane_params.static_friction = self.cfg["env"]["plane"]["staticFriction"] 225 | plane_params.dynamic_friction = self.cfg["env"]["plane"]["dynamicFriction"] 226 | self.gym.add_ground(self.sim, plane_params) 227 | 228 | def load_roller_asset(self): 229 | assets_root_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../assets") 230 | asset_roller_path = "urdf/bongo_board/roller.urdf" 231 | asset_roller_options = gymapi.AssetOptions() 232 | asset_roller_options.fix_base_link = False 233 | asset_roller_options.replace_cylinder_with_capsule = True 234 | 235 | self.asset_roller = self.gym.load_asset(self.sim, assets_root_dir, asset_roller_path, asset_roller_options) 236 | if self.debug_mode: 237 | balance_board_utils.print_asset_info(self.asset_roller,"Roller", self.gym) 238 | input("Here is the info of Roller, press any key to continue") 239 | 240 | self.roller_start_pose = gymapi.Transform() 241 | self.roller_start_pose.p.z = 0.06 242 | 243 | def load_robinion_asset(self): 244 | assets_root_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../assets") 245 | asset_robinion_path = "urdf/robinion_meshes/fixed_robinion_torso_center.urdf" 246 | asset_robinion_options = gymapi.AssetOptions() 247 | asset_robinion_options.fix_base_link = False 248 | asset_robinion_options.collapse_fixed_joints = False 249 | 250 | #Use vhacd to overcome the triangle meshes 251 | asset_robinion_options.vhacd_enabled = True 252 | asset_robinion_options.vhacd_params.resolution = 300000 253 | asset_robinion_options.vhacd_params.max_convex_hulls = 10 254 | asset_robinion_options.vhacd_params.max_num_vertices_per_ch = 64 255 | 256 | self.asset_robinion = self.gym.load_asset(self.sim, assets_root_dir, asset_robinion_path, asset_robinion_options) 257 | if self.debug_mode: 258 | balance_board_utils.print_asset_info(self.asset_robinion, "Robinion", self.gym) 259 | input(f"Here is the info of Robinion, press any key to continue") 260 | self.num_dof = self.gym.get_asset_dof_count(self.asset_robinion) 261 | 262 | self.robinion_start_pose = gymapi.Transform() 263 | self.robinion_start_pose.p.z = 0.72 264 | self.robinion_start_pose.r = gymapi.Quat(0.0, 0.0, -0.707107, 0.707107) 265 | 266 | def Generate_dict_for_indexing(self): 267 | #Rigid body dict 268 | self.robinion_rigid_body_dict = self.gym.get_asset_rigid_body_dict(self.asset_robinion) 269 | for rigid_body_name in self.robinion_rigid_body_dict.keys(): 270 | self.robinion_rigid_body_dict[rigid_body_name] += 1 #0 is roller 271 | #Dof dict 272 | self.dof_dict = self.gym.get_asset_dof_dict(self.asset_robinion) 273 | 274 | self.mode_joints_index_dict = {} 275 | for mode_joints in self.mode_joints_dict: 276 | controlled_dof_index = [] 277 | for dof_name in self.mode_joints_dict[mode_joints]: 278 | controlled_dof_index.append(self.dof_dict[dof_name]) 279 | controlled_dof_index = sorted(controlled_dof_index) 280 | self.mode_joints_index_dict[mode_joints + "_index"] = controlled_dof_index 281 | self.mode_joints_index_dict["fixed_" + mode_joints + "_index"] = list(set(range(self.num_dof)) - set(controlled_dof_index)) 282 | 283 | self.controlled_joints_index = self.mode_joints_index_dict[self.mode_joint_name + "_dof_index"] 284 | self.fixed_joints_index = self.mode_joints_index_dict["fixed_" + self.mode_joint_name + "_dof_index"] 285 | 286 | #For parallel pitch mode 287 | if self.p_control_mode == ControlMode.parallel_pitch: 288 | self.l_knee_pitch_index = self.dof_dict["l_knee_pitch_joint"] 289 | self.r_knee_pitch_index = self.dof_dict["r_knee_pitch_joint"] 290 | self.l_ankle_pitch_index = self.dof_dict["l_ankle_pitch_joint"] 291 | self.r_ankle_pitch_index = self.dof_dict["r_ankle_pitch_joint"] 292 | self.l_knee_pitch_index_index = self.controlled_joints_index.index(self.dof_dict["l_knee_pitch_joint"]) 293 | self.r_knee_pitch_index_index = self.controlled_joints_index.index(self.dof_dict["r_knee_pitch_joint"]) 294 | 295 | knee_ankle_list = [self.l_knee_pitch_index, self.l_ankle_pitch_index, self.r_knee_pitch_index, self.r_ankle_pitch_index] 296 | self.list_transfer_hip_to_knee_ankle = [] 297 | 298 | index = -1 299 | for dof_index in self.controlled_joints_index: 300 | if dof_index in knee_ankle_list: 301 | self.list_transfer_hip_to_knee_ankle.append(index) 302 | else: 303 | index += 1 304 | self.list_transfer_hip_to_knee_ankle.append(index) 305 | 306 | self.parallel_pitch_without_knee_ankle = self.controlled_joints_index.copy() 307 | self.parallel_pitch_without_knee_ankle.remove(self.dof_dict["l_knee_pitch_joint"]) 308 | self.parallel_pitch_without_knee_ankle.remove(self.dof_dict["r_knee_pitch_joint"]) 309 | self.parallel_pitch_without_knee_ankle.remove(self.dof_dict["l_ankle_pitch_joint"]) 310 | self.parallel_pitch_without_knee_ankle.remove(self.dof_dict["r_ankle_pitch_joint"]) 311 | 312 | def _create_envs(self, num_envs, spacing, num_per_row): 313 | self.load_roller_asset() 314 | self.load_robinion_asset() 315 | self.Generate_dict_for_indexing() 316 | 317 | lower = gymapi.Vec3(-spacing, -spacing, 0.0) 318 | upper = gymapi.Vec3(spacing, spacing, spacing) 319 | 320 | self.envs = [] 321 | self.rollers = [] 322 | self.robinions = [] 323 | 324 | for i in range(self.num_envs): 325 | env_ptr = self.gym.create_env(self.sim, lower, upper, num_per_row) 326 | roller = self.gym.create_actor(env_ptr, self.asset_roller, self.roller_start_pose, "roller", i, 0, 0) 327 | robinion = self.gym.create_actor(env_ptr, self.asset_robinion, self.robinion_start_pose, "actor", i, 0, 0) 328 | 329 | dof_props = self.gym.get_actor_dof_properties(env_ptr, robinion) 330 | dof_props["driveMode"].fill(gymapi.DOF_MODE_POS) 331 | dof_props["stiffness"][self.controlled_joints_index] = self.p_controlled_joints_stiffness 332 | dof_props["damping"][self.controlled_joints_index] = self.p_controlled_joints_damping 333 | dof_props["stiffness"][self.fixed_joints_index] = self.p_fixed_joints_stiffness 334 | dof_props["damping"][self.fixed_joints_index] = self.p_fixed_joints_damping 335 | 336 | self.gym.set_actor_dof_properties(env_ptr, robinion, dof_props) 337 | self.rollers.append(roller) 338 | self.robinions.append(robinion) 339 | self.envs.append(env_ptr) 340 | 341 | self.dof_props = dof_props 342 | self.upper_limit = torch.tensor(self.dof_props["upper"], 343 | device=self.device, dtype=torch.float32) 344 | self.lower_limit = torch.tensor(self.dof_props["lower"], 345 | device=self.device, dtype=torch.float32) 346 | self.dof_range = self.upper_limit - self.lower_limit 347 | 348 | if self.debug_mode == True: 349 | balance_board_utils.print_actor_info(self.gym, self.envs[0], self.robinions[0]) 350 | 351 | def compute_reward(self): 352 | self.rew_buf[:], self.reset_buf[:], self.log_reward_tensor = compute_reward( 353 | self.reset_buf, 354 | self.root_tensor, 355 | self.rigid_body_state, 356 | self.robinion_rigid_body_dict, 357 | self.roller_id, 358 | self.robot_id, 359 | self.p_bongo_board_reward_weight, 360 | self.p_robot_reset_height_threshold, 361 | self.max_episode_length, 362 | self.progress_buf, 363 | self.obs_buf, 364 | self.log_reward_tensor 365 | ) 366 | self.episode_sums["bongo_board"] += self.log_reward_tensor[0] 367 | self.episode_sums["bongo_vel"] += self.log_reward_tensor[2] 368 | self.episode_sums["upright"] += self.log_reward_tensor[1] 369 | self.episode_sums["l_foot_position"] += self.log_reward_tensor[3] 370 | 371 | def compute_observations(self): 372 | self.gym.refresh_actor_root_state_tensor(self.sim) 373 | self.gym.refresh_dof_state_tensor(self.sim) 374 | self.gym.refresh_rigid_body_state_tensor(self.sim) 375 | self.obs_buf[:] = compute_observations_sim( 376 | self.root_tensor, 377 | self.rigid_body_state, 378 | self.robinion_rigid_body_dict, 379 | self.dof_pos, 380 | self.dof_vel, 381 | self.robot_id, 382 | self.roller_id, 383 | self.z_basis_vec, 384 | self.controlled_joints_index, 385 | self.previous_actions, 386 | ) 387 | 388 | def reset_envs(self, env_ids): 389 | if self.randomize: 390 | self.apply_randomizations(self.randomization_params) 391 | robot_reset_indexes = self.robot_id[env_ids] 392 | roller_reset_indexes = self.roller_id[env_ids] 393 | actor_reset_indexes = torch.cat([ 394 | robot_reset_indexes, roller_reset_indexes 395 | ]) 396 | actor_reset_indexes_int32 = actor_reset_indexes.to(torch.int32) 397 | 398 | self.root_tensor[actor_reset_indexes, :] = self.reset_root_tensor[actor_reset_indexes, :] 399 | roller_reset_height = (torch.rand((len(env_ids),), device=self.device) * \ 400 | self.p_reset_range_height) + self.p_min_roller_height 401 | self.root_positions[roller_reset_indexes, 2] = roller_reset_height 402 | self.root_positions[robot_reset_indexes, 2] = roller_reset_height + self.p_robot_roller_reset_offset_height 403 | 404 | self.gym.set_actor_root_state_tensor_indexed(self.sim, 405 | gymtorch.unwrap_tensor(self.root_tensor), 406 | gymtorch.unwrap_tensor(actor_reset_indexes_int32), 407 | len(actor_reset_indexes)) 408 | 409 | robot_reset_indexes_int32 = robot_reset_indexes.to(dtype=torch.int32) 410 | positions = torch_rand_float(-0.03, 0.03, (len(env_ids), self.num_dof), device=self.device)#0.03 411 | velocities = torch_rand_float(-0.1, 0.1, (len(env_ids), self.num_dof), device=self.device)#0.1 412 | self.dof_pos[env_ids] = tensor_clamp(self.reset_dof_pos[env_ids] + positions, self.lower_limit, self.upper_limit) 413 | self.dof_vel[env_ids] = velocities 414 | self.gym.set_dof_state_tensor_indexed(self.sim, 415 | gymtorch.unwrap_tensor(self.dof_state_tensor), 416 | gymtorch.unwrap_tensor(robot_reset_indexes_int32), 417 | len(robot_reset_indexes_int32)) 418 | 419 | self.progress_buf[env_ids] = 0 420 | self.reset_buf[env_ids] = 0 421 | 422 | # fill extras 423 | self.extras["episode"] = {} 424 | for key in self.episode_sums.keys(): 425 | self.extras["episode"]['rew_' + key] = self.episode_sums[key][env_ids] 426 | self.episode_sums[key][env_ids] = 0. 427 | 428 | def pre_physics_step(self, actions): 429 | if self.p_control_mode == ControlMode.free: 430 | actions = self.dof_range[self.controlled_joints_index]/2*actions+((self.upper_limit+self.lower_limit)/2)[self.controlled_joints_index] 431 | 432 | elif self.p_control_mode == ControlMode.parallel_pitch: 433 | rotating_degree_pitch_only = actions * self.dof_range[self.parallel_pitch_without_knee_ankle]/2 #multiply the dof range except for knees and ankles 434 | rotating_degree = rotating_degree_pitch_only[:, self.list_transfer_hip_to_knee_ankle] * self.parallel_pitch_weight_tensor #Mapping and mutiply the weight for knee 435 | actions = rotating_degree + ((self.upper_limit + self.lower_limit)/2)[self.controlled_joints_index] #Add the middle of the joint 436 | 437 | elif self.p_control_mode == ControlMode.inverse_kinematics: 438 | pass 439 | 440 | self.pd_targets_tensor[:, self.controlled_joints_index] = actions 441 | self.pd_targets_tensor = saturate(self.pd_targets_tensor, 442 | self.lower_limit, self.upper_limit) 443 | self.gym.set_dof_position_target_tensor(self.sim, 444 | gymtorch.unwrap_tensor(self.pd_targets_tensor)) 445 | 446 | self.previous_actions = actions.clone() 447 | 448 | def post_physics_step(self): 449 | self.progress_buf += 1 450 | self.randomize_buf += 1 451 | 452 | reset_env_ids = self.reset_buf.nonzero(as_tuple=False).flatten() 453 | #Compute disturbance tolerance 454 | if len(reset_env_ids) > 0: 455 | self.reset_envs(reset_env_ids) 456 | if self.tolerance_disturbance_mode: 457 | self.total_force += sum(self.force_magnitude[reset_env_ids]) 458 | print(self.total_force) 459 | self.force_magnitude[reset_env_ids] = 0 460 | self.fall_time += len(reset_env_ids) 461 | print(self.fall_time) 462 | print(self.total_force / self.fall_time) 463 | if self.fall_time >= 2048: 464 | print(self.total_force / self.fall_time) 465 | input() 466 | 467 | self.compute_observations() 468 | self.compute_reward() 469 | 470 | #Add 10N every 2 seconds 471 | if self.tolerance_disturbance_mode and self.tolerance_count % 120 == 0: 472 | print("Apply force:", self.force_magnitude) 473 | self.gym.apply_rigid_body_force_tensors(self.sim, gymtorch.unwrap_tensor(self.forces), None, gymapi.GLOBAL_SPACE) 474 | self.force_magnitude += 10 475 | self.forces[:, 1, :] = normalize(torch.randn((self.num_envs,3), device = self.device, dtype=torch.float32)) 476 | self.forces[:, 1, :] *= self.force_magnitude.unsqueeze(-1) 477 | 478 | if self.tolerance_disturbance_mode: 479 | self.tolerance_count += 1 480 | 481 | #Compute average angular velocity of the board 482 | if self.board_pitch_mode: 483 | self.board_pitch_ang_vel_sum += self.rigid_body_state[0, self.robinion_rigid_body_dict["board"], 11] 484 | self.step_count += 1 485 | print(self.board_pitch_ang_vel_sum) 486 | self.average_board_pitch_ang_vel = self.board_pitch_ang_vel_sum / self.step_count 487 | print("step count", self.step_count, "Average ang vel:", self.average_board_pitch_ang_vel) 488 | if self.step_count == 3600: 489 | input() 490 | 491 | ##################################################################### 492 | ###=========================jit functions=========================### 493 | ##################################################################### 494 | @torch.jit.script 495 | def compute_reward(reset_buf, root_tensor, rigid_body_state, robinion_rigid_body_dict, roller_ids, robot_actor_ids, 496 | p_bongo_board_reward_weight, p_robot_reset_height_threshold, max_episode_length, progress_buf, obs_buf, log_reward_tensor): 497 | # type: (Tensor, Tensor, Tensor, Dict[str, int], Tensor, Tensor, float, float, int, Tensor, Tensor, Tensor) -> Tuple[Tensor, Tensor, Tensor] 498 | 499 | ones = torch.ones_like(reset_buf) 500 | robot_torso_height = root_tensor[robot_actor_ids, 2] 501 | bongo_board_angvels = rigid_body_state[:, robinion_rigid_body_dict["board"], 7:10] 502 | 503 | bongo_board_rpy = get_euler_xyz(rigid_body_state[:, robinion_rigid_body_dict["board"], 3:7]) 504 | pitch = bongo_board_rpy[0] 505 | pitch = torch.where(pitch > np.pi, pitch % np.pi - np.pi, pitch) 506 | 507 | up_projection = obs_buf[:, -1] 508 | up_projection = torch.clip(up_projection, 0.0, 1.0) 509 | 510 | l_foot_position = rigid_body_state[:, robinion_rigid_body_dict["l_foot_link"], 0:3] 511 | Det_box_position = rigid_body_state[:, robinion_rigid_body_dict["DET_BOX_l_foot"], 0:3] 512 | l_foot_dist = torch.norm(l_foot_position - Det_box_position, p=2, dim=1) 513 | 514 | bongo_board_rew = 1 - (p_bongo_board_reward_weight * pitch ** 2) 515 | bongo_board_vel_rew = -torch.norm(bongo_board_angvels, dim=1) 516 | upright_rew = -(1 - up_projection) 517 | l_foot_dist_reward = 1 - 20 * l_foot_dist**2 518 | rew_buf = bongo_board_rew + upright_rew + bongo_board_vel_rew + l_foot_dist 519 | 520 | log_reward_tensor[0] = bongo_board_rew 521 | log_reward_tensor[1] = upright_rew 522 | log_reward_tensor[2] = bongo_board_vel_rew 523 | log_reward_tensor[3] = l_foot_dist_reward 524 | 525 | yaw = bongo_board_rpy[2] 526 | yaw = torch.where(yaw > np.pi, yaw % np.pi - np.pi, yaw) 527 | yaw_degree = yaw * 180 / np.pi 528 | tolerate_degree = 5 529 | 530 | reset_buf[progress_buf >= max_episode_length] = 1 531 | reset_buf = torch.where(robot_torso_height < p_robot_reset_height_threshold, ones, reset_buf) 532 | reset_buf = torch.where(rigid_body_state[:, robinion_rigid_body_dict["L_detection_box"], 2] < 0.0 , ones, reset_buf) 533 | reset_buf = torch.where(rigid_body_state[:, robinion_rigid_body_dict["R_detection_box"], 2] < 0.0 , ones, reset_buf) 534 | reset_buf = torch.where(l_foot_dist > 0.08, ones, reset_buf) 535 | reset_buf = torch.where(yaw_degree < -90 - tolerate_degree, ones, reset_buf) 536 | reset_buf = torch.where(yaw_degree > -90 + tolerate_degree, ones, reset_buf) 537 | reset_buf[progress_buf == 0] = 0 538 | #Reset punishment 539 | rew_buf = torch.where(robot_torso_height < p_robot_reset_height_threshold, ones * -1.0, rew_buf) 540 | rew_buf = torch.where(rigid_body_state[:, robinion_rigid_body_dict["L_detection_box"], 2] < 0.0 , ones * -1.0, rew_buf) 541 | rew_buf = torch.where(rigid_body_state[:, robinion_rigid_body_dict["R_detection_box"], 2] < 0.0 , ones * -1.0, rew_buf) 542 | rew_buf = torch.where(l_foot_dist > 0.08, ones * -1.0, rew_buf) 543 | rew_buf = torch.where(yaw_degree < -90 - tolerate_degree, ones * -1.0, rew_buf) 544 | rew_buf = torch.where(yaw_degree > -90 + tolerate_degree, ones * -1.0, rew_buf) 545 | 546 | return rew_buf, reset_buf, log_reward_tensor 547 | 548 | @torch.jit.script 549 | def compute_observations_sim(root_tensor, rigid_body_state, robinion_rigid_body_dict, dof_pos, dof_vel, 550 | robot_actor_ids, roller_ids, z_basis_vec, parallel_pitch_controlled_dof_index, previous_actions): 551 | # type: (Tensor, Tensor, Dict[str, int], Tensor, Tensor, Tensor, Tensor, Tensor, List[int], Tensor) -> (Tensor) 552 | 553 | robot_angvel = root_tensor[robot_actor_ids, 10:] 554 | 555 | robot_quat = root_tensor[robot_actor_ids, 3:7] 556 | robot_rpy = get_euler_xyz(robot_quat) 557 | 558 | robot_position = root_tensor[robot_actor_ids, :3] 559 | bongo_board_position = rigid_body_state[:, robinion_rigid_body_dict["board"], 0:3] 560 | roller_position = root_tensor[roller_ids, 0:3] 561 | 562 | torso_position = rigid_body_state[:, robinion_rigid_body_dict["torso_link"], 0:3] 563 | l_foot_position = rigid_body_state[:, robinion_rigid_body_dict["l_foot_link"], 0:3] 564 | r_foot_position = rigid_body_state[:, robinion_rigid_body_dict["r_foot_link"], 0:3] 565 | l_foot_relative_to_torso = l_foot_position - torso_position 566 | r_foot_relative_to_torso = r_foot_position - torso_position 567 | 568 | bongo_board_rpy = get_euler_xyz(rigid_body_state[:, robinion_rigid_body_dict["board"], 3:7]) 569 | bongo_board_angle = bongo_board_rpy[0] 570 | bongo_board_angle = torch.where(bongo_board_angle > np.pi, bongo_board_angle % np.pi - np.pi, bongo_board_angle) 571 | 572 | l_foot_position = rigid_body_state[:, robinion_rigid_body_dict["l_foot_link"], 0:3] 573 | Det_box_position = rigid_body_state[:, robinion_rigid_body_dict["DET_BOX_l_foot"], 0:3] 574 | l_foot_dist = torch.norm(l_foot_position - Det_box_position, p=2, dim=1) 575 | 576 | up_vec = get_basis_vector(robot_quat, z_basis_vec) 577 | up_projection = up_vec[:, 2] 578 | 579 | obs = torch.cat([ 580 | dof_pos[:, parallel_pitch_controlled_dof_index], # Only observe joints that we control. 581 | dof_vel[:, parallel_pitch_controlled_dof_index], # Only observe joints that we control. 582 | previous_actions, 583 | robot_angvel, 584 | robot_rpy[0].unsqueeze(-1), 585 | robot_rpy[1].unsqueeze(-1), 586 | robot_rpy[2].unsqueeze(-1), 587 | robot_position, 588 | bongo_board_position, 589 | roller_position, 590 | l_foot_relative_to_torso, 591 | r_foot_relative_to_torso, 592 | bongo_board_angle.unsqueeze(-1), 593 | l_foot_dist.unsqueeze(-1), 594 | up_projection.unsqueeze(-1) 595 | ], dim=1) 596 | 597 | return obs -------------------------------------------------------------------------------- /Docs/Demo.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Docs/Demo.gif -------------------------------------------------------------------------------- /Docs/Reinforcement Learning and Action Space Shaping for a Humanoid Agent in a Highly Dynamic Environment.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/NTNU-ERC/Robinion-Balance-Board-PPO/f6943b3dca10776c904ab9fa661cc8ddd9dedd30/Docs/Reinforcement Learning and Action Space Shaping for a Humanoid Agent in a Highly Dynamic Environment.pdf -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Humanoid robot playing balance board with PPO (NVIDIA Isaac Gym) 2 | This repository contains the code and configuration files for humanoid robot playing balance board in the NVIDIA Isaac Gym simulator. This work was done as part of the paper titled "Reinforcement Learning and Action Space Shaping for a Humanoid Agent in a Highly Dynamic Environment." 3 | 4 | Demo: 5 | 6 | ![Demo](Docs/Demo.gif) 7 | 8 | ## Training Details 9 | 10 | The agent aims to remain stable on the balance board while avoiding letting itself or the board touch the gorund. We designed different control schemes for our agent and compare their performance. Therefore, the observation space and the action space will be different according to the schemes we applied. 11 | 12 | The reward function is conditioned on board’s angle to the ground, the orthogonal projection of the agent’s orientation to the z-axis (upwards direction), the angular velocity of the board, and the distance between the agent’s left foot and an optimal position on top of the board. 13 | 14 | Observation Space: 15 | Feature | Dimensionality | 16 | --- | --- | 17 | Position of Controlled Joints | [6-23]* | 18 | Velocity of Controlled Joints | [6-23]* | 19 | Previous Actions | [6-23]* | 20 | Agent’s Position | 3 21 | Agent’s Angular Velocity | 3 22 | Agent’s Orientation | 3 23 | Left and Right foot position (Relative to CoM) | 6 24 | Distance of Left Foot to Ideal Position | 1 25 | Upwards Projection of Robot Torso | 1 26 | Board Position | 3 27 | Board Angle | 1 28 | Roller Position | 3 29 | 30 | Action Space was the number of the joints the agent can control. 31 | 32 | The agent was trained with PPO using the NVIDIA Isaac Gym simulator. Training for 300 million timesteps which takes under 40 minutes of real-time on a single RTX 2080 Super. 33 | 34 | For more information, you can check our paper under `Docs/`. 35 | 36 | ### Files 37 | 38 | `Code/tasks/fixed_robinion_balance_board_sim.py` contains the implementation of the environment. This file should be placed under the `isaacgymenvs/tasks/` folder in the Isaac Gym simulator. 39 | 40 | `Code/cfg/task/FixedRobinionBalanceBoardSim.yaml` contains the parameters for the environment. This file should be placed under `isaacgymenvs/cfg/task`. 41 | 42 | `Code/cfg/train/FixedRobinionBalanceBoardSimPPO.yaml` contains the parameters for the RL algorithm. This file should be placed under `isaacgymenvs/cfg/train`. 43 | 44 | `Assets/` contains the robot and balance board models and their URDF description. 45 | --------------------------------------------------------------------------------