Example Request - Cartpole/Ant using Warp instead of dFlex

[korzen]

Hi, first of all, thanks a lot for your great piece of software! We are really excited to apply it in our research in surgical simulation. However, we ran into problems while trying to switch dFlex to its successor - Warp.

Would it be possible to provide us with a minimal reference Cartpole and/or Ant SHAC example using Warp? That would be very helpful not only for our group but also for other users.

Thank you

[mmacklin]

Hi @korzen, thanks for the message, we are working on exactly what you asked for and should have something to share soon.

Adding @ViktorM who may be able to provide more information.

Best,
Miles

[ViktorM]

Hi @korzen,

I have these environments implemented with Warp internally. Can train them with PPO and am working to get them trained with SHAC. About the release timeline, I need to discuss it with @mmacklin

Best,
Viktor

[korzen]

Hi @mmacklin and @ViktorM and thanks a lot for your reply. Please let us know about the estimated "official" release date. We will keep digging on our side, but if you have got any ad-hoc code snippet, which you could share right away, we would be very grateful.

In parallel, we will proceed with porting my surgical simulations from Unity's Burst to Warp.

Thanks a lot again for you awesome work and we are really looking forward to your sample code

[mmacklin]

We'll aim to get the main RL examples shipped and training with PPO within the next month, from there we'll look at doing the full SHAC training using gradients.

Looking forward to what you do with Warp, please let us know if you have any other issues in the meantime.

Cheers,
Miles

[korzen]

Hi @eanswer was this really completed? Could you please point me to implementation of SHAC usign Warp as simulation backend? Thanks!

[dyumanaditya]

Hello,

Is there any news on these warp compatible environments?

I'm trying to convert the cartpole env to work with warp instead of dflex and I found that the Model.state is now represented by warp arrays instead of as torch tensors like in dflex -- is there a reason for this?

Would it be better in terms of efficiency to operate on these states as warp arrays during environment simulation (this would involve a lot of code changes), or would it be better to convert them into torch tensors to retain the additional functionality (easier to code but not sure about efficiency)?

Any guidance on this would be very helpful,
thank you!

[etaoxing]

For any interested, released some Warp envs for first-order RL.