|
--- |
|
datasets: |
|
- commaai/commabody |
|
pipeline_tag: robotics |
|
--- |
|
|
|
This model has been trained on a larger version (194 minutes total) of the commabody dataset. |
|
It includes a [vqgan](https://github.com/CompVis/taming-transformers) encoder/decoder fine tuned from imagenet. It compresses images of size 160x256 to 10x16 tokens. |
|
|
|
It also includes a GPT2 model trained to predict the next frame, wheel speeds and actions. It can be used either as a simulator or as a policy. More details in [our blog post](https://blog.comma.ai/a-drive-in-the-office/). |
|
|
|
<video title="imagined rollouts from 3s of context" controls> |
|
<source src="https://blog.comma.ai/img/body_patrol/rollouts.webm" type="video/webm"> |
|
</video> |
|
|
|
You can run it on a comma body using [our example script in body-jim](https://github.com/commaai/body-jim/blob/master/examples/roam.py). |
|
|
