EmbodiedAgent

Model release for paper: EmbodiedAgent: A Scalable Hierarchical Approach to Overcome Practical Challenge in Multi-Robot Control

Results

Model	Infer type	ASR_{top-k}	Expert grading	UE	PoE	PlE	SE	EE	RPAS
GPT-4o	FPS	9.03	31.25	0.00	0.00	6.25	93.75	0.00	20.14
OpenAI-o1	FPS	6.74	34.38	0.00	0.00	3.12	96.88	0.00	20.56
Claude-3.5-Sonnet	FPS	37.5	32.19	0.00	0.00	0.00	0.00	62.5	34.85
Deepseek-R1	FPS	61.87	64.84	0.00	18.75	15.62	0.00	18.75	63.36
Deepseek-R1-Distill-Llama3.3-70B	NAP	33.70	60.00	3.12	59.38	3.12	0.00	21.88	46.85
LLaMA-3.1-8B	NAP	16.56	43.44	3.12	71.88	50.00	18.75	3.12	30.00
LLaMA-3.1-70B	NAP	27.50	40.94	0.00	9.38	87.5	62.5	21.88	34.22
LLaMA-3.1-405B	NAP	34.24	61.09	28.12	46.88	6.25	0.00	0.00	47.67
LLaMA-3.3-70B	NAP	46.02	60.31	0.00	48.48	39.39	15.15	6.06	53.26
Gemma-2-9B	NAP	17.36	29.06	31.25	34.38	25.0	0.00	6.25	23.21
Qwen2.5-7B	NAP	9.17	10.94	3.12	15.62	90.62	0.00	6.25	10.06
Qwen2.5-72B	NAP	40.42	50.62	0.00	56.25	18.75	0.00	6.25	45.52
MAP-Neo-7B-Multiplan	FPS	46.39	44.22	21.88	28.12	25.0	6.25	3.12	45.31
EmbodiedAgent (Ours)	NAP	74.01	69.69	9.38	15.62	3.12	9.38	0.00	71.85

Links

Code: EmbodiedAgent

Dataset: MultiPlan+

Model weight: EmbodiedAgent

License

EmbodiedAgent is released under the llama3.1 License. See the LICENSE file for more details.