metadata

pipeline_tag: robotics
library_name: transformers
license: apache-2.0
tags:
  - robotics
  - agent
  - computer-vision
  - llm

STEVE-R1: Towards Long Reasoning Computer-use Agents

We evaluate the performance of the STEVE-R1 agent on both in-domain WindowsAgentArena (Windows 11 OS) and out-of-domain OSWorld (Ubuntu OS) benchmarks. The evaluation involves 16 attempts per task, with task completion rates recorded as the primary metric. In the in-domain Windows 11 setting, the STEVE-R1 agent demonstrated a 14% higher task completion rate compared to the previous open-source state-of-the-art model, UI-TARS-7B-DPO. Furthermore, in the out-of-domain Ubuntu OS environment, where STEVE-R1 was not explicitly trained, it still achieved a 7% higher task completion rate than UI-TARS-7B-DPO.

Release

Currently only the SFT STEVE-R1 model with step-verified training data is released. RL tunning is in progress.
🔥 An improved version STEVE-R1 is released with long reasoning ability and long image context. We extend the model context length to 128K with at most 32 screenshot inputs for a single task. The model response length is greatly improved with deepseek-R1 distillation, see the examples. We release the training data, models, and evaluation trajectories.
We release the paper of STEVE: Step Verification Pipeline for Computer-use Agent Training. We propose a single-frame computer-use 7B agent trained with SFT & step-verified KTO.

Trajectory Data

... (rest of the original README content)

Citation

To be added.