ReadPaper Blog
SIEVE: Structure-Aware Data Selection for Imitation Learning with VLA Models
SIEVE addresses the problem that large robot demonstration datasets can make Vision-Language-Action models less efficient to train because they contain redundancy, noise, suboptimal behavior, and uneven task coverage. The paper proposes a structure-aware data selection method that discovers reusable visuo-motor primitives, allocates selection budgets by primitive-transition exposure, and chooses medoid demonstrations for stable behavior cloning. Its experiments report that this structural view can outperform competitive selection baselines and even surpass full-data training while using only half the demonstrations and half the training steps.
Source: SIEVE: Structure-Aware Data Selection for Imitation Learning with VLA Models
Why more data can still fail
The paper starts from a practical tension in Vision-Language-Action imitation learning: scaling robot demonstrations does not guarantee better control policies. Large demonstration pools often repeat near-identical behaviors, include noisy or suboptimal human actions, and cover tasks unevenly, so behavior cloning can waste computation while absorbing inconsistent supervision. SIEVE frames data selection as the task of retaining a compact subset that is more useful for learning than the unfiltered dataset. The motivation is especially important for VLA models because they are trained on costly long-horizon manipulation demonstrations that combine vision, language instructions, and robot actions. The paper argues that useful data should expose reusable behavioral regularities rather than merely increase the number of trajectories.
The gap in old selection methods
The paper identifies a granularity mismatch in prior imitation-learning data selection methods. Trajectory-level approaches use global signals such as representation similarity, reliability, downstream feedback, or validation-loss influence, but a single score can hide which sub-behaviors inside a long-horizon trajectory are actually useful. State-action-level approaches use local signals such as mutual information, task progress, or joint state-action similarity, but these criteria can be too myopic to capture coherent task semantics and long-range execution structure. SIEVE positions its contribution between these extremes by evaluating demonstrations through reusable mid-level structures. This framing makes primitive composition, rather than whole-trajectory quality or isolated action predictability, the central signal for VLA data selection.
SIEVE’s core idea
SIEVE’s core claim is that robot demonstrations can be understood as compositions of reusable visuo-motor primitives connected by transition interfaces. The paper connects this idea to the Minimum Description Length principle, where repeated patterns are valuable because a bounded learner can internalize them as shared subprograms rather than relearning them from redundant examples. In imitation learning, this means a demonstration is useful when it exposes primitives, composition patterns, and transitions that recur across tasks and support long-horizon behavior. The method also emphasizes that behavior cloning needs stable and predictable action targets, so not every example of a useful pattern is equally desirable. SIEVE therefore separates the question of which structures to preserve from the question of which concrete trajectories best represent those structures.
How SIEVE selects data
The method proceeds in three stages: Primitive Discovery, Structural Exposure Allocation, and Learning-Friendly Trajectory Selection. Primitive Discovery segments trajectories at physically grounded interaction boundaries such as end-effector grasp or release state flips, represents the resulting segments with visual-motor features, and clusters them into a vocabulary of reusable primitives. Each trajectory is then represented as a sequence of primitives, defining a composition pattern along with adjacent primitive transitions. Structural Exposure Allocation assigns the selection budget across composition-pattern buckets to maximize reuse-aware primitive and transition exposure under diminishing returns, so common reusable structures are covered without overspending on duplicates. Finally, Learning-Friendly Trajectory Selection chooses medoid trajectories within each bucket using trajectory representations, favoring central and stable demonstrations that should provide cleaner supervision for behavior cloning.
What the paper claims
The paper reports experiments across multiple datasets, benchmarks, and VLA models showing that SIEVE consistently outperforms competitive data selection baselines. Its most striking reported result is that SIEVE can surpass full-data training while using only 50% of the demonstrations and 50% of the training steps. This evidence supports the paper’s thesis that reusable primitive-transition structure is a stronger selection signal than raw dataset size for efficient VLA imitation learning. The implication is not simply that robot-learning datasets should be smaller, but that their selected subsets should expose compositional behavioral regularities while avoiding redundant or unstable demonstrations. By treating demonstrations as structured programs of primitives and transitions, SIEVE offers a principled route toward more compute-efficient and learning-friendly VLA policy training.
