CounterAlign: Counterfactual Supervision for
Vision-Language-Action Models


Haru Kondoh1,2   Kei Ota2   Asako Kanezaki1,3,4   Yueh-Hua Wu2

1Institute of Science Tokyo   2AI Robot Association (AIRoA)   3RIKEN AIP   4Tohoku University

[Paper]   [Code (Coming Soon)]  

Summary Video

Audio note: Turn on sound to hear the narration.

Motivation

Robots need contrast—not just demonstrations.

Behavior cloning Limited generalization

Costly robot data · Good actions only

Online RL Unsafe and sample-inefficient

Risky exploration · Too many interactions

Offline RL Promising, but hard to scale

A reward per task · Fixed-dataset ceiling

CounterAlign Make offline data teach more.
Expert demos Counterfactuals Mixture dataset

Learn the reward from data and scale supervision without new robot rollouts.

Abstract

Behavior cloning (BC) shows a Vision-Language-Action (VLA) model what the expert did, but not which plausible alternatives conflict with the instruction. CounterAlign turns successful demonstrations into corrective supervision through counterfactual relabeling, then combines it with adversarial training against policy-generated actions. The resulting instruction-grounded reward drives offline RL—without additional rollouts or annotations—and improves robustness in simulation and real-robot experiments.

DataExpert demos only
SignalRelabeling & adversarial learning
TrainingReward-guided offline RL
Overview of CounterAlign
Figure 1 | Overview of CounterAlign. Through adversarial and relabeling-based discriminator training, CounterAlign converts expert demonstrations into an instruction-grounded reward. Relabeling creates counterfactual negative or unlabeled tuples that teach the reward model which instruction–action pairings are semantically inconsistent. The learned reward is then used to train a VLA policy with offline RL, improving robustness to object position and task perturbations without additional data collection or annotation.

Method

CounterAlign combines an action-fidelity score learned by contrasting expert and policy actions with an instruction–action alignment score learned from relabeled expert data. Their weighted sum forms a dense reward for offline RL, while near-positive instruction relabels broaden critic training.

1Relabel

Build similarity-filtered negative and unlabeled tuples.

2Score

Learn action fidelity and semantic alignment.

3Optimize

Train the critic and VLA policy with offline RL.

Discriminators for adversarial training and relabeling
Figure 2 | The two discriminators share the same architecture but are trained on different types of samples. (a) The adversarial discriminator is trained on original expert samples and samples whose actions are replaced with policy-generated actions. (b) The relabeling discriminator is trained on samples whose instructions or actions are replaced with other instructions or actions from the dataset.

Simulation Results

On LIBERO-PRO, CounterAlign delivers its clearest gains under position and task perturbations, where success depends on instruction-grounded action selection rather than trajectory replay.

Method LIBERO-Spatial LIBERO-Object LIBERO-Goal LIBERO-10
Obj Pos Sem Task Obj Pos Sem Task Obj Pos Sem Task Obj Pos Sem Task
OpenVLA-OFT 0.30 0.14 0.65 0.0 0.67 0.10 0.89 0.0 0.85 0.0 0.44 0.03 0.06 0.0 0.39 0.0
VLA-Adapter 0.98 0.0 0.98 0.49 0.89 0.0 0.99 0.08 0.61 0.0 0.75 0.12 0.47 0.0 0.91 0.10
xVLA 0.90 0.0 0.69 0.39 0.92 0.05 0.98 0.0 0.72 0.10 0.94 0.08 0.61 0.10 0.71 0.20
π0.5 0.99 0.53 0.97 0.55 0.89 0.19 0.95 0.10 0.90 0.29 0.95 0.17 0.66 0.06 0.91 0.17
CounterAlign 0.98 0.60 0.97 0.63 0.80 0.51 0.99 0.26 0.82 0.41 0.94 0.46 0.64 0.11 0.97 0.29
Table 1 | Success rate comparisons on LIBERO-PRO. CounterAlign yields its largest improvements in the Pos and Task settings, where the policy cannot succeed by simply replaying actions observed in the training data.

Real-Robot Experiments

Across all four TX-G2 tasks with perturbed object poses, CounterAlign consistently achieves the highest mean success rate.

TX-G2 experimental setup
Figure 3 | TX-G2 experimental setup.
Method Task 1 Task 2 Task 3 Task 4
VLA-Adapter 0.00 0.00 0.00 0.00
xVLA 0.00 0.08 0.05 0.05
π0.5 0.05 0.48 0.80 0.60
CounterAlign 0.20 0.53 0.93 0.85
Table 3 | Success rate comparisons on the TX-G2. Mean success rates over the primitive actions that make up each task. CounterAlign consistently outperforms the baselines under object-pose perturbations in the real-robot setting.

Rollout Videos

In-Distribution Layouts and Tasks

These videos compare π0.5 and CounterAlign on in-distribution tabletop object layouts and tasks, where both methods succeed.

π0.5 CounterAlign
Instruction: Pick up the dishes from the desk and place them in the wooden dish rack.
π0.5 CounterAlign
Instruction: Pick up the bowls from the desk and stack them.
π0.5 CounterAlign
Instruction: Pick up the clothes from the desk and place them into the basket.

Perturbed Tabletop Object Layouts and Tasks

These rollouts use tabletop object layouts and tasks that differ from the training distribution, where π0.5 fails and CounterAlign succeeds.

π0.5 CounterAlign
Instruction: Pick up the light blue spoon from the desk and place it on the green dish.
π0.5 CounterAlign
Instruction: Pick up the yellow bowl from the desk and place it into the basket.
π0.5 CounterAlign
Instruction: Pick up the clothes from the desk and place them into the basket.

BibTeX

@misc{kondoh2026counteralign,
  author       = {Haru Kondoh and Kei Ota and Asako Kanezaki and Yueh-Hua Wu},
  title        = {{CounterAlign}: Counterfactual Supervision for Vision-Language-Action Models},
  year         = {2026},
}

Acknowledgments

This work is based on results obtained from a project, JPNP25015, commissioned by the New Energy and Industrial Technology Development Organization (NEDO).