Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints

Figure 1 — Same image: the LVLM answers "Yes" to "Has the batter hit the ball?" but "towards" to "Is the ball flying towards or away?"—contradictory.

1. Background and Motivation

LVLMs are strong at perception but reliable visual reasoning remains unsolved. Existing benchmarks focus on symbolic math/science or simple vision tasks: math-science suites (MMBench, MathVista) let models pass via symbolic reasoning or domain text rather than faithful visual interpretation; vision-centric suites (BLINK) are too shallow ("solvable in a blink") to test complex reasoning.

Critically, models often give inconsistent answers to semantically equivalent queries (Fig. 1), indicating reliance on spurious correlations rather than genuine understanding—yet current benchmarks barely examine this robustness. This work fills both gaps: complex visual reasoning evaluation and cross-query consistency evaluation.

2. Core Method

2.1 ConVBench

Each image is paired with two logically equivalent questions whose answers should align, spanning six categories: action & state, complex counting, spatial relations, causal & intent, commonsense, temporal perception. Built via a two-stage pipeline: an LVLM generates equivalent QA pairs, then human annotators validate—accept if correct, minimally edit if needed, discard otherwise—ensuring reliability without exhaustive manual authoring.

Figure 3 — ConVBench six task categories; each image paired with two logically equivalent questions.

2.2 Two Metrics

• Logical Consistency (Cons.): indicator $C_I=1$ if both questions answered correctly or both incorrectly, else 0.
• Robust Accuracy (Acc.): $A_I=1$ if both answered correctly, else 0—strict correctness.

High consistency but near-zero accuracy means "consistently wrong"; only both high means robust reasoning.

2.3 ConVLM: Consistency-Constrained RL

Objective: maximize accuracy while enforcing cross-query consistency. Let $\mathcal{I}$ be images, $\pi_\theta(o|I,q)$ the policy, $\mathcal{G}(I)$ equivalent pairs, $r_{acc}(I,q,o)\in[0,1]$, $r_{con}(o_1,o_2)\in\{0,1\}$:

$$\max_\theta\ \mathbb{E}_{I,q,o}[r_{acc}(I,q,o)] \quad \text{s.t.}\ \mathbb{E}_{I,(q_1,q_2),(o_1,o_2)}[r_{con}(o_1,o_2)] \ge \tau$$

Practically optimize the Lagrangian relaxation:

$$\max_\theta\ \mathbb{E}_{I,q,o}[r_{acc}(I,q,o)] + \gamma\,\mathbb{E}_{I,(q_1,q_2),(o_1,o_2)}[r_{con}(o_1,o_2)]$$

Figure 2 — ConVLM pipeline: (i) logically equivalent question generation (proposer GPT-4.1 from image+caption+objects); (ii) consistency-based RL (GRPO + dual reward).

2.4 Logically Equivalent Question Generation

Manual labeling is costly. A proposer model (model-agnostic) auto-generates; this work uses GPT-4.1 conditioned on image, caption, object info (boxes + categories):

$$\{(q_1,a_1),(q_2,a_2)\}=\mathcal{P}(P(I,C,O))$$

$q_1,q_2$ are logically equivalent—$a_1\leftarrow f(I,q_2,a_2,q_1)$ or $a_2\leftarrow f(I,q_1,a_1,q_2)$.

2.5 Dual-Reward GRPO

GRPO objective (with clipping and KL penalty):

$$\mathcal{J}_{\mathrm{GRPO}}(\theta)=\mathbb{E}\!\left[\frac{1}{n}\sum_{i=1}^{n}\min\!\big(r_i(\theta)\hat{A}_i,\ \mathrm{clip}(r_i(\theta),1-\epsilon,1+\epsilon)\hat{A}_i\big)-\beta D_{\mathrm{KL}}(\pi_\theta\|\pi_{\mathrm{ref}})\right]$$

Standard accuracy reward is binary (1 if correct else 0), brittle under noisy pseudo-answers. So add a consistency reward: noise-tolerant, encourages agreement across equivalent queries, while accuracy anchors correctness. Two variants, with $G$ samples each for $q_1,q_2$:

• Sample-wise: $r_{con}=c(o^1_i,o^2_i)$, $c$ returns 1 if consistent else 0.
• Group-wise: $r_{con}=\sum_{j=1}^{G}c(o^1_i,o^2_j)/G$.

Total reward $r=r_{acc}+\gamma r_{con}$. Reward hierarchy: both correct+consistent is highest; both wrong+inconsistent lowest; two intermediate cases (correct-but-inconsistent, wrong-but-consistent) governed by $\gamma$.

flowchart TB
    IMG["Image I"] --> PROP["Proposer (GPT-4.1)
generate logically
equivalent QA pairs"]
    CAP["Caption C"] --> PROP
    OBJ["Objects O"] --> PROP
    PROP --> QA1["q1, a1"]
    PROP --> QA2["q2, a2 (equivalent)"]
    QA1 --> GRPO["GRPO RL training"]
    QA2 --> GRPO
    GRPO --> RACC["accuracy reward r_acc"]
    GRPO --> RCON["consistency reward r_con"]
    RACC --> RALL["r = r_acc + gamma*r_con"]
    RCON --> RALL
    RALL --> MODEL["ConVLM: robust+consistent"]
    style RCON fill:#dbeafe,stroke:#2563eb
    style RALL fill:#dcfce7,stroke:#16a34a

3. Experiments

8×A100(40G), OpenRLHF, batch 256, AdamW lr 1e-7, $G=8$, $\beta=0$, $\gamma=0.5$. Trained on 5,000 MSCOCO images.

3.1 ConVBench Main Results

7B reaches 73.36% avg consistency, 66.83% accuracy; 3B reaches 69.34%/64.44%. Beats Claude-3.5-Sonnet-V2 by +11.12% consistency, +20.94% accuracy. Both variants surpass comparable/larger open-source baselines.

3.2 Ablation

Accuracy reward contributes +2.22/+2.70 on ConVBench, +4.70 on V*Bench; consistency reward +1.93/+4.18 and +3.96—consistency training improves both consistency and accuracy. Generated data contributes the most (+19.45/+22.75).

3.3 Consistency Reward Form

Both substantially beat baseline; SampleCon slightly stronger, hence adopted.

3.4 Cross-Benchmark Generalization

ConVLM-7B beats Gemini-2.5-Pro on V*Bench (84.90 vs 79.20), showing consistency training improves generalization rather than overfitting.

4. Main Contributions

• ConVBench: first vision-centric benchmark using paired logically equivalent questions to jointly evaluate complexity and consistency across six categories.
• Dual metrics: logical consistency and robust accuracy, separating "consistently wrong" from robust reasoning.
• ConVLM: GRPO + dual reward (accuracy + consistency), auto-generated pseudo-labels, noise-tolerant, effective without strict answer supervision.
• SOTA: 7B beats Claude-3.5-Sonnet-V2 by +11.12% consistency, +20.94% accuracy, and generalizes to V*Bench beating Gemini-2.5-Pro.

5.

优势函数归一化

$$ \hat{A}_{i}=\frac{r_{i}-\text{mean}(r)}{\text{std}(r)} $$

Limitations and Future Work

Our analysis: the binary consistency judge $c(\cdot)$ is coarse for open-text answers—synonymous paraphrases may be judged inconsistent; a finer semantic consistency judge may help. The proposer relies on GPT-4.1, bound by its quality/cost; equivalent quality of cheap open proposers is under-verified. ConVBench's six categories are natural-image scenes; medical/remote-sensing professional reasoning is uncovered. ConVLM-3B's InfoVQA ANLS (69.75) is below Qwen2.5-VL-7B (80.70), showing consistency training isn't universally superior for document/info-extraction tasks—the generalization boundary needs finer characterization. Fixed $\gamma=0.5$; adaptive or curriculum tuning unexplored.

6. Conclusion

The core idea: use consistency constraints across logically equivalent questions to force genuine visual reasoning. ConVBench pairs two equivalent questions per image and uses dual metrics (consistency + robust accuracy) to test both "answer correctly" and "answer consistently." ConVLM trains with GRPO plus a dual reward—accuracy anchors correctness, consistency tolerates noise and drives cross-query agreement—effective even with noisy pseudo-answers. The 7B model beats Claude-3.5-Sonnet-V2 by +11.12% consistency and +20.94% accuracy, and beats Gemini-2.5-Pro on V*Bench, showing consistency training improves general reasoning rather than overfitting.