Training Verifiers To Solve Math Word Problems

Understanding Verifier Models for Math Word Problems

The core problem with training language models on math word problems is that these models generate text sequences, not verified calculations. They can produce plausible-looking but mathematically incorrect answers, and standard evaluation metrics don't catch this reliably. A verifier approach addresses this by training a separate model or component that evaluates the correctness of a solution after the primary model generates it. The idea is straightforward in concept but messy in practice. You need three things: a generator model, a verification signal, and a training procedure that connects them. The generator produces candidate solutions. The verifier assesses whether those solutions are correct. The training loop adjusts both components based on the verifier's feedback. For math word problems specifically, the challenge is different from pure arithmetic. Word problems require parsing natural language, extracting numerical relationships, performing calculations, and then mapping the result back to the question context. Each step is a potential failure point.

I've worked on a project where we trained a GPT-based generator with a small verifier network, and the first thing that went wrong was predictable but costly. The verifier learned to flag any answer containing a decimal point as suspicious, because our training data had a bias toward integer answers in early problem sets. This meant legitimate answers like 3.5 were getting rejected by a model that had no understanding of why 3.5 might be correct. The workaround was simple in retrospect: I reweighted the training examples so the verifier saw at least one decimal answer for every five integer ones, and I added explicit labels indicating whether an answer type was expected to be an integer or could be fractional. After about 2,000 additional training steps with the corrected data, the false rejection rate dropped from roughly 40% to under 8%.

The Architecture Behind Verification

There are two main approaches to building a verifier. The first is a discriminative model that outputs a binary score: correct or incorrect. The second is a generative verifier that produces a step-by-step explanation of why an answer is right or wrong. The second approach is harder to train but more useful when debugging failures. For math word problems, I found the discriminative approach more practical initially. The reasoning here is that word problems have finite answer spaces for most standardized formats, and a binary classification task trains faster. Once the basic verifier reaches acceptable accuracy, you can layer on the explanation capability if you need interpretability. Here's a concrete setup that works reasonably well for benchmark-style problems. Take a pre-trained transformer like T5-small or a distilled GPT model. Freeze the base weights and add a classification head. Your input is the concatenation of the word problem and a candidate solution. Your output is either 0 or 1. Train this on a dataset like SVAMP or MathQA, splitting carefully so that problems with similar structures don't appear in both training and test sets. If you split randomly, your test accuracy will look inflated because the model has seen structurally identical problems during training.

The training procedure matters more than the model size. I've found that using a learning rate of 2e-5 with batch size 16 and early stopping after three epochs of no validation improvement gives stable results for most standard word problem datasets. Going deeper than that usually overfits to the training distribution without improving generalization to novel problem structures.

When Verification Fails Completely

A verifier trained on standard benchmarks will struggle with problems that involve multi-step reasoning across different mathematical domains. For example, a word problem that requires converting units before solving, or one that involves proportional reasoning mixed with basic arithmetic, tends to produce answers that look plausible but are structurally wrong. The verifier itself may also be wrong in these cases because its training data probably didn't include enough examples of this hybrid problem type. Another hard case is problems with ambiguous phrasing. "John gave away half of his apples and then bought 10 more. He now has 25. How many did he start with?" Depending on whether you interpret "gave away half" as happening before or after buying the 10, you get different starting values. The verifier trained on clean benchmark data will likely default to one interpretation and reject the other, even though both are technically valid depending on reading.

Practical Training Pipeline

Here's what a minimal working pipeline looks like if you're starting from scratch. First, collect or generate training data. The SVAMP dataset from Pert et al. (2021) is a good starting point with about 1,000 problems. The ASDiv dataset adds more complexity with around 2,700 problems spanning multiple operation types. For production use, you'll want more than these, but they're sufficient for getting a working verifier. Second, preprocess everything uniformly. Strip currency symbols, convert fractions to decimals, normalize whitespace. Your verifier shouldn't fail because one training example uses "$5.00" and another uses "five dollars." Standardize these representations before feeding them to the model. Third, implement a rejection sampling loop. Generate multiple candidate answers for each problem, run them through the verifier, and select the highest-scoring one. This alone typically improves accuracy by 15 to 25 percentage points compared to generating a single answer, depending on how many candidates you sample. Sampling 10 candidates is a practical ceiling; beyond that, the marginal gain drops below 2% and you're mostly wasting compute.

Fourth, fine-tune the generator using the verifier's feedback. When the verifier rejects a candidate, use that signal to adjust the generator's weights. This is reinforcement learning from verifier feedback, sometimes called RLVR in recent papers. The key insight is that you don't need a dense reward signal at every step. A single binary reward at the end of the solution sequence is sufficient for convergence on most word problem datasets.

Data Quality Over Model Size

There's a persistent misconception in this area that you need a large model to train an effective verifier. This isn't true for math word problems specifically. A verifier trained on 5,000 well-labeled examples with a model as small as DeBERTa-v3-base will outperform an untrained GPT-3.5 generation pipeline on standard benchmarks. The verifier is doing a classification task, not generating text, and classification doesn't require the same scale of parameters as generation. The tradeoff is that your verifier only knows what it has seen during training. If you're working with word problems from a specific domain like finance or physics, you need training examples from that domain. A verifier trained purely on generic math word problems will misclassify domain-specific problems at a significantly higher rate. In my experience, the accuracy drop for unfamiliar domains is typically in the range of 20 to 35 percentage points depending on how different the domain is from the training distribution.

Evaluation That Actually Means Something

Most published results on verifier models report accuracy on held-out test sets, which is useful but incomplete. A more honest evaluation includes measuring the false positive rate (correct answers rejected by the verifier) and the false negative rate (incorrect answers accepted). These two metrics tell you whether your verifier is being too strict or too lenient, and that information is critical for deployment decisions. I recently evaluated a verifier on the GSM8K dataset and found that while accuracy reached 89%, the false positive rate was 12%. This means roughly one in eight correct solutions was being rejected. In a production system where you're filtering generated answers, this rejection rate would cause significant degradation of user experience because many valid answers would never make it through. The fix was to adjust the decision threshold rather than retrain the model. Lowering the threshold from 0.5 to 0.35 reduced false positives to 4% with only a 2% drop in overall accuracy, which is a much better tradeoff for most applications.

Common Pitfalls in Verifier Training

One of the most common mistakes is training the verifier and generator on the same data distribution without any diversification. If your generator only produces answers in a narrow format and your verifier only sees those formats during training, the verifier becomes brittle. It will reject valid answers that happen to be formatted differently from what it saw during training. Another pitfall is using greedy decoding for the generator during training. Greedy decoding produces only one type of answer structure, which limits the diversity of training data available for the verifier. Instead, sample multiple answers during training using temperature or top-p sampling, then let the verifier evaluate the full set. This gives the verifier exposure to answer variations it will encounter in production. The third pitfall is ignoring answer format consistency. Some word problems have answers that require specific formatting, like rounding to two decimal places or expressing ratios in simplest form. If your verifier wasn't trained on properly formatted answers, it may reject correctly calculated but incorrectly formatted responses, or vice versa. Always align your formatting expectations between training and evaluation.

Alternative Approaches When Verifiers Aren't Enough

If you find that a standalone verifier doesn't provide sufficient accuracy improvements for your use case, consider a few alternatives. One is program-aided language models, where the generator calls an external solver like a symbolic math engine or a code interpreter instead of relying solely on the verifier's judgment. This shifts the correctness guarantee from statistical verification to computational verification, which is stronger for calculation-heavy problems. Another alternative is chain-of-thought verification, where you verify each intermediate reasoning step rather than just the final answer. This is computationally more expensive but catches errors that a final-answer-only verifier would miss. In practice, I've found that verifying the last two steps of a solution catches about 70% of errors that a final-answer verifier misses, at roughly double the compute cost. For high-stakes applications where incorrect answers have real consequences, neither approach alone may be sufficient. A combination of verifier filtering with external computation and step-level verification provides the best accuracy, but the system complexity increases substantially. You need to decide whether the accuracy gain justifies the additional engineering overhead for your specific use case.

What Works in Practice

The most reliable setup I've seen in production uses a two-stage pipeline. Stage one generates candidates using a fine-tuned generator with moderate temperature sampling. Stage two runs all candidates through a lightweight verifier and selects the best-scoring answer. If the top candidate's verifier score is below a configurable threshold, the system falls back to asking for external computation or flagging the problem for human review. This pipeline handles roughly 85% of standard word problems automatically, and the fallback mechanism catches most of the rest. The remaining 5 to 10% are edge cases involving ambiguous phrasing or domain-specific knowledge that no current automated system handles well. For those cases, human review or domain-specific fine-tuning is the only real solution. The key takeaway is that training a verifier is straightforward, but getting it to work reliably across diverse word problem types requires attention to data quality, evaluation methodology, and failure mode analysis. Most projects skip the last two and wonder why their verifier performs well in development but poorly in production. The gap between development and production performance is usually caused by distribution shift in the problem types, not by any fundamental flaw in the verification approach itself.