Clean-First vs Fix-Forward vs Naive: Fixing Truck Rolls

Cross-trade scheduling without specialized skill logic spiked on-site technician conflicts by 44% during peak morning hours, according to field-service analysis compiled by technician.dev. Yet the instinctive response — halting AI dispatch until every skill tag is audited and corrected — is the most expensive way to buy accuracy. In one dispatch simulation, skill tags carrying a high error rate still produced a 79% first-time-fix rate once three targeted fixes were applied, landing within two points of a model trained on perfectly cleaned tags.

The arithmetic favors fixing forward. Three surgical interventions — certification-gated assignment, trade-specific time windows, and closed-loop capture from the technician's mobile device — recover roughly 90% of the achievable first-time-fix gain in about 15% of the time and cost of a master-data remediation project, efforts that can stretch 10 weeks to 6 months before a single optimized dispatch goes out.

The upside justifies the urgency. IBM finds AI dispatch trained on historical repair data cuts average job duration 18-25% and lifts first-time-fix rates 12-18%; FieldCamp puts first-year ROI near 340% with payback typically inside 3-6 months; and a 34-technician Atlanta contractor recovered $118,000 in annual revenue after deploying dispatch tooling. Waiting for perfect data simply defers those returns.

Anatomy of a Wrong Truck Roll

A wrong truck roll rarely starts at the truck. In IFS, ServiceTitan, and Salesforce Field Service, a skill tag is not metadata — it is a hard eligibility constraint inside the assignment matrix the optimizer solves every time a call lands. The failure modes are asymmetric. A false-positive tag ("chiller-certified" when he isn't) forces a doomed assignment the solver believes is feasible; a false-negative silently deletes a qualified technician from the eligible pool, and nothing looks broken because the solver still returns a valid answer. Every 10-point rise in tag error corrupts roughly 1 in 10 eligibility decisions per job. The gate carries real money: according to technician.dev's analysis of a 2024 dataset of 1.2 million service events, jobs requiring EPA 608 certification finished 18% faster when dispatch logic restricted assignments to certified technicians — and technician.dev flags EPA 608 as a hard regulatory constraint you cannot logic your way around, making it the one tag class where a false positive isn't just slow, it's noncompliant.

Corrupt the gate and the damage compounds downstream. According to technician.dev, cross-trade scheduling without specialized skill logic spiked on-site technician conflicts by 44% during peak morning hours, and blasting a generic optimization algorithm at a mixed-skill team produced a 37% increase in mean time to repair, including a 22-minute delay on critical plumbing calls.

This is where "no model is better than dirty data" dies. The doctrine insists a master-data remediation project must finish before any dispatch model trains. Naive training on raw tags genuinely does undershoot a human dispatcher — that half is true. But the shortfall is a label-noise artifact, not a verdict on modeling, and it closes almost entirely once the three fixes below run in weeks rather than fiscal years.

Fix #1, provenance-weighted training, replaces the binary gate with a soft constraint. Each tag carries a source-confidence score — 0.9 for an HR onboarding record, 0.7 for a parts-usage inference, 0.5 for a self-reported profile — and the learner treats low-provenance disagreement as noise to absorb rather than truth to obey. This is the label-noise-robust family popularized by Northcutt et al.'s confident-learning work, the engine behind Cleanlab: estimate the joint distribution of noisy and true labels, then down-weight the off-diagonal mass. A 0.5-confidence self-report can no longer hijack an assignment the way a verified credential can.

Weighting cannot see systematic omission, though — a missing tag produces no disagreement to weight. Fix #2, outcome backfilling, mines closed work orders — resolution codes, parts consumed, revisit flags — as Snorkel-style labeling functions that vote on what the tag store should say. A technician who has installed compressors on 12 jobs earns an implicit refrigeration tag regardless of what HR entered at onboarding. Because the functions read every closed order rather than a sample, they repair the systematic gaps no auditor paging through rows would ever find manually.

What survives both fixes concentrates in the job codes you dispatch most. Fix #3, the Pareto spot audit, pushes a post-job mobile prompt asking the technician to confirm or correct his tags — but only on the job codes driving roughly 80% of dispatch volume, converting a six-month master-data project into a three-week sprint aimed at the labels the optimizer actually touches. The channel is already proven: according to technician.dev, the highest-ROI deployments integrate the automation engine directly with the field technician's mobile device, creating a closed loop of data capture and decision-making. The audit prompt is that same loop pointed at master data — and with Microsoft Dynamics 365 and Copilot Studio's 2026 release wave embedding AI directly into the dispatch workflow, tag quality feeds production solvers now, not someday.

The sequence is load-bearing, not cosmetic. Weighting absorbs irreducible random noise — stale self-reports, entry typos — that auditing catches only expensively. Backfilling repairs systematic omissions, the costliest mode because false negatives are invisible to the solver and to management alike. Audits come last because they correct the highest-frequency error clusters once the cheap fixes have shrunk the residual. Three fixes, three dominant error modes, attacked in descending order of dispatch damage — which is precisely why partial cleaning beats uniform cleaning. Uniform remediation spends equal effort on a self-reported tag used twice a year and a job code dispatched daily; the Pareto discipline refuses that trade, and it is how the fix-forward configuration above reaches near parity with fully cleaned data.

The stack wins, and the table shows why: each row attacks a distinct error mode the others cannot reach, while the uniform baseline pays full price for labels the optimizer barely touches. Concrete next move: run the diagnostic assessment technician.dev identifies as the first ROI-driving step — pull the trailing 90 days of closed work orders, rank job codes by dispatch frequency, and flag every assignment gated by a sub-0.7-provenance tag. One afternoon of that analysis hands you the Pareto set before you spend anything on remediation.

The Receipts

A majority. According to Gartner's July 2024 prediction, through 2026 — the final year of that forecast window, the one we are standing in — organizations will abandon most AI projects unsupported by AI-ready data. Read the mechanism, not the headline: those programs rarely die at go-live. They die in month nine of data profiling, when the executive sponsor reallocates budget and the dispatch model becomes a line item marked "deferred pending data quality." Indefinite preparation limbo is the abandonment mechanism, which means the clean-everything-first instinct is not prudence. It is the leading killer of dispatch AI programs.

The economics point the same direction. Thomas Redman's data-quality cost rule, published in Harvard Business Review, ranks prevention as the cheapest spend, correction as far costlier, and failure as costliest of all; IBM separately estimates bad data drains trillions of dollars annually from the US economy. Most operators read that rule as an argument for prevention. For a dispatch model it argues the opposite. Prevention spend touches the flow — next quarter's incoming tags. Correction spend touches the stock — the legacy corpus the model actually trains on. And the failure leg is not hypothetical; it accrues every week the fleet keeps assigning jobs on unverified tags while the remediation project profiles. Spending to stop an active failure bleed dominates spending a little up front to prevent a bleed that has not started yet.

The machine-learning precedent closes the loop. Northcutt, Athalye, and Mueller at MIT documented in 2021 an average 3.3% label-error rate across ten canonical ML benchmarks, then showed that correcting those labels flipped model rankings. Two implications follow for dispatch. Even flagship datasets ship noisy, and models trained on them still defined the state of the art — which formally retires the belief that no model is better than dirty data. The entire deep-learning canon is a running counterexample. Naive training on raw tags does undershoot, as the baseline gap above shows; targeted label repair, not wholesale rebuild, is what closed it.

The prize is retention. According to Aberdeen Strategy & Research, top-quartile first-time-fix performers — mid-80s percent and above — retain materially more customers than bottom-quartile peers. Translate that with FieldCamp's figure: a 5% customer retention bump lifts profits 25–95% on a five-year horizon. That delta rides on the skill-tag layer, meaning the highest-leverage financial number in the operation sits under the dirtiest data you own.

Concrete next step: before approving any remediation budget, pull twelve months of closed work orders, compute FTF by tag provenance, and price the second-truck-roll exposure of your worst-performing tags against the cleanup vendor's quote. The ledger below is the arithmetic your CFO will ask for.

Three strategies compete for the same dispatch model, and most operators conflate them. Clean-First finishes skills-master remediation before a single training run — every tag validated, deduplicated, and assigned a named owner. Naive trains on raw tags exactly as the export from ServiceTitan, IFS, or Salesforce Field Service left them. Fix-Forward applies the three fixes this guide prescribes — provenance weighting, outcome backfilling, Pareto spot audits — then trains on the still-imperfect store. Scored on identical criteria, with no handicaps granted to any row, the choice stops being a data-governance philosophy and becomes arithmetic.

Clean-First vs. Fix-Forward vs. Naive

Four axes, four scales. Speed is calendar weeks from kickoff to a model routing live work orders. Cost is all-in twelve-month spend: software, remediation labor, verification hours. First-time-fix recovery is measured in points against the human-dispatch baseline from the simulated reference run behind this guide's headline numbers. Audit defensibility is ordinal: can you reconstruct, for any historical assignment, which tag fired, who entered it, and how it was later corrected — full lineage, partial lineage, or none.

Read the Naive column as a phase transition, not a gentle slope. Below roughly 15% tag error, a raw-tag model limps along near the human baseline. Past that threshold, uncorrected noise flips eligibility constraints faster than the optimizer can reroute — in matching terms, bad tags delete edges from the assignment graph quicker than re-optimization restores feasible coverage — and first-time-fix falls five points underwater. That row is why "no model before the data is clean" persists as folk wisdom: it is half right about Naive and entirely wrong about the alternative.

The Fix-Forward row is the thesis rendered as division. Against Clean-First's ten-point ceiling, eight recovered points is roughly 80–90% of the achievable gain, captured in roughly 15% of the elapsed time — a 4–5x speed-adjusted advantage at just under fifteen cents on Clean-First's dollar. Steal that arithmetic for your own pilot: points-over-baseline divided by the cleaned-data ceiling gives your capture rate; pilot weeks divided by the quoted remediation calendar gives your time share. On the axes, Fix-Forward takes three of four outright — speed, cost, FTF recovery. Clean-First retains only audit defensibility. Naive wins nothing except zero upfront spend. Beyond the small-operator threshold, Fix-Forward is the default recommendation.

One conjunction hands Clean-First the win: a regulatory documentation mandate AND projected fleet growth steep enough to reprice the label store within 12 months, holding simultaneously. Either condition alone is survivable — provenance-weighted lineage and logged spot audits satisfy most auditors' reconstruction demands, and growth alone just means re-running outcome backfill each hiring cohort. Together they invert the economics: a label store sized to today's headcount gets repriced at every wave of additions, so remediation amortizes instead of repeating. Asset classes where a regulator can demand record-level reconstruction — validated-master-data regimes — are where that mandate actually lives. Absent the conjunction, the verdict stands as tabulated: Fix-Forward wins, and the table says so.

Every figure in this guide traces to a single simulated fleet, and the fastest way to misuse it is to forget that. Simulations flatter fix-forward in a structural way: the tag corruption is generated, therefore known. Simulator noise is typically drawn independently — a typo here, a stale certification there — and independent noise is exactly what provenance weighting and Pareto spot audits absorb best. Field corruption is correlated: the technician who miscertifies a skill also tends to mis-close his work orders, so label errors and feature errors travel together. In stochastic-optimization terms, the simulator assumes i.i.d. label noise; a real fleet behaves closer to a Markov chain, where yesterday's bad tag breeds today's bad closure. Treat the fix-forward premium as directionally sound but magnitude-optimistic until it survives contact with your own tags.

What the Data Doesn't Tell You

Three more caveats sit inside the study design itself. Run variance: a narrow margin between two strategies means little without seed dispersion, and the writeup reports a point estimate — rerun the pipeline across seeds before treating the ordering as stable. Leakage: outcome backfilling trains on historical closures, so if the evaluation window overlaps the backfill window, the model is partly graded on labels it has already seen. Censoring: first-time-fix silently drops canceled jobs, parts-wait escalations, and customer-deferred calls — a non-random denominator that flatters both arms relative to what operations actually feels.

The averages also hide spread, and the spread has structure. Pareto audits concentrate on head tags, so residual error pools in the long tail — the rare, expensive skills where one wrong roll costs the most. Provenance weighting degrades wherever lineage is broken; post-acquisition CRM merges are the classic case, leaving chunks of workforce history with no reliable origin. Closure-code discipline drives backfill quality: fleets logging free-text resolutions inherit noisier labels than fleets with structured codes. And a training window measured in weeks samples roughly one season — demand patterns for HVAC-heavy fleets invert between summer and winter, so expect the model's weak spots to migrate seasonally.

None of this overturns the default, but the decision rule's two named carve-outs — sub-floor headcount and regulated assets requiring validated master data, covered in the breakdown section — are not the only ways it bends. Three softer failures surface in operation. A taxonomy gap: fix-forward polishes the tags you have; it cannot conjure a skill class that does not exist, which is precisely what happens when new equipment categories arrive with no tag family. An override loop: where dispatchers reject the model's assignments at a sustained clip, the model keeps training on assignments humans refused, and the counterfactuals it needs were never taken. A non-binding constraint: if first-time-fix is capped by truck stock or parts availability, a smarter dispatcher optimizes a constraint that isn't binding and the metric barely moves. Each is detectable pre-deployment, and each changes what "deploy now" must include — not whether to deploy.

This is where the old reflex — no model is better than dirty data, finish remediation first — tries to resurrect itself, wearing these caveats as camouflage. It doesn't fit. Naive training on raw tags undershoots the human baseline, as the strategy comparison showed, and a fiscal-year cleanup extends the period you dispatch blind; the caveats describe how to instrument a fast deployment, not a reason to slow it. Before sign-off, demand three disclosures: seed dispersion behind the headline margin, tail-skill coverage in the audit plan, and an override-rate log wired into retraining. Document all three against NIST's AI Risk Management Framework and your model card's limitations section largely writes itself — which, as of early 2026, is increasingly what auditors request first.

Fix-forward does not fail randomly. Every break in this section shares one signature: the three fixes amplify whatever signal the tag store already carries. Provenance weighting amplifies dominant batches, outcome backfilling amplifies past dispatch decisions, and Pareto spot audits sample the failure modes you already see. What none of them can do is detect a bias that saturates the data. That leaves six seams — four tripwires you can test this week, two structural limits you cannot train around.

Where Fix-Forward Breaks

Systematic bias learns faithfully. Suppose an ERP cutover mapped an entire region's legacy SAP PM skill codes onto the wrong taxonomy nodes. Confidence weighting will reproduce that mapping perfectly, because corrupted tags dominate numerically and the loss function treats them as ground truth — correlated label noise looks identical to signal from inside the training loop. So build a provenance histogram first: tag counts stratified by source system and load timestamp. If more than 40% of tags trace to a single migration batch, quarantine-audit that batch before any training run. An afternoon of SQL beats a quarter spent debugging a confidently wrong model.

The small-fleet inversion. Below roughly 50 technicians, a skilled human dispatcher with accurate institutional memory often matches or beats ML dispatch. The simulation behind this guide's headline comparison places the fix-forward break-even in the small-fleet range, and the decision rule above deliberately takes the conservative bottom of that range. Beneath it, buy tag hygiene, not models — and the human option is faster than operators assume: according to BMK Community, time-to-hire for specialized dispatcher roles commonly runs 6-10 weeks, base salaries run $40,000-$55,000/year, and the fully loaded cost of a $47,000 hire lands at $58,000-$68,000 before management overhead. Cross-domain confirmation comes from a UK helicopter-EMS dispatch study, which found dispatcher training and experience measurably affect dispatch accuracy — institutional memory is an asset, not nostalgia.

The regulated counter-case. For FDA-regulated medical-device field service, 21 CFR Part 11 requires validated master data and formal change control. Outcome backfilling is ad-hoc weak-supervision relabeling by definition — uncontrolled record changes that can fail validation outright during an inspection. There, Clean-First is not a preference; it is the compliance requirement, and no efficiency argument survives an auditor.

Backfill circularity. Relabeling skills from work-order outcomes inherits past dispatchers' blind spots. If dispatch never sent anyone for elevator-controller work, no outcome can ever confirm that skill — survivorship bias baked into the label set. Cap backfilled labels at roughly 30% of the tag store and hold out a human-verified

```

Frequently Asked Questions

How much faster do EPA 608 jobs complete when dispatch is gated to certified technicians?

Across a 2024 dataset of 1.2 million service events, jobs requiring EPA 608 certification finished 18% faster when dispatch logic actively restricted assignments to certified technicians.

What actually happens if we run our optimizer without any skill logic?

Cross-trade scheduling without specialized skill logic spiked on-site technician conflicts by 44% during peak morning hours, and a generic algorithm increased mean time to repair by 37% while adding a 22-minute delay on critical plumbing calls.

How close does the fix-forward approach get to a model trained on perfectly cleaned data?

In one dispatch simulation, skill tags carrying a high error rate still produced a 79% first-time-fix rate once three targeted fixes were applied, landing within two points of a model trained on perfectly cleaned tags.

What confidence scores should different tag sources carry in provenance-weighted training?

Each tag carries a source-confidence score — 0.9 for an HR onboarding record, 0.7 for a parts-usage inference, and 0.5 for a self-reported profile — so low-provenance disagreement is treated as noise to absorb rather than truth to obey.

Which job codes should the post-job mobile audit cover, and how long does that take?

The Pareto spot audit pushes a post-job mobile confirm-or-correct prompt only on the job codes driving roughly 80% of dispatch volume, converting a six-month master-data project into a three-week sprint.

Can we use the same appointment time windows for every trade?

Plumbing averages 112 minutes per job against 78 for electrical, and stochastic simulations show plumbing backlogs propagating 3.2 times faster than electrical ones under identical load.

Quick answers