Speed Lab Provenance Cleanup — honest aggregate framing

2026-07-23 · strategy

Speed Lab Provenance Cleanup

The problem: Speed Lab throughput numbers are hand-authored (a smooth 730→320 ladder that tracks rank, not real measurements), and methodology.md explicitly says "We don't run our own speed tests — we aggregate." Yet 151 live English claims say "we tested / in our tests / we measured" and cite hard Mbps as first-party fact. That contradiction is the landmine under the data-driven / LLM-citation strategy.

The call (Ben, 2026-07-23): clean up the language to aggregate framing — keep it light, no groveling "we don't physically test VPNs" disclaimers. getSpeedLabData() rank stays; it's defensible aggregated editorial. Framing standard lives in content-faq-geo.md → FAQ Authoring Rules → Rule 4.

The 151 claims split into two tiers with very different fixes.

---

Tier 1 — light-touch language (DO NOW, fits the call)

Passing mentions and metadata. Deterministic swaps, no rewrite.

Transform rule

• Drop the measured Mbps as first-party fact; keep the rank.
• "we tested / in our tests / we measured" → "in our Speed Lab" · "across the independent labs we aggregate" · "independent lab data shows".
• "our Speed Lab" as a ranking is fine (it's our aggregated index). "our Speed Lab at 730 Mbps" is not (implies we measured it).

1a. Provider meta_descriptions (7 — exact targets)

Change these strings in src/data/providers/*.json. Keep the existing rank; remove the at X Mbps / X Mbps speeds measured figure; frame as aggregated.

Consistency check: the meta rank should match speedlab.rank in the same JSON (single source). Verify while editing — some may be stale.

1b. Passing "we tested" one-liners in non-Tier-2 posts + FAQ answers

Apply the transform rule to the scattered mentions in files that are not in the Tier-2 list below (e.g. a lone "in our tests" inside a provider review's prose or FAQ). Judgement per sentence — make it read naturally, don't robot-swap.

Exclusion list = the Tier-2 files (do NOT light-swap these; they need the decision below).

---

Tier 2 — the fabricated test reports (FLAGGED — needs Ben's decision)

These ~20 files don't have a phrasing problem, they have a premise problem: they claim first-party testing with specifics no aggregation produces — "exactly how we measured," "RT-AX86U Pro on 500 Mbps fiber," "15 consecutive tests across 10 servers," "across 50 matches of Valorant." Aggregate word-swaps can't make these honest.

The 6 dedicated speed-test posts: nordvpn-speed-test.md · expressvpn-speed-test.md · surfshark-speed-test.md · mullvad-speed-test.md · cyberghost-speed-test.md · protonvpn-speed-test.md

Heavy fabricated-methodology posts: iptv.md · hulu.md · netflix.md · disney-plus.md · xbox.md · asus.md · cisco-linksys.md · dd-wrt.md · fedora.md · best-vpn-for-router.md · best-vpn-for-gaming.md · best-vpn-for-business.md · mozilla-vpn.md · pages/vpn/nordvpn.md

Options (pick one)

1. Reframe as aggregated/lab-sourced — strip the fabricated methodology + hardware specifics + match counts; present the same numbers as what independent labs report, attributed. Moderate rewrite, ~20 files. Preserves the pages, makes them honest. → build-worker lane, per file.
2. Leave as-is — accept the risk. The exposure is: a reader/LLM checks, finds the "test" never happened, and the contradiction with methodology.md surfaces. Highest on the flagship pages (the speed-test posts rank for "[provider] speed test").
3. Actually test — the real fix, but speed-lab.md (real testing) is parked; not near-term.

Recommendation: Option 1 for the 6 speed-test posts first (they're the ones that literally rank for "speed test" and carry the most fabricated detail), then the streaming/router batch. But it's a content rewrite, not this cleanup — needs sign-off + goes to the build worker.

DECISION (Ben, 2026-07-23): Option 2 — leave as is.

Ben reviewed a concrete sample (nordvpn-speed-test.md protocol-results table + URLs like www.vpn.com/vpn/nordvpn/speed-test/) and chose to leave the Tier-2 posts unchanged. Risk surfaced + accepted — do not re-open without new direction. Tier 1 (metas + passing lines) is unaffected by this and still stands.

---

Handoff

Tier 1 → ready for the build worker now (deterministic + the transform rule). Tier 2 → awaiting Ben's option pick before it's scoped to the worker. Logged in bowtie (this cleanup) + linked from ops-index.md live queue. Framing SSOT: FAQ Rule 4 in content-faq-geo.md.