Home › How we test
Methodology · Published October 2026How we test
- Oct 2026 — Rewritten as the global multi-category methodology hub: three published methods, per-vertical status table, update cadence.
1. The promise: rankings you can audit
Our brand promise is one sentence: “Rankings you can audit.” On this site that means something concrete — every score on every ranking page is traceable to published evidence you can check yourself: a documented test protocol, a dated community sample with linked threads, a named third-party audit, or a screenshot of the checkout page. If a score can't be traced, it doesn't ship — that's the house rule.
Most review sites write “we test rigorously” and move on. We publish the protocol, the sample rules, the shill filters, and the honest status of each category — including what we haven't tested yet. That's the whole page, below.
2. Our methods
Different products need different evidence. We use three methods — matched to what each category actually allows — instead of faking one method for everything:
Method 1: Hands-on testing
Where testing is physically possible, we buy the product and run a published protocol — then publish the raw data. This is how our VPN protocol works (full-price purchases, ~214 timed speed runs per VPN, leak tests, kill-switch drops, streaming unblocks, live-chat support probes, renewal-price screenshots). The protocol is documented and frozen; the lab rig that executes it is being built now.
Method 2: Community-sentiment aggregation
Where testing is impossible or dishonest at scale — a fragrance's sillage, a moisturizer's feel — we read thousands of real user comments, count them by strict rules, filter shills and AI spam, and publish the sample with dated disclosure. A human reads every counted comment; AI may find threads, never count mentions. Read the full protocol.
Method 3: Third-party lab data
Where independent labs already publish rigorous results, we cite them instead of re-running them. Our supplements scores are built on published purity and label-accuracy data from Labdoor and ConsumerLab plus our own documentary audit — we don't run a chemistry lab, and we don't pretend to. Read the full protocol.
Honest note on method 1: the VPN protocol is documented and frozen — the lab rig that executes it is being built now. Current VPN scores are launch scores (see below); measured speed, leak, and streaming results publish with the first lab report. We will not write “we tested” about results we haven't measured.
Related: why we publish ‘lab pending’ instead of faking test scores.
3. Per-category status — what's true today
Not every category is at the same stage, and we won't blur that. Here is the honest, current state of every vertical we cover:
| Category | Method | Status (Oct 2026) |
|---|---|---|
| Privacy / VPN /best-vpn · /privacy | Hands-on protocol + audit scoring | 10-step protocol documented and frozen; lab rig being built now. Current scores are launch scores from verified audits, pricing, and features — speed, leak, and streaming rows show “lab pending” until the first lab report publishes. |
| Secure email /learn | Audit scoring (launch) | Ranked on encryption claims, independent audit history, jurisdiction, and pricing. End-to-end hands-on account testing is queued — not yet run. |
| Privacy software (antivirus, password managers, cloud storage) /privacy | Audit scoring (launch) | Ranked on verified audits, feature verification, and pricing. Hands-on platform installs are queued — not yet run. |
| Learning (language apps, courses) /learn | Audit scoring (launch) | Ranked on curriculum structure, pricing, refund policies, and verified platform data. Hands-on app walkthroughs are queued — not yet run. |
| Gear (routers, espresso) /gear | Audit scoring (launch) | Ranked on verified specs, documentary review of independent tests, and pricing. Physical hands-on testing is on the roadmap — not yet possible. |
| Health / supplements /supplements | Third-party lab data + audit scoring | Scored on published Labdoor/ConsumerLab purity and label-accuracy data plus documentary audit of claims and pricing. We never lab-test ingestibles ourselves, and community sentiment is excluded by our liability rule. |
| Fragrance /fragrance | Community-sentiment aggregation | Rankings built from disclosed Reddit/community tallies under our published consensus protocol — tallied blind to commissions, with shill filtering and full disclosure blocks on every ranking page. |
What “launch score” means
A launch score is a score built on verifiable facts, not fabricated tests: published independent audits, official pricing pages, documented features, and cited third-party lab results. Every claim in the audit has a source you can open. Rows that would need hands-on measurement we can't honestly do yet — measured VPN speeds, lab-measured purity — show “lab pending”instead of a number. The label is the whole point: you can see exactly how far the evidence goes, and where it stops.
4. The roadmap: building toward hands-on testing everywhere
The direction is ambitious and public: hands-on testing in every category where testing is physically possible. Where it isn't possible or honest — you can't lab-smell a fragrance, and we won't pretend to — the method is disclosed community consensus, permanently. Here is where each category stands on that road:
- Privacy / VPN — protocol live, rig building. The 10-step protocol is frozen and published; when the rig runs, launch scores become measured scores and “lab pending” rows get real numbers.
- Privacy software, secure email, learning — next protocols.These are the natural next hands-on protocols after VPNs: software and digital services can be tested on real accounts without a physical lab, so they move first.
- Gear — needs a hardware bench. Routers and espresso machines need real devices on real test benches. That means capital expenditure, so this comes later — the scores stay audit-based and labeled until then.
- Supplements — always cited labs, never ours. We will not run a chemistry lab; independent third-party purity data is the right evidence and stays the method.
- Fragrance — always community consensus. Sentiment aggregation is the honest ceiling for subjective products. The protocol keeps versioning; the method doesn't change.
Protocol versions are public: when a protocol changes, the version number changes, and the changelog is public. We will never back-relabel a launch score as a test score.
5. How money works
- Affiliate-funded, never sponsored. Reader purchases through our affiliate links fund the testing work. There is no price at which a brand can buy a rank, and commercial terms are never part of a ranking decision.
- Rankings are commission-blind. The score or tally is finalized and locked before anyone checks which products have affiliate programs or what they pay. For sentiment rankings we document the lock timestamp and the first affiliate-lookup timestamp — if the lookup came first, the ranking doesn't ship, we redo it. Scores come from the methodology, not the payout.
- Purchased vs. loaned — stated per review. VPN subscriptions in our test protocol are bought at full price: no review units, no press discounts, no favors. For other categories, the purchase or loan arrangement is stated on each individual review page — never assumed, never left vague.
- No-pay options are never hidden. Providers with no affiliate program are still mentioned where relevant — we'd rather lose the commission than hide the option.
6. Update cadence & re-testing triggers
A ranking is a snapshot with a date on it. Here is exactly when we revisit it:
| Trigger | What happens |
|---|---|
| Quarterly re-check | Every ranking is re-examined every 3 months. Sentiment rankings get 5 fresh threads per category compared against the published tally; if the top 3 hold, we update the sample date and move on. |
| Price changes | A price, renewal term, or plan change on a ranked product triggers an immediate re-audit of its pricing rows — renewal hikes are the industry's open secret, and we don't let stale prices stand. |
| New third-party audits | A fresh independent audit (a VPN no-logs audit, a new Labdoor purity test) triggers a re-score of the affected rows on publication. |
| 14-day controversy triggers | Recalls, reformulations, ownership changes, a brand caught astroturfing, a viral thread materially shifting opinion, or a correction request that survives our review: full re-sample within 14 days, public correction within 72 hours if the ranking changes. |
Every ranking carries a sample date, and changelog dates on this site are real edit dates — no fake “updated” stamps. If a re-test disagrees with a published ranking, we correct publicly within 72 hours and keep a public corrections log. Being wrong is survivable; hiding it isn't.
What we don't do
- No sponsored placements. There is no price at which a product can buy a rank.
- No fabricated tests. If we haven't run the protocol on it, we don't rank it as tested — it gets a launch score or “lab pending.”
- No invented numbers. Every number traces to a source of truth: a tally sheet, an audit report, a screenshot. An invented number is indistinguishable from fraud.
- No fake authors. Every review names who did the work. Fabricated credentials are an instant credibility death.
- No crowd opinion on ingestibles. Supplements are scored on lab data and documentary audit only — never on community sentiment, no matter how rigorous the sample.