LongCat Benchmark Methodology and Evidence Policy

Why this methodology exists

Benchmark numbers circulate widely, often without clear labelling of who ran the test, under what conditions, and with what limitations. A score attributed to a model tells readers nothing about whether it was measured by the model's publisher, by a third party, or by an independent tester using disclosed methods.

This methodology establishes a consistent evidence-labelling system for any benchmark-related content published on this site. The goal is to make it possible for a reader to determine, at a glance, where each number came from and what degree of independence it represents.

Evidence labels

Every benchmark number or performance claim that appears on this site will carry exactly one of the following labels:

Disclosure checklist for independent tests

Before this site publishes any result labelled “Independently tested by this site,” the following information will be disclosed on the same page:

• Model and version: Exact model identifier, version, and checkpoint or API snapshot used.
• Provider and access method: Whether the model was accessed through an API, a local deployment, or another method, and who provided the access.
• Test date: The date or date range when the test was run.
• Prompts and tasks: The exact prompts, task descriptions, and evaluation datasets used, with links to reproducible materials where possible.
• Settings: Inference parameters (temperature, top-p, max tokens, etc.), hardware specifications, and any configuration that could materially affect the result.
• Scoring method: How scores were computed, including any normalization, aggregation, or filtering applied.
• Limitations: Known limitations of the test setup, including what the test does not measure, potential sources of bias, and why the results should not be overgeneralised.

If any item on this checklist cannot be disclosed, the reason will be stated. A result that cannot meet the full checklist will be labelled accordingly (e.g., publisher-reported or third-party reported) and will not carry the independent-test label.

Current status

How vendor-reported claims are labelled

When this site references benchmark numbers published by the LongCat team (for example, in the LongCat-2.0 blog post or GitHub README), those numbers will be explicitly labelled as “Publisher-reported.” They will not be presented as independent findings, and the original source will be cited.

This site does not reproduce publisher-reported benchmark tables in full. Instead, it links to the publisher's own page and describes the type of benchmarks reported, without restating individual scores.

Corrections and source policy

If a benchmark result published on this site is found to contain a factual error (for example, an incorrect number, misattributed source, or flawed methodology description), the correction will be documented. See the site's corrections policy and source policy for the full process.

Sources

• LongCat-2.0 blog post

  Publisher documentation · Published 2026-06-30 · Accessed 2026-07-19

  Publisher blog post that includes vendor-reported benchmark scores. Referenced here as an example of how publisher-reported claims are labelled under this methodology.

• LongCat-2.0 GitHub Repository

  Primary source · Accessed 2026-07-19

  Primary source for the model card, which includes a benchmark results table with scoring and labelling notes. Referenced to illustrate the distinction between publisher-reported and independently tested results.