How to read a report

A sample report shows the output for a real paper, and can be read alongside this page.

What it does

You give the paper that introduced a resource: a model, a dataset, a tool, from any field. WhoChecked finds that paper in OpenAlex, reads every work OpenAlex records as citing it (up to 10,000), and for each one reports two things: how much it reads like an evaluation, and how many authors it shares with the original paper.

Several papers of one resource

A resource can have more than one paper: a preprint and its journal version, a first paper and a later one. OpenAlex can split the citations between them, so you can give up to 5, one a line, and get one report. A work citing two of them is listed once and says which it cites. Overlap is measured against the authors of all of them. The papers' citations of each other are left out. If one paper cannot be found, or OpenAlex has no author list for it, its authors are unknown, so a work sharing nobody with the others is shown as unknown rather than independent.

What it does not do

• It does not say whether the resource is good. It says who checked it and whether they were at arm's length.
• It does not decide which works are evaluations. The score ranks; it does not classify. A high score means evaluation words, a zero means none were found. Neither is a verdict, so read the works.
• It does not treat a gap as evidence. A missing author list, a missing abstract or a paper OpenAlex lacks is shown as unknown, never as "not independently evaluated".

The score

The score counts benchmark and critique words, a title hit outweighing an abstract hit, and demotes reviews and errata. It ranks; it does not say which works are evaluations. The words are English: a paper in another language scores zero, and that zero means unknown. Each row shows the score's parts and the words that matched, so you can see why it ranks where it does.

Author overlap

The share of a citing work's authors who also wrote the original paper:

• independent: none shared
• partial: some, under half
• same-group: half or more
• unknown: an author list is missing on one side

Authors are matched on surname + first initial, accent- and case-folded. This is lossy: two different people who share both count as overlap, and a name written surname-first ("Xu T") does not match the same name written given-first ("Tam Xu"). A shared author marked weak matched on an initial alone, on a surname of two letters or fewer, or with forenames or middle initials that differ; the overlap counts it either way.

Measured on 74 hand-labelled name pairs: 8 of 61 matches joined two different people, and every one of the eight was a match marked weak. A false match counts as overlap, so it makes a resource look less independently evaluated than it is, not more.

Reuse terms

If you give a github.com link, the report states the licence GitHub detects in that repository and the date it asked. WhoChecked does not look for the repository itself, and it does not say whether you may reuse anything. GitHub's detection reads the repository's licence file and covers what is in the repository; weights or data may carry other terms. A repository with no licence detected, one GitHub cannot find, and a request that fails are each shown as unknown.

If you give a Hugging Face model link, or a Zenodo link or DOI, the report states the licence declared there and the date it asked, and says if the files are gated, restricted or embargoed. That licence is what the uploader typed into a form: nothing has read a licence file or checked it against the files. No licence declared, a record that cannot be found, and a request that fails are each shown as unknown.

What it cannot see

A work OpenAlex holds no reference list for cites nothing there, so it cannot appear below however squarely it evaluates this paper. Share of 2020-2025 works with no reference list, of all works / of works cited 5+ times (OpenAlex, measured 2026-11-02): journal 34% / 3%; repository (preprints) 82% / 44%; conference 53% / 15%; no venue recorded 86% / 53%. Evaluations published only as preprints or conference papers are the likeliest to be missing.

So a short list of evaluations is not proof that there are no others, especially in fields that publish mainly as preprints or conference papers.

Your query

A check runs as a job with a random link. Your query is sent in the form, not the address, so it does not appear in the link, in server access logs, or in what a site you visit next is told. No cookie is set and nothing about you is recorded. The job, query and report included, is deleted 7 days after it was submitted. Anyone you give the link to can open the report until then. The citing works fetched for a paper are kept for reuse by later checks of the same paper, also for at most 7 days, and never with anything about who asked. So a report can be up to 7 days behind OpenAlex, and it states the date its citing works were fetched. A repository or a weights link you give is handled like your query: it is sent to GitHub, Hugging Face or Zenodo with nothing about who asked, kept only in the job, and deleted with it.

Rate limits

Every check spends requests against WhoChecked's daily allowance from OpenAlex, and one visitor could otherwise spend it for everyone. So:

• At most 30 papers an hour, counted across all visitors together, so a check of three papers uses three. A check over the limit is refused with the number of minutes until a place frees, and nothing is kept of it.
• The limit is shared, not per person, because a per-person limit would mean recording who you are, by your network address, and WhoChecked records nothing about who submits.
• When the daily allowance runs low, checks wait rather than fail. They stay in the queue and start when OpenAlex resets it, and the job's page says when that is.
• Checks run one at a time, oldest first. A paper checked recently is answered from what was already fetched, which costs almost nothing.

Source

The code, the measurements behind every number above, and the design are at github.com/JimYuhaoWu/WhoChecked.