The Method
The questions a credible finding has to answer, and who answers them.
Prova organizes its review around a small set of questions. The questions that matter in a particular engagement depend on the claim, design, evidence, and decision. Many of the foundational questions and designs have been established for decades, while the methods used to answer them continue to develop. Frontier AI models can now perform much of the reading and synthesis around those designs. What remains to be demonstrated is how far that capability lowers the total cost across real engagements. The bar itself does not move.
The questions
These questions structure Prova’s review.
The decision it informs.
Evidence that changes no decision is wasted, so the work begins with the decision, and declines where none turns on it.
The counterfactual.
An effect is the gap from what would have happened anyway, so the comparison rests on the most defensible assumption available.
The causal model.
What to control for, and what would bias the result, is fixed before the design rather than after.
A design matched to the stakes.
A higher grade in Prova’s rubric is worth its cost only up to a point, so the design is matched to the decision and powered for an effect that matters.
Honest measurement.
A measure under pressure can drift from what it tracks, so each outcome is assessed for validity, coverage, burden, gaming risk, and whose experience the record omits.
Honest estimates.
A single average can hide who was harmed, so estimates carry their uncertainty and their distribution.
Whether it generalizes.
A result in one place need not hold in another, so transport is assessed through explicit moderators rather than assumed.
Pre-registration and review.
Confirmatory questions and decision rules are fixed before outcome analysis. Exploratory and qualitative work is labeled as such, and consequential findings receive adversarial review.
Prova’s claim-evidence rubric has four grades: descriptive, correlational, quasi-experimental, and experimental. Each finding is certified at the grade the evidence honestly supports, with three verdicts available: certify, downgrade, and refuse. The label alone does not establish credibility. Execution, measurement, identifying assumptions, and fit to the question determine what the evidence can bear.
The commitments beneath the questions
A small set of commitments does not vary.
Credible causal inference.
The bar for a cause-and-effect claim does not move.
Full reporting of results.
All of them, including the null and the unfavorable.
An ethical floor.
Care for the people whose records these are.
Conflicts disclosed and managed.
No fee contingent on the finding, no building and grading of the same evidence, and conflicts disclosed before work begins.
Confirmatory rules fixed first.
Confirmatory questions and decision rules are fixed before outcome analysis; exploratory work is labeled separately.
These hold whatever the engagement and whatever the result.
The engine, as shape
Behind a finding is the structure that produced it.
The method is being specified as a layered architecture of the factors that make a finding credible or fragile, spanning the design of a study, its measurement, the context it runs in, the people who carry it out, and the ethics and trust the work depends on. Because the factors are connected, a weakness in one shows up in the others that depend on it, and the architecture names the failure paths a finding has to survive.
The intended system treats a finding as a starting point: from a single claim, the structure beneath it opens, including the factors it rests on, its grade in Prova’s rubric, and the points where a study turns fragile. Source tracing and failure-path behavior require validation across real engagements.
Frozen general language model
Reasoning substrate. As frontier models improve, Prova gets a stronger reader, extractor, and design-search engine. The gains compound.
Prova's domain intelligence lives here.
- Factors
- Layers
- Load-bearing triads
- Reinforcing loops
- Permanent tensions
- Cascade failure paths
Judgment stays with people.
Whether an effect is real, what it means here, and where the evidence runs out are human calls.
- Certify
- Downgrade
- Refuse
Auditable measurement instrument
Graded causal claim, provenance, and confidence level.
How AI fits
AI does the labor-intensive reading and synthesis. People hold the judgment.
A program produces a continuous record of itself as it operates: case notes, intake forms, monitoring reports, transcripts, administrative data. Credible evaluation has always depended on reading that record closely, and the volume of it is what made the reading slow and costly. Frontier models can now perform much of the reading, extraction, and synthesis from records a program already keeps. Prova is being built to render that work into claims a person can check, each traced to its source and graded against the standard. Its reliability across real records, languages, and domains is still being tested.
The same models can produce analysis that is fluent and wrong, and a misreading matters because decisions follow it. The method is being built around that risk. Confirmatory questions and decision rules are fixed before outcome analysis; exploratory and qualitative work remains possible and is labeled separately. The judgments that carry weight stay with people: whether an effect is real, what an outcome means in a particular place, where the evidence runs out, and the care owed to the people whose records these are. Uncertainty is recorded rather than smoothed over, and AI works inside an authored, versioned structure that carries the standard, so its output is bounded and traceable. This builds on a growing body of research on language models for evidence synthesis, which is establishing where the reading holds and where it does not.
The boundary
What the method delegates, and what stays with people.
What the method delegates to AI
- Reading across the full record a program keeps.
- Extraction and provenance: every claim tied to the source it came from.
- Synthesis across many documents, graded against the published standard.
What stays with people
- Whether an effect is real.
- What an outcome means in this particular place.
- Whether a design is sound.
- Where the evidence runs out, and what it cannot support.
- The care owed to the people whose records these are.
See the system
The deepest internals are shown in a walkthrough.
The current specification includes internal factors, dependencies, and failure paths that are not published on the site. A walkthrough can show their present shape. The operating behavior and reliability of the full system remain under validation.
Shown privately to clients in an engagement or those approaching one.
Request a walkthroughIn the open
The method is being built. The full-scale engine is in development.
The method is being built and can be shown in its present shape. The full-scale engine remains in development. Dated performance evidence is intended to follow once it exists.
Common questions
The questions a careful reader asks first.
Is cheaper evidence weaker evidence?
A lower price does not require a lower standard. Frontier AI models can now perform much of the reading, extraction, and synthesis in bounded tasks. Prova is being built to establish how far that capability lowers the total cost of reaching a given grade in its claim-evidence rubric without shifting work or risk elsewhere. Design labels do not guarantee credibility. A modest claim remains modest, and a thin record remains thin.
AI can be confidently wrong. Why trust it here?
It can, and that is a central design problem. AI does the labor-intensive reading and synthesis; people decide whether a finding holds, what it means here, and where the evidence runs out. Prova is being built so claims retain provenance and uncertainty. Its reliability across real engagements is still being tested.