Schema annotation layer
Cryptic columns and coded values (e.g. 1=Customer, 2=Supplier) are scanned, suggested to you, and taught to the model upon your approval. On legacy schemas this is the main driver of accuracy.
ACCURACY & METHODOLOGY
We don't claim accuracy. We measure it.
A generated query isn't correct just because it ran. Our tests compare the DATA a query returns against a reference answer — a query that runs without errors but returns the wrong answer counts as a failure.
638/661
97%
Result accuracy — full golden set
661 real questions across four database engines (PostgreSQL, SQL Server, MySQL, Oracle), run through the product's own code path (3 September 2026). The score is not SQL text similarity: it is whether the returned data matches the reference answer, so a query that runs without error but returns wrong data counts as a failure. The set deliberately includes cryptic legacy schemas.
A distribution, not a single number
One aggregate ratio is a claim; a distribution is evidence. Below are the segments of the same run. We show the denominators too, because a high ratio on a small sample says little.
| Segment | Correct / questions |
|---|---|
| BY DATABASE ENGINE | |
| PostgreSQL | 22/22 |
| SQL Server | 92/93 |
| MySQL | 266/276 |
| Oracle | 258/270 |
| BY QUESTION TYPE | |
| Single-step questions | 187/188 |
| Multi-step analytical questions | 451/473 |
| BY SCHEMA TYPE | |
| Meaningfully named schema | 506/528 |
| Cryptic schema, annotation layer active | 132/133 |
Two notes. The cryptic sets in the schema row are measured with the annotation layer active, so this result does not mean “cryptic schemas are easier”; it means the annotation layer closes the gap. On a cryptic schema with no annotations the ratio drops. Second, the PostgreSQL segment has a small denominator (22 questions); do not read that row with the same weight as the others.
How we measure
We use a test set of hundreds of Turkish questions over realistic schemas from different business domains (sales, healthcare, manufacturing, legacy ERP), run across four database engines. Part of the set deliberately uses cryptic 'bad' schemas full of coded columns, because that is what the real world looks like. Those schemas are measured with the annotation layer active: what we measure is a deployment where an operator has entered the column meanings.
- 01For every question, PerSight's production pipeline generates a SQL query — there is no special shortcut for tests.
- 02The generated query and a hand-written reference query are each executed against a live test database.
- 03The data returned by both is compared: the question passes only if rows and values match. A query that runs but returns wrong data fails.
- 04Deliberately unanswerable questions are part of the set: instead of fabricating a query, the model must honestly say 'I can't answer' — fabrication counts as failure.
Four layers against the wrong-result problem
The most insidious failure of natural-language query tools is the query that runs cleanly but returns the wrong answer. Its main cause is not language understanding — it is schema misinterpretation. PerSight approaches this with four layers:
Relationship discipline
Tables are only joined through relationships defined in the schema; speculative joins based on name similarity are blocked by rules.
Self-repairing queries
A query that fails at runtime is sent back to the model together with the error message and repaired once — most transient failures never reach the user.
Honest refusal
If a question can't be answered from your schema, PerSight says what's missing instead of fabricating a plausible-looking query. An honest 'I can't answer' beats a wrong answer.
An honest note
This score is measured on our own test set; it is not an independent benchmark. Real-world accuracy depends on your schema quality and how much you use the annotation layer. In our own measurement the difficulty gap now comes less from schema cleanliness than from question complexity: multi-step analytical questions are markedly harder than single-step ones. We gladly share the methodology and the test set with enterprise teams who want to scrutinize it.
Frequently asked questions
What does 97 percent actually mean?
97 percent of the generated queries returned exactly the same data as the reference answer. The criterion is result equality: even a query that runs without errors counts as a failure if the data is wrong.
Why not 100 percent?
The remaining questions mostly fail on the hardest analytical scenarios and on schemas we deliberately keep cryptic. Our criterion is strict: every query that runs but returns wrong data is a failure. We measure progress by hardening the set, not by inflating the score.
Will this hold on my schema?
It depends on your schema quality. On schemas with descriptive table and column names the rate can exceed our test score; on cryptic schemas it rises significantly as you approve the annotation layer. Our tests measure both kinds.
How would I notice a wrong result?
The generated SQL is always visible on screen with a plain-language explanation; you can edit and approve it before it runs. If a question can't be answered from your schema, PerSight says so instead of fabricating a query. And every query is read-only — even a wrong one can't touch your data.
Try it on your own data
The best test is your own schema. Create a free account and ask your first question against your own database.