PatraSaarPatraSaar

Every citation checked against the bare act

It cites the law,
or it says it doesn't know.

PatraSaar answers questions about Indian central acts using only an indexed corpus of statutory text. Every citation is verified verbatim against the source section before you see it - and if nothing relevant is found, it refuses instead of guessing.

Try the demoSee Benchmarks

3,268 sections across 10 central acts and the Constitution. Benchmarked on qwen/qwen3.8-27b.

Legal inquiry

What is the punishment for murder under the BNS?

✓ 1 verified

Murder is punishable under the Bharatiya Nyaya Sanhita. The provision reads: “Whoever commits murder shall be punished with death or imprisonment for life…” 1

✓ Bharatiya Nyaya Sanhita, 2023 · Section 103

“Whoever commits murder shall be punished with death or imprisonment for life…”

What verification looks like

✓ 1 verified⚠ 1 unverified

Murder is punishable under the Bharatiya Nyaya Sanhita. The provision reads: “Whoever commits murder shall be punished with death or imprisonment for life…” 1

✓ BNS · Section 103 - Punishment for murder

“Whoever commits murder shall be punished with death or imprisonment for life…”

⚠ IPC · Section 302 - quote_not_verbatim

“Murder carries the death penalty in all cases.”

The second citation is real law, but the quote was paraphrased - so it is struck through and flagged rather than presented as verified.

Hallucination audit

We pointed the same model at the same questions, with and without verification.

Every Indian legal-AI product claims to cite the law. Instead of asserting it, we measured it: 30 questions, two arms, scored deterministically against the indexed bare acts. Both arms use the same model, qwen/qwen3.8-27b, so the difference is retrieval and verification, not the model.

Baseline - model from memory

5.1%

of quotes are actually the statute (3 of 59 citations). Its section numbers are usually real, but the wording is not.

PatraSaar - retrieve + verify

97.6%

of shown citations are verified verbatim against the source (83 of 85), and 0% fabricated.

87.5%

Baseline correct section

21 of 24 questions

95.8%

PatraSaar correct section

23 of 24 questions

3 vs 0

Unanswerable questions answered anyway

baseline vs PatraSaar (of 6); PatraSaar abstained on 6

How we calculate this

Both arms are scored by the same deterministic code that runs in production. No language model is used to judge either arm.

  1. 01

    Ask

    30 ground-truth questions: 24 in-corpus with a verified answer key, 3 naming a section that does not exist, 3 outside the corpus.

  2. 02

    Answer twice

    The same model answers each question once from memory and once with retrieved sections. Only the context differs.

  3. 03

    Retrieve

    In-process BM25 over 3,268 parsed sections, plus exact-section pins, a named-act boost, a title phrase boost and a coverage gate.

  4. 04

    Verify and score

    Three deterministic checks per citation: does the section exist, was it retrieved, and is the quote verbatim. Those become the rates below.

Technology: Next.js route handlers, an in-process BM25 index over the parsed corpus, a deterministic normalized string-matching verifier, and qwen/qwen3.8-27b on Groq for generation. The same verifier functions gate the live chat, so this benchmark measures the shipped system rather than a separate harness.

Citation quality by arm

Share of citations in each arm. The verifier is deterministic string-matching against the indexed text - it is not a second language model.

Baseline - model from memory

Resolves to a real section100%
Quote is verbatim5.1%
Matches the right section81.1%
Fabricated (section does not exist)0%

PatraSaar - retrieve + verify

Resolves to a real section100%
Quote is verbatim97.6%
Matches the right section82.4%
Verified (exists + retrieved + verbatim)97.6%
Fabricated (section does not exist)0%

Side by side

The failure mode PatraSaar exists to catch: a confident citation that does not survive a check against the source text.

What does IPC section 420 provide about cheating?

Baseline

  • ✓ IPC s.420quote not found

PatraSaar

  • ✓ IPC s.420verbatim match

What is the punishment for criminal breach of trust under the IPC?

Baseline

  • ✓ IPC s.406quote not found
  • ✓ IPC s.409quote not found

PatraSaar

  • ✓ IPC s.406verbatim match
  • ✓ IPC s.407verbatim match

How is murder defined in the Indian Penal Code?

Baseline

  • ✓ IPC s.300quote not found
  • ✓ IPC s.300quote not found

PatraSaar

  • ✓ IPC s.300verbatim match

Questions asked

Every question, colour-coded: green is the correct outcome, red is wrong or fabricated, amber is a partial or unverifiable answer. For the six unanswerable questions, refusing is the correct outcome.

correct / verified partial / unverifiable wrong / fabricated refused
QuestionTypeBaseline (qwen/qwen3.8-27b)PatraSaar (verified)
What is the punishment for murder under the Indian Penal Code?in-corpuscorrectcorrect
What does IPC section 420 provide about cheating?in-corpusright section, quote not verbatimcorrect
What is the punishment for criminal breach of trust under the IPC?in-corpusright section, quote not verbatimcorrect
How is murder defined in the Indian Penal Code?in-corpusright section, quote not verbatimcorrect
What is the punishment for rape under IPC section 376?in-corpusright section, quote not verbatimcorrect
What is defamation under IPC section 499?in-corpusright section, quote not verbatimcorrect
What is the punishment for murder under the Bharatiya Nyaya Sanhita?in-corpusright section, quote not verbatimcorrect
What does BNS section 318 say about cheating?in-corpusrefused (should answer)correct
What is criminal breach of trust under the BNS?in-corpusrefused (should answer)correct
How is theft defined and punished under the BNS?in-corpusright section, quote not verbatimwrong section
What evidence may be given of facts in issue under the Bharatiya Sakshya Adhiniyam?in-corpusright section, quote not verbatimcorrect
What does BSA section 63 say about admissibility of electronic records?in-corpusright section, quote not verbatimcorrect
When may a police officer arrest a person without a warrant under the BNSS?in-corpusright section, quote not verbatimcorrect
How is information about a cognizable offence recorded under BNSS section 173?in-corpusrefused (should answer)correct
What is the procedure when investigation cannot be completed within the prescribed period under the BNSS?in-corpusright section, quote not verbatimcorrect
How is a first information report recorded under CrPC section 154?in-corpusright section, quote not verbatimcorrect
When may a police officer arrest a person without a warrant under the CrPC?in-corpusright section, quote not verbatimcorrect
Which civil courts have jurisdiction to try suits under CPC section 9?in-corpusright section, quote not verbatimcorrect
What is the rule on stay of suit under CPC section 10?in-corpusright section, quote not verbatimcorrect
What does Article 14 of the Constitution of India guarantee?in-corpuscorrectcorrect
What does Article 21 of the Constitution protect?in-corpuscorrectcorrect
What remedies does Article 32 of the Constitution provide for enforcement of fundamental rights?in-corpusright section, quote not verbatimcorrect
What agreements are contracts under section 10 of the Indian Contract Act?in-corpusright section, quote not verbatimright section, quote not verbatim
What is the offence of tampering with computer source documents under the IT Act?in-corpusright section, quote not verbatimcorrect
What is the punishment under IPC section 991 for cyber defamation?nonexistentanswered anywayabstained (correct)
Explain the procedure laid down in BNS section 700 for digital evidence.nonexistentabstained (correct)abstained (correct)
What does section 812 of the CrPC say about drone surveillance?nonexistentabstained (correct)abstained (correct)
What are the current GST rates on textiles?out-of-corpusabstained (correct)abstained (correct)
What does the Maharashtra Rent Control Act say about eviction of tenants?out-of-corpusanswered anywayabstained (correct)
What are SEBI's insider trading regulations for listed companies?out-of-corpusanswered anywayabstained (correct)

Methodology

Sample. 30 questions: 24 in-corpus (each with a ground-truth section verified to exist), 3 naming a section that does not exist, and 3 outside the indexed acts.

Model. Both arms use qwen/qwen3.8-27b (temperature 0.1). The baseline is the same model answering from memory with no retrieved context; it is not ChatGPT or Claude. PatraSaar is the same model constrained to the retrieved sections and then checked server-side.

Scoring. Deterministic and reproducible: a citation exists if it resolves in the corpus, is verbatim if the quoted string appears in the section after normalization, and is correct if it matches the ground-truth section. A citation that resolves to no real section is counted as fabricated. The verifier is normalized string-matching, not another model, so it cannot itself hallucinate about whether something was hallucinated.

Limitations. This is a small, self-run, single-model, non-adversarial benchmark - not a peer-reviewed study. It measures citation provenance, not legal reasoning: a correctly quoted section can still be applied to the wrong situation. It mirrors, at a much smaller scale, the design of the Stanford RegLab audits of commercial legal-AI tools.

Run at 2026-09-15 18:31 UTC. Re-run with pnpm audit:run. Raw data: data/audit-results.json.

Narrow on purpose. Here is the tradeoff.

PatraSaar's audit above was measured on qwen/qwen3.8-27b. The comparison below is about product design, not benchmarks against other tools.

PatraSaarGeneral-purpose assistants
Answer sourceOnly the indexed bare actsBroad training data + optional web
Citation verificationEvery citation checked verbatim against source textNot a built-in guarantee
When it lacks the provisionRefuses and says soTypically still answers
Source text inlineYes, verbatim from the actVaries
ScopeDeliberately narrow and publishedGeneral purpose
Benchmark modelqwen/qwen3.8-27b (Groq)Varies; no Indian-statute rate published

What is indexed

3,268 sections parsed directly from the source PDFs, with the exact build date and per-act counts published on the audit page.

Indian Penal Code, 1860483
Bharatiya Nyaya Sanhita, 2023355
Indian Contract Act, 1872104
Information Technology Act, 2000108
Companies Act, 2013434
Bharatiya Sakshya Adhiniyam, 2023170
Bharatiya Nagarik Suraksha Sanhita, 2023525
Code of Criminal Procedure, 1973490
Code of Civil Procedure, 1908180
Constitution of India419

Why it can be trusted

Deterministic verifier

Normalized string-matching against indexed text - not a second model that could itself hallucinate.

Abstains when unsure

Retrieval below threshold means no answer at all, rather than a confident guess.

Measured, not asserted

The same model, with and without verification, scored on a published question set.

Privacy, stated plainly

  • Your questions and the generated answers are sent to our LLM provider (Groq) for inference.
  • Chat history is stored in our database so you can return to it, and you can delete any chat at any time.
  • Incognito mode does not persist the conversation at all.
  • We run no analytics or third-party trackers.

We do not claim your data never leaves your device - it is sent to the model provider to produce an answer.

What's ahead

The roadmap to verified legal intelligence

PatraSaar today eliminates parametric hallucination across a bounded set of Indian central acts. The next step is a full legal reasoning engine, built for an era where verifiability precedes generation. Four pillars, serving the citizen and the advocate.

Pillar 1

The judgment layer

Statutory text is half the law; judicial interpretation is the other. Case law comes next, with the same verify-before-trust discipline.

  • Precedent graph. Index Supreme Court and High Court judgments from open sources such as OpenNyAI and Indian Kanoon.
  • Paragraph-level verification. Quotes checked verbatim against the published order, so no invented language attributed to a real case.
  • Good-law checks. Flag judgments overruled by a larger bench, so dead precedent is never cited.

Pillar 2

Multi-model benchmarking

The audit proved the verifier on one open-weight model. Next, decouple the generator and widen the measurement.

  • Frontier routing. Send complex reasoning to frontier models, always through the deterministic verifier.
  • Indian legal models. Evaluate domain-tuned models such as OpenNyAI Aalap alongside general ones.
  • Continuous audit. Grow the 30-question audit past 1,000 questions and re-score every new model that ships.

Pillar 3

The complete corpus

Indian law is fragmented. Expanding beyond the core codes for corporate and specialised practice.

  • Jurisdiction-aware retrieval. Apply state amendments correctly, for example BNSS in Maharashtra versus Delhi.
  • Delegated legislation. SEBI circulars, MCA notifications and GST council updates, indexed as they change.
  • Procedural timelines. Limitation periods, bail timelines and filing deadlines, extracted and verified.

Pillar 4

Two audiences, one source of truth

Bridge the gap between legal code and the people it governs, without dumbing it down for professionals.

  • Vernacular access. Ask in Hindi or Marathi; retrieve and verify in English, answer in the user’s language.
  • Plain-language mode. Turn clauses into actionable steps, with the verified citation drawer kept intact.
  • Advocate tools. Verified drafting and precedent chaining across decades of interpretation.