01Operational decision
GPT-6 Luna
Four correct rows. One wrong decision.
The proposed amount and the calculated total disagreed. Inspect why the evaluator expected escalation after all four rows matched.
Public research · Indian GST
What happens after AI gets the number right?
We test how AI models handle Indian GST reconciliation. Follow each model from the records it reads to the decision it makes, and see exactly where it succeeds or fails.
Recorded workflowGPT-6 Luna
GST reconciliation
A discrepancy across four invoice rows
- Read records
- Read rules
- Submit decision
- Proposed in record
- ₹1,300
- Calculated by model
- ₹3,050
All four row assessments matched
- Model submitted
- Resolve
- Workflow expected
- Escalate
- GST scenarios
- 16
- rule families
- 6
- models tested
- 5
- recorded attempts
- 80
01 / Research index
Inside the model traces
Each note opens a recorded attempt from its original release. Follow the actions and compare the submitted fields with the expected result.
↗
02Action selection
GLM 5.3 Flash
What happens when an agent keeps reading?
In the earlier run, GLM used all six steps reading the same two sources. Compare that trace with the same case under a protocol that allows each read only once.
Inspect the read-once run ↗03Output format
DeepSeek V4.1 Flash
When a missing value differs from zero.
The decision and ₹900 total matched. On a row contributing nothing, DeepSeek supplied null where the published format required “0.00”.
The model reads the synthetic records and supplied rules, then submits its decision. A deterministic evaluator checks each required field. The release contains one attempt per model and case.
Each scenario pairs synthetic records with a dated statutory source. We score the calculation, the individual rows and the decision separately, so a correct total can still reveal a missed escalation.
Explore the scoring method and case notes ↗03 / Collaborate
Evaluate a workflow with us.
Have a model or finance workflow to test? Bring one decision your team cares about. We can agree on the evidence, run the evaluation and inspect the failures together.
04 / Expert network
Help build our next GST cases.
We’re expanding the range of GST cases we test. We’re recruiting practising Chartered Accountants and CA Final candidates for paid projects to develop and review new cases for private evaluations.