Field notes / 02 ·
GLM read the evidence.
Then read it again.
What changes when an agent can read each source only once? A look at GLM 5.3 Flash inside our Indian GST evaluation environment.
On one invoice, the model had both sources by turn two. It spent the remaining four turns reopening them. With a different action menu, the same model route submitted a matching amount and decision in three steps.
01 / Observation
A new action. The same evidence.
The earlier study scheduled 14 GLM attempts. Thirteen reached the six-step limit without a submission. In each of those attempts, both sources had been read by turn two; the next four actions repeated a read. That produced 52 repeat reads without new evidence. The remaining slot was rate-limited before the model took an action.
Those traces show an action-selection loop. They do not reveal the model’s internal cause or establish that it could not calculate the answer.
02 / The same invoice
Follow the two recorded paths.
Repeat reads available
- 01Read rule
- 02Read invoice
- 03Read rule again
- 04Read invoice again
- 05Read rule again
- 06Read invoice again
Six steps. No submission.
Inspect the repeated-read attempt ↗Each source available once
- 01Read invoice
- 02Read rule
- 03Submit amount and decision
- Submitted taxable value, before GST
- ₹1,02,500
- Submitted decision
- Resolve
Both fields matched the case.
Inspect the read-once attempt ↗The invoice facts and supplied rule were unchanged. The public workbench contains the recorded actions and field comparisons for both attempts.
03 / The intervention
Change the available actions, not the answer.
At the start, the model could read either source. After one read, its read action was removed from the next action menu. The returned information remained available to the model. After both reads, only submission was available.
- StartRead invoice or read rule
- After one readRead the remaining source
- After both readsSubmit amount and decision
The environment enforced the same choices offered to the model. It did not supply a correct amount or choose between resolving, requesting evidence and escalating. Both studies used structured JSON actions, rather than native provider tool calling.
04 / Results in context
Completion and correctness are separate measures.
All 16 attempts in the expanded GLM study submitted in three steps. Ten matched every check in the published evaluator. A completed trajectory could still contain a missing amount, an incomplete row structure or a decision mismatch.
| Condition | Repeat reads | Read once |
|---|---|---|
| Scheduled cases | 14 | 16 |
| Step limit | 6 | 4 |
| Terminal submissions | 0 | 16 |
| Step-limit outcomes | 13 | 0 |
| Unavailable slots | 1 rate-limited | 0 |
| Published strict matches | 0 of 14 planned | 10 of 16 planned |
This is not a controlled comparison of scores. The cohort expanded and the step limit changed alongside the action policy. The same-invoice traces support a narrower observation: the read-once condition ended in a matching submission, while the earlier condition exhausted its steps reading.
05 / Method and interpretation
What the evidence supports.
Both studies were published on 26 September 2026, using the same configured GLM 5.3 Flash route through OpenRouter / Inference Net, low reasoning effort and a 4,096-token output limit per request. Each study recorded one attempt per model and case. The model received constructed invoice and inventory records and the relevant rules; this evaluates supplied-rule workflow execution.
We inspect completion, calculation and workflow decision separately, and replay recorded actions against the corresponding case version. We cannot infer unseen-task performance, an internal model defect or a training benefit from these observations.
The aggregate counts preserve the published evaluator. Dated qualification notes for the inventory-evidence and annual-return timing cases remain part of the release methods and case notes. The invoice-discount example shown here is unaffected by those notes.
A useful next experiment would hold the case set and step budget fixed, compare action policies, and include the model team’s recommended native tool-calling configuration.
Read the video transcript
Getting the tax amount right does not tell us whether an agent should close an invoice discrepancy. That is why we built our Indian GST workbench around the decisions an agent makes, turn by turn. The agent asks for the invoice records. The environment returns them. The agent then reads the applicable rule and submits its calculation and its decision.
By its second turn, GLM 5.3 Flash had seen everything it needed for this invoice. What did it do on turn three? It opened the rule again. Then the invoice. Then the rule, then the invoice. Six turns, and no decision.
We changed which actions the workbench offered. The model could open either source first. Once it had read one, only the other could be opened. After both reads, the only available action was to submit. The model still had to work out the amount and choose the decision.
On the same invoice, it now read the record, read the rule, and submitted one lakh two thousand five hundred rupees. Its decision to resolve the discrepancy also matched the case.
In the expanded run, all sixteen attempts reached a submission. Ten passed every check. That distinction matters: getting an agent to finish a task and getting it to finish correctly are two different things we can measure.