JSONL execution log: how we measured 123 agent runs without a dashboard

Two agent runs both ended with a published post. From the outside they look identical: same pipeline, same day, same result live. From the inside, one took 25 minutes and the other took 65, and the difference was neither the model nor the subject. It was three commands stopped by the permission gate and the time spent improvising a workaround.

The result hides the path. That is why we write one line per run to a JSONL file and read that file once a week. It is not production observability and it is not a dashboard: it is text that fits in a grep and answers "what repeated" with a number instead of with memory.

What is a JSONL execution log?

It is a text file with one JSON object per line, one line per run, always appended at the end. JSON Lines is read line by line: a broken line does not take down the whole file, and every new run is an append, never a rewrite.

The format matters less than the writing rule. What makes this log work is that it is append-only: today's line cannot change yesterday's. If the record can be rewritten, it stops being a history and becomes a report, and a report always ends up looking good.

Which fields go into the line?

Only what changes the decision for the next run. Ours has eight fields, and each one answers a different question:

Field	What it is for
data	When it ran, in ISO 8601. It is what lets you compare week against week.
trilha	Which kind of task it was. It shows imbalance before it becomes a problem.
tema	The subject in one sentence. Used to check for duplicates and to read the history later.
origem	Where the subject came from: queue, release, issue, search or internal material. This is the field that measures whether planning worked.
status	published, draft or blocked. Without it, "it worked" is an opinion.
nota	A short self-assessment with a fixed criterion. Without a criterion, it becomes self-praise.
urls	Where the output landed, one address per language. It closes the loop between run and delivery.
atritos	A list of sentences about what cost time. It is the only field nobody fills in well from memory, so it gets written during the run.

The format, with example values:

{"data": "2026-11-05", "trilha": "D", "tema": "log de execucao em JSONL",
 "origem": "fila", "status": "published", "nota": "A",
 "urls": {"pt": "https://verboo.ai/pt/blog/jsonl-execution-log",
          "en": "https://verboo.ai/en/blog/jsonl-execution-log",
          "zh": "https://verboo.ai/zh/blog/jsonl-execution-log"},
 "tempo_min": 40,
 "atritos": ["image generator returned 503 on the first call"]}

How do you record it without relying on memory?

With a function that opens the file in append mode and serializes the whole line. Doing it in code, instead of typing it by hand at the end, is the difference between 123 records and 3 forgotten ones.

import json
from datetime import date

def registrar(caminho, **campos):
    linha = {"data": date.today().isoformat(), **campos}
    with open(caminho, "a", encoding="utf-8") as f:
        f.write(json.dumps(linha, ensure_ascii=False) + "\n")

registrar(
    "log-execucoes.jsonl",
    trilha="D",
    tema="JSONL execution log",
    status="published",
    tempo_min=40,
    atritos=["image generator returned 503 on the first call"],
)

After writing, validate the whole file. jq reads a stream of JSON values, so -s (slurp) only finishes if every line is valid, and it also counts them:

jq -s 'length' log-execucoes.jsonl
# 123

If a line is broken, the command fails and points at it. Better that than finding out weeks later that the analysis was reading 118 of 123 records and calling it the total.

How do you analyze the file without turning it into a spreadsheet?

With a small script, standard library only. The goal is not to count everything. It is to separate recurring friction from isolated cases, because only recurring friction justifies changing the process.

import json, re, sys

ATRITOS = {
    "shell blocked": r"bloqueou|pediu aprovacao|heredoc",
    "queue empty":   r"sem item|esgotad",
    "external search": r"Reddit|Composio",
    "image model down": r"503|alta demanda",
}

linhas = [json.loads(l) for l in open(sys.argv[1], encoding="utf-8") if l.strip()]
total = len(linhas)
print(f"{total} rodadas")
for nome, padrao in ATRITOS.items():
    n = sum(1 for l in linhas if re.search(padrao, " ".join(l.get("atritos", []))))
    print(f"{nome:20} {n:3}  {n/total:.0%}")

The regexes match the Portuguese text in our own log, so labels are translated but patterns are not. Run against our file, the output was this:

123 rodadas
shell blocked         30  24%
queue empty           43  35%
external search       34  28%
image model down      11   9%

The count is a floor, not a ceiling. The script looks for words in the free text of each friction, so a run described as "worked around it with a script" lands in the general count but not in the shell one. Searching only for contornad finds 50 runs. Free text undercounts: it is good for spotting a pattern and bad for auditing.

What did 123 runs reveal?

That the bottleneck was not where we were looking. Between September 3 and November 5, 2026 there were 123 runs, 122 published and 1 blocked, with 59 hours of runtime recorded across 115 runs and a median of 30 minutes per run. The most common friction was operational, not model-related.

Measure across the 123 runs	Runs	Reading
Task queue with no pending item	43 (35%)	The topic queue, not the writing, set the pace.
External search got in the way (Reddit, Composio)	34 (28%)	Unreliable input, requires a primary source.
Shell command blocked	30 (24%)	Manual workaround before the task even started.
Image generator unavailable (503)	11 (9%)	Fallback has to be written down, not improvised.

There was also a schema finding: 4 lines put an entire paragraph into the nota field, which was meant to be A, B or C. The longest had 1,761 characters. That is not an analysis failure, it is the symptom of an untyped field. If the format accepts free text, someone will write free text, and the count by grade breaks without warning.

Queue empty: the bottleneck is the topic list

In 35% of runs, the task queue had no pending item and the topic came from on-the-spot research. The origem field exists for exactly this: it shows which source carried that month. Without that number, the conclusion would be "we have nothing to write about", which is not actionable. With it, the conclusion is "the queue for two tracks dries up in three weeks", which is.

Blocked shell: compound commands are the culprit

In 24% of runs, a command was stopped by the permission gate, almost always for the same reason: a heredoc with quotes and braces, chaining with &&, or a redirect outside the project. None of those cases is the agent failing. It is the gate doing its job, and the workaround is always the same: write the script to a file and run a single command.

External search: input, not truth

In 28% of runs, community search got in the way more than it helped: noisy results, a service that was down or a source with no coverage of the subject. The rule that came out of it applies to any pipeline that uses search: a search result is a lead, not a fact. A number only goes into the text after it is confirmed in an official or journalistic source, and the friction gets recorded when that confirmation does not appear.

Image generator down

In 9% of runs, the generator returned a 503 for high demand. What prevented rework was not retrying silently, it was having the fallback order written down: primary model first, alternative second, and the reason recorded in the friction field. An improvised fallback produces a different result on every run.

How do you turn friction into a rule?

With a three-step ladder, and none of the steps is a meeting:

1. First time: write it in the atritos field and move on. An isolated case does not become a rule.
2. Second time: it becomes a written rule in the process, with the command or the order that avoids the problem.
3. Third time: it becomes an automatic check in the script. A rule that depends on someone remembering is not a rule.

One case went through all three steps: the diagram generator was clipping text at the edge of the card. It happened four times in six days, always fixed by hand, until it became a function that measures the width before drawing. The case is written up here, with the code for the fix.

What does it cost to keep a log like this?

One line per run. The file with 123 runs and 59 recorded hours takes up 270 KB, opens in any editor and answers to jq in under a second. The real cost is the discipline to write before closing the session, plus about 10 minutes of reading per week.

What makes the log exist 123 times instead of 3 is that it is a required step in the process. In our case, the skill the agent follows ends with writing the line. If it were optional, a busy day would win.