Model Promised JSON, Parser Returned Null: Five Runs, Three Causes
You ask a model for JSON, it promises JSON, and your parser returns null. Here's why it happens and what actually works when dealing with long-form content.
👁 114 reads
You ask a model for JSON, the model promises JSON, and `json_decode` returns `null`. I ran into this problem this week while building an automation that writes articles - and in the process, I understood why an entire category of new tools is emerging around this exact issue.
Bottom Line
A text-generating model can never guarantee structure - it can only promise it. JSON mode and structured outputs dramatically improve reliability, but they still force format onto a system that fundamentally produces tokens sequentially. For short responses, that’s enough. For long ones - where a field contains an entire HTML article - things start to break. Three things that actually help: a tolerant parser, shorter fields, and avoiding one giant object.
What Actually Breaks
My case: an automation requests an object with four fields, where the last one contains a 1,200-word article in HTML. The model returned a response that started correctly - in the logs I could see a clean JSON opening with title and description. But the parser returned nothing.
There are three causes, and they occur together:
- Truncation at token limit. A long article doesn’t fit within the limit, the response ends mid-string, the JSON has no closing. The content is complete, the structure isn’t.
- Raw newline characters in strings. In valid JSON they must be encoded as
n. The model inserts actual newlines. - Quotes in HTML.
class="pw-lead"inside a string closes it prematurely if not properly escaped.
The Dead End I Walked Into
Since JSON was breaking, I thought I’d abandon it and ask for a delimiter format: ===TITLE===, ===HTML=== and so on. I wrote a parser, deployed it, ran it. The model still returned JSON.
The reason turned out to be trivial but worth remembering: the function handling the request was forcing JSON mode at the API level. The model had no choice - my instruction about delimiters was ignored because the format was imposed at a lower layer. I wasted three runs on this before I checked what the function itself was doing, instead of tweaking the prompt.
Takeaway: when a model stubbornly ignores your format instruction, first check whether something in the transport layer is forcing a different one.
What Worked
A two-stage parser. First, regular `json_decode`. When that fails - extracting fields character by character, handling escape sequences like n, t, and uXXXX, stopping at the first unescaped quote.
The key property: this parser recovers content from truncated responses. If the JSON was cut off after the article field, all earlier fields are complete and can be read. A standard decoder rejects everything.
| json_decode | Tolerant parser | |
|---|---|---|
| Valid JSON | works | works |
| Truncated at limit | null | recovers complete fields |
| Raw newline characters | null | recovers |
| Garbage before object | null | recovers |
After this change, the same automation passed on the first attempt and produced a 1,228-word article.
Four Things I Now Always Do
- Log the end of the response, not the beginning. The beginning always looks fine. Only the last 100 characters tell you whether the response was truncated.
- Long content doesn’t go in JSON. If a field is supposed to contain more than a few hundred words, a separate call or a format without escaping is better.
- Validate content, not just format. Just because JSON parsed doesn’t mean the content makes sense - I have a separate test that checks whether all expected numbers are in the text.
- Three attempts, then log it. After three failures, the model won’t “fix itself” on the fourth try. Better to record the failure than publish garbage.
Why a Separate Category of Tools Is Emerging
This problem is common enough that models have started appearing that don’t generate text at all. Instead of asking a model for JSON and parsing it, you ask a typed question and get back a value with a probability distribution - no text stage, so no parsing stage.
I described one such case separately: TypeSafe and the Jev model, where the API has three question types and returns decisions instead of strings. This is an observation, not a recommendation - I haven’t tested it on my own task yet. But the direction is logical: if the output is meant for code to consume, the intermediate text stage is a cost, not a feature.
When to Use Which Approach
| Situation | What to choose |
|---|---|
| Short response, few fields | JSON mode is enough |
| Long content in a field | Separate call for content plus tolerant parser |
| Classification, scoring, choice from list | Model returning type instead of text |
| Thousands of small decisions | Decision model - cost and time make the difference there |
Common Questions
Why does the model return invalid JSON when I asked for valid JSON?
Because underneath it generates sequential tokens, not structure. JSON mode improves reliability but doesn’t guarantee it - especially when the response gets truncated at the limit.
Do structured outputs solve the problem?
They help a lot with short and medium responses. With long content in a single field, there’s still a risk of truncation.
How do I check if a response was truncated?
Log its last 100 characters and length. A truncated response ends mid-sentence or mid-string, without closing.
The model ignores my format. What do I do?
Check whether the transport layer is forcing a different one. In my case, the wrapper function was imposing JSON mode at the API level - the instruction in the prompt had no chance.
How I Know This
All cases come from building an editorial automation on this site in September 2026. Five failed runs, three different causes, each diagnosed from logs. The two-stage parser is running in production - the first article after the fix passed on the first attempt and had 1,228 words.
More on this from QUANTSLANT
guides·LM Studio & MCP: How Much VRAM You Really Need for Local AIMemory thresholds from 8 to 24 GB, what you can actually run at each level, and four things local models won't do that tutorials don't mention.