Token-Oriented Object Notation 'TOON' Aims to Optimize LLM Prompts, Challenging JSON Efficiency

Source
#llms #prompt-engineering #tokenization #json #TOON

The growing practice of “JSON prompting,” where developers structure LLM instructions as JSON for perceived better instruction adherence, is facing scrutiny over its token efficiency. While aiming for consistency, benchmarks reveal that standard JSON and YAML formats often result in significantly higher token counts compared to plain text. For instance, an example instruction saw its token count jump from 55 (plain text) to 88 (YAML) and 115 (JSON). This inefficiency impacts both inference costs and potentially model performance, leading to the introduction of TOON (Token Oriented Object Notation). TOON is presented as a lossless, drop-in replacement for JSON specifically designed for LLM inputs, claiming 40-60% token savings by optimizing data serialization for tokenizers.

Initial evaluations of TOON demonstrate its effectiveness, particularly in scenarios involving large, flat arrays of data, where it can achieve substantial token reductions. For uniform top-level arrays of primitives, TOON exhibited over 50% token savings compared to pretty-printed JSON. However, its advantages diminish with deeply nested or non-uniform data structures, where even YAML or flattened JSON can sometimes show comparable or better token counts. The creators position TOON as a translation layer: use JSON programmatically, then convert to TOON for LLM input to optimize token usage. Beyond token cost, benchmarks suggest TOON can lead to a meaningful increase in retrieval accuracy (around 4%). Furthermore, model-specific performance varied widely, with Gemini 2.5 Flash and the newly released GPT-5 Nano consistently outperforming other models like Claude Haiku 4.5 and Groq in large-context data retrieval tasks, highlighting critical differences in LLM architecture for handling extensive inputs. The broader “JSON prompting” trend itself remains a subject of mixed community opinions, with studies showing both perceived benefits and high failure rates, suggesting TOON’s primary value lies in efficient data transfer rather than an inherent prompting hack.