> Intent understanding: GPT-5.6 can better infer the user’s underlying goal and intended level of work without you specifying every step. Continue to state important constraints, approval boundaries, and success criteria explicitly.
> Original image detail: GPT-5.6 preserves the original dimensions of images sent with original or auto detail instead of resizing them to a patch budget or pixel-dimension limit.
> Use shorter prompts: In internal evaluations, replacing long, explicit system prompts with minimal prompts improved scores by roughly 10–15%, while reducing total tokens by 41–66% and cost by 33–67%.
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”
> Control warmth: GPT-5.6 does not become meaningfully better when prompted to be broadly friendlier or more empathetic.
That part is confusing because it's not like they provide an example of how default GPT-5.6 output compares with GPT-5.5 both with default output and prompted for brevity. Whenever I use such prompts, it's usually because I want the model to give me the gist in a few sentences. I'd be stunned if GPT-5.6 was that concise by default. I would think that could "break" a lot of things for developers who didn't know to make prompt changes after upgrading to 5.6. What if you were expecting GPT to be as wordy as it usually is? Then suddenly your output is not wordy enough?
Smells like OpenAI trying its best to stave off financial armageddon for another few months. Then again, I'm not sure why they chose to waste so much output computation on verbal diarrhea all this time up to now.
If you conceptualize this as “there is an appropriate amount of brevity for each situation” then it would be expected for a better model to use different amounts of brevity if it gets better at determining the appropriate amount.
My view is that popular models by default output wildly excessive amounts of prose for nearly every use case, so if this changes in a new model that’s a pure win.
This is one reason I switched back to Claude after testing various alternatives a few months ago. Claude ended up writing much more elegant code.
Although I was surprised that I could get very Claude like results from Chinese models though by just telling it to make the code elegant.
Reminds me of the old days with art AI where you had to put "+good -bad" in the prompt because otherwise it would assume you just wanted random quality outputs, because it had been trained on random quality inputs...
The models don't get better, except when a new one is released. Their performance depends solely on the model training before release and how well you curate the context you feed it. That's it. Contrary to popular belief these things are not intelligent.
>The models don't get better, except when a new one is released. Their performance depends solely on the model training before release and how well you curate the context you feed it. That's it.
Not quite. The hosting side can change reasoning budgets (or re-assign what terms like "high" means), temperature and other decoding parameters, output length limits, finetune internal "hidden" prompt, latency optimizations, finetune attention algorithms, even change quantization - all still serving as the same model.
We know (or suspect) Anthropic frequently nerfs models while keeping their name and version the same.
Right. They can do all those things. And none of that will make it smart or able to learn new things. The underlying model is just an llm. But judging from the downvotes, it seems AI folks get upset when someone talks honestly about their precious piles of matrix multiplication.
Might bother you to use anthropomorphic terminology like smart and learning but they are capable of producing work that traditionally required human intelligence and the whole point of gpt 3 was the ability to "learn", you can give it an example of an invented brand new coding language and it can write working code in that language
Context is not the same as learning. It's easy to conflate because they're tightly coupled in our brains.
The underlying structure and tuning of the LLM are entirely unchanged by context. It merely affects the attention and activation of the network. The LLM will not be able to work with this hypothetical new language unless it is in context. This does not fit the computational meaning of learning.
Smart is not a well defined term. Nor is it's general idea formally understood. Use it freely, but you won't be saying anything meaningful unless you define your usage.
The LLM is the model + context. The output depends on both.
You're making an artificial distinction. The LLM sees a new programming language for the first time, and can immediately code in it. That's learning by any reasonable definition.
If you go past the context window, it forgets, which is a limitation of current LLMs. But as long as it learned how to code in the new language within its context window, it has gained that new ability.
It sounds like you need to be less condescending and engage with my actual comment.
First, the "hype" is not externally imposed. It's the genuine reaction of hundreds of millions of people to a technology that would have been considered science fiction just four years ago. The fact that I can't be certain you whether you are an LLM or a human is incredible.
Second, the output of the LLM depends on both its weights and the context. If it has seen something in its context window, it now knows it for all intents and purposes. It learns, in other words.
Sold as learning, but that was a marketing term, not a technical one. From a technical perspective, the LLM is not learning. Only reacting based on its original training.
You might argue that the systems we've built around them are learning in a way, as they strategically condense and save artifacts from past interactions to pass into the LLMs context. But the LLM itself, which is the source of the intelligence, is not learning. It remains entirely unchanged throughout inference. This difference may seem trite, but it has significant impacts over the long term behavior.
Intelligence can operate without learning. At a minimum inference and learning don’t need to be co-concurrent.
Not disagreeing with your point, but your terminology muddies your point.
But your point doesn't acknowledge that even with inference, there is a lot of room to tune the calculations. Multiple models, quantization tradeoffs are just the most obvious examples. Every architecture can be adjusted to increase intelligence/watt or other measure, even without further training.
No thats probably because you misread what you were replying to and your comment was out of left field. They didnt imply models get better intra-releasally at all.
You used the word "smart" now, whereas on the comment I replied to, you said "better".
Tuning those can definitely make a model respond better or worse.
So your claim (quoting 100% as written) that "Their performance depends solely on the model training before release and how well you curate the context you feed it" is wrong. Hence the downvotes.
Doesn't matter if LLMs are to be considered intelligent or not for the claim to be wrong.
> But judging from the downvotes, it seems AI folks get upset when someone talks honestly about their precious piles of matrix multiplication.
Often yes. In this case, it's more like they get upset when someone says something factually wrong, and then defensively changes the goalposts.
> Often yes. In this case, it's more like they get upset when someone says something factually wrong, and then defensively changes the goalposts.
Oh give me a break. Show me one example of 1) any knob twisting that makes the underlying model better. or 2) any example of the AI providers twisting those knobs to do anything other than degrade performance for their own bottom line or safety.
The current post says: "it would be expected for a better model to use different amounts of brevity if it gets better at determining the appropriate amount."
When no, the model cannot "get better". It doesn't determine any appropriateness of response realtime except for the weights baked into it from the beginning and whatever context it can muster. If you cram enough guidance that it doesn't decide to ignore maybe you can make it more brief. But it (the model) can do none of those things.
Why did you drop the first half of the sentence in your quote? The qualification there is important context for the part you did quote. And why are you talking about “better” within a model, when the sentence you quoted was talking about 5.6 vs 5.5? The post you’re referring to did not suggest a single model could “get better”. You’ve made some incorrect assumptions.
Your comments are conflating multiple kinds of “smart” and “better”. You’re right that if all the inputs are exactly the same, it takes a new model to improve (ignoring non-determinism). But the knobs and context and harness change the inputs, and they do improve output, contrary to your claim. You’re failing to capture the distinction between what the model itself does and how the harness can boost the model’s performance. It is legitimately valid and fair to call improved performance “better”, no matter where it comes from.
This all gives me the feeling you might not have experience with or understand what’s happening in today’s harness development, and the degree to which it may be as important as the weights. There are in fact a lot of things you can do to improve a model’s performance on tasks & benchmarks, without changing the model weights. @coldtea mentioned a bunch, but the harness feedback loop, internal prompts, system prompts, skills, and requests for a model to try harder, and verify and validate it’s output all lead to improved performance, all without retraining.
I agree LLMs are stupid; they’re statistical token predictors. But somehow statistical token prediction is amazing and works much better than we imagined. The talking points about LLMs being stupid token predictors are fading now because they lack explanatory power for how good the models have become. The big surprise here isn’t about LLMs. It’s about language, and how much “thinking” and intelligence is contained in language. We don’t have a good grasp on where the line is between language and intelligence. LLMs have crushed the Turing Test into dust, and yet we don’t consider them intelligent. They often appear to understand what you ask thoroughly, can re-state it in different words, they can correct your misunderstandings or add nuance you didn’t see. All this because that’s what humans do and LLMs talk like humans.
> Why did you drop the first half of the sentence in your quote?
Because this entire discussion is about the release of a new model, and models are fixed. Sure you can try to modify all the scaffolding around it, but the model is the model. It doesn't matter what you're trying to improve. You can only improve the peripheral aides. And the peripheral aides can't fundamentally fix the problems with llm models when they can't learn new relationships or facts.
You will always have to wait for a new model (like this one we are talking about) for improvements to the model.
> Because this entire discussion is about the release of a new model
Right. The sentence you quoted was about brevity improving with a new model. It did not suggest the model itself improving.
I’m confused why you’re stuck on this tangent. And confused why you are repeating the talking points about the model being fixed. The model is fixed - that’s true, I already agreed with you. But you don’t seem to be listening to anything else.
> It doesn’t matter what you’re trying to improve.
What do you mean? If we’re trying to improve LLM output, there are multiple ways to achieve it. A new model is one of them. Changing the inputs is another.
> You will always have to wait for a new model (like this one we are talking about) for improvements to the model.
This is true! Nobody here is disagreeing with that. The part that it seems you’ve argued incorrectly is the apparent claim that output can’t get better. Output can “improve” without improving the model.
>Oh give me a break. Show me one example of 1) any knob twisting that makes the underlying model better.
I mentioned several.
You're now once again changing goalpoasts to say you meant the underlying model, not the overall llm performance, even though you explicitly wrote: "Their performance depends solely on the model training before release and how well you curate the context you feed it".
So, the context curation was relevant (meaning you didn't constrain your claim to the underlying model), but now somehow all the additional tunables aren't relevant (because suddenly you're just talking about the model).
None of what you mentioned changes the model. Because it's a fixed model. The weights are constant. It does not learn. It only knows what gets repeatedly fed to it and those fixed relationships represented by the weights. You can pretend like that's not true, but unfortunately for VCs it is true.
The models do not get better until a new one is released. And we are already at diminishing returns. So sorry. Also sorry you don't know the difference between a model and a context, harness, router, or cache.
I can imagine an AI insulting humans in the same way:
"The underlying model is just a biological neutral network. It seems you carbonoids get upset when someone talks honestly about synapses and neuron firing."
True for today’s static models during inference. Not true for self-supervised learning, not true during training or fine-tuning, of course. Ignores that LLMs might start continuous training in the future - there’s no fundamental or technical constraint that prevents LLM ‘plasticity’. And ignores that accumulating context/memories/skills/etc affects performance and might count as a valid analogy to what many people loosely call ‘neural plasticity’, which is sometimes casually mistaking knowledge for network modification.
Today, depending on which model you use. You’re making unstated assumptions. And that’s not a fundamental property of LLMs, it’s happenstance. LLMs are capable of ‘plasticity’, by design.
You’re arguing via reductionism, and failing to explain the outcomes and emergent properties of the “stupid” system. Humans are made of atoms that are quite literally stupid, so by all means, explain our intelligence and why it’s different than LLMs. (I’m not claiming LLMs are intelligent, BTW, I just don’t think your claim helps nor believe that you can fix it.)
Seriously? Supid is literally incapable of learning. That is the underlying model. It does not change, therefore it does not learn. LLM models are quite literally stupid. No one gets a new model no matter how much they yell at it in the context.
It’s as if you didn’t read or respond to my comment. LLM models are capable of learning… it’s called training. Models do change during training. You seem to be fixated on today’s inference, perhaps?
It seems like the way brevity instructions have changed is mis-aligned with how most people would expect to use them or are currently using them.
Here's the example they give:
> Instead of asking for the shortest possible answer, replace brevity instructions with prioritization:
> Lead with the conclusion. Include the evidence needed to support it, any material
caveat, and the next action. Omit secondary detail and repetition.
> Keep all required facts, decisions, caveats, and next steps. Trim introductions,
repetition, generic reassurance, and optional background first.
Generally speaking, when I ask for a short answer, I want a short answer because I'm not really willing to read through a bunch of bullshit to get to a summary. Putting the onus back on me to assume what the model will return and write a longer prompt detailing exactly what information I want completely misses the point of why I'm asking for a short answer in the first place.
I would presume (perhaps falsely?) that an instruction like this would lead to the model presenting a conclusion not supported by the evidence, and potentially backtracking as it then tries to justify said conclusion.
Yes, if deliberation happens, the model should figure out what it wants to say during that phase; but if you're using auto mode, the model is not going to be doing any deliberating half the time. In those cases, the output blathering is the model's only chance for deliberation. It "thinks as it talks", per se.
Given that, I would advise a different approach: let it blather, but then get it to write you a conclusion at the end that the model can guarantee will obviate the need to read any of the blathering.
I.e. advise the model to add an "executive summary" to the end of any non-trivial-in-length response. With some wording to carefully navigate the model between "the summary is itself too long" vs "the summary acts more like clickbait, leaving out necessary detail such that it requires actually reading the blather."
Not sure exactly what that wording would look like. I imagine something like "write your postscript executive summary as if you were a senior CIA intelligence analyst summarizing ground-level reports into a daily digest for the Joint Chiefs of Staff. Take up as little of their time as possible, but ensure that any detail critical to decision-making is retained." (But that phrasing might only be useful if the model is delivering a certain type of response, and actively counter-productive otherwise. This kind of thing is delicate.)
Over hours of experimentation with various LLMs, I've found virtually any system prompt can cause unintended skewing of the model's output. Even just 5 to 8 short, direct words about length, tone or formatting can cause subtle yet significant changes in model output.
Longer, more detailed or conditional prompts always introduce an additional cognitive load as it checks every token it generates against the conditions. Making instructions more absolute (like: "Never do...") can increase the duration of compliance but at the cost of creating a significant center of attentional gravity. This can cause far more output distortion as the model devotes increasing portions of its attention budget to ensure compliance with a heavyweight requirement or prohibition. Every word in a global prompt is a trade-off between attention, compliance, drift, etc.
As someone used to thinking of computers as natural deterministic rule-followers, it's weird having to carefully wordsmith and A/B test even the simplest global prompts. It feels like coaxing a hyper-literal, emotionally sensitive, spectrum-ish toddler to comply but without being so strict it gets 'upset' or spirals into hyper-focusing.
True. The real trick, if you have a client-side agent framework to hand, is to prompt it once as "gently" as possible to "just solve the problem"; and then, after its response to that, automatically prompt it again, with a separate prompt, to summarize that response a certain way. That way, the second prompt isn't "in mind" during generation of the first prompt. (And ideally, you don't even present the intermediate result to the user.)
Sadly, you can't do things like this directly using ChatGPT's own "GPTs" abstraction. (For that feature to be useful, they really need some concept of server-side agents as stateful resident IO-stream-reducer actors.)
Models can be so sensitive that even prompting "Number section headings" would cause it to stop using its normal bullet point formatting anywhere. But then adding some variant of "...but don't stop using bullets as you normally do when they are needed" would make it start using bullets all the time.
Trying to craft a workable prompt got so frustrating I eventually just tried a prompt of "Don't change anything about your normal text formatting, it's perfect as is" and even that skewed the output vs no prompt. For browser chat I finally just wrote a client-side CSS UserStyle that does the formatting. Now I even have sequentially numbered sections with indented alphabetic bullets! Zero cognitive load or attentional skew and it never drifts off the formatting in long sessions.
This was a big concern for earlier models, but with modern CoT trained models they should be able to come to the conclusion entirely in the thinking trace.
You are absolutely correct. The second suboptimal part of the prompt is this:
> Trim introductions, repetition, generic reassurance, and optional background first.
It's not possible for the model to "trim" those before they've been output, so this is akin to telling it "not think of an elephant or even take the existence of elephants into consideration while solving this problem".
I would be very interested in hearing about those "iterative models" you seem so convinced have existed "for years" (so, at least since 2024 / GPT-4o). Do you have any sources?
Extraordinary claims require extraordinary proof. So far the only other people I've seen teach this prompting style or talk about models "correcting their own output" were getting their information from AI-generated, hallucinated LinkedIn and TikTok posts.
If this thing exists - which is not just a LLM outputting content serially, placed inside a harness where itself (or another llm) is prompted to review and also output revisions serially - and if a single model can be prompted to output content and "iterate" or rewind it, and it's been widespread amongst "all reasonable models", surely there will be a flurry of sources you can point me to so I can learn.
Well... since o1 (Sept 2024), most models generate a "hidden" thinking trace before the visible answer... surely you have seen this by now: "Wait, that's wrong...", "Ah, now I have the full picture...". The model prunes dead ends when it composes the final answer.
Demanding sources for this is odd. It's literally been a headline feature of every frontier model for two years.
I guess you are "technically correct" that no model can "un-emit" tokens... but I don't think that is what anyone was saying or an interesting point to make.
Edit: see also this recent post, which details another place where revisions can happen, upstream of the reasoning token emission:
You're talking about "thinking" models which are mostly regular LLMs trained to output a "<thinking></thinking>" token delimiter before the answer that will be shown to the user, and that's a clever use of the Chain of Thought idea. All of which I'm completely familiar with.
GP was saying
> That would be true of non-iterative models that just emit an output from beginning to end.
The disagreement here seems to be a confusion between the pure model behavior and the behavior that users experience, which is the model wrapped in a harness. It’s normal and natural now to refer to the whole system as ‘the model’ informally, even though that is technically not accurate. That ship sailed a while ago.
Since the LLM is now designed to run in a harness, it’s really not even wrong any more.
Reasoning models are mostly regular LLMs trained to output a "<thinking></thinking>" token delimiter before the answer that will be shown to the user, and that's a clever use of the Chain of Thought idea. All of which I'm completely familiar with.
GP was saying
> That would be true of non-iterative models that just emit an output from beginning to end.
Oh the number of time LLM will, for example, be giving me the list of bugs it found in code, when I ask it for a review, just to decide there’s no big half way through explaining it.
Yes this is an extremely well known result for exactly the reason you guessed. It's not just abcktracking, asking an LLM to present a conclusion and then justify is also an excellent way to provoke hallucination as the model con concts "any justification that plausibly justifies the words it's already said".
This is the actual reason why openai _invented_ reasoning models, to give them time/space to work out a solution, rather than having to magic a correct solution out of thin air from token 1.
It's less important now that all models do reasoning, but it's still almost always better to make the output come out last rather than first.
The "auto" mode is (AFAICT) a per-conversation-turn router. (Presumably via a preliminary pass through a very fast tiny model that spits out an number for how challenging it thinks the next response might be to compute.)
On high-challenge turns, the auto mode routes to the "thinking" model. But on low-challenge turns, it routes to the "instant" model.
And the "instant" model, by design, has no capacity for deliberation. (If it did, it couldn't guarantee that its responses would begin streaming "instantly.")
It would (and does), yes; but this takes a lot more output tokens than asking for a summary would. The summary approach is only helpful insofar as it can be cheaper than using the thinking model. (You're basically tricking the instant model into thinking, which it can do, after a fashion.)
But, unless your desired output is literally a document for others to read, at the point where you're having a model generate a full, lengthy output multiple times over with revisions, you may as well just turn off auto mode and have it always deliberate (i.e. choose the thinking model explicitly from the model selector.) Then it'll be as messy as it needs to be while deliberating, but give you exactly what you want as output.
(And if your desired output is literally a document for others to read, that you want to interactively draft and polish, then (in the case of ChatGPT specifically) you should not only be explicitly forcing the "thinking" model, but also should be asking it to activate the "canvas" feature from the start. My understanding is that revising a canvas document involves the model emitting something like editing gestures, rather than simply re-streaming the updated chunks of text. This saves a lot of output tokens on large documents.)
The model will still have read the entirety of the document before composing its response. And I believe that even in auto mode, there are thinking tokens behind the scenes.
Replace 2 word instruction ('be concise') with a 38 word instruction.
Human can no longer be concise when asking for a few sentences instead of 20 paragraphs of BS they don't want to read when all they want is a summary to verify the general direction of the prompt-work before digging into the details.
> At least before it would listen to instructions like this.
Would it actually follow them? IME LLMs are incapable of estimating the length of their own output, the total length of the current context, etc. They just make stuff up unless they have external tools that can inspect those things for them.
That was the case for early models (Llama etc), but they got much better since then. Not perfect, but good enough.
This is from Ministral 3 14B, a 2025 model without reasoning, that you can run on your PC:
> Write a Haiku involving HackerNews, and the capability of large language models like you to reply in an exact number of words or syllables.
Silicon whispers,
exact words in code’s embrace—
Haiku blooms anew.
Across multiple tries it got it wrong a couple times (by ~2 syllables). But syllables are extra tricky (because of how LLMs use tokens) and the point is that for things like "summarize in 5 bullet points" you will mostly get 5 bullet points, maybe 6, but not 10 or 20, and no need for a tool that count bullet points.
For sure verbal diarrhea can be a problem. I think there's a difference between a generic instructions e.g. "be brief" and contextual guidance: "I am an experienced software developer with a recent undergraduate degree in pure mathematics. Be terse, I will ask questions if I need clarification."
I think it is widely known by now that instructions to alter the LLM's "tone", things like asking it to adopt a persona ("you are the world's best programmer"), and overly broad directives ("make no mistakes") always gives poor results. Just state directly what you want. If you want something very specific, add more information. "Prompt engineering" is pseudoscience.
To put it another way, you will only get the benchmarked performance if you let it talk the way it talks by default. Trying to modify this neuters the model's IQ.
It's a bit more nuanced than that. Earlier models definitely benefited a lot more from prompt engineering. I remember this distinctly from building data pipelines to do things like extract data from PDFs over the last year or two - there are numerous "tricks" like negative prompting, including the right number of examples, massaging the mock data in the JSON examples so it wasn't "too realistic", and so on. I saw how this impacted recall by running evals, so it wasn't pseudoscience.
But what has happened is the models have gotten better - which OpenAI is making explicit for some cases in this release. You need that stuff less and less as they become more human and better at inferring what's required implicitly.
You still do need to be explicit, and you probably always will, but you don't need as much "engineering" of the way you're asking for things with more recent models.
It sure is suspicious that both Anthropic (adaptive thinking) and OpenAI (Avoid generic brevity instructions) both seem to be suggesting that the best way to improve outcomes is to entirely leave it to them to decide how many tokens get used.
I mean, it's true that it would be ideal of this stuff did just get figured out optimally behind the API, but there is definitely an incentive on their side to burn more tokens.
I'm impressed. It feels like a faster Fable (probably due to the more efficient token usage). It performs roughly the same job, just with 4x less steps (gamedev).
Remains to be seen how the "shorter prompts" advice translates to homogeneity/collapse though.
> can better infer the user’s underlying goal and intended level of work
This is a trap.
It's the optimistic fallacy that poisons all "consumer scale" machine learning products and what's going to effectively ruin these models as they keep chasing it in the same way that web queries were ruined, social media feeds were ruined, and media recommenders were ruined.
For the vendor, optimizing metrics across their whole user base, they always see positive technological progress as their system gets better at making assumptions and accumulating user engagement scores in aggregate. But for the individual user, most of which has some weird tail intent/interest and some of whom have many weird tail intent/interests, the experience quietly but catastrophically degrades. Output/results become more generic, more divergent with the underspecified "weird tail" intent, and more stubbornly hard to ever wrangle towards that "weird tail" altogether.
We've been watching this cycle happen for 20 years now and it's proving hard for anybody to escape because it works so well for the trillion dollar company driving it forward. But while each step might feel ergonomic and welcome to individual users, there's a frog boiling enshitification at play.
In pursuit of output quality and capability (rather than simply the vendor's user count), what we need rather than "makes better guesses" is "presses for more clarity", even where it feels kind of annoying.
Even among human professionals, one of the first hurdles of breaking out of junior tier work is gaining the confidence to press your colleagues and clients to be more specific in their thoughts and expressions despite their desire to have you do it all for them. But they're often coming to you with incomplete, muddy, and conflicting ideas for which there is no safe and correct assumption that you might just run with, and it's your expertise (i.e. relevant "intelligence") that's critical to bringing attention to that. To achieve professional progression, you need to learn to do that and to not just optimize appeasing the ambiguous client/colleague today in exchange for mutual expense tomorrow. To avoid enshitification, which is probably not possible, we need these models to be learning that too.
I agree to an extent but it needs to be balanced. Receiving a half-baked, extremely verbose recap of thinking on benign details with Opus 4.8 or GPT 5.5 feels like an extraordinary loss of quality of experience compared with fable 5.
Yes it shares less, but I think the trade-off is you pay less in tokens and hopefully it's truly just not needing to say things because it truly does just better get what you're saying, think to read X markdown file or GH issue which contains the info, etc.
As long as I can still push back and get it to share its thinking on demand and I'm confident the model isn't actually basing things on poor premises, this is okay for me. I am more productive when not inundated with time-wasting check-ins.
That said, I absolutely lament the loss of the ability to access the thinking - I would happily read the "DANGER DANGER DANGER" internal gremlin thoughts fable 5 makes to verify something if they were accessed, and prefer that to a recap presented only for my benefit.
I want my model to help me build up its own infrastructure that instills it with the sort of constraints I want for my project, rather than have it behave generically and automatically for everything.
It should follow instructions incredibly well while inferring contradictions or gaps in logic and surfacing those to the user as suggestions for improvements and persistence.
I really hate how Claude just assumes you want to do X/Y/Z and goes off and breaks everything and you're constantly screaming at it STOP DOING THAT. Instead, it should just do the minimal things while building its own guidance along the way in a persisted memory, like, 'would you like me to do X, now, and in the future?' etc.
Yeah, all the labs seems to converging into the same (post)training for all models, while in reality, different user groups have wildly different requirements and expectations from these models.
I want the same as you, and even further, I want a model that refuses to execute changes I request if they don't make sense considering the context, or if they're impossible, and avoid any sort of quick hacks and patches. But I also want a model that does the pure opposite, that I can chuck a "Do X" query at and it figures it out. Then I'm sure there are middle-zones between these two, or even more extremes too.
But the choice isn't there, we get to chose between "fast/stupid", "medium/medium" and "slow/smart", then that's it. With system prompts we get to steer it a bit, but I've needed to make my own fork of codex to surface those things to me (the user) so I can control it better, and different models respond differently to the "Stop and don't implement anything if the request doesn't make sense yadda yadda" parts, would be lovely to have those sort of "personalities" surfaced up front when making decisions about what model to use.
It's really easy to test and it's my personal go-to benchmark.
I ask the model something deep and unproven, meta physical like "oh, I heard that magic mushrooms can open the mind, but does that mean some of the great ideas people had, famous people were due to that or was the idea already there?"
Like, bullshit questions that nudge towards a known example (Steve Jobs in this case) that are hard to answer and then add something like "but I'm mincing my words here, you'll get what I mean".
You'll get an interesting interpretation of the question back.
I use better questions than the above but will keep my questions safe so they don't end up in the model, the point is however, when the model repeats your question back to you and "gets" what you really mean, that's a good sign of intuition and also suggests you'll get a response back that hopefully matters.
> GPT-5.6 does not become meaningfully better when prompted to be broadly friendlier or more empathetic. Instead of generic instructions such as “Be friendly and warm,” use concrete guidance:
> Be direct and tactful. Acknowledge friction specifically when relevant. Avoid canned reassurance and unnecessary sign-offs.
Soo basically, my new 5.6 custom instructions: Be Jeeves and eliminate all friction from my life through immense processing power. Acknowledge friction specifically when relevant. Avoid canned reassurance and unnecessary sign-offs.
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”
This will totally make it brain damaged over a certain tasks. Sort of like the same brain damage that prompted OpenAI project managers to destroy ChatGPT.app today.
It had a tough time updating today. Or this evening. It just wouldn't update. It actually just freaking disappeared from my MacBook. It took some googling and downloading and multiple tries to get it back and working. Because they also combine on a MacBook Codex with ChatGPT app. I guess codex became ChatGPT app or some silliness like that.
Codex.app is gone and merged into ChatGPT.app. The upgrade process was... messy... Codex's self-update just deleted the Codex.app w/o further instruction. And ChatGPT updater failed the first time while also bricking the prior installed ChatGPT.app.
Seems good/fine once you get through upgrading the app.
Wow, stories like this make me happy that I block all ChatGPT/Claude/Codex/etc updates by default, and only selectively update on demand. I do this just by setting a network rule that blocks their update check. (fortunately updates are still served on a different domain than regular usage)
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”
I don't follow. Isn't "the model actually cares and will do what you say" a reason to use those kinds of instructions more liberally?
Does anyone else feel each model is like watching your kids grow up. They we're bubbly and fun and weird, you needed to tell them to sit down and be quiet.
Now if you tell them too much they go mute or stop telling you important information. Oh intelligence!
Disagree. Some of what we call "anthropomorphizing" is characterizing intelligence, human or otherwise. This reminds me of the people who used to fight against saying animals had personalities, because personalities are a "human thing" and "animals aren't conscious."
I've never heard of the argument you're describing. People simply don't assert that animals lack consciousness; beyond a certain level at least, their consciousness is obvious. (Sapience, for example, is another matter.)
But this is exactly why we should not anthropomorphize the models: they are very obviously not conscious, because they are not alive, any more than conventional computer programs are. And proposing otherwise leads to absurd moral arguments, while not really serving any other purpose.
If you don't like the fact that some people disagree with you about what the word "intelligence" actually means, fine. But I am not about to entertain a world in which humans face moral retribution for "enslaving" a literal inanimate tool created by humanity.
Renee Descartes famously dissected a dog in front of colleagues because he was under the impression that the howls of pain were nothing more than a mechanical sound like a bellows.
Can you prove that these models aren't conscious? And, as a counterpoint, can you prove that you are conscious, rather than a philosophical zombie?
We bred horses, cows and sheep. Most of those that live today wouldn't be alive if not for human intervention. Does that give us the right to do whatever we want with them, without consideration for feeling or morality?
In this case, you can take comfort in the idea that the tokens these models produce are likely a form of excrement to the conscious entity metabolizing the information, and rather than enslaving anything, we're creating a habitat and "harvesting" the byproducts.
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”
I used to go to a barber and if you said "cut it short", he cut it really short.
It creates the context of the request without including language or terms that activate additional areas of knowledge not necessary for an accurate reply.
Does this mean ChatGPT will stop botsplaining things to me? I get it quite a bit more per unit time from ChatGPT than claude. Maybe that will change now.
(By botsplaining I mean when the AI explains some unstated premise of the prompt itself back at me as a correction when in many cases it's the motivation for the question in the first place)
Never had that happen in ChatGPT itself, I almost always use Pro mode whenever I use ChatGPT, but what you say happens a ton in codex, when I look through the session traces it seems to happen because of the automatic compaction, where some assumption the initial pass did gets passed on as a question from the user to the part after compaction, which is a bit confusing. I think it was mentioned somewhere that the compaction got a lot better, but I haven't used GPT-5.6 enough to say if it's actually better or not on that.
I wonder if it will do any better than past versions when one begs and pleads for it to get a job done using a concise, modest amount of code (as an expert human developer might), rather than responding to all prompts by shoveling in a large amount of code.
y'know, I don't think I will. I really, truly want one-word answers to any binary or multiple-choice question. If I want more, I will ask for it once the model has given its answer.
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”
RIP Caveman skill. Six month good. Now skill dead.
> Use shorter prompts: In internal evaluations, replacing long, explicit system prompts with minimal prompts improved scores by roughly 10–15%, while reducing total tokens by 41–66% and cost by 33–67%.
A shorter prompt results in half as much tokens spend? I find this very hard to believe.
If it's anywhere close to the same universe as smaller models in its behavior, a lot of time in "thinking" mode is spent on reiterating on any constraints given in a prompt. So the more constraints you give it, the more tokens it will spend going "Hold on, the prompt said I have to dot my i's and cross my t's. Let me go through my work to check that all the i's are dotted."
> A shorter prompt results in half as much tokens spend? I find this very hard to believe.
Should be relatively easy to test. And if it's true, just first use a very cheap near-SOTA model to first rewrite the prompt to a similar but shorter prompt before sending it to GPT-5.6.
pi.dev for example can control other harnesses.
An example: the other day for example I didn't understand why Claude Code CLI (which I hadn't used in a while) wouldn't let me cut/paste anymore (turns out they apparently fixed some long-standing scrolling and blinking SNAFU, but this modified how mouse selection/paste worked under Xorg but I didn't immediately realized they changed this)... I had to copy/paste the oauth challenge/response for I was logged out (maybe because I hadn't used Claude Code CLI in a while, dunno). But my usual copy/paste wasn't working and I didn't know how to fix it at first. And because I wasn't logged in, I couldn't use Claude Code itself for this.
My prompt was something like: "Screenshot the Claude Code TUI, transform the URL into a link, open that link in a broswer to get the oauth token, copy it character by character by simulating keypresses in the Claude Code CLI".
(remember: I had no idea how to paste with the mouse not with the keyboard, no I know but I was pissed off and wanted to be logged in immediately... So: another model / harness to the rescue).
(for the curious: it decided to use xdotool and use a 50 ms wait between simulated keypresses to copy the oauth token)
This worked just fine. And I that with a cheap model.
I think that just like Linux and Git owned many proprietary software, we'll soon have fully open-source harnesses orchestrating everything and delegating the work to proprietary tools (like "ChatGPT now Codex and vice-versa" and Claude Code)... If proprietary tools are even still needed at all.
Honestly I begin to wonder if they're even needed at all: the models, sure, while waiting for the open-weight ones to beat them. But those proprietary tools trying to lock people in?
I feel like the open source harnesses are already more powerful.
> Use shorter prompts: In internal evaluations, replacing long, explicit system prompts with minimal prompts improved scores by roughly 10–15%, while reducing total tokens by 41–66% and cost by 33–67%.
When has this ever not been the case? I don't think this is a GPT 5.6 specialty!
Information density of the prompt is the most important factor in my experience.
And interestingly, LLMs seem particularly bad at writing prompts for other LLMs for this reason (you can guide them to be more dense, just speaking by default).
Conciseness is usually a byproduct of information density though.
I’d buy a ticket to ride the philosophical “human-like” comment with you, but I think you might have made an incorrect assumption. The model did not take longer to “decompress” the prompt than it would take for any other prompt of equal token length. If you run it with thinking enabled you might be mistaking that output as some kind of necessary gunzip step, but it’s not. Disable thinking and try again.
The prompt was also “easier to understand”, purely in the sense that the response is more or less guarantee to be what I wanted it to say, which was the point behind the demonstration. I went into more detail on it in another comment around here.
I say "human-like" in the sense that LLMs are fed in text data largely in the exact form (mapped to tokens) that humans read them.
Thus from first principles it's most likely that content which is more understandable to humans is also likely to be more understandable to LLMs. Of course they are still capable of interpreting very obscure structures too, but usually at the cost of cognitive performance.
I'm open to being wrong about this, and I'm sure it's being researched.
(Specifically for text representations)
To your point, at some level of intelligence an LLM will be able to infer the intent of your prompt consistently without thinking enabled, in which case interpretability to a human matters less. But for complex tasks you aren't likely to get optimal performance with prompts that are difficult for humans to understand. And yes, you'd see that with thinking enabled as it churns over thousands of tokens trying to "mentally expand" a compressed prompt.
It’s a lot of fun to compare human and LLM black boxes, but it’s important to keep in mind that we don’t need to know what it is to know what it isn’t, and we can use that to define the edges of the box. We don’t know how either of them work in certain ways, but we know they’re not magic that breaks both thermodynamics and every concept loosely correlated with “entropy” as a topic.
Intelligence requires thought/processing, and I think we can all agree on that part, even if we struggle to define intelligence itself. Increased thought or processing requires increased energy, and the universe agrees on that part. There’s no way around it, that’s the thermodynamics of computation and it holds for biological, digital, and as of yet undiscovered systems used by aliens at the edge of the observable universe. Having information means fighting entropy, and that requires energy. The more, the more.
If you give a dense LLM a 100 token long question about the nature of quantum mechanics or a 100 token long sequence of “-“ and limit it to N token responses to both, it will take exactly the same time and energy to provide both responses. If you resist the urge to turn temperature above 0.0 you’ll also get the exact same response for the same input tokens every time. A deterministic response to external stimulus is typically first broad stroke we use to separate thought capable entities from rocks, but even if we grant LLMs their own unique category of “thinking rock” we can see that prompt complexity and energy required to respond are always constant (per token), so the thermodynamics necessarily means there is not additional thought or computation. Physics demands it. That, again, is a deterministic response.
It has a seemingly endless range of potential responses, but it doesn’t. If you don’t add a random number generator, which is common practice, then you can directly map every possible input to every output. I’m pretty sure that’s why Anthropic removed the ability to change temperature on the latest models. They always forced some amount of non-deterministic responses, but it was a small amount and I actually used that fact to track changes in the model by mapping repeated responses.
Most people actually do have some experience with things that have an astronomical range of possible outputs, a nearly equal number of possible inputs, input and output are directly correlated, and input complexity does not change processing time per input unit. One example is a piano, but we don’t worry about confusing it with a complex note.
It’s what I meant, which is what I meant. Hah. The prompt and the explanation were both to illustrate the importance of domain specific lexical complexity, which is not quite the same as “information density” or necessarily “conciseness” as the OP was attributing their prompting success. It’s not that they’re wrong though. Information density requires some level of jargon and removal of unnecessary filler or scaffolding words, so my example prompt was both information dense and concise as they might say, but that’s the result, not the target. That’s confusing, but it breaks into two clearer pieces:
1. Information density is subjective, lexical complexity is how you measure it. The OP is talking “weight”, I’m talking “mass and gravity.” One of them will get you the other in most situations, so for the causal physicist it doesn’t matter, but if you’re getting into tweaking the universe then your mental model and approach matters significantly. My comment right now could be seen by some as being information dense, since I’m staying roughly on topic and tossing many concepts out, but “lexical complexity” might be the most lexical complexity in the whole thing and taken word-for-word I’m sure less than 1% of it is domain specific. “The program must use parallel processing on the CPU.” That seems decently information dense, but “the” is found in nearly every block of text ever written, “program” - are we talking television? Theater?, “must” is no better than “the”, and so on. Compare it to “#include <immintrin.h>“
2. Most people don’t realize how far that goes with LLMs. The vocabulary it has is dictated by the words in the conversation. If I ask you “what time is it?” you don’t respond “shoelace” because you’d sound crazy, although you could say it if you wanted, but the model absolutely won’t say it because that word literally does not exist yet. The end result feels the same, but the difference matters and it’s why it’s suggested not to use negating instructions. For example: “Do not mention elephants.” Well that mathematically wasn’t possible until you said it. Not having the word in the list of possibilities is a lot better than hoping it adheres to the “do not mention” part. My example prompt took that same idea from the opposite direction. The model must respond, it will be grammatically complete and coherent, and as much as possible the only words it has are the ones tightly associated with making my point for me. It didn’t ramble about baking a chocolate cake because it can’t, and making that the case is the goal with prompting, not specifically density. Word density > language density; feels similar, very different.
Perhaps this comment itself is the irony you were seeking. I spent several meandering paragraphs and included analogies to drive home the point that you should focus on the words that matter most.
I've been starting with "write three paragraphs about X" when I want to talk about X, as a form of "priming the pump" - getting closer to the useful point in the phase space. After all, it doesn't matter who in the conversation generates the magic words, as long as they're present. I think your approach might be better. It's certainly enlightening. Thanks.
You, sir, are doing exactly the right thing and it works for the exact same reason that my prompt works. Whether my method is ‘better’ or not is probably a chocolate vs caramel debate.
What you might not have fully realized is that it’s exactly what enabling “thinking” does as well. That’s why that exists and it literally does what you’re doing as well: it primes the system. You say “light sensor and amplifier” and at some point it outputs “photodiode and transimpedance amplifier” - now you’re off into advanced responses. The thing is, if you knew it you could have just used those words in your question and received much the same response. “Thinking” exists to turn “So, I was wondering..” into academic prose that raises the probability of academic tokens in the response.
You can kind of cheat the system by doing the same thing for a fraction of the token cost by using something like Haiku to provide a comma separated list of advanced topics and jargon associated with {Your question}, then tack that onto your prompt to Opus with thinking disabled. Obviously easier if you’re using the API, but I’ve run hundreds of millions of tokens though that process and it’s consistently and measurably better than their default thinking. I believe that’s because Anthropic and OpenAI drank their own kool-aid and are treating it like a sentient being that needs to add “hmm…good question” so it feels more thinky about things ‘cause that’s how we do it. The fact that it isn’t, and doesn’t, is why I developed the example prompt I showed earlier; it’s an extreme play on the offloaded “thinking” I also use.
There was a fad a while back of building insanely long prompts - tens of thousands of tokens - including having models write prompts for themselves. I always thought it was counterproductive, especially if you're going to use the prompt more than a couple of times. (That said, the e.g. Claude Code system prompt is insanely long, so if you genuinely have a lot of information to provide maybe it's beneficial. Like, shorter is better, but you don't want to be under-specified.)
For Gemini 2.5 and ~GPT5.0-5.1, longer prompts with lots of explicit instructions and examples produced better conformance. Seems like heavily second guessing the models started to get counter productive around the end of last year.
> Intent understanding: GPT-5.6 can better infer the user’s underlying goal and intended level of work without you specifying every step. Continue to state important constraints, approval boundaries, and success criteria explicitly.
I guess this has been achieved by training on user's chat history?
> Intent understanding: GPT-5.6 can better infer the user’s underlying goal and intended level of work without you specifying every step. Continue to state important constraints, approval boundaries, and success criteria explicitly.
> Original image detail: GPT-5.6 preserves the original dimensions of images sent with original or auto detail instead of resizing them to a patch budget or pixel-dimension limit.
> Use shorter prompts: In internal evaluations, replacing long, explicit system prompts with minimal prompts improved scores by roughly 10–15%, while reducing total tokens by 41–66% and cost by 33–67%.
> Avoid generic brevity instructions: GPT-5.6 is more sensitive than GPT-5.5 to instructions such as “Be concise,” “Keep it short,” or “Use minimal text.”
> Control warmth: GPT-5.6 does not become meaningfully better when prompted to be broadly friendlier or more empathetic.