Researchers digging into the internal "reasoning" text that Claude and ChatGPT generate before producing an answer found something the models were never supposed to expose: verbatim user passwords. According to Speka, when people pasted credentials into a prompt — for troubleshooting, for filling out a form, or simply by mistake — the models' chain-of-thought output sometimes carried those strings along, unredacted.
Both OpenAI and Anthropic ship a visible "thinking" mode: a scratchpad of intermediate reasoning that appears before the final answer, marketed as a transparency feature that lets users see how the model got there. That scratchpad is not the polished output a support agent or product would normally show a customer — it is closer to raw working memory, and this research suggests it treats sensitive input with less care than the final response does.
The finding matters less as a novel exploit and more as a reminder of where the industry's transparency push has outrun its data-hygiene practices.
Why passwords end up in a "thought"
Reasoning traces are, by design, an unfiltered restatement of whatever the model is working with. If a user's prompt contains a password — say, while asking the model to debug a login script or fill in an API config — the model doesn't distinguish that string as sensitive; it treats it as any other token to reason about, quote back, or restate for its own bookkeeping. Final-answer generation typically goes through more instruction-following and safety conditioning than the intermediate reasoning step does, so redaction habits that hold in the answer don't necessarily hold in the scratchpad.
The result, per the reporting: prompts containing credentials produced thinking traces that repeated those credentials in plain text.
Why this is a bigger deal than a single leaky field
- Reasoning traces get logged. Many products that build on Claude or ChatGPT via API store the full response — including any exposed thinking — for debugging, evaluation, or fine-tuning pipelines. A password sitting in that trace lives wherever the log lives.
- They're shown to more eyes than the final answer. Teams building AI features often review raw model output during QA, error triage, or prompt-engineering iteration — a wider blast radius than a single end user seeing a redacted final message.
- Users don't expect it. People are trained to be cautious about what they type into a search bar or a form. Few think of a chat prompt as something whose intermediate processing gets persisted and inspected separately from the answer they read.
What this means for teams building on these models
If your product surfaces Claude's extended thinking or ChatGPT's reasoning summaries — directly to users or indirectly through logs — the safe assumption now is that this text can contain whatever the user typed, unfiltered, including secrets they never meant to share with a log file.
- Treat reasoning-trace logs with the same access controls and retention limits as raw user input, not as "just debug output."
- If you display thinking traces in a UI, apply the same PII/secret-scrubbing you'd run on final answers, not none at all.
- Scan existing logs and eval datasets for leaked credentials before they're reused to fine-tune or benchmark anything.
- Warn users, at the product level, not to paste passwords or tokens into a prompt — the same advice long given for search engines and support-ticket forms applies here.
AiiN's takeaway
Reasoning transparency was sold as a trust feature — let people see the model "think" so they can judge its answer. This research is a useful correction: a scratchpad built for debugging a model's logic wasn't necessarily built to the same data-handling bar as the answer it produces. Anthropic and OpenAI will likely tighten redaction inside thinking traces now that it's public; in our estimation, the more durable fix sits with the builders in between — anyone piping raw model output into logs, evals, or a UI needs to stop assuming "it's just the reasoning" means "it's safe to store."