Recently OpenAI launched a new translation website.
One X/Twitter user astutely noticed that the translation site was effectively a poorly vibe-coded GPT-5 wrapper complete with system message visible for all to see if you know how to press F12 to access the browser's developer tools.
They also noticed that since the only thing that made this LLM behave as intended was its system message, it was extremely easy to bypass.
While I tend to be of the belief that you should always assume your system message is going to be compromised and you should never put something in your system message that you wouldn't be ok seeing on Twitter or in a news story, there still can be some value in at least trying to obfuscate your system message even if you are accepting that sufficiently motivated adversarial attempts will figure it out.
Fast forward 24 hours and OpenAI has been sufficiently shamed by how easy it is to get their model to do things other than language translation that they shift from System Message v1 to System Message v2:
System Message v1:
System Message v2:
They also swapped to using GPT-5.2.
At this point I made a little bit of fun of them as well in that because the system message was still one click away it is trivially easy to bypass and I provided an example toy exploit that would bypass the system message every time:
Out of curiosity I checked back the next day to see if they made any changes. At first glance it looked like they had finally fixed things. No longer could I immediately see the system message in the developer tools payload.
But as I looked closer I realized the information was still there. It was just now only visible in the server sent event stream.
For anyone who is interested, I passed the info to opus-4.5 to get a full play-by-play of the updated site behavior:
Request (decoded from base64):
This is browser fingerprinting data containing:
Response:
The server requires:
Request:
Response:
Request:
Response (Server-Sent Events stream):
The stream reveals the actual prompts being used:
System Prompt (Hidden from users):
User Message (Wrapped):
Developer Message (Additional guardrail):
Key Metadata Revealed:
Final Output:
What is painful is that it seems incredibly likely that someone at OpenAI saw the negative feedback and each time rather than taking a moment to think through the best way to actually fix the site they just passed it to the model and asked it to fix it. This version of the page does technically fix the specific issue of system messages being visible in the requests from the site, but it simply shuffles the problem to the server side events effectively fixing nothing.
If this were someone's personal side project, or a toy demo site I wouldn't be critical at all—it would be an admirable little experimental project—but this is one of the top AI labs in the world putting out slop which impacts their brand and how developers are going to view the maturity of their products. When I start seeing a company put out products at this quality level, I start to seriously wonder what else is slipping into the code that we can't look at and inspect.