Auditing an app built to be insecure

I keep a test application around that was built to be deliberately insecure. It has, in one place, the kinds of flaws I find scattered across real AI-built apps, so it is a good way to show what a security review actually turns up. The app runs. You can sign up, log in, use the dashboard. Nothing about using it suggests anything is wrong, which is the whole point.

In the last post I listed the categories of holes I find in AI-built apps. This is one app, reviewed start to finish, so you can see how they stack up when they are all in the same place.

The secrets were sitting in the browser

The first thing a review looks at is what the app hands to every visitor. This one shipped its secret keys straight into the JavaScript that runs in the browser, where anyone can read them by opening the developer tools. Not one key. A Stripe secret key, an OpenAI key, an AWS secret access key, a SendGrid key, a database connection string with its password, and the secret used to sign the app's own login tokens.

Any one of these is serious. The Stripe secret key is full control of the payment account: charges, refunds, customer data. The database string is the entire database. The token-signing secret is worse than either, because with it you can forge a valid login for any user, including an administrator, without ever knowing a password.

Here is the part that separates a real review from a naive one. Not everything that looks like a key is a problem. The same bundle also carried a Google Maps key, a Stripe publishable key, and a Firebase config. Those are meant to live in the browser. Their safety comes from restrictions and server-side rules, not from being hidden. A review that flags all of them is noise that trains people to ignore it. A review that flags none of them misses the ones that end the business. Knowing which is which is most of the job.

An endpoint gave them away too

I did not even need the bundle. The app had an endpoint, reachable without logging in, that returned its entire configuration, every secret above, as a tidy response to anyone who asked. There was also a debug endpoint that reported the server's internals, and another that revealed internal file paths. None of it required a password.

Anyone could pull the whole user list

A separate endpoint returned every user, their email and their role, with no authentication at all. That is a ready-made phishing list, and it points straight at the accounts worth attacking, because it tells you which ones are administrators.

You could read another user's private records by changing a number

Signed in as an ordinary user, I asked for my own sensitive record at an address ending in one number, then changed the number. The next user's record came back, the kind of data you never want moving: government IDs and card details. The app had checked that I was logged in. It never checked that the record was mine. Because the numbers count upward, there is no guessing involved. You walk the list.

The quieter doors

Some findings are less dramatic on their own and matter because of what they add up to. This app accepted requests from any website while carrying the user's session, so a logged-in visitor who opened a bad link in another tab could have their account used against them. Its login cookie could be read by any script on the page. Its password reset had no rate limiting, so it could be hammered. It shipped its source maps to production, sent none of the standard protective headers, and left an open redirect in place.

Individually, low severity. Together they tell you that no security work was done at any point, which is usually true, and usually means the serious findings above are present too.

The findings a scan will never give you

Here is the turn that matters most. Some of what I have described, an automated scanner will catch: the exposed keys, the missing headers, the source maps. Those are worth catching. But the findings that actually sink a business are usually the ones a scan cannot confirm.

Whether that broken record check really lets one user read another's data takes two accounts and a real test, not a single automated request. The app also had an admin-only statistics endpoint that checked your token was valid but never checked that you were actually an admin, so any logged-in user could reach it. No scanner sees that, because it needs someone to log in as an ordinary user and try. Whether a cross-site scripting hole is genuinely exploitable takes running it and watching what happens.

These are the expensive ones, and they have something in common. They live in the logic of who is allowed to do what, and they only appear when a person sits with the app and tests the relationships between accounts. A scan can tell you a door is unlocked. It cannot tell you the vault is open, because to know that, someone has to walk in.

What it takes to fix

Through all of this, the app worked. That is the uncomfortable part, and the whole reason this is hard to catch: from the seat of the person who built it, everything is fine. Finding these issues took one review. Fixing them is the other job, and the harder one, because every fix touches how the app reads and writes data, and the work is doing that without breaking what already works, and making sure a later change does not quietly undo it.

That is what we do at Thunkle. We review an app for exactly these issues, fix them, and make sure the fixes hold as it changes. If your app was built fast and has real users, you can request a review or reach us at thunkle.ai. Better to find this in a test than to have someone else find it in production.