I asked an AI to break into my own production site. It found twelve ways in. I patched all of them in one evening. Here's what happened, what the AI got wrong, and why I'd do it again — next time before the deploy instead of months after.

This is the first post in a short series on building with AI: using the same tools I'm shipping with to harden, measure, and inspect the things I've already built. The theme is simple — the process is the proof. So let's start with the most uncomfortable kind of inspection: pointing the tool at yourself.

The setup

The site is the usual solo-developer stack: Django behind Gunicorn behind Nginx, Redis for caching and queues, several apps sharing a single VPS. It had been running in production for months. That's the uncomfortable part of the story — it was already live and serving traffic before anyone looked hard at how it was actually configured.

I'm one person. A professional penetration test runs into the thousands of dollars, and I wasn't going to commission one for a personal site. But I also didn't want to pretend the risk wasn't there. So I asked a narrower, more honest question: can an AI catch the misconfigurations that actually get you owned — the default settings, the forgotten flags, the service quietly listening on a port it shouldn't be?

The tool was the OpenClaw security-auditor skill, and it works in two phases. First a deterministic recon script enumerates the boring-but-critical facts: running services, open ports, the Nginx config, Django settings, the output of a dependency audit, the state of the TLS certificates. Then that recon output goes to GPT-5.5 for analysis.

The split matters more than it looks. The recon is reproducible and can't hallucinate — it reports what's on the box. The model only reasons over facts it was handed, not facts it imagined. That structure is what keeps an AI security review from becoming an AI security fiction. And I used GPT-5.5 specifically because it's security-tuned. Security analysis is not the place to reach for the cheapest model on the shelf.

The twelve findings

The audit came back with twelve findings, ranked by severity. Two were genuinely alarming.

Critical. Gunicorn was running as root. Any code-execution bug anywhere in the stack would have handed an attacker the entire machine. The fix was straightforward: create a dedicated gunicorn user and start the service as that user instead. The second critical was a hardcoded SECRET_KEY fallback — Django's django-insecure- prefix still sitting right there in settings. I removed the fallback entirely. SECRET_KEY now raises ImproperlyConfigured if the environment variable is missing, so the app refuses to boot insecure rather than booting quietly wrong.

High. DEBUG=True was the default in two of the apps' settings — exactly the kind of thing that leaks stack traces and environment data to anyone who triggers an error. I flipped those defaults to False. Redis was listening with no authentication, which means anyone able to reach it on the network could read or write the cache; I added requirepass. And a dependency audit surfaced eleven critical vulnerabilities across Django, Werkzeug, requests, and Gunicorn. I upgraded all of them, taking the audit count from 122 down to 62.

Medium. No fail2ban, so brute-force attempts went completely unthrottled — I installed it and configured three jails (sshd, nginx-http-auth, nginx-botsearch). No Content-Security-Policy header and no Permissions-Policy header — added both. And the admin panel sat at the entirely predictable /admin/. I moved it to a non-obvious path and left a honeypot at /admin/ that returns a 404 to anyone poking at the old address.

Low and informational. Two findings I looked at and deliberately declined — more on those in a second — plus one clean bill of health on certbot's auto-renewal, which was already working.

Twelve findings, and the two that would have actually sunk me were configuration sins, not exotic code exploits. That's the pattern worth internalizing: the stuff that gets small sites owned is almost never clever. It's a service running as root and a default that nobody flipped.

What the AI got wrong

This is the part worth dwelling on, because it's where the "just run the tool and do what it says" mindset falls apart.

The AI told me to set X-Frame-Options to DENY. A sensible default — except this site embeds Plotly Dash dashboards in iframes, and DENY would have broken every one of them. I kept SAMEORIGIN. It told me to set CSRF_COOKIE_HTTPONLY=True. Also a sensible default — except my front end reads the CSRF token client-side for AJAX calls by design, and True would have broken that flow. I declined it too.

It also flagged all 62 remaining dependency vulnerabilities with the same urgency, even though every one of them lives in a package that never touches a web request — image-processing and data libraries used only in offline jobs. The model correctly deprioritized them in its summary, but it still listed them, and that's the point: not every vulnerability matters equally, and a tool that enumerates them all is handing you a filtering job, not a finished verdict. It re-suggested some Nginx hardening that was already in place, too.

The pattern is consistent. The AI is thorough and it is fast, but it does not know your architecture. It can tell you what the textbook says; it can't tell you which textbook rule your particular system depends on bending. That judgment is still yours, and the moment you outsource it entirely is the moment the tool starts breaking things in the name of securing them.

The patch process

The patching went in three phases and took about forty-five minutes of actual work.

Phase one was the critical and high-severity infrastructure: the Gunicorn user, the SECRET_KEY fix, Redis auth, fail2ban. After each change I confirmed every site still returned 200 — security work that takes the site down is its own kind of failure. Phase two was dependency upgrades, the admin rename, and CSP deployed in Report-Only mode first, so I could watch the browser console for violations before enforcing anything. Phase three switched CSP to enforcement, pinned setuptools (a Dash dependency that breaks on newer versions), and finished the remaining package upgrades.

The audit took longer to read and reason about than the fixes took to apply. That's exactly the right ratio. The thinking is the expensive part; the typing is cheap.

What I'd do differently

A few honest lessons, the kind you only get by doing it on a live system instead of a demo:

• Run the audit before going to production, not months after. This site had been serving traffic the whole time it was misconfigured. The fixes were easy; the window of exposure was not.
• Always stage CSP in Report-Only first. Going straight to enforcement is the fastest way to break a working page and not know why.
• Don't blindly accept every finding. Two of the AI's suggestions were wrong for my stack. You have to know your own system well enough to reject intelligently.
• Don't reach for the cheap model. Security analysis rewards the better reasoner. This is the wrong corner to economize in.

The repeatable process

None of this was a one-off. The security-auditor skill is in the OpenClaw registry, and the recon script it runs collects the same facts every time: running services, open ports, Nginx config, Django settings, the dependency audit, the TLS certificate state. That output goes to GPT-5.5, and the result is a prioritized list — each finding with a severity, an explanation, and a suggested fix. The whole thing is designed to be run again quarterly, not admired once.

That's what makes it useful rather than impressive. A demo finds problems in an app built to have them. This found real problems in a system I'd actually shipped, and it'll find the next batch the next time I run it.

The honest verdict

I'm not going to claim AI security auditing replaces a professional pen test — it doesn't, and anyone selling you that is selling you something. A skilled human attacker chains together things no checklist anticipates.

But for a solo developer or a small team running production Django, that's not the threat that gets you first. What gets you first is the service running as root, the default left flipped on, the admin panel sitting where everyone knows to look. AI catches that class of problem reliably, explains it clearly, and hands you patch-ready fixes — in an evening, not a billing cycle.

The twelve findings on my site are patched. The script that found them is open. The only question worth asking now is the one I put off for months: what would it find on yours?

Next in this series: Weibull analysis for manufacturing people who think they hate statistics — and, after that, why I publish my AI usage data instead of hiding it.