1. Scope first
Before any access, we agree in writing what is in scope: which repo, which Supabase project (ideally staging), which features and which deliverables. Anything outside it is not touched.
2. Test as every kind of user
Most serious problems in AI-built apps are about who can do what. We test each rule as several actors:
- Anonymous visitor, with only the public key every browser has.
- Outsider, a logged-in user with no relationship to the data.
- Other tenant, a real customer of the app trying to reach another customer's data.
- Member and owner, to confirm legitimate use still works.
3. Every finding becomes a control
A control is a small automated check with a clear pass or fail: it tries one attack, or confirms one normal action. In the sample report there are 53 controls (38 database, 9 edge function, 6 code): 39 attack attempts and 14 normal-use checks. A finding is only reported if its control reproduces it.
4. Severity
| Level | Meaning |
|---|---|
| Critical | Anyone can reach other customers' data, take over accounts or roles, or fake payments. Blocks launch. |
| High | Serious damage or exposure that needs an account, a specific condition or another finding to exploit. |
| Medium | Information leaks, abuse paths or business-logic gaps that cost money or trust but don't expose core data alone. |
Granting yourself an admin, owner or staff role, or entering another tenant, is always critical.
5. Evidence, reproduction, fix
Each finding includes the real output of the test, steps your developer can rerun, the expected behavior, and the fix as a diff. Fixes arrive as pull requests; nothing reaches production unless you merge it.
6. Verify and protect
After the fixes, every control runs again on the fixed code. The controls are committed to your repo with a single command that exits with an error on any failure, so your CI can block a change that reopens a hole.
7. Standards and limits
Findings reference OWASP ASVS 5.0 where it applies. This is not a penetration test of live systems, not a compliance certification (HIPAA, SOC 2, ISO, GDPR) and not load testing. Apps may hold sensitive data such as health notes; we test who can technically reach it, not regulatory compliance.
8. How we checked the method itself
Before taking clients we ran it on practice apps with 26 seeded flaws and 5 traps. It found 8 of 8 critical and 9 of 9 high flaws in both runs, with no false alarms on the traps. Medium flaws were 8 of 9 and 6 of 9, which is why every fix is confirmed by a control. This is an internal benchmark, not an independent certification.