The security review
What we check before a new version goes live
Every time a new version goes live, it first goes through a security review: all of the code is read and the running system is tested, with a deliberate attempt to break in. Each problem we find is reproduced where it can be, fixed, and covered by a test that catches it if it ever comes back.
This page is technical, for you or your developers: what the review covers, what its report contains, and the review of our own booking product, step by step. For where your system runs and who looks after it, see how we host it.
The security review
What every security review covers
The same eight areas on every build, checked in the code and against the running system.
Before every production deployment we review the whole repository and test the running system. Each finding is reproduced where it can be, fixed, and guarded by a test.
| Area | What is checked |
|---|---|
| Authentication and sessions | How people sign in, and how their sessions are handled. |
| Tenant isolation | One client's data invisible to another, enforced in the database. |
| Authorisation on every route | Is this record yours, not only are you logged in. |
| Input handling | Uploads, URL ingestion, tokens and oversized bodies. |
| Secrets | Keys separated by purpose, sealed at rest, nothing in the repository. |
| Public endpoints | Rate limits and budgets. |
| Dependencies | Pinned and hashed. |
| Deployment | Headers, exposed ports, and what answers from outside. |
Our own review, not an independent audit. It is done by the team that built the system. If you need an independent, certified audit, an outside tester can be brought in.
The report
What the report contains
Every finding is written up the same way. Findings are acted on when they are reproduced, not when they are argued.
One finding, as it appears in the report
- Severity
- Blocker
- High
- Medium
- Low
- Reproduction
- How it was reproduced against a live instance, or traced to a line where it cannot be.
- Fix
- The change that closes it, read by the expert like every other change.
- Guarding test
- A test that failed before the fix and passes after it, kept in the suite.
- Still open
- What remains, with an owner, the risk it carries and a mitigation plan.
Then the launch-readiness audit
The audit that follows lists every finding to closure, by date.
Reproduced first
A finding is reproduced against a live instance where it can be, then fixed, then guarded. Where it cannot be reproduced, it is traced to the line that causes it.
A worked example
Our own booking product, audited twice in 18 days
The same review, run on the booking and enquiry receptionist, a product we own and operate. The first audit found 33 problems and all were closed. Then the code changed, so it was audited again, and the second audit said it was not ready.
-
Repository review
Six ways in, two of them reproduced on a running system
A review of the whole repository. Among the findings:
- One patient could cancel another patient's appointment, because the cancel checked the appointment's number and not whose it was.
- An uploaded HTML file opened in a frame with no sandbox, one preview away from an admin's login.
- A malformed login token got a server error instead of a refusal.
- One key did two jobs: signing sessions and sealing secrets.
All fixed that week, each with a test that failed before the fix.
-
Go-live audit
33 findings, all closed the same day
Every one closed in the code that day.
-
18 days
Still building
About 32,000 lines change
Work on the product went on. By 14 September about 32,000 lines had changed since the version the audit had read. An audit covers the code it read, so none of the new work had been reviewed.
So the whole audit was run again.
-
Re-audit
The verdict: not ready
Four lanes read the new code in parallel: security and tenancy, the Linux deployment path, the test suites, and the path a customer's message runs. They found nine blockers. Among the findings:
- One business's admin could delete another business's opening hours.
- A shared test link let a stranger act as the owner.
- The test meant to prove that every route needs a login was looking at 1 route of 172.
- A customer was told the team had been informed when nobody had been.
Each was reproduced or traced to a line, then fixed, then guarded with a test that fails on the old code. Most of what was found was fixed the same day, and every test passed.
1,685backend tests passing48calendar tests passingThe audit's own line that night: "still not ready, for fewer reasons".
-
That evening
Deployment
Onto its server, with the open list written down
The product went onto its production server that evening. Some of the last items could only be closed there, and were. By then eight of the nine blockers were closed, and the ninth needed only its release tag. It is in early access today. What is still open sits on one list, each item with its status and what will close it, and the list is worked one item at a time.
What this means for your build: the review is not a certificate handed over once. An audit covers only the code it read, so when the code changes it runs again, and you see what it found, what was fixed and what is still open.
Technical brief
The detail, in two pages
The workflow, the test strategy, the security scope, the hosting options and the hand-over list.