What a good penetration test actually looks like

Most organizations buy their first penetration test because someone told them to — an auditor, a customer, a regulator. That’s a perfectly good reason. But it also means many buyers can’t tell the difference between a genuine assessment and a vulnerability scan exported to PDF with a consultancy logo on the cover. After years of delivering tests (and reviewing other people’s reports during PCI DSS audits), here is what I believe a good penetration test looks like — from the buyer’s side of the table.

It starts with scoping, not tooling

A test without a well-defined scope answers a question nobody asked. Before any traffic is sent, you and the tester should agree on:

Assets in scope — exact hostnames, IP ranges, application URLs, API endpoints, mobile app builds. “Our website” is not a scope.
The question being answered. “Can an unauthenticated internet attacker reach cardholder data?” is a different test from “what can a malicious insider with a standard workstation do?” Both are valid; they are not interchangeable.
Testing perspective — black box (no knowledge), grey box (credentials and documentation), or white box (source code access). Grey box is usually the best value: you pay for finding vulnerabilities, not for the tester spending two days rediscovering your architecture.
Exclusions and fragile systems — production payment flows, legacy hosts that fall over when scanned, third-party services you don’t own and can’t authorize testing against.

If a vendor doesn’t push back on a vague scope, that tells you something about the test you’re going to get.

Rules of engagement protect both sides

The rules of engagement (RoE) document is not bureaucracy — it’s the thing that makes the test legal and safe. It should name the testing window, source IP addresses, emergency contacts on both sides, escalation procedure if something breaks, and how exploitation is handled: is the tester allowed to exploit, or only to identify? May they pivot from a compromised host? What happens if they stumble on evidence of a prior, real compromise? (It happens more often than you’d think, and the answer should be “testing pauses, incident response starts.”)

Methodology: what the tester should actually be doing

A real test is structured. The frameworks differ in vocabulary — PTES, OWASP WSTG for web, OSSTMM, MITRE ATT&CK for scenario mapping — but the phases are consistent:

Reconnaissance and enumeration. Mapping the attack surface: subdomains, exposed services, technology fingerprinting, leaked credentials in public breach data. On internal tests: AD enumeration, share crawling, listening for broadcast protocols.
Vulnerability identification. Yes, scanners are used here — Nessus, Burp, Nuclei. That’s normal. The difference is that scanner output is a starting point, not the deliverable.
Exploitation. Manually validating findings and chaining them. A medium-severity IDOR plus a low-severity information leak is often a critical finding when combined. Chaining is where the value of a human tester lives; no scanner does it.
Post-exploitation. Answering “so what?” — what data was reachable, what lateral movement was possible, how far was domain admin or the crown-jewel system. This is what turns a finding list into a risk statement your management can act on.
Cleanup. Removing accounts, shells, and files created during the test, documented in the report.

The report is the product

You are not paying for the hacking; you are paying for the report. A good one has:

An executive summary in business language — what an attacker can achieve, not a CVE inventory. It should be readable by someone who will never open Burp Suite.
Findings with reproduction steps. Every finding should include evidence (sanitized screenshots, request/response pairs) and steps your own team can follow to reproduce it. If your developers can’t reproduce a finding, they can’t fix it.
Risk ratings with reasoning — CVSS is a fine baseline, but the rating should reflect your context. An “informational” internal-IP disclosure means something different on a PCI DSS-scoped payment page.
Actionable remediation — specific to your stack, not “apply input validation” copy-pasted under every web finding.
An attack narrative. The chain from initial access to final impact, told as a story. This is the single most useful section for engineering teams and the one most often missing.

Retesting closes the loop

A test that ends at report delivery is half a test. Agree upfront on a remediation retest — typically within 60–90 days — where the tester verifies the fixes and issues an updated report. If you’re doing this for compliance (PCI DSS Requirement 11.4 explicitly requires that exploitable vulnerabilities found during penetration testing are corrected and testing repeated), the retest isn’t optional anyway.

Red flags when choosing a vendor

A quote produced without a scoping call.
“Unlimited IPs, fixed price, 3-day turnaround.” Real testing scales with scope.
No named testers or their certifications on request (OSCP, CPTS, CRTO and similar actually indicate hands-on ability).
A sample report they refuse to share (sanitized), or one that reads like scanner output.
No mention of rules of engagement, safe-testing procedures, or insurance.

A penetration test is one of the few security purchases where you’re explicitly paying someone to tell you bad news. Choose a vendor who is set up to find it, document it so you can fix it, and come back to confirm you did.

If you’re planning a penetration test and want to talk through scope first, see the services page or get in touch.