CyberLearn
← Back to foundations
Foundations

HTTP Headers Scanner

Audit a URL's response headers for missing or weak security controls

Log in to track progress
3–5hPython

What you'll learn

  • HTTP fundamentals
  • Security headers (CSP, HSTS, X-Frame-Options)
  • httpx requests
  • Scored audits
██╗  ██╗███████╗ █████╗ ██████╗ ███████╗██████╗ ███████╗
██║  ██║██╔════╝██╔══██╗██╔══██╗██╔════╝██╔══██╗██╔════╝
███████║█████╗  ███████║██║  ██║█████╗  ██████╔╝███████╗
██╔══██║██╔══╝  ██╔══██║██║  ██║██╔══╝  ██╔══██╗╚════██║
██║  ██║███████╗██║  ██║██████╔╝███████╗██║  ██║███████║
╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝╚═════╝ ╚══════╝╚═╝  ╚═╝╚══════╝

Cybersecurity Projects Tier: Foundations Python 3.13 License: AGPLv3 Tests Lint HTTP Client

Fetch a URL once and grade its HTTP security headers A through F using the same weighted-rubric model as Mozilla Observatory.

This is a quick overview, security theory, architecture, and full walkthroughs are in the learn modules.

[!NOTE] Foundations tier, this project is built for someone who has never written Python before. The source code is heavily commented as a teaching aid, the learn/ folder explains every concept from zero, and the whole tool is one readable file. If you already know Python, the natural next step is PROJECTS/foundations/password-manager, the hardest foundations project, which adds Argon2id, AES-GCM, and on-disk state.

What It Does

  • Performs one polite HTTPS request to the URL you provide and inspects the response headers
  • Grades six security-critical headers against a weighted rubric (high = 30 pts, medium = 15 pts, low = 5 pts)
  • Reports each finding as ok, weak, or missing with a one-line explanation of why
  • Computes a 0 to 100 score and maps it to an A through F letter grade (90+ = A, 80+ = B, etc.)
  • Catches subtly broken values like Strict-Transport-Security: max-age=0 (header present, actively disabled) and flags them as weak, not ok
  • Follows redirects and grades the final URL, the one your browser would actually land on
  • Prints a colored Rich table plus a grade panel plus a recommendation list for every non-ok finding
  • Returns meaningful exit codes: 0 for A/B, 1 for C/D, 2 for F or network error, useful in CI pipelines

The Headers It Grades

HeaderSeverityWhat it stops
Strict-Transport-SecurityhighSSL stripping on coffee-shop wifi
Content-Security-PolicyhighXSS via injected <script> tags
X-Content-Type-OptionsmediumMIME-sniffing of uploaded files
X-Frame-OptionsmediumClickjacking via hidden iframes
Referrer-PolicylowLeaking secret tokens via the Referer header
Permissions-PolicylowCompromised third-party scripts abusing camera/mic/etc.

Every header maps to a real attack class with a real history. The 01-CONCEPTS.md module walks through each one with concrete attack examples.

Quick Start

./install.sh
just run -- https://example.com
# Grade: B, Score: 85 / 100  (example.com is missing CSP and Permissions-Policy)

[!TIP] This project uses just as a command runner. Type just to see all available commands.

Install: curl -sSf https://just.systems/install.sh | bash -s -- --to ~/.local/bin

Demo URLs

Try these, each demonstrates a different grading path:

URLExpected gradeWhy
https://github.comAComprehensive CSP, HSTS with includeSubDomains, almost every header set
https://web.devAGoogle's own developer-docs site, full modern header set
https://mozilla.orgAMozilla practices what Observatory preaches
https://example.comB / CHas HSTS, but missing CSP, Permissions-Policy, and others
http://neverssl.comFIntentionally serves plain HTTP, no security headers at all
just run -- https://github.com
just run -- https://example.com
just run -- https://web.dev --timeout 5
just run -- http://neverssl.com

[!IMPORTANT] Always include the http:// or https:// scheme. The scanner refuses bare hostnames like github.com because it cannot guess which scheme you meant, and guessing wrong is exactly the SSL-stripping problem HSTS exists to prevent.

Sample Output

                  Headers for https://github.com/ (HTTP 200)
┌─────────────────────────────┬─────────┬──────────┬─────────────────────────┐
│ header                      │ status  │ severity │ note                    │
├─────────────────────────────┼─────────┼──────────┼─────────────────────────┤
│ Strict-Transport-Security   │ ok      │ high     │ Present and contains... │
│ Content-Security-Policy     │ ok      │ high     │ Present                 │
│ X-Content-Type-Options      │ ok      │ medium   │ Present and contains... │
│ X-Frame-Options             │ ok      │ medium   │ Present                 │
│ Referrer-Policy             │ ok      │ low      │ Present                 │
│ Permissions-Policy          │ missing │ low      │ Header ... is not set   │
└─────────────────────────────┴─────────┴──────────┴─────────────────────────┘
╭─ Result ───────────────────╮
│ Grade: A                   │
│ Score: 95 / 100            │
╰────────────────────────────╯

Followed by a Recommendations: block for every non-ok finding, with the exact header value to add.

Exit Codes

The scanner returns shell-friendly exit codes so you can wire it into CI:

GradeExit codeMeaning
A, B0Green light, no action needed
C, D1Worth investigating, often acceptable depending on context
F or network error2Hard fail, must fix
just run -- https://my-deployed-site.com
if [ $? -gt 1 ]; then exit 1; fi   # fail the build only on F or error

Tooling

just            # list available recipes
just test       # run pytest (11 tests, runs in under a second, network-mocked with respx)
just lint       # ruff + mypy --strict + pylint
just format     # yapf
just run -- <url>  # scan a URL

Requirements

  • Python 3.13+, the install script will check.
  • uv, modern Python package manager (auto-installed by ./install.sh).
  • just, command runner (auto-installed by ./install.sh).
  • A working internet connection at runtime (the scanner makes one real HTTPS request per scan, but the test suite mocks the network with respx and runs fully offline).

No compilers, no system libraries. The project is one Python file plus tests.

Learn

This project includes step-by-step learning materials covering security theory, architecture, and implementation, written for someone who has never touched Python before.

ModuleTopic
00 - OverviewQuick start, prerequisites, expected output, common first-run problems
01 - ConceptsWhat HTTP is, what a header is, each security header with the real attack it stops (SSL stripping, clickjacking, MIME sniffing, XSS, referer leakage)
02 - ArchitectureThe four-stage pipeline, dataclasses as value objects, the I/O fence pattern (functional core / imperative shell)
03 - ImplementationFunction-by-function walkthrough, every Python feature explained when first encountered, plus test patterns and tooling
04 - ChallengesTwelve extension ideas, from "add a seventh header rule" up through "wrap it in a FastAPI service with rate limiting"

Real-World Context

This scanner is a teaching-scale version of tools that do the same job at production scale:

Once you understand how this scanner makes decisions, those tools become readable instead of magical. The 04-CHALLENGES.md module includes ideas for growing this project toward what Observatory does.

See Also

License

AGPL 3.0