Skip to content

Manual Test Runbook — G3: Report renderer

Owner: Sagar  |  Time: ~15 min  |  Sandbox: none required — fixture-driven

Purpose

Validate that the Markdown renderer produces a report with all required sections, severity-sorts findings, groups the roadmap by remediation_asset_id, and that Pandoc can convert it to PDF.

Prerequisites

  • Node 20+, npm
  • Pandoc + xelatex installed (brew install pandoc basictex on macOS; apt-get install pandoc texlive-xetex on Linux)

Steps

1. Snapshot test on the Markdown renderer

cd apps/discovery-auditor
npm ci
npm test -- --testPathPattern markdown.test
  • All tests pass (cover, exec summary, severity sort, asset grouping, error appendix)

2. End-to-end PDF generation from a fixture report

node -e "
import('./dist/render/markdown.js').then(async (m) => {
  const fixture = JSON.parse(require('fs').readFileSync('tests/fixture-report.json','utf8'));
  process.stdout.write(m.renderMarkdown(fixture));
}).catch(console.error);
" > /tmp/report.md

pandoc /tmp/report.md -o /tmp/report.pdf --pdf-engine=xelatex -V geometry:margin=1in --toc
open /tmp/report.pdf   # or xdg-open

(Skip this step if you don't have a fixture-report.json; run G4 to get a real one instead.)

  • PDF renders without LaTeX errors
  • Cover page shows subscription name, run ID, operator
  • Executive summary shows secure score and severity counts
  • Findings table is severity-sorted (critical first)
  • "Prioritized Roadmap" section groups by asset (e.g., ### Asset F5, ### Asset N5)
  • Roadmap is sorted by criticality (highest weighted severity first)

3. Visual review checklist (deliverable quality)

  • Cover branding visible (will be added via --metadata-file once branded LaTeX template is built — track as follow-up)
  • No raw {{ }} template placeholders
  • No collector errors leak into the body — only into Appendix
  • Every offender count matches the JSON

4. Failure-mode test — corrupt Markdown

echo "# only a header" | pandoc -o /tmp/bad.pdf --pdf-engine=xelatex
  • Pandoc returns 0 (this is the happy degenerate case — we want the workflow to never block on minimal MD)

Pass criteria

  • Snapshot tests green
  • PDF renders successfully from a real fixture
  • Roadmap correctly groups by remediation_asset_id
  • No template placeholders leak through

Failure modes & escalation

Symptom Likely cause Action
! Package fontspec Error: The font ... cannot be found. xelatex missing fonts Install texlive-fonts-extra (Linux) or mactex full (macOS)
pandoc: pandoc-citeproc: createProcess: posix_spawnp ... not found Stale Pandoc install Reinstall Pandoc from official tarball
Roadmap empty No failing findings (clean sub) Reseed a fixture violation; not a renderer bug

Follow-ups

  • Branded LaTeX template + cover artwork (track as G3 v1.1 — not a M1 blocker).
  • Playwright HTML→PDF fallback if Pandoc becomes a constraint for embedded charts.

Sign-off

  • Tester: ___  |  Date: _  |  Result: PASS / FAIL / N/A
  • Notes: