QA Skills Library / web-visual-qa

Web UI Visual QA

Sohail maintains his own QA protocol set and expects them executed, not described. The deliverable is screenshots plus a list of concrete defects you found and fixed — "I checked it" is not a result.

The Iron Law

Never declare a page or component "ready" without rendering it, capturing real screenshots at desktop 1440 / tablet 768 / mobile 390, and performing a self visual audit of those images.

Describing the protocol instead of running it does not count. Neither does a green HTTP 200.

The loop

  1. Render + audit the console. Zero JS exceptions, zero failed requests, no hydration errors.

On a static single-file page the last surviving error is almost always the favicon — Failed to load resource: the server responded with a status of 404 () with no other detail. A favicon is not a real defect, but it is indistinguishable in the console from one, so a report of "zero console errors" is not true until it is gone. Inline it and the page reaches a genuine zero with no extra request:

html <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='8' fill='%23b45309'/%3E%3C/svg%3E">

Worth doing rather than explaining away: it costs one line and removes the sentence you would otherwise have to write every time you hand over a demo ("one console error, but it's only the favicon") — which also trains the reader to ignore console errors. 2. Screenshot all three viewports (full page). 3. Measure layout at the container level, not the page level — documentElement.scrollWidth - clientWidth catches a page that scrolls sideways, but it happily reports 0 while whole columns are hidden inside an inner overflow-x-auto wrapper. Enumerate every scrolling container; see Scroll affordance below. Total page height catches the "10,000px wall of rows" failure. 4. Look at the images. Use the vision tool on the mobile shot specifically — that is where bidi and wrapping defects actually show. A code review will never find them.

Split before you load. Seeing a screenshot is your own multimodal ability, but the loader takes one image at a time and a full-page capture of a long page is a single giant composite — too coarse to read text, and large enough that some endpoints reject the payload. Thumbnail it, and for anything multi-page render one image per page and review them individually rather than one stitched sheet. A full_page capture of a 9,000px dashboard is a thumbnail of nothing: crop the region you actually need to judge, or split it into segments. 5. When scope allows edits, fix, then re-run the same script and compare numbers before/after (for a read-only audit, report defects without editing code). Re-verification is part of the job, not optional polish.

Run scripts/visual_qa.py — it does steps 1–3 and prints a per-viewport verdict. It accepts a URL, an output prefix, and optional basic-auth credentials.

python3 ~/.hermes/scripts/visual_qa.py https://example.com /tmp/vqa [user] [pass]

Local paths inside an SSH tunnel are a different class — use the browser toolset there for interaction (click/type/JS eval) and this script for measured multi-viewport capture.

Reveal-animated pages: a full_page screenshot comes back mostly black

A marketing site built from hero + scroll-reveal sections (Reveal / IntersectionObserver / a parallax hook) renders everything below the fold at opacity: 0 until it scrolls into view. Playwright's full_page=True captures the whole document without ever scrolling it, so those sections never reveal — and you get a 1,000px+ screenshot of pure black between the hero and the footer that reads exactly like "the site is missing all its content". That was reported as a defect in one session while the page was in fact complete.

Scroll the whole document before you capture, then return to the top:

async def autoscroll(page, step=500, pause=140):
    y = 0
    h = await page.evaluate("document.body.scrollHeight")
    while y < h:
        y += step
        await page.evaluate(f"window.scrollTo(0,{y})")
        await page.wait_for_timeout(pause)
    await page.evaluate("window.scrollTo(0,0)")
    await page.wait_for_timeout(600)

Make it a number rather than an eyeball — count what is still hidden after the scroll:

() => [...document.querySelectorAll('*')]
    .filter(e => getComputedStyle(e).opacity === '0' && e.getBoundingClientRect().height > 120).length

The byte-size tell does not catch this one. "Blank page ≈ 15–20 KB" applies to a failed render; an animation-gapped capture is large — the visible sections are real photos and gradients — so a 500 KB all-black-gap shot passes the size, textlen, and pageerror checks while the picture is wrong. Only the vision pass on the actual PNG catches it, which is why step 4 of the loop is not optional polish.

scripts/reveal_aware_capture.py runs the autoscroll + hidden-count + pageerror capture + full-page shots in one pass for a list of paths and viewports.

"errors = 0" does not mean the interaction happened

Proving a flow (add-to-cart toast, cart drawer, checkout step) needs the right element clicked and the right frame captured. Two failure modes, both from one session, both invisible in the console:

Two habits close both holes. First, enumerate the candidate controls before clicking and assert the one you want exists (and how many there are):

() => [...document.querySelectorAll('button,a')]
     .map(b => ({txt:(b.innerText||'').slice(0,18), aria:b.getAttribute('aria-label'),
                 cls:(b.className||'').slice(0,30)}))
     .filter(x => x.aria || /add|cart|سلة|أضف/i.test(x.txt))

Second, assert the post-condition in DOM terms — the state change the flow is supposed to cause, read off the element that owns it (the header cart badge going from empty to 1, the toast text present) — and only then look at the PNG with the vision tool and describe what you actually see. Writing the report sentence from errors=0 alone is how a "verified" flow turns out to have captured the wrong screen.

What this caught on a real page (proof it earns its keep)

A dashboard that looked fine in code was, on mobile, 9,176px tall with every row expanded, Latin numerals scrambled inside Arabic lines, and a cramped header. After fixing: 2,236px, correct number order, no overflow. Three of the three defects were invisible to code review and visible in one screenshot. See references/rtl-arabic-ui-pitfalls.md.

Blank SPA: when the console hands you an empty error

A page that renders nothing while document.title is right and the JS bundle answers HTTP 200 is a runtime crash during mount, not a loading problem. The trap: the console read comes back as {"message":"", "source":"exception"} — no message, no stack — and calling location.reload() from that same context kills the target (CDP error: Inspected target navigated or closed), destroying the only evidence. You cannot attach a listener after the fact either: the bundle already ran and failed.

Re-install the listeners before the module executes, in a probe page that loads the same bundle, dropped into the served build dir (works with wrangler pages dev dist):

<!doctype html><html><body><div id="root"></div>
<script>
window.__errs = [];
addEventListener('error', e => window.__errs.push({m:e.message, f:e.filename, l:e.lineno, c:e.colno,
                                                    st:(e.error && e.error.stack) || ''}));
addEventListener('unhandledrejection', e => window.__errs.push({m:String((e.reason && e.reason.stack) || e.reason)}));
</script>
<script type="module" src="/assets/<hash>.js"></script>
</body></html>

Navigate to it, then read window.__errs — full stack, one shot, no reload.

Pitfall: an SPA _redirects rule (/* /index.html 200) rewrites /probe.html → /probe and can serve index.html instead of your file. Assert window.__errs !== undefined before trusting anything it reports — a probe that silently never ran looks exactly like "no errors".

Then prove the cause, don't just name it. A second probe patches window.fetch before the bundle and rewrites the suspect response; if the page then mounts, causality is established rather than inferred:

const _f = window.fetch;
window.fetch = (...a) => _f(...a).then(r => String(a[0]).includes('/api/settings')
  ? r.json().then(j => (j.whatsappNumber = String(j.whatsappNumber),
      new Response(JSON.stringify(j), {status: 200, headers: {'Content-Type': 'application/json'}})))
  : r);
  ```

  **Playwright-native variant: `page.route()` — no probe file, no `_redirects` hazard.** The probe-HTML
  trick needs a file dropped into the served build dir and then a check that it actually ran. If you
  are already driving the page from a Playwright capture script, intercept at the network layer
  instead — same causality, and it re-runs identically every time:

  ```python
  async def patch_json_field(pg, url_glob, key, fn):   # rewrite one field on one API route
  async def h(route):
    r = await route.fetch()
    try:
        j = await r.json()
    except Exception:
        return await route.fulfill(response=r)    # not JSON — pass through untouched
    j[key] = fn(j.get(key))
    return await route.fulfill(status=r.status, body=json.dumps(j),
                               headers={"content-type": "application/json"})
  await pg.route(url_glob, h)

  await patch_json_field(pg, "**/api/settings", "whatsappNumber", str)   # then capture
  ```

  Ship it as a **mode flag on the capture script** (`raw` vs `fixed`) writing two parallel output
  trees, so the deliverable is the broken reality *and* the rendered-after-patch comparison side by
  side — `qa/ui/<vp>/<page>.png` (blank) next to `qa/ui/fixed/<vp>/<page>.png` (full content). Far more
  persuasive than prose, and it self-checks: **if `fixed` is also blank you misdiagnosed**, and you find
  out before reporting instead of after.

  **Triage signal: which route survives.** When every route is blank *except one*, the crash is in a
  **shared component, not the router**. Real case: `/`, `/perfumes`, `/skincare`, `/consult`,
  `/perfume-finder` all blank at all three viewports while `/admin` rendered a working login form.
  `/admin` renders a stripped-down shell that never mounts the Header/Footer — and therefore never
  calls `whatsappNumber` — so it survived. Ask *"what does the surviving route NOT render?"*; that is
  your suspect, and it explains why one type error killed the entire React tree.

  Concrete end-to-end recipe (blank census, error grouping, `FINDINGS.md` shape, verification checks):
  `references/multi-page-regression-capture.md`.

  ### The usual root cause is a JSON *type*, not a rendering bug

A backend that stores settings as text and does `JSON.parse(row.value)` silently converts the digit
string `"201000000000"` into the **number** `201000000000`. The frontend then runs
`phone.replace(/[^0-9]/g,'')` and throws `TypeError: o.replace is not a function` — the entire SPA
dies on first paint. **Every endpoint still returns 200 the whole time.** So audit types, not
status codes:

```bash
curl -s localhost:8788/api/settings | python3 -c \
  "import sys,json;d=json.load(sys.stdin);[print(f'{k:18}{type(v).__name__:8}{str(v)[:40]}') for k,v in d.items()]"

Any digit-only text field coming back as int/float is this bug. Same family: null where a string is expected, an object where a scalar is, and present-but-wrong fields — a value that is a number only in the seeded row, so it passes every test written against hand-written fixtures.

Related false green: a route that does not exist yet is answered by the SPA fallback with HTTP 200 and an HTML body, which a status-only endpoint sweep records as a pass. Check the Content-Type (or isinstance(body, dict)) on every JSON route.

Fixing it permanently is five layers, not one

Diagnosing the type is the easy half. A one-line client patch (.replace → String().replace) makes the page render, and is then re-introduced by the next seed, the next settings write, or the next hand-written fixture. The fix is a stack, and each layer belongs in the report:

layer what it is proof
1. the live row prepare a dry-run and get explicit approval before rewriting the stored value to the correct JSON type re-read via SQL and the public API
2. the API boundary decode/coerce every settings field as it leaves the Function raw endpoint JSON shows str, not int
3. the client boundary one normalizeSettings() / toText() applied to every field on every read a unit test feeding each bad shape
4. the seed generator the script that emits seed.sql, when the seed is generated regenerate + git diff shows the fix
5. a regression test every value shape, through both the boundary and the display path the new tests plus the pre-existing suite

Two details that are easy to get wrong:

Layer 5 should enumerate shapes, not fields — a bare number, null, "", an already-quoted string ("\"…\""), a boolean — and push each through both the coercion helper and the URL/display builder. That is what catches the second, invisible half of the bug: a coercion that stops the crash but leaves literal quotes in the rendered label.

Then grep for every other unguarded call on the same field before declaring it done (grep -rn '\.replace\|\.trim\|\.split' src/ filtered to settings consumers). The call site that crashed is only the first one on the unlucky path; the next one is waiting behind a flag.

Full worked example with the actual commands, the toText() helper, the generator diff and the shape-parameterised test: references/layered-settings-type-fix.md.

Re-verifying after someone else's hotfix: "type fixed" ≠ "value fixed"

When the fix lands outside your session (a DB/data edit, a client deploy, a fix you dispatched to another agent), you are asked to verify, not to fix. Treat it as verification-only: re-run the project's existing capture script rather than writing a second one (two scripts = two definitions of "blank"), and let only your own QA artifacts change — not project code, not the deploy.

Answer two questions separately, because they disagree in practice:

  1. Does it render? blank_pages / page_errors in summary.md + console-report.json.
  2. Is the value correct? the API type and the value as displayed to a user.

Real case: the type flipped number → string and the whole site came back (0 blanks, 0 pageerrors across 18 shots) — but the string content still had literal quotes baked in: {"whatsappNumber":"\"201000000000\""} (length 14 = 12 digits + 2 quotes).

So probe both, and never let a clean href excuse a dirty label:

await pg.eval_on_selector_all("a[href*='wa.me']", "e=>e.map(x=>x.href)")        # functional
await pg.eval_on_selector_all("a[href*='wa.me']", "e=>e.map(x=>x.outerHTML)")   # displayed

A broad text-walk can contradict the screenshot — trust the exact element + the crop. In that session a querySelectorAll('body *') leaf filter read innerText and reported the clean number, while the same anchor's outerHTML and a cropped footer shot both showed the quotes. Anchor cosmetic checks on the selector that owns the value and read outerHTML/textContent, then corroborate with a crop: two independent channels agreeing beats one DOM walk, and a finding seen in only one of them is a lead, not a reportable defect.

Also confirm the artifacts are fresh before believing them — generated_at vs every PNG's stat -c '%y', the report mode (a fixed-mode report proves nothing about production), and byte size as a content tell (blank full-page ≈ 15–20 KB, real content 250–900 KB; /admin is legitimately small everywhere because it mounts a stripped shell that never touches the crashing field).

Report shape for this pass: a pages × viewports table (blank/not plus textlen/elems as the why), the remaining-errors count per channel (state 0 explicitly), freshness, and the paths to the requested screenshots. Full session detail and the probe recipes: references/post-fix-reverification.md.

Stop shipping the white screen: make the render check a blocking gate

Everything above is forensics you run by hand. The durable fix is to fold the same three assertions (min visible text under #root, any pageerror, any same-origin status >= 400) into the project's gates.sh, so a blank page or an uncaught JS error fails the pipeline before a PR exists.

Full recipe (the wrangler pages dev + local D1 seeding sequence, the CI browser install, why the gate must be separate from typecheck/tests): references/render-smoke-gate.md.

Scope boundary: screenshots, viewport/RTL, layout and console forensics → here. Endpoint sweeps, DB persistence proof, authorisation matrix, cart/coupon/checkout E2E → fullstack-functional-qa.

Audit checklist

Reviewing information hierarchy and interactive feedback

A UI can render, pass every viewport check, show correct numbers — and still be wrong. When the user says "the flow and the UX have errors", they mean the page does not answer the user's question in the right order. Screenshots reveal this; a passing checklist does not.

Audit these five, in this order:

  1. What does the eye land on first, and is it the most important thing on the page? Put a non-product element in the prime slot and the whole page reads wrong. A real case: a "demo scenario — open the simulator" promo banner sat directly above the owner's KPIs, so a company owner opened their dashboard and saw a demo helper instead of their own results. Demo affordances belong below the user's data, or in a slim strip — never above it.
  2. Does the page have a hierarchy, or is everything equal weight? Eight identical KPI cards in a 4×2 grid is no hierarchy: nothing dominates, so the user cannot answer "am I on track?" at a glance. Promote the one or two numbers that answer the page's central question (here: revenue vs target, projected cash) into a visually larger card; demote the rest.
  3. If the page is interactive, is the change visible? This is the most commonly missed defect, and on a simulator it destroys the point of the feature. Sliders updated the numbers but gave no signal that anything moved — no flash, no delta, no colour change — so the user had to remember the previous value. An interactive control whose effect is invisible is a broken feature. Every derived number needs a change affordance: a transient highlight plus an explicit before→after or ▼ −19,600 delta, and semantic colour (a funding gap goes red, a surplus green).
  4. Is the same number shown twice? Duplicated metrics read as two different facts and undermine trust. Real examples from one page: the same pipeline total in two cards; one margin figure in two cards; the discount/deposit pair echoed in three places. Each value belongs in exactly one place — the rest should reference or link, not restate.
  5. How many warnings are competing? Four simultaneous caution banners (simulation-mode notice, a note under the CTA, an approval-threshold rule, a minimum-threshold rule) means none of them is read. Keep the one that changes behaviour; demote the rest to a tooltip or a quiet line.

Also check for inputs disguised as results — a read-only-looking row inside a results card that is actually a driver of the calculation (e.g. "setup cost paid before start, 40,000"). Misplaced inputs make the model of the screen unlearnable; make it editable or label it as an assumption.

The aesthetic bar: "professional" is not "wow", and colour is not the gap

A redesign can land its palette, tokens, dark mode, and component kit correctly and still read as an ordinary admin dashboard. Sohail's client judges by exactly this gap — «الديزاين أهم نقطة… عايزين شكله شيك وبروفيشنال وكأنه موقع واو», and separately «المهم نجيب الشغلانة» — i.e. the visual impression is what wins the contract, not the feature depth.

When a UI passes every functional and viewport check but still feels merely competent, the cause is almost never the colour scheme. Audit these three, which are what separate the two tiers:

  1. Breathing room. Card padding ≥ 24px, ≥ 20px between cards, ≥ 32px page gutters. Dense dashboards read as "internal tool"; the same data with air reads as "product".
  2. One hero number per screen. 48–64px, heaviest weight, accent colour — everything else 25% smaller. Every number at the same size is the single most common reason a dashboard has no hierarchy, and it is invisible to a functional test.
  3. Fewer elements. If a screen has more than ~7 primary elements, split it into tabs or an accordion rather than shrinking them. Deleting a card that adds no decision is a design fix.

State the gap explicitly when reporting, and quantify it — "structurally improved, still not wow, here's what's missing" is a useful report; "looks good" is not.

Turn "still too subtle" into a number you can re-measure

When the owner or an independent reviewer bounces a design fix with an adjective — «الوضع الفاتح لسه شبه Excel», «مش واو لسه», "the shadow doesn't read", "cards still look flat" — do not answer with another screenshot and an opinion. Build a two-number instrument; both numbers are cheap, and both are re-runnable after the fix so the second report is a delta, not a vibe.

1. Token separation. Read the computed custom properties off the live DOM and take the max per-channel delta:

const rs = getComputedStyle(document.documentElement);
const rgb = v => rs.getPropertyValue(v).trim().split(/\s+/).map(Number);
Math.max(...rgb('--card').map((c, i) => Math.abs(c - rgb('--background')[i])));   // target ≥ 8

A delta of 1–4 units is not perceptible, and that number is the whole content of a "still looks flat" complaint about a card-on-page design. The first attempt at such a fix moved #FFFFFF → #FEFDFB (delta 4) and read as unchanged — correctly.

2. Shadow strength. Sample 1px-tall luminance bands at increasing offsets below a card's bottom edge:

OFFSETS = [1, 2, 3, 5, 8, 13]        # all inside the inter-card gap, never the next card
prof = [mean_luminance(img, x0, x1, base + o, base + o + 1) for o in OFFSETS]
strength = max(prof) - min(prof)     # 0 = no shadow visible at 1×

Self-referential — darkest at the edge, rising outward — so it needs no clean background patch and cannot be fooled by page colour. A shadow whose spread is close to minus its blur (0 12px 30px -22px …) collapses this profile to flat and only appears once you zoom the PNG. That is exactly why a before/after crop at 2× is bad evidence: it proves the CSS declaration, not that a human sees anything. Hand over a full-scale number instead.

3. Read the change back from the running service, not from the file. Design tokens (and their siblings: config values, feature flags, permission policy) are read at process start, so after editing, restart and re-read the effective value:

curl -s -u "$U:$P" http://127.0.0.1:4096/config | jq '.permission'     # the live value, not the file

A file that says the right thing while the process still holds the old value looks identical from the source alone — the read-back is the only proof a change is actually live.

Report as before → after numbers, and pixel-diff any theme you were told not to touch (max delta ≤ 2). Run scripts/design_delta_probe.py to get both measurements in one pass.

Verify WHICH build you screenshotted — rendered styles, not source tokens

Grepping the source for the palette proves the tokens are declared, not that they rendered. Two ways this produces a confident false verdict:

So read the computed styles out of the live DOM, and name the build you are looking at:

const rs = getComputedStyle(document.documentElement);
['--color-primary','--color-accent','--color-background'].map(v => [v, rs.getPropertyValue(v)]);
// then sample real elements
getComputedStyle(document.querySelector('aside')).backgroundColor

Corroborate with a size tell: the redesigned page was 474 KB of HTML vs 147 KB for the pre-redesign build — a byte-size difference is often faster evidence than any screenshot that you are looking at the new build at all.

Cheapest and sharpest: compare the content hash of the built bundle to the one the page actually served. npm run build prints dist/assets/index-<hash>.js, and a pageerror names the same assets/index-<hash>.js. Equal hashes ⇒ the deployed preview is the current source, so the defect is live code and not a stale deploy — one line of output, no screenshot, no guesswork:

fresh build: index-DL319oSG.js  |  deployed pageerror: index-DL319oSG.js   ⇒ same build, bug is live

Screenshot staging (the live-build env) when judging a design change; screenshot production only to confirm what users currently see.

Verify a vision model's textual claims against the raw source

When your visual pass reads text off a screenshot — a typo, a wrong label, a mislabelled button — confirm it against the rendered HTML or the source string before reporting it. A VLM reading a dense RTL screenshot hallucinated a spelling error that did not exist (السهولة for السيولة); one grep of the live page showed the correct text four times. Reporting it would have sent the user chasing a non-bug, and a wrong finding costs more trust than a missing one.

curl -s https://<host>/ | grep -oE 'السيولة المتوقعة' | sort | uniq -c

Layout, hierarchy, and "something looks off" judgements are what a vision pass is for. Treat its content claims as a lead to verify, never as a finding.

Your own probe can lie too — anchor measurements to the container, not the value

Every section above warns that the delegated agent's report can be wrong. The inverse also happens, and it is worse: a verifier that measures the wrong element produces a confident false negative, which sends the owner back to redo work that is already correct (or, worse, gets reported to them as "not done").

A real case. The acceptance criterion was "the three pipeline numbers now sit in one compact row". The probe collected the first DOM occurrence of 120,000, 90,000 and 60,000, read their y positions, and printed sameRow: false, spanY: 1070. Verdict: not fixed. Wrong. Each number appears several times on that page — 120,000 first inside a decision card at y=998, 90,000 first inside the pipeline row at y=2048 — so the probe had compared three unrelated elements.

Select the structural container the criterion describes, then measure its children:

// ✗ text-anchored: grabs the first match of each value, wherever on the page it lives
document.querySelectorAll('*').forEach(e => {
  if (!e.children.length && e.textContent.includes(n)) { /* record rect */ }
});

// ✓ container-anchored: find the row the criterion is about, then measure its cells
let row = null;
document.querySelectorAll('div[class*=grid-cols-3]').forEach(g => {
  if (g.querySelectorAll(':scope > a').length === 3 && !row) row = g;
});
const r = row.getBoundingClientRect();
row.querySelectorAll(':scope > a').forEach(a => { /* per-cell x / w / h */ });

Rule: for any criterion shaped "X is now one row / one card / N equal columns / a compact strip", locate the container first and measure its children. Never measure free-floating values and infer the layout from their coordinates.

Two habits that catch this before it reaches the user:

Report the raw numbers and the selector you measured, so a wrong anchor is visible in the report instead of hidden behind a verdict.

Scroll affordance: clipped content is a defect even when the page doesn't overflow

documentElement.scrollWidth - clientWidth === 0 is a false negative for the most common real layout bug on this class of dashboard. A kanban board, a wide table, or a tab strip almost always lives inside an overflow-x-auto wrapper, so the inner content is clipped while the page stays perfectly contained. Measured on a live pipeline page after a "0 overflow" report:

Viewport Container Hidden Scrollbar rendered
1440px overflow-x-auto pb-4 724px (scrollWidth 1808 · clientWidth 1084) 0
390px same 1430px (1788 · 358) 0

Two whole kanban stages — the won and lost columns, i.e. the closed deals — were unreachable on desktop, and an independent reviewer found it in one measurement. Enumerate scrolling containers per viewport:

() => [...document.querySelectorAll('*')].flatMap(e => {
  const cs = getComputedStyle(e);
  if (!/auto|scroll/.test(cs.overflowX)) return [];
  const hidden = e.scrollWidth - e.clientWidth;
  if (hidden < 30) return [];
  return [{ cls: e.className.toString().slice(0, 70), hidden,
            scrollbar: e.offsetHeight - e.clientHeight }];
});

Judge on two signals, never one:

Same pass, same script, cheap counts worth a table: tap targets under ~40px (icon buttons, language/theme toggles — a 24px toggle is a real defect on the phone the client will use) and body text under 13px.

Third cheap count: truncated <input> values. A text field inside a flex row shares the row with buttons, so its content is silently clipped — and because an input has no ellipsis and no scrollbar until focused, the user simply cannot read the value they typed. This is worse than a clipped container: the data looks wrong, not just cut off. The owner of a shop app reported it from the phone («لو اسم البند كبير مش بيظهر على التليفون»); measured on the live page with a 50-character value:

() => { const el = document.querySelector('.line-name-input'); const r = el.getBoundingClientRect();
        return { clientW: Math.round(r.width), scrollW: el.scrollWidth,
                 hidden: el.scrollWidth - el.clientWidth }; }
Viewport Field width Hidden
390px 90px 277px — roughly three-quarters of the name invisible
768px 468px 0 — fine, which is why nobody caught it on a laptop

Count them across the page the same way as tap targets, and probe each of the three viewports — the tablet passes while the phone fails, so a single-width check reports success.

The fix is layout, not typography. text-overflow: ellipsis does not apply usefully to an <input>, and shrinking the font is not a fix. Let the field have the row to itself on small screens and push the controls beneath it:

@media (max-width: 600px) {
  .line-row       { flex-wrap: wrap; row-gap: 8px; }
  .line-info      { flex: 1 1 100%; min-width: 0; }   /* name gets the full width */
  .line-qty-control { flex: 0 0 auto; margin-inline-start: auto; }
}

Measured effect: field width 90px → 288px, hidden 277px → 120px. A field four times wider does not always fit a pathological name — say so rather than claiming a total fix, mirror the full value into title so it is readable on hover/long-press, and offer the real wrap-to-two-lines fix as a follow-up the owner can choose.

Fix at the component level, not per page. When the pattern repeats — a kanban board and three tables on one screen all clipping with no hint — the answer is one shared scrollable-container primitive with a default affordance, not a per-page patch. Run scripts/scroll_affordance_audit.py <url> [viewport ...] to get the whole table in one pass.

When you write a value into a CSS variable, prove something reads it. A .js config block that sets --accent is a silent no-op if the stylesheet actually consumes --amber (or --brand, --primary). Nothing errors, the page renders, and the "branding is now configurable" claim is false. One grep before you report success:

grep -c 'var(--accent' style.css     # 0 ⇒ nothing consumes it; find the real variable name
grep -oE '^\s+--[a-z-]+:' style.css | sort -u    # the variables this file actually declares

Then set every name the app really uses, and confirm the computed value in the live DOM (getComputedStyle(document.documentElement).getPropertyValue('--amber')).

Tablet is its own viewport, and "correct" can still be stretched

Zero overflow, correct numbers, no console errors — and the owner still bounces it: «هي كدة على التابلت هيبقى تمام؟» with a photo. The failure is not clipping, it is no constraint: at 768/1024 a single-column layout keeps stretching, so rows become long thin bars with dead space and the stat row spreads to unusable columns. Measured on a live page before the fix:

Viewport Line row width Stat columns
1024px 966px 326px each
768px 710px, 66px tall —

Desktop and mobile both looked fine, which is exactly why a two-viewport check misses it. Add tablet to the loop and measure container width and column count, not just overflow:

() => { const row = document.querySelector('.line-row'),
             lc  = document.querySelector('.lines-container');
        return { rowW: Math.round(row.getBoundingClientRect().width),
                 cols: getComputedStyle(lc).gridTemplateColumns }; }

Two fixes, either acceptable — state which you applied:

Acceptance number worth writing into the brief: a repeated row should not exceed ~820px at tablet width, and a stat row should not stretch past ~3 fixed columns.

Concurrent / multi-account testing

Isolated Playwright browser contexts are separate cookie jars — that is how you simulate 2–4 logged-in users side by side in one script. Never rely on one context with cookie clearing. See references/concurrent-account-testing.md for the failure-mode matrix that matters (disconnect mid-round, late join, double-click, server-authority races).

Playwright traps that cost real time

wait_until="networkidle" never fires on an app that holds a live connection or a polling loop. A page with SSE / WebSocket (agent UIs, anything with push updates) never goes idle — and neither does an admin dashboard on setInterval fetch that re-hits its APIs every few seconds for ever. The navigation times out at 60s with an error that points at the URL, not the wait condition; the symptom reads like "the site is unreachable" when the site is perfectly fine. Polling is the more common of the two, so reach for domcontentloaded on any dashboard-shaped route without first proving it settles.

# ✗ times out on any SSE/WebSocket page
await page.goto(url, wait_until="networkidle")
# ✓ what you actually want
await page.goto(url, wait_until="domcontentloaded")
await page.wait_for_timeout(2500)   # let the client hydrate, then assert on content

The same property is why a reverse proxy in front of such an app needs flush_interval -1 (Caddy) — otherwise the proxy buffers and the stream never arrives.

add_init_script is scoped to the context, so it authenticates every page you open in it. To screenshot an admin dashboard whose real login endpoint is down, the standard trick is to seed a session token into localStorage with context.add_init_script(...). The trap: the very next shot you take — "the login gate, as an anonymous visitor", via context.new_page() — runs with that same seeded token and silently renders the dashboard again, so admin-login-gate.png and admin-dashboard.png come out byte-identical.

Two habits catch it:

Take the session-dependent shots (the dashboard and its tabs) in the seeded context, and prove each tab actually switched by comparing len(page.inner_text("body")) before/after the click — a tab whose length never moves did not switch. Full worked script: references/authed-dashboard-capture.md.

fetch() throws Request cannot be constructed from a URL that includes credentials when the page was loaded as user:pass@host — a relative fetch('/api/x') resolves against a document URL carrying credentials and the browser refuses. This only reproduces under credential-in-URL navigation, so it is easy to miss locally. When auditing an app that fetches on load:

// ✓ absolute from origin — immune to credential-in-URL navigation
const API = location.origin;
fetch(API + '/api/overview')

If you find this, the fix belongs in the page (build API URLs from location.origin), not in the audit script — the app is genuinely broken for anyone who logs in via URL.

page.screenshot(clip=…) is viewport-relative unless you also pass full_page=True. Passing a clip whose y is below the fold does not scroll — it raises Error: Clipped area is either empty or outside the resulting image, with no hint that the missing ingredient is full_page. This bites exactly when you are trying to capture one region of a long page to hand to the user:

# ✗ clip y=1976 on a 1000px viewport → "Clipped area is either empty or outside"
await pg.screenshot(path=p, clip={"x": 0, "y": 1976, "width": 1440, "height": 360})
# ✓ full_page gives one tall capture, then the clip selects the region from it
await pg.screenshot(path=p, full_page=True, clip={"x": 0, "y": 1950, "width": 1440, "height": 420})
# ✓ equally valid: scroll the element into view first, then clip viewport coords

Region clips are the right delivery format — a full_page shot of a 2,500px dashboard is unreadable when sent to a phone. Clip to the thing the request was about (the row, the tab, the card).

Setup on a fresh machine

The toolchain is not preinstalled. One command covers it:

pip install playwright beautifulsoup4 lxml pytest pytest-asyncio && playwright install chromium

Chromium may already exist under ~/.cache/ms-playwright/ (other tools download it). Check there before re-downloading. Reusable Playwright code belongs in scripts/, run it through execute_code/terminal rather than re-deriving the API each session.

Honest capability statements

When asked what you can reach, answer with a verified table, not a promise — and pair every gap with its setup step, never as a permanent refusal:

Surface Status To enable
Static HTML, JSON APIs works —
JS/SPA pages, authenticated flows works (Playwright + browser toolset) pip install playwright
Network interception, multi-context works (Playwright) —
Arbitrary in-page JS eval, console read works (browser toolset) —
Public social pages (Facebook et al.) metadata only — see below credentials for interactive content
Viewing a local image file needs the loader tool auxiliary.vision provider — see hermes-prompt-internals → references/auxiliary-provider-gating.md
Android device QA needs a device install adb, attach device over USB/TCP
Telegram bot QA needs the MCP server add telegram-mcp to mcp_servers
Windows desktop QA needs a Windows host —

State which of these you have personally executed this session versus which are untested projections, and say which is which.

Social platforms are a different reach class — measure before promising

Verified against facebook.com (logged out): the site loads and the cookie dialog is dismissible, but the useful boundary is the login wall, not the network.

What Reality
Site loads, cookie/consent flow works — dismiss it, then read the shell
Public page metadata works: page name, follower count, intro text, review summary, photo count, tab labels
The post feed itself login-walled — a See more from … dialog covers it; dismissing it yields metadata only
Logging into a personal account needs the user's credentials and risks a platform checkpoint/lockout on automated login

Practical guidance to pass on:

raw Markdown · all skills