Docs / The test suite

The test suite#

How to run the tests, what the suite covers, the diagnostic/trace tools, and the VICE cross-checking workflow. Run the whole thing any time with npm test.

Current count: more than 2,400 labeled tests across about 350 registered spec files in test/all-test.js's default TESTS array, run by npm test. This is counted from the emitted ok pass lines; a few files also contain unlabelled internal assertions.

Note that the test suite is managed by AI. There might be duplicate tests and tests that don't make sense. They have not been reviewed in detail.

Running the suite#

# Full default suite (357 files, 6-way parallel)
npm test                         # alias for: node test/all-test.js
node test/all-test.js

# Tune concurrency
node test/all-test.js --jobs=8   # more parallelism
node test/all-test.js --jobs=1   # force sequential (isolated failure debugging)

# A single test file (fastest iteration during development)
node test/sid-spec-test.js
node test/vic2-sprite-render-spec-test.js

all-test.js spawns each spec file as its own Node subprocess so failures are isolated and the runner prints PASS / FAIL (ms) per file plus an overall summary; on failure it echoes the last 15 lines of the offending file. Exit code is non-zero if any file fails. Every file the tests use that is not part of the repository — the ROMs, hardware test programs, and demo fixtures — resolves through test/external-assets.json: edit the paths there, or set the per-entry env var, to match your machine. Spec tests that depend on such a file SKIP with a note when it's absent, so the suite passes without the optional fixtures (the ROMs are the exception — the runner checks them up front).

Test categories#

Diagnostic / trace tools (not gated)#

Run on demand to dump telemetry. Output is usually written to /tmp/ or printed to stdout.

The reference-demo screenshot pass (test/commit-screenshots.mjs) runs the fixed DEMOS table headlessly and writes timestamped framebuffer PNGs to investigation/commitscreenshots/. It is a human visual check, not a spec test: the filename is plain .mjs, all-test.js does not gather it, and it makes no assertions. It never deletes old screenshots; successive runs accumulate so a before/after pair can coexist. After each run, it diffs every new shot against the previous run's matching shot and prints which demo seconds changed; changed shots also get diff-<demo>-sNN.png overlays with unchanged pixels dimmed and changed pixels tinted magenta. Run it only when explicitly requested, or as the visual baseline/after pair for a render or timing change.

node test/commit-screenshots.mjs

The demo crash/hang status board (test/demo-status.mjs) boots each tracked open-bug demo (disk 1) headless and classifies the outcome — CRASH (JAM/KIL opcode + PC + time), runs clean, or DISPLAY FROZEN (possible silent hang; framebuffer-based, since PC-sampling can't tell a silent hang from a healthy interrupt-driven spin) — comparing each to its expected status (✓ / ✗ CHANGED, so it doubles as a regression detector). Each demo loads via the chunked keyboard buffer (the UI path) and runs 3 minutes, saving a screenshot every 10 s plus the end frame (<demo>-<YYYYMMDD-HHMMSS>[-sNNN].png, to a git-ignored output directory created on demand) so a demo's progression — and exactly where it visually breaks — is visible. The demo disk images resolve through the collection roots in test/external-assets.json. SID defaults to 8580 to match the UI. Crash disks are booted directly (e.g. Mojo disk 4). It's a *.mjs tool, so the all-test.js runner skips it (like commit-screenshots.mjs).

# Crash/hang status board over the open-bug demos (screenshots + ✓/✗ vs expected)
node test/demo-status.mjs                 # all tracked demos
node test/demo-status.mjs coma next       # filter by demo name
SID=6581 node test/demo-status.mjs        # force the original 6581 SID

# Convert raw Float32 audio to listenable WAV
node test/f32-to-wav.js /path/to/audio.f32 /tmp/out.wav

# Exercise the worklet's power-cycle / reset / cycle-sync paths
node test/sid-power-cycle-trace.js

# Run the cycle-exact $D41B OSC3 demo PRG headlessly + verify output variety
node test/osc3-cycle-test.js

Cross-checking against VICE (reference oracle)#

VICE (x64sc) is the ground-truth oracle for VIC-II / CPU / sprite / timing questions — install it from your platform's package manager (e.g. brew install vice) or the VICE project. When writing one-off capture scripts, reuse existing connection boilerplate rather than re-deriving the monitor protocol.

Always pass -VICIImodel 0. x64sc -pal defaults to VICIIModel=1 = the 8565 ("new" VIC), but this project models the 6569 ("old" VIC). The 8565 samples registers one cycle earlier, so an uncorrected compare shows a spurious 1-cycle / 8-pixel offset that is not a bug. Verify in the monitor with resourceget "VICIIModel" (0=6569, 1=8565, 3=6567 NTSC).

Always run scripted VICE launches headless. Set SDL_VIDEODRIVER=dummy in spawned VICE processes so no window appears on the user's desktop. If ports or monitor sockets conflict, kill stray x64sc processes before retrying.

Two conventions that bite every time:

• VICE's monitor CYC column is 0-based (0–62); our cycleInLine and Bauer's spec are 1-based (1–63). So VICE CYC = our cycleInLine − 1 — a write VICE reports at CYC 12 lands at our cycleInLine 13, the same physical cycle. Don't mistake the off-by-one for a timing bug.
• Match the two runs by demo state (which scene / pose / digit is on screen), never by absolute frame or cycle count — our boot timing differs from VICE's, so the runs drift apart in wall-clock time.
• VICE flag polarity is inverted from common CLI intuition: -NAME enables a resource and +NAME disables it. For example, -drive8truedrive turns true drive on.

Capture style A — headless one-shot screenshot (fast, fully deterministic; best for pixel diffs):

x64sc -VICIImodel 0 -warp -autostart-warp +drive8truedrive -autostartprgmode 1 \
  -limitcycles <N> -exitscreenshot /tmp/vice.png -autostart "/path/to/demo.prg"

The PAL screenshot is 384×272 — the same crop as our frameBuffer — so pixels align directly. -limitcycles + -exitscreenshot is reproducible; warp + a hand-driven monitor is not. Autostarted PRGs need enough budget to clear boot, injection, RUN, and settle: about 9M cycles for a 9 s run. Around 3M cycles often lands on bare READY. before the program has run.

Capture style B — monitor (breakpoints, register/memory watches, single-step): launch with a monitor socket and script it over TCP.

• Remote text monitor (-remotemonitor -remotemonitoraddress ip4://127.0.0.1:PORT): connect, wait for boot or a -limitcycles halt, send newline-terminated commands. break exec $XXXX sets a checkpoint; on each hit r dumps registers including LIN/CYC (raster + cycle-in-line), m d011 d011 reads memory, save "<file>" 0 <start> <end> dumps memory while stopped, x resumes, screenshot "out.png" 2 saves a PNG.
• Binary monitor (-binarymonitor -binarymonitoraddress ip4://127.0.0.1:PORT): a framed binary protocol — set exec/store/load checkpoints, advance one instruction (then read regs for LIN/CYC), or store-watch a register. Lower-level but scriptable to thousands of samples (per-instruction (PC, LIN, CYC) traces and per-line register store-watches).

The workflow that works:

1. Reach the scene in VICE — checkpoint the demo's inner-loop routine and advance N hits to a stable frame.
2. Capture VICE ground truth: a per-instruction (PC, LIN, CYC) trace, and/or per-line register stores with their CYC, and/or a screenshot.
3. Build the same trace headlessly from our emulator — drive machine._runMasterCycle() in a loop (19656 cycles = one PAL frame) and record (cpu.pc, vic2.raster, vic2.cycleInLine − 1) at each cpu.atInstructionBoundary(). (Boot with ~200 warm-up frames before loadPRG + injectRun, or KERNAL init clobbers the injected RUN.)
4. Diff by PC, not by time — the instruction stream is identical. A divergent CYC for the same PC points straight at the cycle where our timing differs; a CYC gap on one side is a CPU stall (bad-line or sprite-DMA BA) the other side doesn't have.

For testprogs suites that ship a references/ directory of VICE screenshots, use the shared comparator instead of re-rolling a render/diff script:

node test/ref-compare.mjs <prg> <refPng> [boot=200] [run=80] [refPalette=pepto|colodore]

test/ref-compare.mjs boots the KERNAL, loads and runs the PRG, and compares in palette-independent colour-index space. It handles the two common false positives for these references:

• Palette mismatch. Testprogs VICE screenshots are usually Pepto, while the emulator default is Colodore. Raw RGB diffs are palette noise; the shared tool quantizes both sides to their own 16-colour palette. If you capture a raw PNG yourself for RGB diffing or eyeballing, set the emulator palette to Pepto first with setVicPalette('pepto').
• Crop offset. VICE PNG row 0 is raster 16 while our framebuffer row 0 is raster 15. The comparator searches small dx,dy offsets; PERFECT (1-line crop offset) is a pass.

VICE's external Pepto palette still runs through its gamma/contrast curve. Tiny residual index diffs confined to 1px features, where position matches and only colour differs, are usually this curve rather than a rendering bug. For those cases, read the VIC registers over the monitor (m d027 d02e, masking colour-register reads with & 0x0f) before chasing pixels.

PRG-building helpers#

A few specs need a tiny 6502 program injected into a fresh machine. The build-*.mjs scripts emit those PRGs on demand:

node test/build-osc3-cycle-test-prg.mjs       # → test/osc3-cycle-test.prg
node test/build-sid-feature-test-prg.mjs      # → test/sid-feature-test.prg
node test/build-raster-prgs.mjs               # → test/*-raster.prg
node test/build-vice-prgs.mjs                 # → test/vice-*.prg

Test runner registration#

Every new test must be listed in test/all-test.js's TESTS array. The runner won't find files that aren't explicitly registered.

The rest of this document is the DevTools debug/inspection surface — console helpers and live model toggles for triaging behavior in the browser.

Debug console (DevTools)#

The running machine is the machine global (window.machine); the trace and inspection helpers below are c64Trace / c64Vic / c64Bus.

// Machine lifecycle. softReset = a /RESET-line pulse: preserves RAM (the KERNAL
// re-inits screen + zero page). No UI button, and allowSoft:true is REQUIRED (a
// bare softReset() throws) so a soft reset never fires by accident.
machine.softReset({ allowSoft: true })
machine.reset()                          // cold boot / power cycle (regenerates RAM)

// VIC frame trace — enrich the Cmd+S snapshot with whole-frame + per-raster
// data (see "VIC frame trace and state snapshots" below for workflow + perf).
c64Trace.enable() / .disable() / .status()

// Raster-scroller jitter capture — per-frame IRQ-accept / soft-vec entry / $F7
// / $D020 / $D021 / $D012 / BA-AEC-release cycles across N frames, with
// frame-to-frame variance analysis. Auto-downloads a JSON report on dump.
c64Trace.jitterStart(60)        // arm an N-frame capture, then let the demo run
c64Trace.jitterDump()           // print + download the report (also returns raw)

// SID write trace — capture every SID register write for inspection
c64Trace.sidStart(20000)       // capture next N writes
c64Trace.sidDump(0x18)          // pretty-print first 40 $D418 writes
c64Trace.sidStats()             // per-register count + Hz rate summary

// VIC-II model toggles (off by default — unstable / variant-specific)
c64Vic.bankDelay(true|false)    // NMOS: DDR-driven single-bit 0→1 that decreases
                                // VIC bank by 1 or 2 delays one cycle (VIC-Addendum
                                // "Video bank and C64C"). Unstable on real chips.
c64Vic.bankGlitch(true|false)   // C64C / 8565: VIC-bank 10↔01 transitions blip
                                // through bank 3 for one cycle. Only active when
                                // vicVariant='8565'.

// Render performance toggle (ON by default — byte-identical optimisation)
c64Vic.batchRender(true|false)  // _fixupColumns fast path: re-render ONLY the
                                // cycles whose mid-line mode (ECM/BMM/MCM) or bg
                                // ($D021-$D024) lookahead window changed, instead
                                // of re-rendering the whole line twice. Proven
                                // pixel-identical (orbit fb hash + spec suite +
                                // vic2-fixup-batch-equivalence-spec-test); biggest
                                // win on heavy mid-line-write demos like Orbit
                                // Untold (~54→65 fps). Flip OFF to A/B if a render
                                // regression is ever suspected.
c64Vic.captureDedup(true|false) // _captureCycleState fast path (ON by default —
                                // byte-identical optimisation): alias the previous
                                // cycle's row + sprite + register snapshot buffers
                                // when the source is unchanged (version-counter
                                // tracked; the register snapshot's version bumps on
                                // every CPU $D0xx write) instead of re-copying ~10
                                // typed arrays every visible cycle. With batchRender,
                                // Orbit ~65→77 fps.
                                // c64Vic.captureDedupVerify(true) adds a per-cycle
                                // assert that the alias still matches the live source
                                // (catches a missed version bump). Flip OFF to A/B.
c64Vic.spriteSkipIdle(true|false) // _renderSpriteSegmentForSprite fast path (ON by
                                // default — byte-identical): return early on cycles
                                // where a started sprite is steady and paints nothing
                                // (no segment overlap, no end-of-line wrap), plus a
                                // never-started loop-level skip. On sprite-heavy demos
                                // ~74% of per-cycle sprite calls paint nothing. Flip
                                // OFF to A/B.

// Shared external-data-bus model — see next section for the full table
c64Bus.status()                 // dump every flag + the live latch bytes
c64Bus.openBus('disabled')      // 'vice-compatible' (default) | 'disabled' | 'random'
c64Bus.colorRam(true|false)     // composed Color-RAM read re-drives the latch
c64Bus.portZeroOne(true|false)  // $00/$01 RAM-under-port quirk (default off)
c64Bus.refresh(true|false)      // VIC r-access drives the bus
c64Bus.spriteIdle(true|false)   // sprite idle fetch leaks vs all-$FF
c64Bus.cpuInternal(true|false)  // KIND_INTERNAL cycles fire a discarded read
c64Bus.traceStart(1024)         // enable per-cycle bus trace ring
c64Bus.traceStop()              // disable + free
c64Bus.traceDump(64)            // print + return last N entries (oldest first)

VIC frame trace and state snapshots (Cmd+S)#

Cmd+S (macOS) / Ctrl+S (Windows/Linux) downloads a debug snapshot of the machine: a c64-snapshot-<timestamp>.json state dump plus a sibling c64-snapshot-<timestamp>.png of the rendered frame. It is handled before the "is the machine running" gate, so a JAMmed or paused machine can still be inspected. The JSON embeds the same PNG as framebufferPng, so the one file is self-contained; the sibling .png is just for quick preview. (Bound in input.js; the download itself is downloadSnapshot() in media.js.)

The snapshot's vicFrameDebug block is where the VIC frame trace lands: per-pixel borderBuffer / graphicsPriorityBuffer / spriteOwnerBuffer maps plus per-line register/flag traces (frameTraceHBorder, frameTraceVBorder, frameTraceLineD011, frameTraceLineD016, frameTraceLineD015, …), indexed raster * 64 + cycle. Combined with framebufferPng this answers questions like "is this side-border garbage the border being open or closed?" pixel by pixel.

Enable the trace first for a whole-frame map. With the trace off, vicFrameDebug falls back to the VIC's line-sized live buffers — only the last rendered line is meaningful — and the snapshot records traceEnabled: false. c64Trace.enable() switches the VIC to accumulating each rendered line into a full-frame map, so the snapshot then covers the entire frame. The intended loop:

c64Trace.enable()    // start accumulating whole-frame trace data
// …run the demo to the exact moment of interest…
// press Cmd+S / Ctrl+S to download the snapshot (JSON + PNG)
c64Trace.disable()   // stop — restore the fast path
c64Trace.status()    // check whether it's currently capturing

Performance. Leaving the trace off costs nothing measurable — the off-path is a single boolean check per raster, and it is in fact the optimized path: the line-batch renderer, capture-state dedup, and sprite-idle skip (the c64Vic.* toggles above) are all active only while frameTraceEnabled is false. Turning the trace on deliberately disables those optimizations (the renderer then runs live per-cycle on every line and re-copies the capture snapshots every visible cycle) and adds ~5 extra per-cycle field computations plus whole-frame map accumulation. Expect a visible frame-rate drop on heavy demos while it is on — which is why it is a flip-on-then-off tool, not a default. c64Trace.disable() restores full speed immediately.

Shared external-data-bus model#

The emulator models a shared 8-bit external data bus (memory.externalDataBus8) that is updated by every CPU read/write and every VIC chip-bus fetch. Open-bus CPU reads ($DE00–$DFFF without a cartridge, plus the upper nybble of Color RAM at $D800–$DBFF) sample the latch instead of returning a fixed $FF. The VIC's vicInternalBus is a separate latch that feeds sprite idle fetch (VIC-Addendum §"Sprite idle fetch"); both latches are driven by _vicBusRead simultaneously.

Each behavior is gated by a flag so you can bisect a suspected regression or fall back to the legacy model:

All of these can be set live from the DevTools console, e.g. machine.mem.openBusMode = 'disabled'. They take effect on the next master cycle — no restart needed.

Per-cycle bus trace#

A debug-only ring buffer that records phi1/phi2 owner, BA/AEC/RDY, the current CPU microop kind, and both bus latches per master cycle. Off by default because it allocates one entry per cycle (~985 KHz).

machine.enableBusTrace(1024)     // start capturing into a 1024-entry ring
machine.busTraceSnapshot(64)     // return the most recent 64 entries (oldest first)
machine.disableBusTrace()        // stop + free the ring

Useful when you suspect an open-bus or BA/AEC timing issue. Each entry has frame, raster, cycle, ba, aec, rdy, cpuBlocked, cpuOp, phi2Owner, externalDataBus8, vicInternalBus8.