Tools and Scripts

Reference for every executable in the documentation repository. This page covers the nine Windows batch wrappers at the repository root, the conventions every command-line tool shares, and the configuration files the tools read. The tools themselves — the Node and Python scripts under scripts/, the twinBASIC program imagestrip, tbdocs and its flags, and the PDF render driver — are on seven pages, grouped by what they work on. For the day-to-day workflow, see Building and Deployment. To modify the build pipeline, see tbdocs Internals.

Tool groups
Command-line conventions
Batch wrappers at the repository root
Configuration files

Tool groups

Build and Generation Tools – tbdocs, the PDF render driver, the build comparison, and the generators of committed files.
Site Checks – the link, freshness, diagram and accessibility checks over a built site.
Toolchain Tests – the gates test.bat runs, which test the toolchain rather than the pages.
Compiler Harness – tbbuild, tbrun, impexp, and the probes of the harness itself.
Code Samples and Attributes – check_examples and the attribute census, probe generator and sweep.
IDE Automation – the add-in and IDE test runners, the help add-in tryout, the screenshot tool, and the IDE’s embedded images.
Defect Reproducers – bug_repro and vb6run.

Command-line conventions

Every Node tool answers --help or -h by printing its usage to standard output and exiting 0, without doing any work.

Every Node tool refuses an unusable command line the same way: it reports the problem on standard error and exits 2, before doing any work. A crash also exits 2. The refused cases are:

an unknown flag, a flag without its value or with an empty one, and an unexpected argument;
a number that is not a number, is out of range, or is a fraction where a whole number is needed;
a regular expression that does not compile;
a URL that is not an absolute http or https one;
a date that is not ISO 8601;
a value outside a fixed set;
options that exclude each other.

A tool that reads its command line through lib/cli.mjs and takes search terms, file names or folder names accepts one that starts with a dash after --.

Batch wrappers at the repository root

All nine sit at the repository root, beside package.json. Each uses @pushd "%~dp0" to run from that root wherever it is invoked, and each entry below gives the POSIX equivalent of what it runs. The equivalents have no pushd, so run them from the repository root: tbdocs’s --src docs, check_publish_policy.mjs’s default source root and every path given to render-book.mjs resolve against the working directory.

examples.bat, addin-test.bat, ide-test.bat and try-help-addin.bat have no POSIX equivalent. They need a twinBASIC install and drive the IDE, so they run only on Windows and are not part of the site build. Four other tools are Windows-specific for the same reason and are not part of the build either:

scripts/tbbuild.mjs and scripts/tbrun.mjs drive the twinBASIC IDE.
census_attributes.mjs unpacks the packages of a twinBASIC install. It is cross-platform when given an already-exported tree with --exported.
imagestrip is a twinBASIC program.

tbdocs and every gate in both wrappers is a Node script. CI runs all of them on ubuntu-latest except check_tree_fresh.mjs, which guards against a failure CI cannot have.

build.bat

build.bat [extra tbdocs flags]

POSIX:

node builder/tbdocs.mjs --src docs --check-audit-index [extra tbdocs flags]

Renders the documentation, forwarding extra arguments through %*. Produces _site/, _site-offline/ and _site-pdf/, unless --no-offline / --no-pdf or the also_build_offline / also_build_pdf keys in _config.yml turn one off. A local build also writes the help archive, unless --no-help-archive is given.

Keep --check-audit-index when running tbdocs by hand. It implies --check; without it the same three trees are written, no link check runs, and the build reports success.

Every run prints its own time (Done in …, with the page and static-file counts). The time depends on page count, core count and which passes ran: --no-check, --no-offline, --no-pdf and --no-help-archive each remove one.

Exit codes, those of tbdocs: 0 nothing to report; 1 the build or its check found a problem: a link or integrity failure, a failed build step, a fall in the page count, or a symbol-index URL lost; 2 a refused command line, a build stopped by the stall watchdog, or a crash.

serve.bat

serve.bat [extra tbdocs flags]

POSIX:

node builder/tbdocs.mjs --src docs --serve [extra tbdocs flags]

Starts a live preview, forwarding extra arguments through %*. After an initial build it serves the site on port 4000 (--port <N> changes it), rebuilds on each change under docs/, and reloads a connected browser after each successful rebuild. The offline and PDF passes are skipped. Ctrl+C stops it. Only failures (4xx, 5xx, server exceptions) are logged. An edit under builder/ does not reach a running preview; restart it — see why serve.bat does not show a builder change.

Exit codes, those of tbdocs: 0 the server was stopped with Ctrl+C, 2 a refused command line, a failed first build, a port already in use, or a crash.

check.bat

check.bat

The gates that read the built site; run build.bat first. Link and integrity checking is in build.bat (see tbdocs), and tests of the toolchain are in test.bat. Four steps, each stopping the run if it fails:

scripts/check_tree_fresh.mjs — refuses a tree older than the sources that produced it.
scripts/check_a11y.mjs — the puppeteer + axe-core accessibility scan.
scripts/check_dot_fit.mjs — re-renders every committed diagram with the real webfont and fails if a label sits outside its box.
scripts/pick_a11y_sample.mjs --check — verifies the sample still covers every markup construct the site uses.

POSIX — four commands, chained so the run stops where check.bat would:

node scripts/check_tree_fresh.mjs \
  && node scripts/check_a11y.mjs \
  && node scripts/check_dot_fit.mjs \
  && node scripts/pick_a11y_sample.mjs --check

Exit codes: 0 every step passed; otherwise the code of the step that stopped the run, as that step’s entry gives it.

A local pass of check_a11y.mjs does not predict CI’s. Its target-size rule measures rendered boxes, and an inline element’s height depends on the font system-ui resolves to on the machine running the scan. Both workflows install fonts-liberation, and the site’s padding is calibrated against the smallest face in that band. Larger local metrics clear controls that CI then fails. See Building and Deployment.

test.bat

test.bat

The tests the toolchain has to pass. Twenty-five steps, each stopping the run if it fails:

scripts/check_publish_policy.mjs — verifies the publish allowlist still refuses the types it should. Needs neither a browser nor a built tree.
scripts/check_gate_lists.mjs — verifies the two gate lists on this page still match the wrappers that run them.
scripts/check_ci_workflows.mjs — verifies both CI workflows run the gates the wrappers run, and build as build.bat does.
scripts/check_lint.mjs — runs Biome’s linter and formatter check over the tooling and fails on any finding, warnings and layout included.
test/search.test.mjs — unit tests for the site search: what the search entries hold, and that the copies of the search client agree.
test/render.test.mjs — unit tests for the markdown-it plugins, on inputs no page holds.
test/strftime.test.mjs — unit tests for the footer’s date formatter, which no build calls.
test/png.test.mjs — unit tests for the pictures of a bug reproducer: PNG decoding, comparing and the side-by-side image, the files behind images and expect.imagesDiffer, and the refusals of bug_repro.mjs.
test/compact-image.test.mjs — unit tests for what an embedded image keeps: an editor’s metadata goes, a notice of ownership and terms stays. The half that runs imagestrip is skipped unless its exe is named.
test/svgshot.test.mjs — unit tests that an SVG screenshot is written on one line, that its underlines are placed as Chromium places them, that a background image is sized by its bytes, that a frame’s page is placed inside its frame, that a web font is found and cut as the page uses it, that the page’s own <svg> icons are found and kept apart, and that a colour filter is baked into the colours it paints.
test/example-batches.test.mjs — runs check_examples.mjs’s probes, which test how samples are batched and how a crashed batch is cut down, without an IDE.
test/ports.test.mjs — unit tests for how the harness claims the DevTools ports its IDEs use, and for the temp folder and profile an IDE on a port leaves behind, without an IDE.
test/addin-project.test.mjs — unit tests for how the build packs the help add-in’s project file: exactly the files git tracks in add-in/, never Resources/HELP/, the same bytes from any checkout.
test/indexer.test.mjs — unit tests for the package indexer: the VB6 preprocessing, the files of a package, the api.json built from them (sorted by code unit, nested types by their path, values and fields in declaration order, the same bytes from the same files) and the differ that compares two of them.
scripts/check_regex_safety.mjs — refuses a regex that can backtrack exponentially, written as a literal or built from constants.
scripts/check_code_regions.mjs — verifies no pre-render rewrite alters the contents of a code fence or code span, that the rewrites over rendered HTML leave a raw <pre> or <code> alone, and that lib/markdown.mjs and lib/frontmatter.mjs pass their probes.
scripts/check_page_baseline.mjs — verifies the page-count drift guard still refuses a fall.
scripts/check_book_coverage.mjs — verifies the build still warns about a page docs/_book.yml does not mention.
scripts/check_symbol_index.mjs — verifies the symbol index still places each kind of symbol, and its drift guard still refuses a lost URL.
scripts/check_twin_parsers.mjs — verifies the scanners of twinBASIC source and of the attribute reference still read the shapes each once misread.
scripts/check_attribute_sweep.mjs — verifies the logic of the attribute sweep: its site skeletons, how it reads a probe’s diagnostics, how it batches probes, and how it compares the answers with Attributes.md.
scripts/check_cli.mjs — verifies lib/cli.mjs, the command-line parser, and each tool’s recorded command-line errors.
scripts/check_pdf_shims_equiv.mjs — verifies the book’s pdf-lib shims write what stock pdf-lib writes, patch the members of pdf-lib it lists, have each of them in their shim’s table of targets, and run.
scripts/check_impexp_parity.mjs — verifies the two editions of the impexp tool pass the same built-in tests, and exit, print and write the same for one sequence of commands. Without Python it reports itself skipped and passes, except in CI.
scripts/check_axe_patch_equiv.mjs — verifies the vendored axe source patch still produces identical colour values.

POSIX:

node scripts/check_publish_policy.mjs \
  && node scripts/check_gate_lists.mjs \
  && node scripts/check_ci_workflows.mjs \
  && node scripts/check_lint.mjs \
  && node --test test/search.test.mjs \
  && node --test test/render.test.mjs \
  && node --test test/strftime.test.mjs \
  && node --test test/png.test.mjs \
  && node --test test/compact-image.test.mjs \
  && node --test test/svgshot.test.mjs \
  && node --test test/example-batches.test.mjs \
  && node --test test/ports.test.mjs \
  && node --test test/addin-project.test.mjs \
  && node --test test/indexer.test.mjs \
  && node scripts/check_regex_safety.mjs \
  && node scripts/check_code_regions.mjs \
  && node scripts/check_page_baseline.mjs \
  && node scripts/check_book_coverage.mjs \
  && node scripts/check_symbol_index.mjs \
  && node scripts/check_twin_parsers.mjs \
  && node scripts/check_attribute_sweep.mjs \
  && node scripts/check_cli.mjs \
  && node scripts/check_pdf_shims_equiv.mjs \
  && node scripts/check_impexp_parity.mjs \
  && node scripts/check_axe_patch_equiv.mjs

Exit codes: 0 every step passed; otherwise the code of the step that stopped the run, as that step’s entry gives it.

Twenty-two of the twenty-five cannot be affected by an edit confined to docs/, which is why they are separate from check.bat. Run this one when the change touches builder/, scripts/, lib/, book/, eval/, wisdom/, indexer/ or test/, the site’s scripts in docs/assets/js/, the help add-in’s source in add-in/, a wrapper, or a workflow. Both CI workflows run all twenty-five unconditionally.

The three exceptions are check_code_regions.mjs, check_gate_lists.mjs, which reads this page, and check_lint.mjs, which lints the site’s scripts in docs/assets/js/.

check_code_regions.mjs has two halves. Its corpus sweep tokenises every markdown file under docs/, so a page that provokes a rewrite into altering a code region fails it. Its fixed probes run against their own sources whatever the tree holds, and cover the mirror fault, where a rewrite silently stops firing; the sweep cannot see that one. After adding a page with an unusual code construct, run test.bat and read the built page too.

A gate belongs in test.bat rather than check.bat if it would still mean something against an empty docs/. check_axe_patch_equiv.mjs needs a built page and Chromium only because its probe needs some document to run inside; it tests the axe patch.

book.bat

book.bat

POSIX — three commands, not one:

node scripts/check_tree_fresh.mjs --tree docs/_site-pdf --marker book.html
mkdir -p docs/_pdf
node book/render-book.mjs docs/_site-pdf/book.html \
  -o "docs/_pdf/twinBASIC Book.pdf" \
  --outline-tags h1,h2,h3,h4 \
  --additional-script perf/detach-pages.js

Renders the PDF book from docs\_site-pdf\book.html into docs\_pdf\twinBASIC Book.pdf with node book\render-book.mjs (see below). To rename the PDF, change the -o argument in book.bat and in .github/workflows/tbdocs-gh-pages.yml.

Run build.bat first to populate _site-pdf/; book.bat checks this with the pre-flight below.

The npm install guard tests for the package, not for the browser. book.bat checks only that node_modules\puppeteer\package.json exists. An install run with --ignore-scripts or PUPPETEER_SKIP_DOWNLOAD, or a puppeteer cache cleared afterwards, passes that test with no Chromium to render with. npx puppeteer browsers install chrome fixes it. PDF Generation gives the error.

render-book.mjs does not create the output directory. A missing docs/_pdf/ fails with ENOENT at the end of the render, after the page-breaking pass has run, so book.bat and the deploy workflow create it first.

Do not chain the two as build.bat && book.bat. build.bat sets a non-zero exit code when the link or integrity check finds something, though it still writes all three trees. && reads only the exit code, so a broken link anywhere cancels the render, and the terminal ends on the link findings and no PDF. Run them as two separate commands; see Building and Deployment.

Exit codes: 0 the PDF was written; 1 the tree is stale, or npm install failed; 2 there is no built tree, or the render failed. check_tree_fresh.mjs runs first and its code is returned as it is; render-book.mjs has no 1 of its own.

The pre-flight freshness check

book.bat first runs, before the npm install test and the renderer:

node scripts/check_tree_fresh.mjs --tree docs/_site-pdf --marker book.html

check_tree_fresh.mjs refuses a _site-pdf/ tree older than docs/ or builder/. Without it, editing a page and running book.bat without build.bat spends two minutes rendering the previous book and reports success. The PDF is internally consistent, so nothing flags it. The renderer cannot catch this: it refuses only a missing input (input not found: and the resolved path).

--marker book.html is required and book.bat is the only caller that passes it. The script identifies a tree by its index.html, and _site-pdf/ holds only book.html.

A 2 does not say which half of book.bat failed; the message does. The pre-flight returns 2 for an absent tree and the renderer returns 2 for a failed render. A 1 is unambiguous: a stale tree, or a failed npm install. Every pre-flight failure is prefixed check_tree_fresh:, and these are the two it prints:

check_tree_fresh: docs/_site-pdf/book.html does not exist.
  Run build.bat first -- there is no built tree to check.

check_tree_fresh: docs/_site-pdf is 118s older than docs/Reference/Core/Dim.md.
  Run build.bat first. Scanning a stale tree reports a pass for the
  previous build, which is the one thing these gates must never do.

The check has one known false positive: it watches all of docs/ and builder/, so editing a builder/PLAN-*.md marks every tree stale although the build does not read it. Rebuild.

examples.bat

examples.bat [flags]

One invocation of check_examples.mjs, with every flag passed straight through:

node scripts/check_examples.mjs [flags]

Compiles the documentation’s own twinBASIC code samples — every ```tb fence marked check_build — and reports the ones the compiler refuses, against the line in the page they came from. A sample also marked check_run is run, and what it prints is checked against the page. To mark a sample, and for what a pull request that changes one shows, see Authoring Pages.

It is not one of the gates, and must not become one. It is absent from build.bat, check.bat, test.bat and both CI workflows, because it needs a twinBASIC install (npm install alone must suffice to build the docs) and Windows with a private desktop and a CDP-reachable WebView2 (the CI box has neither). An IDE cold start also takes 6 to 8 seconds, against four for a whole site build. A person runs it, as with sweep_a11y.mjs.

Exit codes, those of check_examples.mjs: 0 clean, 1 a sample does not compile, or does not run as its page says, 2 the harness failed.

addin-test.bat

addin-test.bat [flags]

One invocation of addin_test.mjs, with every flag passed straight through:

node scripts/addin_test.mjs [flags]

Tests twinBASIC IDE add-ins by machine: it builds each add-in under test, loads it into an IDE, operates the IDE as a person would, and checks what the add-in did.

It is not one of the gates, for the reasons examples.bat is not, and is absent from build.bat, check.bat, test.bat and both CI workflows.

Exit codes, those of addin_test.mjs: 0 every lane passed and the registry is as it was found, 1 a lane failed, 2 the harness failed, 3 the registry or a work folder was not put back.

ide-test.bat

ide-test.bat [flags]

One invocation of ide_test.mjs, with every flag passed straight through:

node scripts/ide_test.mjs [flags]

Tests the twinBASIC IDE itself by machine: it opens a project in an IDE, operates the IDE as a person would (the debugger, Export Project, the Packages dialog), and checks what the IDE did. It is addin-test.bat’s runner for scenarios that test the IDE rather than an add-in.

It is not one of the gates, for the reasons examples.bat is not, and is absent from build.bat, check.bat, test.bat and both CI workflows.

Exit codes, those of ide_test.mjs: 0 every lane passed and the registry is as it was found, 1 a lane failed, 2 the harness failed, 3 the registry or a work folder was not put back.

try-help-addin.bat

try-help-addin.bat [flags]

One invocation of try_help_addin.mjs, with every flag passed straight through:

node scripts/try_help_addin.mjs [flags]

Opens an IDE on your desktop with the help add-in built and loaded, to try it by hand, and puts the registry back once the IDE is closed. It is not one of the gates: it needs a twinBASIC install and Windows.

Exit codes, those of try_help_addin.mjs: 0 the IDE was closed and the registry is as it was found, 1 the add-in did not build or the project does not compile, 2 the tool could not run, 3 the registry or the work folder was not put back.

Configuration files

The build also reads these declarative files.

File	Effect
`docs/_config.yml`	Site config. `tbdocs` reads `url`, `baseurl`, `title`, `logo`, `also_build_offline`, `also_build_pdf`, `offline_exclude`, `exclude` (which filters the source walk but is not the publish safety net — see `check_publish_policy.mjs`), the footer / aux-link knobs, the GitHub edit-link knobs, and the download-link knobs (`gh_offline_link`, `gh_offline_link_url`, `gh_pdf_link_url`). Jekyll-only keys (`markdown`, `kramdown`, `theme`, `highlighter`, the `defaults` block, the `compress_html` block) are ignored.
`docs/_book.yml`	The PDF book’s chapter manifest. Entries are resolved to pages via the selector schema (`page` / `pages` / `nav_page` / `nav_pages` / `no_descent`) and control PDF outline behaviour via `landing_page:`, `landing_is_target:`, `no_outline_entry:`, `no_heading_shift:`, and `outline_closed:`. Its `left_out:` list names the pages not in the book, each with a `reason:`; the build warns about a page that is in neither. The file header documents the full schema.
`builder/themes/Light.theme`, `Dark.theme`, `Classic.theme`	twinBASIC IDE theme files, vendored from the BETA installer. `builder/highlight-theme.mjs` parses them into the palette behind the renderer’s scope-to-class mapping and the generated `tb-highlight.css`. Refresh them from the installer when the IDE adds palette entries.
`builder/twinbasic.tmLanguage.json`	TextMate grammar for the twinBASIC language. Shiki uses it to tokenise every ```tb code block.