Sabline 8.7.0

Changelog: 9.x

Every release of this major version, newest first, as CHANGELOG.md records it.

9.0.0-alpha.4 - The job that was read, and then run #

9.0.0-alpha.3 published its crate and made no GitHub release. The two hard things worked: release.yml dispatched publish-crate.yml, waited for the run with gh run watch --exit-status, and sabline-rt 9.0.0-alpha.3 went to crates.io through OIDC with no stored credential. Then the release job fell over on one line:

Text
Post https://uploads.github.com/.../assets: read assets/rt: is a
directory

The crate job gave upload-artifact two paths, so the artifact kept the directories they had in common - rt/target/package/... - and the release step globbed assets/*, handing gh a directory. gh refuses one, and then deletes the release it had just created, so there is no GitHub release under v9.0.0-alpha.3 although the tag and the crate stand.

Two fixes, and the second is the one that matters:

  • the step names files rather than globbing a tree (find assets -type f), so a directory can never reach gh whatever the artifact's shape, and it attaches what an existing release lacks rather than doing nothing - the same shape the ordinary release's job has always had;
  • the artifact is one flat directory, so assets/ holds the two files and nothing else.

The part worth keeping #

Three releases were spent on three defects, and all three were in workflow code I had verified by reading it. A skip that travels past a job which rescued itself; a registry that refuses a token minted under workflow_run; a glob that hands a directory to a command that refuses one. Every one of them takes seconds to see in a run and is invisible on the page.

This project already knew that. check_release.py has run the Marketplace job's and the pin-moving job's own steps, in bash, against stand-ins since 8.2.1 and 8.4 - because both had failed in ways reading did not catch. The jobs added for 9.0 did not get the same treatment.

They do now. run_prerelease_github_job lays the assets out from the crate job's own upload-artifact path, so the fixture tests the artifact this repository actually produces rather than one chosen to pass, and runs the release job's real steps against a gh that refuses a directory exactly as the real one does. Run against the job as 9.0.0-alpha.3 shipped it, the fixture fails with gh refused a directory; against the fix it passes. Writing it found two more things a reading had not: the notes step needs the checkout's CHANGELOG, and the "nothing else moved" step needs the registry to say the crate is there.

check_release.py: 175 passed, 0 broken.

compatibility: a pre-release publishes the sabline-rt crate and a GitHub release marked pre-release and nothing else, so nothing a user of 8.6.0 has is touched - not PyPI, not npm, not the VS Code Marketplace, not the MCP registry, not the Action pins, and not any "latest" anywhere.

v9.0.0-alpha.1, v9.0.0-alpha.2 and v9.0.0-alpha.3 stay where they are, neither moved nor deleted nor retagged. alpha.1's crate is on crates.io because a person put it there, which is how a crate's first version always gets there; alpha.3's is there because the workflow put it there, which is how every one after it will. None of the three has a GitHub release, and the entries below say why each.

9.0.0-alpha.3 - The crate is published by a workflow of its own #

9.0.0-alpha.2 tagged and did not publish either. Its crates_io job ran this time - the skip alpha.2 fixed stayed fixed - and failed at the one place nothing had exercised:

Text
Failed to retrieve token from Cargo registry. Status: 400.
Trusted Publishing does not support the `workflow_run` event trigger
due to security concerns.

crates.io refuses a Trusted Publishing token minted under workflow_run, and release.yml runs on workflow_run. It refuses it on purpose: crates.io removed workflow_run and pull_request_target because both have been behind supply-chain incidents. The triggers it allows are push, release and workflow_dispatch. push on a tag is the one this project cannot use - no workflow here listens for tags, precisely so that a tag pushed by hand publishes nothing.

So the crate's publish is publish-crate.yml, a workflow_dispatch workflow of its own, and release.yml starts it: the crates_io job tags, asks whether crates.io has the version, dispatches that workflow if it does not, waits for the run it started with gh run watch --exit-status, and asks crates.io itself afterwards. A publish that did not happen cannot be mistaken for one that did. release.yml still decides what is released; that workflow is the arm that reaches crates.io for it, and it stores no credential either.

A workflow anyone with write access can start by hand is only safe if it refuses everything release.yml would not have asked for. Before a credential is touched, and with no if: of its own to be stepped around, release_checks.py crate-publishable gives one of three answers:

  • red - the version is not a pre-release, or the checkout's own rt/Cargo.toml does not say it, or its tag is missing or names another commit. release.yml only ever tags the commit whose tests passed, so a tag that names this commit is the statement that they did.
  • publish - go ahead. Every step that touches a credential or the registry is conditioned on this answer and on nothing else.
  • skip - crates.io has it already. Not a refusal and not a failure: there is nothing left to do, the publish steps are skipped, and the run ends green having published nothing. A re-run lands here, and so does a version that landed while somebody was typing.

check_release.py holds every one of those to a fixture, the workflow's shape included: the guard comes before the token and before the publish, the guard has no condition, each step after it carries the guard's own answer and nothing else, the last step runs whichever way the guard went, and nothing in that workflow publishes anywhere but crates.io.

What a person does once, and only once: the trusted publisher on crates.io names a workflow file, and it now names publish-crate.yml rather than release.yml. RELEASING.md says where to change it. No token is involved and none is stored; the entry simply pointed at the wrong file.

compatibility: a pre-release publishes the sabline-rt crate and a GitHub release marked pre-release and nothing else, so nothing a user of 8.6.0 has is touched - not PyPI, not npm, not the VS Code Marketplace, not the MCP registry, not the Action pins, and not any "latest" anywhere. The Python package's version files still say 8.6.0, and the release gate refuses a pre-release whose commit moved one.

v9.0.0-alpha.1 and v9.0.0-alpha.2 stay where they are, tags and all, neither moved nor deleted nor retagged - a published thing stays where it is (3.4 was not retagged; 6.0.0 was yanked, not deleted). alpha.1's crate is on crates.io because a person put it there, which is how a crate's first version always gets there. Neither has a GitHub release, and the two entries below say why. What alpha.1 was meant to be, this is.

9.0.0-alpha.2 - The parser, twice, and a release that publishes #

9.0.0-alpha.1's content, released. The alpha before it tagged v9.0.0-alpha.1 and then published nothing, because of a defect in release.yml that this entry is mostly about. Its crate reached crates.io only because a person put it there by hand, which is how a crate's first version always reaches crates.io - trusted publishing can only be configured on a crate that already exists. What 9.0.0-alpha.1 was meant to be, 9.0.0-alpha.2 is; the CHANGELOG entry below it says what exists and what does not, and none of that changed.

v9.0.0-alpha.1 is not moved, deleted or retagged, for the reason 3.4 was not retagged and 6.0.0 was yanked rather than deleted: a published thing stays where it is. The tag stands, its crate stands, and this entry is the record of why there is no GitHub release under it.

What was wrong #

GitHub propagates a skip along needs transitively, and a job that rescues itself with !cancelled() does not rescue the jobs after it. Its own result is success, and the skip still reaches them.

9.0.0-alpha.1 added the crate job to what tag waits on. On a pre-release the seven Python build jobs are skipped; tag rescued itself and ran; and every job below tag inherited the skip - so crates_io did not run, and prerelease_github after it did not either. The run ended green having tagged and published nothing.

The same defect would have broken an ordinary release worse. There crate is the skipped job, so pypi, npm, vscode, github_release, mcp_registry, attach_attestation and consistency would all have been skipped: a release that tags and publishes nothing, on the path every release takes.

crates_io, pypi, npm and vscode now carry !cancelled() and check the job before them by result. Every job further down already had that shape - the trap was known, and 9.0.0-alpha.1 walked into it by changing what tag waits on.

Why the fixtures did not catch it #

check_release.py runs release.yml job by job against a stand-in, and 137 of its checks were green on the commit that shipped this defect. Its decides implemented the semantics a reader would assume - the implicit success() covering a job's direct needs - and GitHub's covers the transitive closure. The simulation agreed with the mistake, so it had nothing to say.

decides now models the closure. Against the workflow as 9.0.0-alpha.1 shipped it reproduces both failures exactly, the real one included. And thirteen structural checks were added: every job below tag must carry a status check function in its condition, which is the rule that keeps the next change to tag's needs from doing this again.

compatibility: a pre-release publishes the sabline-rt crate and a GitHub release marked pre-release and nothing else, so nothing a user of 8.6.0 has is touched by this release either - not PyPI, not npm, not the VS Code Marketplace, not the MCP registry, not the Action pins, and not any "latest" anywhere. The Python package's version files still say 8.6.0, and the release gate refuses a pre-release whose commit moved one.

9.0.0-alpha.1 - The parser, twice #

The first alpha of 9.0. decisions/0002-runtime-in-rust.md says why there is a second runtime and plan/9.0.md is the ladder; this is its first rung. sabline-rt reads a Sabline program and builds the same tree the Python parser builds. That is all it does.

This is a pre-release. It publishes the sabline-rt crate to crates.io and a GitHub release marked pre-release, and nothing else. 8.6.0 is still what pip install sabline-lang gives, what npm gives, what the Marketplace lists and what the MCP registry serves; the Action pins still name 8.6.0's commit; sabline --version still says 8.6.0. The Python package's version files were not touched, and the release gate refuses a pre-release whose commit moved one.

What exists #

rt/, a Cargo workspace holding one crate, sabline-rt. Edition 2021, minimum Rust 1.82, no dependencies at all. A lexer, a parser and a canonical dump of the tree, in fourteen files. #![forbid(unsafe_code)] at the workspace root, so there is none; rt/README.md states the rule for the day a later alpha needs some - a // SAFETY: comment naming the invariant and a test that would fail if it broke - and check_rt.py enforces it, against fixtures of its own so the scan is known to catch each way of getting it wrong rather than only to run.

One canonical AST dump, written the same way by both. sabline ast --json <file> from the Python package, sabline-rt ast <file> from the crate: keys sorted, no whitespace, ASCII only, bytes rather than text; a whole number as decimal text, because the reference's integer literals are arbitrary precision; a float as the shortest decimal that reads back as the same double, written d[.ddd]eE, because neither language's default printing is the other's. rt/README.md states the format. It is not a stable interface, it is not in tests/api/golden.json, and nothing but the gate reads it; sabline ast is not in sabline --help, and answers --help for itself.

The absence of indentation is a decision rather than a default, and it was made from a measurement: two spaces a level makes a document O(depth squared), and the dump of the deepest program the parser accepts - 40 KB of source, which the adversarial corpus holds on purpose - was 352 MB of mostly spaces, which the gate would have written twice on every leg of CI. Compact, the same document is 316 KB and the gate takes 10 seconds instead of 16.

The agreement gate, check_agreement.py, over every Sabline source this project has:

CorpusPrograms
examples/, including lib, ops and runner112
stdlib/14
benchmark/corpus/85
tests/error_messages/84
sabline-spec's conformance corpus216
the lie corpus (check_prover_lies.py)146
check_sandbox.py's escapes and honest programs58
check_refusals.py's wrong programs24
every example cut at fifteen points1,680
the adversarial corpus (agreement_edges.py)113
paths that are not files2
total2,534

2,534 programs, 2,534 agreements, 0 differences. 1,395 of them parse and 1,139 are refused, between them reaching every code the lexer and the parser give: E000 (158), E001 (2), E002 (4), E100 (409), E101 (539), E102 (11), E407 (3), E511 (3), E512 (6), E562 (4). The six corpora plan/9.0.md names hold no program the lexer or the parser refuses - every one of them parses and is refused later or not at all - so the other five are there because a gate that only ever compared trees would say nothing about the codes, the messages, the fixes and the lines, which is half of what this alpha claims.

The gate cannot be turned off, and it is proven to go red. check_gate.py reads the gate's syntax tree and fails on os.environ, getenv, a configuration file or any use of the command line beyond the corpus paths; runs it with eight plausible disabling variables set and requires the number of compared programs to be unchanged; and builds sabline-rt three times with one difference injected each time - an error code, an error message, a field of the tree - and requires the gate to go red for each, over the same number of programs, so that it found a difference rather than stopped early. check_workflows.py holds the agreement job to having no if:, no continue-on-error, no matrix and no environment.

Fuzzing. cargo fuzz has two targets, parse and dump, on arbitrary bytes; fuzz_parsers.py --target agreement feeds the same generated input to both parsers and requires the same answer, coverage- guided as the rest of that file is. Both run briefly on every push and longer in monthly.yml. No input may make sabline-rt panic, abort, overflow its stack or hang: the library publishes PARSE_STACK and on_parse_stack, every entry point that parses runs the parse there, and tests/limits.rs holds the deepest program the parser accepts against that number. A stack overflow is not a panic - it is the process going away with no message - so it is held by a measurement and not by hope.

CI. cargo build, cargo test, cargo clippy -D warnings, cargo fmt --check and check_rt.py on every leg of the matrix, arm64 and Windows included, with Swatinem/rust-cache; and four legs of their own: agreement (the gate, its own test's three injections, and two minutes of the differential fuzzer) at 169 s, rt_fuzz (a minute of each cargo fuzz target) at 144 s, supply_chain (cargo deny check - advisories, licenses, bans and sources, with a committed rt/deny.toml naming all five target triples a release builds for) at 30 s, and msrv (a build and test on 1.82, the version rt/Cargo.toml states and check_rt.py holds it to) at 22 s.

The cost, measured against the last green run of main before this branch: the eighteen matrix legs together went from 20,250 s to 21,016 s, up 3.8%, and the median leg from 938 s to 1,051 s. Wall clock did not move - 2,781 s to 2,719 s, which is noise - because the four new legs run beside a matrix whose slowest leg is three quarters of an hour. Per-leg noise on these runners is larger than the change: three legs got faster. cargo deny runs once rather than eighteen times, because it reads the lockfile and deny.toml already names every platform a release builds for; running it on each leg would tell nobody anything one run does not.

What does not exist #

sabline-rt does not check types, does not check effects, does not check termination, does not prove anything, does not run a program, does not hold a budget, does not ask the operating system for anything and does not write a receipt. It cannot tell you whether a program is safe to run. The Python package is what does that and is the reference: where the two disagree, the Python package is right and sabline-rt has a defect, until the demotion criteria in plan/9.0.md are met, and they are not. plan/9.0.md's alphas 2 to 8 are the rest, and none of them has started.

Nothing calls sabline-rt. sabline run, sabline.run and sabline.Pool are the Python interpreter, exactly as in 8.6.0. There is no C ABI and no binding. The crate is published so that the alpha is a shippable tag with a version anyone can point at, which is what plan/9.0.md asks an alpha to be.

What the reference was found to do that the spec does not say #

Writing the parser twice is what finds these, and four are worth the record. Three are copied into sabline-rt as they are, because the Python package is the reference and a second implementation that fixed things quietly would be a second specification. One was changed, in Python.

  • \d is every Unicode decimal digit. The NUMBER and FLOAT patterns are \d+ and \d+\.\d+, and \d in a str pattern is category Nd, not the ten ASCII digits - so let x = ١٢ is Num(12), because int() reads each character by its decimal value, and let x = ١.٢ is FloatNum(1.2). [A-Za-z_] is ASCII in the same regular expression, so a non-ASCII letter is E000 and not an identifier: the two rules disagree about what a character is. SPEC.md says nothing about either. rt/crates/sabline-rt/src/unicode_nd.rs carries the set, generated from CPython's own \d by scripts/gen_unicode_nd.py, and it follows the Unicode version of the CPython that generated it - so two CPythons this project supports can disagree with each other about whether a recently assigned code point is a digit. No program in any corpus is affected.

  • open(path, encoding="utf-8") in the loader decides three things the lexer then depends on, and none of them is written anywhere: strict UTF-8, so bytes that are not UTF-8 are E512 - with a message that says "cannot import" about a file nobody imported, because the entry file goes through that branch; universal newlines, so \r\n and a lone \r are both \n before the lexer sees one, which means the lexer's own [ \t\r]+ never sees a \r from a file; and a byte-order mark left in place, because utf-8 is not utf-8-sig, so a file that begins with one is E000 on line 1.

  • Parser.lambda_n is a class attribute the parser never resets, so the generated name of a lifted function value - fn#N, for#N - depends on how many the process has parsed before, not on the file. sabline ast --json sets it to zero for each file so that a dump is a function of that file alone; nothing else in the package does.

  • E000's message was not stable across the Pythons this project supports, and that one was fixed. It was f"unexpected character {source[pos]!r}", and repr writes a character raw when the Unicode database calls it printable - so the same program gave one message on a CPython with one Unicode version and another on a CPython with another, and a second implementation could match at most one of them. It is !a now, which escapes every character outside ASCII and asks the database nothing. The gate found it: unexpected character 'é' against unexpected character '\xe9', on the first run that had a corpus with a non-ASCII character in it. Message text is prose, which STABILITY.md does not cover, and nothing that was a code, a line or a refusal changed.

The release, and what a pre-release may do #

release_checks.py and release.yml learned what a pre-release is. A version of the form X.Y.Z-alpha.N (or beta, or rc) in rt/Cargo.toml is a pre-release; it needs a CHANGELOG entry of its own, it must be newer than every tag, and the Python package's six version files must still say what the newest ordinary release said - the gate refuses the commit if one moved, because a pre-release that changed what PyPI would serve is not a pre-release of the crate.

A pre-release run tags the commit, builds and packages the crate from the committed lockfile, publishes it to crates.io by trusted publishing (OIDC - no token is stored, and MAINTENANCE.md's secrets table gains no row), and makes a GitHub release marked pre-release. It does not publish to PyPI, npm, the VS Code Marketplace or the MCP registry; it does not move the Action pins; it does not move any "latest" anywhere. A last step asks each of those afterwards and fails if one moved, because reading the workflow and believing it is not the same as checking.

check_release.py holds both halves to fixtures: an alpha publishes the crate, the tag and the pre-release GitHub release and nothing else, with every other job skipped rather than failed; RELEASE_PAUSED stops a pre-release too; a crates.io publish that fails leaves no GitHub release behind it, and re-running the failed jobs finishes it with each publish made once; and an ordinary release is unchanged, with no crate job run.

perf: the numbers this entry would carry are not measured for a pre-release: it publishes no Python artefact, so there is nothing whose time changed. plan/9.0.md's alpha.8 is where sabline-rt gets a performance floor of its own.