# AUIS Revival Roadmap

Last updated: 2026-08-17

This document is intended primarily for AUIS revival participants,
with a summary of what's running, what's active, what's next, and the
high level rationale for the various tradeoffs made.

You can make a deep dive into the details in the documents below:

| Doc | Description |
|---|---|
| `revival.md` | What's been accomplished, told as a narrative, for readers who already know AUIS or are meeting it for the first time |
| `porting-assessment.md` | Bug-class root-cause analysis and the full ANSI C conversion plan |
| `porting-changelog.md` | Dated log of every fix |
| `claude-history/README.md` | Index of retired working docs — task prompts, investigation write-ups, rollout runbooks, and this document's full pre-2026-08-07 history (`roadmap-old.md`) |
| `quickstart.md` | Build and run instructions |
| `mail-quickstart.md` | IMAP/SMTP mail setup |

---

## Status

`ez`, `help`, and `messages` all run. The codebase has been through a
multi-month modernization effort — what that involved and why is in
Key tradeoffs, below — which is now done for its original scope. Most
applications and insets work well; the table below tracks what's
solid versus still rough. Active work is listed under Projects.

---

## Applications and insets

| Component | Status | Notes |
|---|---|---|
| `ez`, `help` | Fully working | |
| `bush`, `org`, `chart`, `layout` | Fully working | |
| `messages` | Mostly working | Runs against a local mailbox and against a live IMAP mirror, but the IMAP side still needs a manual `imapsync` step — see Projects → AMS over IMAP/SMTP |
| `typescript` | Broken | Crashes on launch: PTY allocation fails, and a missing NULL check turns that failure into a crash instead of a clean error |
| `ness` (scripting) | Inert | Blocked on a bison grammar extension specific to the (disabled-on-arm64) Andrew bison fork |
| Core embedded insets — `eq`, `table`, `fad`, `fnote`, `bp`, `srctext`, `figure`, `value`/slider/button, `pushbutton`/`link`, `lset`, `org`, `apt`/`cel`/`arbiter` | Fully working | `figure` has one open cosmetic bug: italic-text sizing is non-monotonic (10pt renders bigger than 12pt). `eq` has one open bug: deleting a multi-line equation's lines only reduces each to a single "zilch" placeholder rather than merging down to one line — a long-standing bug, not new to this port (see `porting-assessment.md` §23) |
| `calc`, `zip` (contrib) | Fully working | |
| `clock` | Fully working | |
| `raster` (as an embedded inset) | Fully working | |
| `contentv` (Table of Contents) | Fully working | An earlier report of it ignoring enumerated headings was a false alarm — root cause was input focus being inside an embedded inset rather than the document itself when the ToC view was opened |
| `convertraster` (standalone CLI) | Fully working | Fully tested 2026-08-08; three bugs found and fixed (see `porting-changelog.md`) |
| `image` (JPEG/TIFF import) | Fully working | Fixed 2026-08-08: TIFF import was totally broken (four LP64 struct/stride bugs in vendored `libtiff`); JPEG/TIFF solid-color render was an unrelated `xgraphic.c` variable mixup — see `porting-changelog.md`. GIF import shares the same render path so is likely also fixed, but wasn't retested |
| `htmlview` | N/A — superseded | The standalone viewer itself hasn't changed; real-world HTML mail rendering now goes through a separate, purpose-built parser+renderer wired into `messages`/`cui` instead — see Projects → HTML mail rendering. Retargeting `htmlview` onto that same parser is an optional later step, not yet done |

---

## Projects

Active work. Each entry starts with a short description and next step;
details will fill in as each project actually unfolds, rather than
being front-loaded here.

### AMS over IMAP/SMTP

- **Description:** Local-mailbox AMS talking to a real IMAP/SMTP
  server through a one-way mirror. Read, browse, send, and writeback
  all work.
- **Next step:** The mirror integration is still manual and clunky
  (a separate `imapsync` step) — thread it into the normal workflow.
  After that, XOAUTH2 auth.

### HTML mail rendering

- **Design doc:** `html-mail-rendering-design.md` — sanitization
  allowlist, table strategy (see below — pivoted mid-project), image
  strategy (`image__ReadOtherFormat` capability check + fallback), and
  the explicit renderer-level fallback contract. Work happens on the
  `html` branch (`~/src/AUIS/html/`), not `trunk/`.
- **Status: core rendering done, verified live against real mail.**
  MIME body decoding was already solid (see `porting-changelog.md`'s
  2026-08-09/2026-08-10 entries); the actual rendering pipeline is now
  built and wired in, in three stages, mirroring how `mimepart.c`
  already sits outside ATK so both `cui` and `messages` can share it:
  1. **Parser** (`htmlpart.c`, 2026-08-15) — ATK-independent
     tokenizer + sanitization-allowlist tree builder. 20/20 tests.
  2. **Plain-text renderer** (`htmltext.c`, 2026-08-15) — walks the
     parse tree to text; what `cui` always uses. 13/13 tests.
  3. **ATK-styled renderer** (`htmlatk.c`, 2026-08-16) — real clickable
     links, inline images, styled text; wired into `text822.c`/`cui`
     the same day, retiring the old `mimepart_HtmlToText` shim.
  Midway through Stage 3, the originally planned table strategy
  (Andrew's `table`/`spread` grid object) turned out not to fit real
  mail: it's a fixed-pixel grid that doesn't reflow with window width,
  the way marketing HTML's near-universal wrapper-table boilerplate
  needs. Pivoted to `lset`/`lpair` instead (percentage-based, does
  reflow), plus a "peeling" heuristic that collapses that wrapper
  boilerplate down to real content — see the design doc's "Table
  strategy" section for the full reasoning and known imperfections.
  That pivot surfaced (and fixed) a real core-ATK bug along the way —
  `BackSpace`'s end-of-file line padding could infinite-loop `^V`
  scrolling against a disproportionately tall embedded view — plus,
  today, two more real-mail rendering bugs (isolated blank spacer rows
  never coalescing, hidden `display:none` preheader text rendering as
  visible body text), both found live against a real National Grid
  message (now saved at `revival/tests/national-grid.html`) and fixed.
- **Next step:** Two loose ends, both deliberately deferred, neither
  blocking further use:
  - A distinct, non-HTML-specific core-ATK bug in scrollbar
    endzone-click handling (jump-to-end cycles through 3 states rather
    than settling) — see Open issues below and `revival.md`.
  - *Optional, later:* retarget `htmlview`'s own standalone viewer
    onto the same shared parser, so there's one HTML engine in the
    tree rather than two. `htmlview`'s composition/authoring side
    (hand-building a document, not parsing untrusted wire HTML) is a
    separate concern and doesn't need to change.
  With core rendering done and verified, the open question is whether
  to keep hardening on the `html` branch or merge it back into `trunk`
  at its current, already-useful state (the `andrew-6.4` precedent) and
  pick up the rest later — not yet decided.

### Coverage inventory

- **Description:** Most apps and insets are confirmed either working
  or broken (see the table above), but a handful have simply never
  been exercised, and aren't distinguished from either group yet.
- **Next step:** Inventory untested apps and insets.

### Application work

- **Description:** Track and fix applications known broken.
- **Next step:** `typescript`'s PTY crash is the one broken
  application left (see table above).

### Inset work

- **Description:** Track and fix insets known broken or buggy.
- **Next step:** `figure` italic-text sizing, `eq`'s multi-line cleanup
  bug (see table above); `convertraster` CLI bugs and `image` JPEG/TIFF
  import fixed 2026-08-08.

---

## Open issues

Smaller items that don't fit the tables above.

**Little annoyances**
- Arrow keys and a "Meta" key aren't bound yet
- ~~`make Clean` transiently deletes `src/atk/adew/Arb`~~ **RESOLVED 2026-08-12**: this undersold it — a build had actually clobbered the tracked `.ez` arbiter document with the generated `arb` launcher (same directory entry on case-insensitive filesystems) back in June, and the corruption had been committed, not transient. Restored from the original import on both branches; `Imakefile` now builds the launcher under a collision-proof name and renames it only at install time. See `revival.md` → "Old bugs never found till now."
- `messages`: intermittent, non-fatal `X_OpenFont BadValue` X errors during normal folder browsing — not root-caused
- `filetype.c DeleteEntry`: a bogus-free risk and an apparently-inverted condition, flagged by the compiler, never observed to actually fire
- `runapp -d` with no app-class argument segfaults instead of printing usage — pre-existing since the 1988 source, not a regression
- **RESOLVED 2026-08-12 (docs):** a fresh checkout has two separate hardcoded-path spots, not one — `site.h`'s `DEFAULT_ANDREWDIR_ENV` was documented, but `config/Makefile`'s `BASEDIR` (baked in from `site.h` by imake at Step 2, and not reliably self-regenerated afterward — see the fossil-mtime caveat elsewhere in this file) wasn't, and silently stays stale if `site.h` gets fixed after Step 2 has already run once. Reported independently by an outside builder hitting exactly this. `quickstart.md`'s "Site configuration" section now covers both.
- Clicking a scrollbar endzone to jump to the end of a document cycles through 3 distinct results rather than settling — root cause identified (`textview`'s `endzone()`/`setframe()`, `src/atk/text/textv.c`, fires a second chained scroll operation on every discrete click, not just a held-down repeat), not yet fixed. Found chasing an HTML-mail-rendering report but is a general ATK scrollbar bug, not HTML-specific. See `revival.md` → "Open issues" for the full writeup.

**Heisenbugs** (intermittent, low reproducibility)
- Xlib display-lock self-deadlock: reproduced once, root cause identified (`_XLockDisplay` re-entered from inside `XRefreshKeyboardMapping`, triggered by a keyboard-mapping-change event) but not yet fixed. Current best explanation for the older, harder-to-pin-down "`^V` scroll hang."
- A rare report of text drawing into the menu area — not reproduced since first observed
- Menu-bar posting sometimes takes a fixed ~510ms quantum to draw (perceived multi-second delays are two or more of these back-to-back, not one longer stall): **root cause confirmed 2026-08-11, not an AUIS defect, no fix available or needed.** Live process sampling of XQuartz's own `X11.bin`, correlated directly against `mdbg`-logged stall timestamps, showed real dispatch time (~1278ms) closely matching independently measured stall time (1533ms across 3 stalls in the same window) inside one specific chain: `ProcClearToBackground` (servicing `DrawMenuItems`'s `XClearWindow`) → `RootlessStartDrawing` → `xp_lock_window` → `SLSConnectionSynchronizeSLSCATransaction` → `_SLSTransactionWaitSource` — XQuartz blocked waiting for macOS's own SkyLight/Core Animation window compositor to synchronize a transaction before it's allowed to lock the popup window's backing store and paint. This is genuine, external, server-side latency in XQuartz's rootless-drawing pipeline, confirmed unrelated to AUIS's `XGrabPointer` usage (already ruled out) and to App Nap or frontmost/background app status (both tested and ruled out as the *why* behind the ~510ms specifically). The *why* the wait is sometimes instant and sometimes ~510ms remains unresolved (some undocumented SkyLight/WindowServer-internal cadence, not app-focus-dependent) but the *where* is proven and outside AUIS's or XQuartz's control — treated as an accepted characteristic of XQuartz on modern macOS, not pursued further absent a second X server or SkyLight documentation. See `porting-assessment.md` → "Intermittent ~500ms menu-bar posting delay" for the full inventory. Diagnostic tracing left in the tree, off by default (`MenuDebugTrace` profile switch; `MENUDBGTRACE` env var for the popup-menu path), now timestamped for future correlation work.

---

## Key tradeoffs

**Modernization strategy** — how the code itself was brought forward:

- **Compiler leniency, not mass modernization, as the base strategy.**
  Leave 1990s K&R source as-is; suppress the resulting compiler noise
  with flags (`-Wno-implicit-int` etc.) instead of rewriting the tree.
  *Why:* one automated mass-conversion attempt was tried first and
  reverted — it introduced bugs faster than it fixed real ones.
  `porting-assessment.md` → "Getting K&R-era source to build under a
  modern compiler."
- **The later ANSI C conversion (see Major milestones) went
  directory-by-directory with the compiler as auditor, not via a
  rewrite tool.** *Why:* `.ch` class-interface files already carried
  full ANSI signatures that the class preprocessor was silently
  discarding; emitting them let the compiler type-check every call
  site tree-wide before any file was hand-edited, catching decades of
  signature drift as compile errors instead of grep hits.
  `porting-assessment.md` §14.
- **`gnu89`, not `gnu99`/`c99`.** *Why:* verified empirically — the
  actual blocker isn't classpp's untyped-dispatch idiom (compiles
  identically under either standard), it's reserved-keyword collisions
  from 35 years of K&R-era identifiers (e.g. a parameter literally
  named `restrict`). A `c99` migration is possible later but needs a
  tree-wide reserved-word audit first. `porting-assessment.md` →
  "Why gnu89."
- **`dlopen()`/`dlsym()` instead of reviving the original
  platform-specific loaders.** *Why:* ~70 lines replacing ~500;
  POSIX-standard, no per-platform loader code to maintain.
- **System `bison`, not the vendored Andrew bison fork.** *Why:* the
  vendored fork hangs on arm64. `mkparser` was patched to match modern
  bison's output format instead.

**Scope decisions** — what's deliberately not part of this build:

- **Console (terminal-emulator) subsystem excluded.** *Why:* its
  interprocess-communication layer would need its own rewrite; set
  aside rather than taken on alongside everything else. Known side
  effect: two icon fonts an unrelated animation inset depends on
  happen to live in Console's own build directory, so they go unbuilt
  too unless Console (or just its font directory) is built standalone.
- **`vui`/`tm` (curses terminal clients on the removed BSD `sgtty`
  API) and `bdffont` left out rather than fixed.** *Why:* not needed
  for the GUI `messages` path; `bdffont` additionally has no working
  build machinery left in this checkout at all.

**Mail architecture** — how AMS talks to the outside world:

- **Local mbox store plus a one-way IMAP mirror (Thunderbird-style
  cache), not a revived AMDS delivery daemon.** *Why:* AMDS's
  shared-filesystem delivery model is a dead end on a modern
  single-user Mac — there's no shared AFS cell to deliver into. The
  local store stays authoritative; an IMAP sync agent mirrors a real
  mailbox into it. Full design: `ams-IMAP-project.md`.
- **Hand-rolled IMAP client, not an existing library.** Decision and
  reasoning recorded in `ams-IMAP-project.md` §8.

---

## Major milestones

- **2026-08-16** — HTML mail rendering's ATK-styled renderer (Stage 3)
  wired into `messages`/`cui`, retiring the old plain-text-only shim —
  real-world mail now gets clickable links, inline images, and styled
  text instead of stripped tags. Table strategy pivoted mid-stage from
  Andrew's `table`/`spread` object to `lset`/`lpair` (the former
  doesn't reflow with window width); a real core-ATK scrolling bug and
  two more real-mail rendering bugs found and fixed the same day,
  verified live against an actual message.
- **2026-08-07** — The ANSI C modernization effort (see Key tradeoffs)
  reached its conclusion: the entire active codebase now compiles
  clean under full strict-C compiler settings, closing out a
  systematic, decades-overdue type-safety pass across the whole tree.
  Dozens of long-latent bugs found and fixed along the way are
  cataloged in `revival.md`.
- **2026-07-23** — AMS-over-IMAP writeback complete: local edits to a
  mirrored mailbox replay back to the real IMAP server. Feature-complete
  except OAuth2.
- **2026-07-17** — First SMTP mail sent by AMS: a scripted client
  composes and submits over TLS to a real mail provider, authenticated,
  end-to-end — roughly 35 years after AMDS last moved a message by
  copying files through a shared filesystem.
- **2026-07-10** — Every method call site in the active tree became
  compiler-checked for the first time, via typed class dispatch — the
  first step of the ANSI C modernization effort above.
- **2026-07-05** — `messages` runs against a local mail store for the
  first time.

---

## Stretch goals

- **Pie menus** — AUIS's menu architecture suits Don Hopkins' pie-menu
  design well; a collaborative implementation project once other work
  settles down.
- **`ez2md` improvements** — table/eq/figure/fad/image/link objects
  still render as placeholder comments in the `.ez`-to-Markdown
  converter; batch-converting the archive's FAQ/newsletters/papers to
  Markdown is unstarted.
- **`c99`/`c11` migration** — not planned for the foreseeable future.
  `gnu89` is load-bearing today only because of ~35 years of K&R-era
  identifiers that collide with later reserved words (`restrict` is the
  one confirmed hit); a migration needs a tree-wide reserved-word audit
  first. `porting-assessment.md` → "Why `gnu89`."

---

## Further reading

Testing procedure material that used to live in this file (the
subsystem dependency lattice, the per-inset test-document table, the
regression checklists) is preserved in `claude-history/README.md` →
"Retired top-level docs" → `roadmap-old.md`. It may get a dedicated
home of its own in a later documentation pass rather than staying only
in the retired file.
