One Recipe, Three Runtimes

What it takes to make a glitch behave the same way in a browser, on a print farm, and on a phone camera at 30fps — and the four places it doesn't, one of them provably.

I build glitch art. For a while the tooling was a folder of exports — you make a thing, you like it, you save the PNG, and the thing that made it is gone. Every piece is a one-off, because the process that produced it was a sequence of mouse gestures nobody wrote down.

The fix sounds obvious and isn't: make the process the artifact. Store the effect stack as data, not as a rendered result. Then the same stack can be re-run at 1080px for a wallpaper, at 6000px for a 24×36 print, or against a live camera feed at 30fps for a lens on someone's phone.

That's the whole project. Two codebases: Aura Labs (the studio — a browser editor wrapped in a macOS app) and Aura Lenses (the delivery layer — a phone camera app, an entitlements API, and the shader engine that runs the same stacks live). What follows is the architecture, the places it holds, and the four places it doesn't — one of them provably.

1. The contract

Everything hangs off one JSON shape:

{
  "version": 1,
  "lab": "glitch",
  "designDim": { "width": 2048, "height": 2048 },
  "layers": [
    { "effect": "sliceGlitch",   "params": { "seed": 13, "bands": 24, "maxShift": 60 },
      "opacity": 1, "blend": "normal", "mask": { "type": "none" } },
    { "effect": "channelShift",  "params": { "amount": 10 } },
    { "effect": "scanlines",     "params": { "gap": 3, "intensity": 0.35 } }
  ]
}

That's a preset called Signal Lost. Three layers, in order: horizontal sync tears, then a composite cable, then a CRT.

The ordering is not incidental, and it's the one piece of aesthetic doctrine baked into the data model. Destructive effects don't commute — glitch-then-mirror and mirror-then-glitch are different images, not different intensities of the same image. So the stack reads as a signal chain:

Memory → transmission → display. Corruption happens to the stored bytes first, then to the signal carrying them, then to the surface displaying it.

Get that order backwards and you get an image where a CRT scanline has been torn by a memory fault, which is nonsense your eye notices before your brain does.

Presets carry that reasoning as metadata the schema ignores:

{
  story: 'Horizontal sync tears in memory, then a composite cable, then a CRT.',
  spine: { effect: 'sliceGlitch', param: 'maxShift' },  // the "more" dial
  seedPolicy: 'fixed'                                   // vs 'edition'
}

Thirty presets ship with that annotation. spine matters more than it looks: every preset has exactly one parameter that means “more of this,” and naming it is what lets a non-technical surface expose a single slider instead of eleven.

2. Runtime one — the browser

The reference implementation is engine.js: about 1,850 lines of plain JavaScript, 61 effects, no dependencies, operating on raw pixel buffers.

Two properties matter. First, effects never mutate their input — every pass returns a new buffer. Second, the pipeline caches by layer signature:

const sig = (L) => (L.enabled === false ? 'off'
  : JSON.stringify([L.effect, L.opacity, L.blend, L.params, L.mask]));

Dragging a slider on layer 7 of an 8-layer stack recomputes layers 7 and 8. Layers 1–6 are reused by reference, which is only safe because of the immutability rule. That's the difference between an editor that feels alive and one that stutters every time you touch anything.

Layers get per-layer opacity, eight blend modes, and area-of-effect masks in five flavors — none, radial, linear, freehand path, and image. The parametric ones are a few floats in JSON and therefore resolution-independent, which becomes the whole ballgame in section 4.

The one thing not written in JavaScript

Vector export runs through a roughly 90-line Rust crate wrapping vtracer, compiled to a ~172KB WebAssembly artifact. One gotcha worth writing down because it cost a day: crates.io only publishes vtracer through 0.6.5, which depends on an old fastrand and has no wasm feature — its random number generator panics under wasm. You need the git tag 0.6.12. Pin the commit in your lockfile or your reproducible build isn't.

Tracing a smooth gradient produces thousands of color-layer paths and multi-megabyte SVGs. Posterize first and the same image traces in under 150ms into a few KB. The trace runs in a Web Worker so a bad input stalls a thread instead of the UI.

3. Runtime two — Node, and the cheat that isn't one

Aura Labs exposes an MCP server so agents can render recipes headlessly. The obvious implementation is to port the effects to the server. The actual implementation is one line:

enginePromise = import(webModuleUrl('engine.js'));
// ...
const out = engine.applyRecipe(base, recipe);

It imports the same file the browser runs. Not a port, not a shared library, not a separate build — the literal engine.js, loaded as a module under Node.

This feels like cheating and it is the single best decision in the codebase. Server output is identical to browser output by construction. There is no drift, because there is nothing to drift from. Every effects port I've seen in a pipeline like this eventually grows a bug where the headless render is 2% different from the preview, and nobody notices until a print comes back wrong.

The cost is real but small: the engine has to stay DOM-free. No canvas as a scratch buffer, no 2D context for compositing, and — the annoying one — no relying on the browser's JPEG encoder. The generation-loss effect needed a hand-written deterministic DCT precisely because Chrome's encoder isn't available in Node and isn't deterministic across versions anyway.

There's a fourth surface running the same file, incidentally: the macOS app's tooling hashes the preset file and compares it against what the installed app is serving. If they differ, the live tools emit a drift warning. When your architecture depends on “it's literally the same file,” you should verify that it's literally the same file.

4. The resolution law

This is the constraint that made the whole thing a pipeline instead of a folder.

Every destructive parameter must be expressed as a fraction of the frame, never a fixed pixel count.

A 200px smear is 18% of a 1080px-wide wallpaper and 3% of a 6000px-wide print. Upscale a finished glitch and the artifacts blur into mud. Re-run at full resolution with unscaled parameters and the artifacts vanish — the tears are still 200px, they're just invisible at that size. Both failure modes look like “the print didn't come out.”

So a recipe carries the size it was authored at, and every runtime rescales pixel-denominated params on the way in:

const designDim = recipe.designDim || Math.max(base.width, base.height);
const scale = Math.max(base.width, base.height) / designDim;
// ...
const p = scaleParams(layer.effect, { ...defaults, ...layer.params }, scale);

Which params scale is a hand-maintained table — channel-shift amount, slice-glitch max shift, scanline gap, wave amplitude, seventeen effects in total.

The subtlety is what's deliberately absent. The 8×8 grid in the JPEG effects is not in that table, because it isn't an aesthetic dimension — it's a property of the codec. A JPEG block is 8×8 at every resolution that has ever existed. Scaling it would mean simulating a codec that doesn't exist. Same for the restart-marker interval and the bitstream offsets.

5. Runtime three — the phone GPU, where the contract breaks

Live camera means a full effect stack per frame at 30fps. The CPU engine renders a 2048px still in a few hundred milliseconds. That is roughly 10× too slow, and it's the wrong shape besides — a per-pixel JavaScript loop on the UI thread is a jank generator.

So the phone path is 63 GLSL fragment shaders, run as ordered passes, each rendering the previous pass's output into the next:

for (final pass in passes) {
  final shader = _programs[pass.effect]!.fragmentShader();
  shader.setFloat(0, current.width.toDouble());
  shader.setFloat(1, current.height.toDouble());
  var i = bindEffectParams(shader, pass);
  shader.setFloat(i++, pass.opacity);
  if (effectUsesTime(pass.effect)) shader.setFloat(i++, time);
  shader.setImageSampler(0, current);
  current = await _drawPass(shader: shader, input: current);
}

Passes are ordered, not flattened — the signal-chain semantics survive. Blur, bloom, and sharpen need multiple passes and get bespoke handling.

Sixty-three shader files cover all sixty-one effects, a few pulling double duty across multipass helpers. On paper, complete coverage. It is not a complete port.

What the live path actually loses

Pixel sort: mathematically impossible in a fragment shader

A real pixel sort finds a contiguous span of pixels above a luminance threshold and sorts them. Sorting is sequential and non-local: the output value at pixel x depends on every other pixel in its span. A fragment shader computes each output pixel independently, with no knowledge of its neighbors' results.

You cannot write this shader. What you can write is something that reads like it:

float lum  = dot(orig.rgb, vec3(0.299, 0.587, 0.114));
float mask = lum < u_threshold ? 1.0 : 0.0;
float smear = mask * 24.0;
vec2  dir  = u_cols > 0.5 ? vec2(0.0, smear) : vec2(smear, 0.0);
vec3  a    = texture(u_texture, (coord - dir) / u_size).rgb;
vec3  b    = texture(u_texture, (coord + dir * 0.5) / u_size).rgb;
vec3  fx   = mix(orig.rgb, (a + b + orig.rgb) / 3.0, mask);

A luminance-keyed directional smear. It has the gesture of a pixel sort — bright rows draining sideways — with none of the mechanism. On a moving camera feed at 30fps you will not catch it. On a still export at print resolution you absolutely will, which is why stills never take this path.

Data bend: the CPU one is real, the GPU one is theater

The CPU version does actual file-level databending. It runs a pinned in-repo JPEG codec, flips entropy bytes in the compressed stream, and decodes the damage. Restart markers every 8 MCUs keep the corruption local instead of destroying the rest of the scan. Headers are untouched, so the file still opens. There is no clean-source fallback — if the corrupt decode fails, the effect fails, because a databend that silently returns the original image is a lie.

The GPU version is nineteen lines:

float off = (hash21(vec2(floor(coord.y / 8.0), u_seed)) * 2.0 - 1.0)
          * (12.0 + stride * 4.0);
vec3 a = texture(u_texture, vec2(coord.x + off,       coord.y) / u_size).rgb;
vec3 b = texture(u_texture, vec2(coord.x - off * 0.5, coord.y) / u_size).gbr;
vec3 fx = mix(a, b, 0.35 + 0.2 * u_format);   // .gbr = the channel slip

That .gbr swizzle is doing the aesthetic work — it's the color-slip you recognize from real databending, produced by a completely different mechanism. It's a good impression. It is not databending, and the preset library says so rather than pretending otherwise.

The recipe is portable. The render is not. The stills path is exact; the live path is an interpretation with a documented gap list.

Those are two different promises, and conflating them is how you end up with a customer whose print doesn't match their screen.

6. The macOS app is not a rewrite, and that's the point

Aura Labs on the desktop is a Flutter shell around a native web view serving the same site. If you want to call that an Electron app with extra steps, go ahead — the interesting part is what got bolted on, because those are the things a browser categorically cannot do.

Native file dialogs. Exports open a real macOS Save panel instead of dumping into Downloads through the browser download flow. Small thing. Enormous quality-of-life difference when you're exporting forty variations.

An authenticated localhost control API. The app runs a local server so agent tooling can drive the live editor — push a recipe, read the current recipe, apply a named preset, toggle the virtual camera. On launch it writes a credentials file with mode 0600 containing a port, a token, and a PID. The token rotates per launch, is deleted on quit, and is required on every control route. Static assets stay ungated. This is what makes an agent a first-class operator of the studio rather than a thing that writes files and hopes.

The virtual camera war story

The goal: pipe glitched frames into Meet, Teams, and FaceTime as a system camera device. On macOS 13+ that means a CMIO Camera Extension — a system extension living inside your app bundle, publishing frames the OS treats as a real capture device. Three things bit, in order.

1. System extensions only activate from /Applications. Not from your build directory, not from a symlink. Running from source gives you a working app with a permanently broken virtual camera and a confusing error. The fix is a status chip that offers to copy the bundle to /Applications and relaunch — plus an install script that syncs the current web assets into the bundle and checksums the preset file against the repo, because an old install silently serving stale presets is a maddening bug to diagnose.

2. The extension can't read your App Group container. The extension runs as _cmiodalassistants, a system user. It cannot enter ~/Library/Group Containers, which is mode 700 on the user's home directory, even when the group folder itself is 755. The documented sharing mechanism does not work for this class of extension. Frames go through a world-traversable path in /Users/Shared instead, with a small binary header:

struct Header {
  var magic: UInt32      // 'AURA'
  var version: UInt32
  var width, height: UInt32
  var stride: UInt32
  var seq: UInt32
  var hostTimeNs: UInt64
  var flags: UInt32      // 1 = live, 0 = idle
}
// 1280×720 BGRA, unmirrored, ≤30fps

3. Liveness needs a timeout, and a generous one. If the app dies mid-call, the extension keeps publishing the last frame forever and your colleagues stare at a frozen glitch. So frames carry a host timestamp, and anything older than two seconds becomes a branded idle slate. Two seconds is a long time for a video pipeline; it's the right number here because JPEG encode plus the web-view bridge can exceed 500ms under load, and a slate that flickers on every hiccup is worse than one that's late.

One more, learned the embarrassing way: deactivating the extension when the user toggles the camera off removes “Aura Labs” from every call app's device list and can wedge macOS's camera assistant. Today it only stops publishing frames. The extension stays installed.

7. Entitlements, geofences, and one hole I'm naming on purpose

A lens is bound to something you bought or somewhere you're standing. Six access modes:

export const LENS_ACCESS_MODES = [
  "entitled",  "entitled_geo",   // owns the product
  "member",    "member_geo",     // tagged member
  "public_geo", "public",        // anyone / anyone in the right place
] as const;

Entitlements come from four sources: membership, product purchase (an order webhook maps a product to a lens), a redemption code printed on packaging, and staff publish. Codes are deliberately multi-redeem — unlimited by default. If someone photographs the card inside a shirt and posts it, that's distribution, not theft. The card is the ad.

Product entitlements are revoked on refund, keyed by order ID — the kind of thing you only remember to build after the first chargeback.

8. What I'd tell someone starting this

Make the process the artifact. The recipe-as-JSON decision is what made every subsequent runtime possible. If the output is the artifact, you have a folder. If the process is the artifact, you have a pipeline — and a pipeline can be pointed at surfaces you hadn't thought of yet.

Share the implementation, not the specification. “Both sides implement the same spec” is how you get drift. Importing the actual file under Node is how you get correctness for free. Pay the DOM-free tax; it's cheaper than the bug.

Scale the choices, never the physics. Aesthetic parameters are fractions of the frame. Codec constants are constants. Confusing the two is most of what goes wrong when work moves between resolutions.

Know exactly where your abstraction is lying. Sixty-three shaders is a number. Four effects that are impressions rather than ports, four blend modes that silently degrade, and zero mask support — that's the truth — and it's the version that belongs in the docs, because the day someone asks why the print doesn't match the phone, the answer needs to already be written down.

Ordering is doctrine, not preference. Memory → transmission → display. It's the one piece of aesthetic reasoning that got encoded into the architecture instead of living in my head, and it's the reason a preset library reads as a coherent body of work rather than thirty accidents.

Break something on purpose. Then write down exactly how.

Aura Labs and Aura Lenses are in active development. Effect counts and gap lists are current as of this writing and will move.