Automation

Run a wasm app with:

/Applications/Playbit.app/Contents/MacOS/Playbit \
    --remote-control program.wasm

In this mode:

• stdin is a JSONL command stream
• stdout is a JSONL result stream for screenshot commands only
• stderr is for logs and fatal protocol errors
• commands are processed in input order

One JSON object per line.

Fatal Errors

The process terminates with an error on stderr when:

• a line is not valid JSON
• command is missing or invalid
• a non-screenshot command has invalid fields
• a non-screenshot command cannot be enqueued or synchronized
• a nonzero objectId does not match the main window

screenshot is the only recoverable command. It writes either a success result or an error result to stdout.

Window Readiness

There is no window_opened event.

Commands that require a window are held until the main window is:

• created
• visible
• has presented at least one frame

Window-dependent commands:

• fence
• screenshot
• pointer_enter
• pointer_leave
• pointer_down
• pointer_up
• pointer_move
• pointer_cancel
• scroll
• gesture_pan
• gesture_pinch
• gesture_rotate
• key_down
• key_up

wait does not require a window.

Queue Semantics

wait

• delays later commands by ms
• uses a timer; it does not block a thread while waiting

fence

• waits until all prior inserted input has been consumed by the guest
• if prior commands caused a frame request, waits for the resulting frame to present
• produces no stdout output
• is not needed immediately before screenshot, since screenshot already implies the same wait

Event commands

• enqueue the requested runtime event
• do not wait for guest consumption or frame presentation
• produce no stdout output

screenshot

• first waits until all prior inserted input has been consumed by the guest
• for format:"png", waits for any resulting frame to present
• for format:"json", waits for the guest barrier after prior input so the latest submitted renderer package reflects the effects of those commands
• then captures a PNG or writes a JSON renderer-package description
• writes one JSON result line to stdout
• this means {"command":"key_down",...} followed by {"command":"screenshot",...} captures state after the key event has been applied; PNG additionally waits for any resulting frame to present before capture
• an explicit fence immediately before screenshot is redundant

When stdin is closed, the app exits after the last queued command completes.

Command Shape

All input lines have this base shape:

interface InputMessage {
    command: string
    id?: string
}

id is opaque. It is only echoed back by screenshot results.

Wait

{"command":"wait","ms":250}

• ms is a non-negative integer number of milliseconds

Fence

{"command":"fence"}

Screenshot

{"command":"screenshot","id":"s1","format":"png","rect":{"x":0,"y":0,"width":640,"height":480}}

Fields:

• output_file optional output path
• format optional, "png" or "json", default "png"
• rect optional capture rect in window content coordinates, top-left origin

Rules:

• If output_file is omitted or "", a temporary file is created under TMPDIR.
• For format:"png", rect is clipped against the window content area.
• For format:"json", rect is ignored.
• format:"json" writes a JSON file describing the most recently submitted renderer package.
• JSON screenshot files also include windowObjectId, the main window's object id. This is useful when testing explicit nonzero objectId input.

JSON screenshot file format:

interface RendererPackageScreenshot {
    windowObjectId: number
    shapes: ShapeItem[]
    texts: TextItem[]
    instructions: InstructionItem[]
    clips: ClipItem[]
}

interface Vec4 {
    x: number
    y: number
    z: number
    w: number
}

interface U16x4 {
    x: number
    y: number
    z: number
    w: number
}

interface ShapeItem {
    bounds: Vec4
    cornerRadius: U16x4
    fillColor: string
    fillColors: [string, string, string, string]
    strokeColor: string
    strokeWidth: number
    flags: number
    blur: number
    opacity: number
}

interface TextItem {
    bounds: Vec4
    textPlan: number
}

interface InstructionItem {
    kind: "shape" | "text" | "clip_push" | "clip_pop"
    runLength?: number
}

interface ClipItem {
    bounds: Vec4
}

Notes:

• bounds and clip rectangles are encoded as {x,y,z,w} matching the runtime vector fields. For rectangles this means (x, y, width, height).
• cornerRadius is encoded as four unsigned 16-bit values {x,y,z,w}.
• Color values are hex strings like "0xFF18120C00000000".
• runLength is present only for "shape" and "text" instructions.
• textPlan is the runtime text-plan handle captured when the renderer package was submitted.
• The JSON file is intended for testing and automation. It reflects the latest submitted renderer package, not a rasterized image.

Example:

{
    "windowObjectId": 123,
    "shapes": [
        {
            "bounds": { "x": 24, "y": 24, "z": 16, "w": 16 },
            "cornerRadius": { "x": 0, "y": 0, "z": 0, "w": 0 },
            "fillColor": "0xFFC4E87000000000",
            "fillColors": [
                "0x0000000000000000",
                "0x0000000000000000",
                "0x0000000000000000",
                "0x0000000000000000"
            ],
            "strokeColor": "0x0000000000000000",
            "strokeWidth": 0,
            "flags": 0,
            "blur": 0,
            "opacity": 1
        }
    ],
    "texts": [],
    "instructions": [
        { "kind": "shape", "runLength": 1 }
    ],
    "clips": []
}

Stdout result:

interface ScreenshotSuccessResult {
    id?: string
    path: string
}

interface ScreenshotErrorResult {
    id?: string
    error: string
}

Examples:

{"id":"s1","path":"/var/folders/.../shot-1.png"}

{"id":"s1","error":"no renderer package available"}

Event Commands

Common fields for all event commands:

interface EventCommand extends InputMessage {
    objectId?: number
    timestamp?: number
    clientId?: number
    deviceId?: number
    modifiers?: string
}

Defaults:

• objectId: 0
• timestamp: current PBSysTime
• clientId: 0
• deviceId: 0
• modifiers: ""

objectId handling:

• 0 targets the main window and leaves the inserted event objectId as 0
• nonzero objectId must equal the main window object id

modifiers is a space-separated list of:

• shift
• ctrl
• alt
• meta
• capslock
• fn

Pointer Events

Commands:

• pointer_enter
• pointer_leave
• pointer_down
• pointer_up
• pointer_move
• pointer_cancel

Shape:

interface PointerEventCommand extends EventCommand {
    command:
        | "pointer_enter" | "pointer_leave" | "pointer_down"
        | "pointer_up" | "pointer_move" | "pointer_cancel"
    pointerId?: number
    flags?: string
    buttons?: number[]
    button?: number
    kind?: "mouse" | "touch" | "trackpad" | "pen"
    clickCount?: number
    x: number
    y: number
    dx?: number
    dy?: number
}

Defaults:

• pointerId: 0
• flags: ""
• buttons: []
• button: omitted, encoded as 0
• kind: "mouse"
• clickCount: 0
• dx: 0
• dy: 0

flags is a space-separated list of:

• primary
• in_contact
• eraser
• inverted
• coalesced
• predicted

buttons and button use 1-based numbering in JSON:

• 1 primary
• 2 secondary
• 3 middle
• 4+ additional buttons

Example:

{"command":"pointer_move","x":120,"y":80,"dx":4,"dy":-2,"flags":"primary"}

Scroll

interface ScrollCommand extends EventCommand {
    command: "scroll"
    kind?: "mouse" | "touch" | "trackpad" | "pen"
    phase?: "changed" | "began" | "ended" | "momentum"
    flags?: string
    x?: number
    y?: number
    dx: number
    dy: number
    wheelZ?: number
}

Defaults:

• kind: "trackpad"
• phase: "changed"
• flags: ""
• x: 0
• y: 0
• wheelZ: 0

flags is a space-separated list of:

• precise
• inverted
• unit_lines
• unit_pages

Example:

{"command":"scroll","x":120,"y":220,"dx":0,"dy":-48,"flags":"precise"}

Gestures

gesture_pan

{"command":"gesture_pan","phase":"changed","x":120,"y":220,"dx":16,"dy":0}

gesture_pinch

{"command":"gesture_pinch","phase":"changed","x":120,"y":220,"scale":1.05}

gesture_rotate

{"command":"gesture_rotate","phase":"changed","x":120,"y":220,"rotation":0.1}

Shared fields:

• phase optional: "changed", "began", "ended", default "changed"
• x, y optional, default 0, 0

gesture_pan defaults dx=0, dy=0.

gesture_pinch defaults scale=1.0.

gesture_rotate defaults rotation=0.

Keyboard

Commands:

• key_down
• key_up

Two payload forms are supported.

Keyed form:

{"command":"key_down","key":"Tab","deviceKey":"Tab"}

IME/text form:

{"command":"key_down","text":"a"}

Shape:

interface KeyboardEventCommand extends EventCommand {
    command: "key_down" | "key_up"
    flags?: string
}

interface KeyedKeyboardEventCommand extends KeyboardEventCommand {
    key: string
    deviceKey: string
}

interface TextKeyboardEventCommand extends KeyboardEventCommand {
    command: "key_down"
    text: string
}

Rules:

• flags is a space-separated list containing repeat
• keyed and text forms are mutually exclusive
• text form is only valid for key_down
• text may contain up to 8 Unicode codepoints
• in text form, keyCode and deviceCode are derived from the first codepoint

Keyboard key names are case-insensitive and use the PBSysKeyboardKey variant suffix name, for example:

• "LeftBracket"
• "Enter"
• "g"
• "G"

Examples

Pipe a whole command stream:

{
    printf '%s\n' '{"command":"pointer_move","x":120,"y":80}'
    printf '%s\n' '{"command":"pointer_down","x":120,"y":80,"buttons":[1],"button":1}'
    printf '%s\n' '{"command":"pointer_up","x":120,"y":80,"buttons":[],"button":1}'
    printf '%s\n' '{"command":"fence"}'
    printf '%s\n' '{"command":"screenshot","id":"shot1"}'
} | /Applications/Playbit.app/Contents/MacOS/Playbit \
    --remote-control program.wasm

Delay before a screenshot:

{
    printf '%s\n' '{"command":"wait","ms":250}'
    printf '%s\n' '{"command":"screenshot","id":"shot2","format":"png"}'
} | /Applications/Playbit.app/Contents/MacOS/Playbit \
    --remote-control program.wasm

Write the latest render commands as JSON:

printf '%s\n' \
    '{"command":"screenshot","id":"pkg1","format":"json","output_file":"/tmp/render-package.json"}' |
    /Applications/Playbit.app/Contents/MacOS/Playbit \
        --remote-control program.wasm