playbit / docs

Automation

Run a program under automation with pb run --script:

# script from a file
pb run program.wasm --script script.jsonl

# inline JSONL argument (first character '{'; may contain multiple lines)
pb run program.wasm --script '{"command":"screenshot","id":"a","format":"png"}'

# one JSONL line per argument
pb run program.wasm --script -- '{"command":"fence"}' '{"command":"screenshot","id":"a"}'

# JSONL lines on stdin
printf '%s\n' '{"command":"screenshot","id":"a"}' | pb run program.wasm --script -

<input> can also be a source file or directory; pb builds it first. The run is synchronous: screenshot results are written to pb's stdout, logs go to stderr, and Playbit.app's exit status propagates to pb run.

pb run --script drives Playbit.app's remote-control mode, which can also be used directly:

/Applications/Playbit.app/Contents/MacOS/Playbit \
    --remote-control program.wasm

--script is accepted as an alias for --remote-control.

In this mode:

One JSON object per line.

Fatal Errors

The process terminates with an error on stderr when:

screenshot is the only recoverable command. It writes either a success result or an error result to stdout.

Window Readiness

There is no window_opened event.

Commands that require a window are held until the main window is:

Window-dependent commands:

wait does not require a window.

Queue Semantics

wait

fence

Event commands

screenshot

When stdin is closed, the app exits after the last queued command completes.

Command Shape

All input lines have this base shape:

interface InputMessage {
    command: string
    id?: string
}

id is opaque. It is only echoed back by screenshot results.

Wait

{"command":"wait","ms":250}

Fence

{"command":"fence"}

Screenshot

{"command":"screenshot","id":"s1","rect":{"x":0,"y":0,"width":640,"height":480}}

By default (no format), a screenshot writes three files and reports all three paths in the result:

Fields:

Rules:

JSON screenshot file format:

interface CanvasCommandScreenshot {
    windowObjectId: number
    dpScale: number
    shapes: ShapeItem[]
    texts: TextItem[]
    instructions: InstructionItem[]
    clips: ClipItem[]
}

interface Vec4 {
    x: number
    y: number
    z: number
    w: number
}

interface Vec2 {
    x: number
    y: number
}

interface F32x4 {
    x: number
    y: number
    z: number
    w: number
}

interface ShapeItem {
    bounds: Vec4
    transformX: Vec2
    transformY: Vec2
    transformO: Vec2
    cornerRadius: F32x4
    fillColor: string
    fillColors: [string, string, string, string]
    strokeColor: string
    strokeWidth: number
    flags: number
    blur: number
    opacity: number
}

interface TextItem {
    bounds: Vec4
    transformX: Vec2
    transformY: Vec2
    transformO: Vec2
    textPlan: number
}

// One entry per submitted command, in stream order. Coordinates are raw
// submission-space values (multiply points by dpScale to compare with the
// point-converted shapes/texts/clips arrays). Colors are wire colors dumped
// verbatim as hex strings: four binary16 channels in r,g,b,a memory order,
// so the u64 reads 0xAAAABBBBGGGGRRRR.
interface InstructionItem {
    kind:
        | "rect"
        | "rounded_rect"
        | "stroke_rect"
        | "complex_rect"
        | "gradient_rect"
        | "text"
        | "clip_push"
        | "clip_pop"
        | "texture_set"
        | "transform_push"
        | "transform_pop"
        | "group_push"
        | "group_pop"
        | "group_mask"
        | "gradient"
        | "mesh_gradient"
    bounds?: { x1: number, y1: number, x2: number, y2: number }
    fillColor?: string
    fillColors?: [string, string, string, string] // gradient_rect
    strokeColor?: string
    strokeWidth?: number
    strokeFlags?: number
    cornerRadius?: F32x4
    opacity?: number
    blur?: number       // complex_rect; group_push (gaussian radius, 0..100)
    uvMin?: Vec2        // complex_rect
    uvMax?: Vec2        // complex_rect
    textPlan?: number   // text
    flags?: number      // text
    texture?: number    // texture_set
    transformX?: Vec2   // transform_push
    transformY?: Vec2   // transform_push
    transformO?: Vec2   // transform_push
    blendMode?: number  // group_push (PBSysCanvasBlendMode)
    maskFlags?: number  // group_mask (PBSysCanvasMaskFlags)
    // gradient (see PBSysCanvasCmdGradient): flags is the raw cmd.flags value
    gradientKind?: number // PBSysCanvasGradientKind
    space?: number        // PBSysCanvasGradientSpace
    spread?: number       // PBSysCanvasGradientSpread
    p0?: Vec2
    p1?: Vec2
    stops?: { offset: number, color: string }[]
    // mesh_gradient (see PBSysCanvasCmdMeshGradient); space as for gradient
    falloff?: number
    points?: { x: number, y: number, weight: number, color: string }[]
}

interface ClipItem {
    bounds: Vec4
    transformX: Vec2
    transformY: Vec2
    transformO: Vec2
}

Notes:

Example:

{
    "windowObjectId": 123,
    "shapes": [
        {
            "bounds": { "x": 24, "y": 24, "z": 40, "w": 40 },
            "transformX": { "x": 16, "y": 0 },
            "transformY": { "x": 0, "y": 16 },
            "transformO": { "x": 24, "y": 24 },
            "cornerRadius": { "x": 0, "y": 0, "z": 0, "w": 0 },
            "fillColor": "0xFFC4E87000000000",
            "fillColors": [
                "0x0000000000000000",
                "0x0000000000000000",
                "0x0000000000000000",
                "0x0000000000000000"
            ],
            "strokeColor": "0x0000000000000000",
            "strokeWidth": 0,
            "flags": 0,
            "blur": 0,
            "opacity": 1
        }
    ],
    "texts": [
        {
            "bounds": { "x": 24, "y": 64, "z": 104, "w": 92 },
            "transformX": { "x": 1, "y": 0 },
            "transformY": { "x": 0, "y": 1 },
            "transformO": { "x": 24, "y": 64 },
            "textPlan": 10
        }
    ],
    "instructions": [],
    "clips": [
        {
            "bounds": { "x": 18, "y": 250, "z": 364, "w": 360 },
            "transformX": { "x": 330, "y": 36 },
            "transformY": { "x": -16, "y": 74 },
            "transformO": { "x": 34, "y": 250 }
        }
    ]
}

Stdout result:

interface ScreenshotSuccessResult {
    id?: string
    path: string
}

interface ScreenshotErrorResult {
    id?: string
    error: string
}

Examples:

{"id":"s1","path":"/path/to/project/_pb/screenshots/260806-143502.123456.png"}

{"id":"s1","error":"no canvas commands available"}

Event Commands

Common fields for all event commands:

interface EventCommand extends InputMessage {
    objectId?: number
    timestamp?: number
    clientId?: number
    deviceId?: number
    modifiers?: string
}

Defaults:

objectId handling:

modifiers is a space-separated list of:

Pointer Events

Commands:

Shape:

interface PointerEventCommand extends EventCommand {
    command:
        | "pointer_enter" | "pointer_leave" | "pointer_down"
        | "pointer_up" | "pointer_move" | "pointer_cancel"
    pointerId?: number
    flags?: string
    buttons?: number[]
    button?: number
    kind?: "mouse" | "touch" | "trackpad" | "pen"
    clickCount?: number
    x: number
    y: number
    dx?: number
    dy?: number
}

Defaults:

flags is a space-separated list of:

buttons and button use 1-based numbering in JSON:

Example:

{"command":"pointer_move","x":120,"y":80,"dx":4,"dy":-2,"flags":"primary"}

Scroll

interface ScrollCommand extends EventCommand {
    command: "scroll"
    kind?: "mouse" | "touch" | "trackpad" | "pen"
    phase?: "changed" | "began" | "ended" | "momentum"
    flags?: string
    x?: number
    y?: number
    dx: number
    dy: number
    wheelZ?: number
}

Defaults:

flags is a space-separated list of:

Example:

{"command":"scroll","x":120,"y":220,"dx":0,"dy":-48,"flags":"precise"}

Gestures

gesture_pan

{"command":"gesture_pan","phase":"changed","x":120,"y":220,"dx":16,"dy":0}

gesture_pinch

{"command":"gesture_pinch","phase":"changed","x":120,"y":220,"scale":1.05}

gesture_rotate

{"command":"gesture_rotate","phase":"changed","x":120,"y":220,"rotation":0.1}

Shared fields:

gesture_pan defaults dx=0, dy=0.

gesture_pinch defaults scale=1.0.

gesture_rotate defaults rotation=0.

Keyboard

Commands:

Two payload forms are supported.

Keyed form:

{"command":"key_down","key":"Tab","deviceKey":"Tab"}

IME/text form:

{"command":"key_down","text":"a"}

Shape:

interface KeyboardEventCommand extends EventCommand {
    command: "key_down" | "key_up"
    flags?: string
}

interface KeyedKeyboardEventCommand extends KeyboardEventCommand {
    key: string
    deviceKey: string
}

interface TextKeyboardEventCommand extends KeyboardEventCommand {
    command: "key_down"
    text: string
}

Rules:

Keyboard key names are case-insensitive and use the PBSysKeyboardKey variant suffix name, for example:

Examples

Click and take a screenshot:

pb run program.wasm --script -- \
    '{"command":"pointer_move","x":120,"y":80}' \
    '{"command":"pointer_down","x":120,"y":80,"buttons":[1],"button":1}' \
    '{"command":"pointer_up","x":120,"y":80,"buttons":[],"button":1}' \
    '{"command":"screenshot","id":"shot1"}'

Delay before a screenshot:

pb run program.wasm --script -- \
    '{"command":"wait","ms":250}' \
    '{"command":"screenshot","id":"shot2","format":"png"}'

Write the latest render commands as JSON:

pb run program.wasm --script \
    '{"command":"screenshot","id":"pkg1","format":"json","output_file":"/tmp/render-package.json"}'

Pipe a whole command stream to Playbit.app directly:

{
    printf '%s\n' '{"command":"pointer_move","x":120,"y":80}'
    printf '%s\n' '{"command":"screenshot","id":"shot1"}'
} | /Applications/Playbit.app/Contents/MacOS/Playbit \
    --remote-control program.wasm