Runtime V2 API Catalog

A read-only audio value for recognizing sounds in an app or game. Use Audio(name) to load a saved sample, or Audio.recent(duration_ms) to copy recently captured audio. Capture and matching run on the device; continuous listening buffers are never uploaded and scripts cannot access raw audio (PCM).

Represents one ARGB color and provides color sampling/comparison helpers.

Build and show a runtime input panel for script-controlled macro parameters.

Draw text on top of the screen with optional interaction tracking (press/move/hold). Setter names such as text/background are methods; read current values through state().

A point on the screen, measured in pixels. Scale/anchor control coordinate mapping across screen sizes. After mapping and offsets, runtime consumers clamp the final point to the current screen's valid pixel range. Point can be destructured as x, y.

A rectangular region on the screen, measured in pixels on the current screen axes. Scale/anchor control coordinate mapping across full-screen sizes without rotating through portrait coordinates or using the app viewport; runtime consumers clip the final region to the current screen after mapping and offsets. ScaleParam controls template matching inside this region. Region can be destructured as x, y, width, height.

Template matching parameters for Region. fast_matching downsamples both the Region image and template.

Static controls for shared screen capture during the current macro run. Automatic caching normally reuses frames for up to 100 ms to reduce capture work. For rapid recognition that needs the freshest available frame, use auto_cache_off(); this increases processing and battery use and does not guarantee faster execution or newly rendered pixels. Explicit cache_on() pins a frame separately and takes priority over automatic-cache settings.

Static persistent key/value storage for the current macro. Normal variables stay shared only between sequential PYTHON_CODE actions within the same flow, including actions inside GROUP wrappers; parallel flows keep isolated variables. Storage keys are shared across all flows, so use Storage when values must survive the next macro run, not as inter-process messaging. Writes are autosaved after 2 seconds of idle time and committed again after the whole macro Run and all background flows end.

One point inside a flat swipe path. The point must be a Point or tuple (x, y). Pass a list of at least two SwipePoint values to swipe([...]) to move one finger through multiple points.

Access runtime system/device info and trigger system-level actions from script.

Template image wrapper. You can scale/crop an existing template, save it as a new template name, or check existence.

TFLite model handle for scripts. The model type must be selected in Template Manager metadata. Supported outputs are YOLOv8/YOLO11 object detection [1,N,4+C]/[1,4+C,N], YOLO26 end-to-end detection [1,N,6]/[N,6], YOLO11/YOLO26 image classification [1,C], and MobileNetV3 image classification [1,C]. Runtime detects FLOAT32, INT8, or UINT8 tensor precision from the model and dequantizes quantized detection and classification outputs before applying thresholds. Classification matches return class confidence and metadata without point or bounds. Inference methods require a local model file; when local file is missing and runtime download fails, script raises a runtime error.

Millisecond stopwatch helper for script timing checks and guarded waits.

Static helper for parallel click/swipe actions created by the UI or code via click(..., parallel=True) and swipe(..., parallel=True). You can collect click/swipe actions and send them together, or inspect click/swipe actions that are active or queued. The helper also supports cancelling active click/swipe actions (Android 8.0+ only).

Read-only audio match result. Score is similarity, not an accuracy probability. find() timestamps use the listening session timeline; match() uses the searched audio value's timeline. Compare timestamps to avoid handling one sound repeatedly. Only use timestamps as detections when matched is True.

Static controls for starting named macro flows in parallel. At most two background flows can run beside the main flow, and only one live invocation of a given named flow is allowed, including while it is paused or stopping. Flow.start() returns immediately after the coordinator accepts the branch; background flows may start peer flows. Variables are isolated per branch and never merge into the caller. The main flow waits for all background flows before Run completes. An invalid or duplicate name, a full background-flow limit, any branch error, System.stop(), or Main Stop ends the whole macro. Each numbered background dock control can temporarily Pause or Resume only that branch; it has no per-branch Stop. Main Pause or Resume applies to every branch, and global Resume also clears branch-local pause flags. Touch gestures from different branches can cancel each other. ScreenCapture and audio capture policies are shared globally. Storage shares the same persistent keys across branches and is not an inter-process messaging channel.

Read-only recognition match result returned by region.find* helpers. In a recognition Success branch, Recognition.result points to the active Match or MatchList.

A read-only list of Match values returned by region.find_any/find_all/find_multi, wait_any/wait_all, and Recognition.result when multiple recognition targets matched.

Used in the recognition success branch in the UI. Recognition.result returns the Match for one recognition target or MatchList when multiple targets matched.

Snapshot of one script/UI touch item in the shared gesture queue.

A key/value collection that keeps insertion order and can be read or updated during script execution.

An ordered collection you can create, read, loop through, destructure, and update while the script runs.

A mutable collection of unique values. CogniTab keeps iteration order stable so script output is easy to read.

Text value collection. Created by string literals or str(value), then read by length/index/loop operations.

An ordered read-only collection. Use tuple literals when values should stay fixed after creation or when returning grouped values for destructuring.