Input

Glassly Miniapp SDK betaThe SDK is in beta, so its APIs may change before general availability.Developing on Mentra Live? We recommend using the Glassly Bluetooth SDK.The developer tools are available in the Glassly App under Settings → Miniapp Developer Settings.Share feedback with an in-app bug report, on Discord, or by email at [email protected].

session.input delivers the physical controls on the glasses: button presses and touchpad gestures. The starter scaffold wires a button press to the display, so a short press shows a text wall.

src/background/index.ts

import { registerMiniapp } from "@glassly/miniapp/background";

registerMiniapp((session) => {
  session.input.onButtonPress((press) => {
    session.display.render([{type: "text", id: "msg", box: {x: 0, y: 0, w: 576, h: 288}, text: `${press.buttonId}: ${press.pressType}`}]);
  });
});

Button events only reach glasses that have a button. Declare a BUTTON hardware requirement in your manifest to gate the miniapp to those glasses.

Button presses

onButtonPress fires on every press of a physical button:

session.input.onButtonPress((press) => {
  if (press.pressType === "long") session.display.render([]);
});

Each ButtonPressData has:

Field	Type	Notes
`buttonId`	`string`	Which button fired.
`pressType`	`"short" \| "long"`	Press duration.

Keyboard

Even G2 glasses running Glassly CFW revision 39 or later can host a Bluetooth keyboard themselves (the user pairs it from the Glassly App under Controllers). onKey delivers every key transition to every subscribed miniapp, the same way button presses are broadcast:

session.input.onKey((e) => {
  if (e.phase !== "down") return;
  if (e.text) type(e.text);            // printable characters, layout applied
  else if (e.key === "Enter") submit();
  else if (e.key === "c" && e.modifiers.ctrl) cancel();
});

Each KeyboardEventData has:

Field	Type	Notes
`phase`	`"down" \| "up"`	Press or release.
`key`	`string`	Logical key under the host layout (US today): `"a"`, `"A"`, `"Enter"`, `"ArrowUp"`, `"F5"`, `"AudioVolumeUp"`.
`code`	`string`	Physical key: `"KeyA"`, `"Digit1"`, `"Space"`, `"ControlLeft"`.
`modifiers`	`{shift, ctrl, alt, meta}`	Modifier state at the time of the event.
`text`	`string?`	The character a plain press types. Down and repeat only; absent for Ctrl/Alt/Meta chords and named keys.
`repeat`	`boolean`	Synthesized auto-repeat while the key stays held.
`timestamp`	`number`	Phone receipt time in ms.

A dropped link releases every held key with matching up events, so a modifier can never stay stuck. On glasses without a keyboard bridge the subscription never fires.

Touch gestures

onTouch subscribes to touchpad gestures. Pass a handler for every gesture, a single gesture name to filter, or an array of names for one subscription across several gestures.

// Every touch gesture.
session.input.onTouch((touch) => {
  session.display.render([{type: "text", id: "msg", box: {x: 0, y: 0, w: 576, h: 288}, text: touch.kind}]);
});

// One gesture.
session.input.onTouch("single_tap", () => {
  session.display.render([{type: "text", id: "msg", box: {x: 0, y: 0, w: 576, h: 288}, text: "Tap"}]);
});

// Several gestures, one subscription.
session.input.onTouch(["swipe_up", "swipe_down"], (touch) => {
  // touch.kind tells you which one.
});

Each TouchData has:

Field	Type	Notes
`kind`	`string`	The gesture. See the values below.

The gesture values the runtime reports:

`kind`	Gesture
`"single_tap"`	A single tap on the touchpad.
`"double_tap"`	Two quick taps.
`"triple_tap"`	Three quick taps.
`"long_press"`	A press and hold.
`"swipe_up"`	A swipe up.
`"swipe_down"`	A swipe down.
`"tap_start"`	The raw finger-down edge. Brackets every gesture on hardware that reports it.
`"tap_end"`	The raw finger-up edge. Brackets every gesture on hardware that reports it.
`"tap_then_hold"`	The G2 2.2.9+ tap-then-hold (Menu) gesture.
`"touch_down"`	The R1 ring’s finger-down, forwarded by the Glassly CFW (GLASSLYCFW/37+). Only the ring sends it; the temples report `tap_start`.

Glasses can send other kind strings, so kind is typed as string. Match the values you handle and ignore the rest.

Claiming gestures

setClaimedGestures(gestures, {scope?}) registers the gestures your miniapp handles. Claims stack: the most recent registration receives the gesture alone, through onTouch, and the host skips its default behavior. This also applies to background miniapps. Subscribing with onTouch alone does not claim a gesture or override another miniapp’s claim.

const offTouch = session.input.onTouch("double_tap", () => {
  // Handle an in-miniapp back gesture.
});
const offClaim = session.input.onClaimChange(({gesture, held}) => {
  console.log(gesture, held ? "received here" : "held by another miniapp");
});

await session.input.setClaimedGestures(["double_tap"], {scope: "page"});

// When the page closes, restore the previous claimant.
await session.input.setClaimedGestures([], {scope: "page"});
offTouch();
offClaim();

Each call replaces the selected scope’s set; omitting scope uses the default scope. Re-registering a nonempty set takes priority again. Use separate scopes for persistent shortcuts and temporary pages so closing a page releases only its claims. Stopping the miniapp releases every scope. onClaimChange reports {gesture: string, held: boolean} to the miniapps gaining and losing a gesture; a handoff between two scopes of the same miniapp does not fire it. One host rule runs first: double-tap leaves a miniapp visibly drawing on the main view if that miniapp has not registered double_tap in any scope. Claim it if you use it while drawing there. Main content covered by a dashboard view is not visible, so the rule does not exit the miniapp underneath. A main miniapp that registered double-tap is not exited even if another claimant currently has priority.

Cleaning up

onButtonPress and every onTouch form return an unsubscribe function:

const off = session.input.onTouch("single_tap", handler);
off(); // stop listening

Calling onTouch with an array returns one unsubscribe function that drops all of the gestures it registered.

Getting Started

Core Concepts

UI Layer

Display

Audio & Speech

Sensors & Input

Camera

Phone & System

Data

Reference

Button presses

Keyboard

Touch gestures

Claiming gestures

Cleaning up

​Button presses

​Keyboard

​Touch gestures

​Claiming gestures

​Cleaning up

Button presses

Keyboard

Touch gestures

Claiming gestures

Cleaning up