AI - Glassly

Glassly Miniapp SDK betaThe SDK is in beta, so its APIs may change before general availability.Developing on Mentra Live? We recommend using the Glassly Bluetooth SDK.The developer tools are available in the Glassly App under Settings → Miniapp Developer Settings.Share feedback with an in-app bug report, on Discord, or by email at [email protected].

session.ai runs LLM completions through the Glassly platform. Your miniapp ships no API key and never sees a provider credential: the platform picks the provider, holds the key, and pays the bill.

const summary = await session.ai.prompt(
  "Summarize this in one sentence: " + text,
  { maxTokens: 128 },
);

AI needs the AI permission in your manifest. Without it every call rejects with PERMISSION_NOT_DECLARED. session.ai.hasPermission reports whether you declared it.

Tiers, not models

A request never names a provider or a model. It says how much model the call deserves:

`tier`	For
`"quick"`	High-volume gating and classification, where latency and cost dominate.
`"balanced"`	Routine generation: summaries, extraction. The default.
`"frontier"`	The user-facing answer, where quality dominates.

The phone resolves each tier to a concrete provider and model from the user’s Settings > AI Models dropdowns, or automatically from whichever keys they saved. Your miniapp keeps working unchanged when models or the user’s preferences move. If the user has saved their own key for the provider that served a call (Settings > API Keys), the platform uses that key and the call isn’t metered against their plan. billedToUserKey on the result says which happened.

Completions

complete(request) runs one turn.

const result = await session.ai.complete({
  messages: [{ role: "user", content: "What's the capital of France?" }],
  system: "Answer in as few words as possible.",
  tier: "quick",
  maxTokens: 64,
});

Field	Type	What it does
`messages`	`AiMessage[]`	The conversation. Each is `{role: "user" \| "assistant", content, toolCalls?, toolResults?}`.
`system`	`string?`	System prompt. Placed correctly for whichever provider serves the call.
`tier`	`AiTier?`	`"quick" \| "balanced" \| "frontier"`. Defaults to `"balanced"`.
`maxTokens`	`number?`	Output cap.
`temperature`	`number?`	Advisory: providers whose current models reject sampling params drop it.
`jsonMode`	`boolean?`	Ask for a JSON object back. You still parse and validate the text.
`tools`	`AiTool[]?`	Tools your miniapp executes. See below.
`webSearch`	`boolean?`	Let the model search the web. See below.

The result:

Field	Type	What it is
`text`	`string`	Concatenated text. Empty when the model only called tools.
`stopReason`	`AiStopReason`	`"end_turn" \| "tool_use" \| "max_tokens" \| "refusal"`.
`toolCalls`	`AiToolCall[]?`	Populated only when `stopReason === "tool_use"`.
`usage`	`AiUsage`	`{inputTokens, outputTokens}`.
`provider`	`AiProvider`	Which of `"anthropic" \| "gemini" \| "openai"` served the call.
`model`	`string`	The model it resolved to.
`billedToUserKey`	`boolean`	`true` when the user’s own key paid, so the call wasn’t metered.
`webSearchCount`	`number?`	Server-side searches the model ran.

stopReason: "refusal" is a successful result, not an error: the model declined. Check it before trusting text.

prompt(text, opts?) is the single-shot convenience wrapper, taking {system, maxTokens, jsonMode, tier} and resolving to a string. It throws on a refusal so callers that just want text don’t have to remember that one.

The tool loop

Pass tools and the model may ask you to run one. That comes back as stopReason: "tool_use" with the calls to execute. Append the assistant turn carrying toolCalls plus a user turn carrying your toolResults, then call again.

const tools = [{
  name: "get_weather",
  description: "Current weather for a city",
  inputSchema: {
    type: "object",
    properties: { city: { type: "string" } },
    required: ["city"],
  },
}];

const messages = [{ role: "user" as const, content: "Weather in Paris?" }];

for (let round = 0; round < 5; round++) {
  const res = await session.ai.complete({ messages, tools });
  if (res.stopReason !== "tool_use" || !res.toolCalls?.length) {
    console.log(res.text);
    break;
  }
  messages.push({ role: "assistant", content: res.text, toolCalls: res.toolCalls });
  messages.push({
    role: "user",
    content: "",
    toolResults: res.toolCalls.map((call) => ({
      toolCallId: call.id,           // echo verbatim, never reorder or invent ids
      content: JSON.stringify(runTool(call)),
    })),
  });
}

Write inputSchema in the lowercase JSON Schema dialect ("object", "string"); Gemini’s uppercase OpenAPI dialect is translated for you. Set isError: true on a tool result when the tool failed, so the model can recover instead of hanging.

The platform holds no conversation state. The whole messages array round-trips on every call, and nothing else will stop a runaway loop, so bound it yourself.

Web search

webSearch: true lets the model search the web when live data would help. It’s a grant, not a command: the model may answer from knowledge, and a provider with no server-side search tool (OpenAI today) ignores the flag. Searches run inside the same call, so there’s no extra round-trip and no tool loop on your side, and webSearchCount on the result reports how many actually ran. Platform-funded searches add a small per-search surcharge on top of tokens.

Visual generation

generate(request, opts?) is a separate, long-running call: the cloud drives an agentic provider loop with server-side web search, code execution, and file retrieval, then returns text plus rendered images. Anthropic-only by design, and markedly more expensive than a completion, so call it deliberately.

const out = await session.ai.generate(
  { prompt: "Draw a monochrome sparkline of these values: " + values.join(",") },
  { onProgress: ({ activity }) => showWaitCard(activity) },
);
// out.text, out.images[] ({data: base64, mimeType, width?, height?})

Expect tens of seconds of latency, up to about 2.5 minutes, so show progress UI. onProgress ticks with a coarse activity — "thinking", "searching", "drawing", "finishing" today. Treat an unknown value as "thinking": the platform may refine the labels without a new SDK, and hosts that predate the progress surface complete the call normally and just never tick. generate rejects with QUOTA_EXCEEDED when the account’s AI credit is spent, and with INTERNAL when the host predates the request type or the cloud has the surface disabled.

What this surface doesn’t do

Streaming, and provider server-side tools other than web search (Anthropic’s code_execution, the Files API). Those are vendor-specific, so exposing them would make the contract non-portable. A miniapp that needs them calls the provider directly with a key from session.keys.

Getting Started

Core Concepts

UI Layer

Display

Audio & Speech

Sensors & Input

Camera

Phone & System

Data

Reference

AI

Tiers, not models

Completions

The tool loop

Web search

Visual generation

What this surface doesn’t do

​Tiers, not models

​Completions

​The tool loop

​Web search

​Visual generation

​What this surface doesn’t do

Tiers, not models

Completions

The tool loop

Web search

Visual generation

What this surface doesn’t do