# ❓ How It Works
# How It Works
Escribano is a local evidence layer for your work. It captures your screen, understands what's happening, and makes it queryable — all on your Mac, nothing in the cloud.
## Overview
```mermaid
flowchart LR
A[Screen capture] --> B[Vision model
on-device]
B --> C[Description
+ entities]
C --> D[(SQLite
+ FTS5)]
D --> E[escribano-query]
E --> F[Human]
E --> G[Agent]
```
Every step runs on-device. No cloud upload, no telemetry on your activity.
---
## The Pipeline
### 1. Capture
A small menu-bar app watches your screen in the background, sampling a frame every few seconds and skipping visual duplicates.
* Runs entirely on your Mac
* Energy-efficient — adaptive sampling, dedupe before any heavy processing
* Pause and resume from the menu bar at any time
### 2. Understand
Each frame is processed by a vision-language model running entirely on your Apple Silicon GPU:
```
Screenshot → [VLM, on-device] → "User is debugging JWT flow in VS Code…"
```
The model extracts:
* A plain-language description of what's happening
* The applications visible on screen
* **Entities** — code files, URLs, error messages, git branches, tools, languages
A second NLP pass normalizes entities and applies PII filtering before anything is written to disk.
### 3. Store
Descriptions and entities are written to a local SQLite database with FTS5 full-text indexing:
| Table | Contents |
|-------|----------|
| **Observations** | Timestamped descriptions plus metadata (apps, file paths, URLs) |
| **Entities** | Tools, languages, frameworks, errors, etc., linked to their observations |
| **Frames** | Optional — retained only if image access is enabled |
Everything lives on your machine. There is no sync, no cloud backup, no remote index.
### 4. Query
You query history through `escribano-query`, the CLI built for both humans and agents:
```bash
escribano-query today # timeline of today's activity
escribano-query search "bug" # find mentions of "bug"
escribano-query entities --kind programming_language # which languages you used
```
The CLI is JSON-first, versioned (`api_version: 2`), and self-describing via `--help --json`. See the [CLI Reference](/doc/cli-reference-escribano-query-VOMrfUt91U) for the full command surface.
---
## Privacy by Design
| Layer | What happens |
|-------|--------------|
| Capture | Screenshots never leave your machine |
| Understanding | VLM runs on the local Apple Silicon GPU |
| Storage | SQLite database on local disk |
| Query | Text output is PII-filtered by default |
| Images | `--images` flag required to surface screenshots |
---
## For Agents
The CLI is designed for programmatic access from coding assistants and other agents:
* **JSON output** — structured, versioned (`api_version: 2`)
* **Self-discovery** — `escribano-query --help --json` returns the command surface
* **Exit codes** — `0` on success (even empty data), non-zero on failure
* **Progressive disclosure** — text by default, `--images` for visual verification
:::info
**Quick install** for popular agents:
:::