Metermark
Start free

Security review: the Metermark agent

For security and IT teams evaluating a deployment.

What it is

A single static binary, metermark, that runs as the signed-in user on a schedule, reads the local history of AI coding tools, sends short prompt samples to Metermark for classification, and uploads per-session labels and numbers to the customer's organization.

What leaves the machine

Per session, one row: who (email), when, which tool and model, token counts and estimated cost, project name, an inferred kind of work, five practice scores, and one label of at most eight words. Per person per day, one usage row per vendor. That is the whole upload; metermark sync --dry-run prints what the next sync would send, verbatim.

For classification only, up to 40 prompt excerpts of at most 400 characters each are sent over HTTPS to metermark.ai, processed in memory, and discarded once the label is produced. They are not written to any log, database or cache on either side; Metermark's own test suite injects a canary string into prompts and sweeps logs and the database for it on every change. content_mode: metadata turns sampling off entirely: sessions are then counted and costed but not labeled.

What never leaves the machine

Source files, diffs, full transcripts, model responses, file contents, environment variables, credentials for other services. The agent does not read them. It sends no telemetry: its only network calls are to metermark.ai, to auth.metermark.ai for an interactive sign-in, and to Metermark's release store for updates. Nothing about a machine is reported to any third-party analytics service.

Privileges and footprint

  • Runs as the user: no root, no setuid, no kernel extension, no Full Disk Access.
  • Reads only the AI tools' own data directories in the user's home, and avoids looking up git remotes inside Documents, Desktop and Downloads so that macOS never prompts.
  • No inbound network listener. The optional metermark serve local API exists for developers and is off unless started by hand.
  • Outbound: metermark.ai:443; auth.metermark.ai:443 only for interactive sign-in. HTTPS_PROXY and NO_PROXY are honoured; a private root certificate is trusted through NODE_EXTRA_CA_CERTS.
  • The local store is a SQLite file in the user's home, readable only by that user, holding the same rows that are uploaded and nothing else.

Identity and authorization

  • Fleet installs authenticate with an organization enrollment key that can only upload and classify; it cannot read anything back (the read API, MCP and key management answer 403). Keys are created by owners and admins and revocable instantly; revocation stops every machine at its next sync.
  • Self-installs authenticate with a user token issued by Metermark's identity provider (Google sign-in today; SAML or OIDC single sign-on on the Enterprise plan), stored readable only by the user, refreshed automatically, and revoked on sign-out.
  • Who may see person-level data in the dashboard, API and MCP is role-gated server-side: members of a company organization see only themselves in person-level views; owners and admins see the organization. Cohort benchmarks never name anyone below a minimum group size.
  • Person-level reads are audit-logged: whenever an owner, admin or API key opens someone else's page, lists or opens their sessions, centres benchmarks on them, asks a question whose answer names them, or exports the organization, one row records who, whom, through which channel (API, MCP, dashboard) and the request id, never what was returned. Members reading themselves are not logged. Owners and admins read the log at GET /api/v1/audit and on the People page.

Supply chain and updates

  • Releases are published with SHA-256 checksums for every asset. The macOS binaries, app and installer package are signed with Hassion Studio, Inc.'s Developer ID and notarized by Apple. The Windows builds are not code-signed yet.
  • Self-update downloads the asset for its platform, verifies the checksum and size, and on macOS the code signature, before replacing the program. Fleets set update_channel: managed to disable self-update and manage versions themselves.
  • Third-party services behind the hosted side: Vercel (hosting), Neon (Postgres), Auth0 (sign-in), Stripe (billing), Resend (email), PostHog (product analytics and error reporting for the website and dashboard), and the classification model provider, which receives only the transient prompt samples described above. Data is stored in the United States.
  • PostHog receives page views, clicks, feature usage and error stack traces from the website and the dashboard, keyed by an internal account id and the organization's plan. Signed-out pages set no analytics cookie. It never receives prompts, session content, emails, or the names of people in an organization. The agent on the laptop sends nothing to PostHog or any other third party.

Data lifecycle

Rows are retained while the organization is active. An individual account deletes itself with DELETE /api/v1/me, which removes its sessions, usage days, keys, quota and audit rows and its Stripe customer at once. A company organization is deleted by Metermark on its owner's request, within 30 days. The sign-in identity is removed as part of either, and until it is, a token that outlives the account cannot re-create it. Billing records are kept. GET /api/v1/export streams the organization's sessions and usage days as one JSON document at any time, under the same role rules as every other read. See https://metermark.ai/privacy.