corvo.rda.run
git.rda.run/corvo/corvo
Install
the facts, for looking up

Reference

Every environment variable, route, status code and column.

Configuration

Everything is configured through environment variables. There is no configuration file: .env exists only for make dev and make pg-up, and is never read in production. Export the variables for real there, with a systemd EnvironmentFile=, an orchestrator secret, or whatever your platform offers.

An empty variable counts as unset and falls back to its default.

Database

variabledefaultmeaning
NINHO_DSN(required)The database owner. Applies migrations, creates partitions, and is the fallback for the two roles below. The server refuses to start without it.
NINHO_INGEST_DSNNINHO_DSNConnection used to write the trail. In production, a user holding the ninho_ingest role.
NINHO_READ_DSNNINHO_DSNConnection used to read the trail. In production, a user holding the ninho_reader role.
NINHO_PARTITION_MONTHS3How many monthly partitions to keep created ahead. Checked at startup and every hour.

Every connection is opened with timezone=UTC. See Database for what the roles may do.

Server

variabledefaultmeaning
NINHO_ADDR:8080Listen address.
NINHO_INGEST_PATH/api/ingestPath the agents POST to.
NINHO_INGEST_TOKENemptyBearer token the agent must send. Empty accepts any agent, and startup logs a warning.

Session

variabledefaultmeaning
NINHO_SESSION_KEYrandomKey that signs the session cookie. When empty, a random 32-byte key is generated at startup and a warning is logged.
NINHO_SESSION_HOURS12Session lifetime. A value that is not a positive integer falls back to the default.
NINHO_READ_TOKENemptyOptional bearer token that reads the API without a sign-in, for scripts.

NINHO_SESSION_KEY has to be set, and identical, on every instance behind a load balancer. The cookie is stateless and signed with HMAC-SHA256: an instance cannot verify a cookie another instance signed with a different key, and the symptom is a sign-in that works followed by 401 on every screen. Changing the key signs everyone out, and nothing else.

Any non-empty string works as a key. Generate one with openssl rand -base64 32.

LDAP

variabledefaultmeaning
NINHO_LDAP_URLemptyldaps://dir.example:636 or ldap://dir.example:389. Empty leaves the UI and the read API open, and startup logs a warning.
NINHO_LDAP_BIND_DN(required with LDAP)Service account that looks up the signing-in user. The lookup is never anonymous, and the server refuses to start without this.
NINHO_LDAP_BIND_PASSWORD(required with LDAP)Its password.
NINHO_LDAP_BASE_DNemptySearch base, for example ou=people,dc=example,dc=com.
NINHO_LDAP_FILTER(uid=%s)Search filter. %s receives the typed username, escaped.
NINHO_LDAP_GROUP_DNemptyWhen set, only members of this group may sign in.
NINHO_LDAP_INSECUREfalseWith an ldap:// URL, skips StartTLS. Development only: the password then crosses the network in the clear.

With an ldap:// URL and NINHO_LDAP_INSECURE unset, the connection is upgraded with StartTLS and a failure to upgrade fails the sign-in. ldaps:// URLs are always TLS. Both require TLS 1.2 or newer and a certificate the system trusts.

See Configure LDAP sign-in for the group check and The authentication model for what each bind is for.

Image build

Read only by make docker, which asks Quay to build. They are not read by the server.

variablemeaning
NINHO_QUAY_TOKENThe password half of the Quay trigger's webhook URL, under "View Credentials". The basic-auth user is the literal string $token.
NINHO_QUAY_TRIGGERThe trigger UUID from that same URL.

Timeouts and limits

These are not configurable. They are listed here because they explain behaviour you may otherwise have to read the source to understand.

whatvaluewhere
Sign-in deadline10 sCovers the whole LDAP exchange, including the dial.
Read header timeout10 sHTTP server. There is no write timeout: it would cut the live tail's stream.
Database ping on startup10 sPer pool.
Shutdown grace15 sAfter SIGINT or SIGTERM.
Ingest body, compressed8 MiBOver it, 413.
Ingest body, decompressed32 MiBThe ceiling that matters against a gzip bomb.
Live tail poll2 sOne query for all connected clients, and none when nobody is connected.
Live tail heartbeat20 sKeeps proxies from closing an idle stream.
Events page size100, max 500limit query parameter.
Session page size1000, max 5000limit query parameter.

HTTP API

Two surfaces on one port: ingest, which the agent writes to, and the read API, which the UI and your scripts read from. They authenticate differently and on purpose, and the ingest path never depends on a session.

All responses are application/json; charset=utf-8 unless noted. All instants are RFC 3339 in UTC.

Ingest

POST /api/ingest

The path is configurable with NINHO_INGEST_PATH. Implements section 4 of corvo's design/api-contract.md.

Authentication: Authorization: Bearer <NINHO_INGEST_TOKEN>, compared in constant time. An empty NINHO_INGEST_TOKEN disables the check and accepts any agent.

Body: a JSON array of envelopes. Content-Encoding: gzip is accepted.

[{"v": 1, "id": "...", "host": "...", "event": { }}]
fieldmeaning
vEnvelope format. Stored as format, clamped into a smallint. An unknown value is not rejected.
idDeduplication key, an opaque string. When empty, host plus the occurrence instant is used.
hostThe machine that produced it.
eventThe whole event, stored as jsonb with no loss.

From event, these become columns: @timestamp, category, record_type, result, session, summary.actor.primary, summary.action, user.ids.auid, user.ids.uid, process.exe, and process.args joined by a space into args_text. Everything else stays in the jsonb and is still queryable there. No field inside event is required, and record_type is not validated against a list: the set of types is each host's filter's choice, and it changes with a SIGHUP.

The read rows also carry auid_name and uid_name, the id turned into the name a person is held accountable by. It comes from summary.actor or user.names when corvo resolved the id on the host. When corvo could not (an agent in a container has no access to the host's /etc/passwd), the name is recovered from /home/<name> in process.cwd, which only names the logged-in user, so it stands in for uid only when uid is that same id. Empty string when nothing was found, and the id then shows on its own.

An unparseable @timestamp falls back to the receive instant. Dropping the event would be worse than storing it with an approximate time.

Envelopes whose host is corvo-check, or whose id starts with corvo-check-, are corvo check probes: they are counted, answered 2xx, and not persisted.

Responses

statuswhenwhat the agent does
204Persisted, including a batch that was entirely probes or duplicates.Drops the batch from its queue.
400Body is not readable, is not a JSON array, or an event is malformed.Dead-letters the batch with this response attached, and the queue unblocks.
401Missing or wrong token.Retries for 15 minutes, which gives a token rotation time to arrive.
413Over 8 MiB compressed or 32 MiB decompressed.Dead-letters.
503The database, or anything else on our side, failed.Keeps the batch and retries.

Getting the 400 and 503 distinction wrong is how audit data disappears without anyone noticing. See The contract and the status codes.

Authentication

POST /api/login

{"username": "ana", "password": "..."}

Body is capped at 8 KiB. The whole LDAP exchange has a 10 second deadline.

200 sets the ninho_session cookie and returns {"auth": true, "user": "ana", "version": "0.1.0+..."}. 401 with a plain text body means an unknown user, a wrong password, or a user outside NINHO_LDAP_GROUP_DN: the three are deliberately indistinguishable to whoever is guessing. The reason is in the server log.

With no directory configured, this returns 200 and {"auth": false, "user": "", "version": "..."} without setting a cookie.

POST /api/logout

204, and clears the cookie.

GET /api/me

{"auth": bool, "user": string, "version": string}. auth is false when no directory is configured, which is how the UI decides between the sign-in screen and the trail. Needs no session.

Reading the trail

Every route below requires a session cookie or Authorization: Bearer <NINHO_READ_TOKEN>, and answers 401 not authenticated without one. When no directory is configured they are all open. All of them except the stream are gzipped when the client asks for it.

A failed query answers 503 unavailable, never a 4xx: the client asked for something valid and our side failed.

GET /api/events

The main query. Keyset pagination, never OFFSET.

parameterdefaultmeaning
host, record_type, resultnoneExact match.
auid, uidnoneExact match on the raw id or on the resolved name (auid_name / uid_name).
qnoneSubstring of args_text, case-insensitive. Never matches process.title.
from, tononeRFC 3339. occurred_at >= from and occurred_at < to.
cursor_at, cursor_idnoneThe pair from the previous page. They travel together: without the id there is no tiebreaker.
limit100, max 500
{
  "events": [ ],
  "next_cursor_at": "2026-01-15T10:30:00Z",
  "next_cursor_id": "..."
}

The two cursor fields are present only when the page came back full, which is the signal that another page may exist. Rows carry the extracted columns and never the jsonb.

Ordering is occurred_at DESC, id DESC.

GET /api/events/{id}

The only route that returns the whole raw event. occurred_at is a required query parameter in RFC 3339, because the table is partitioned by it and the primary key is the (id, occurred_at) pair.

400 without it, 404 when there is no such event.

GET /api/sessions/{host}/{session}

Everything one login session did on one host, oldest first. limit defaults to 1000, capped at 5000.

GET /api/agents

One row per host: first_seen, last_seen, last_event_at, agent_version, events_total, and silence_seconds computed in the database. Ordered by last_seen ascending, so the quietest agent is first.

silence echoes back as silence_threshold for the UI to draw a line with. It defaults to 3600 and is capped at 30 days.

GET /api/overview

parameterdefaultmeaning
top10, max 50Size of each ranking.
silence3600, max 2592000Seconds without a batch before an agent counts as silent.
{
  "per_hour": [{"hour": "...", "total": 0}],
  "top_exe": [{"value": "/usr/bin/rm", "total": 0}],
  "top_auid": [ ], "top_host": [ ],
  "silent_agents": [ ],
  "silence_threshold": 3600
}

Every ranking covers the last 24 hours and is aggregated in the database. top_auid shows summary.actor.primary when there is one and falls back to the raw auid.

GET /api/stream

Server-sent events. One data: line per event, carrying the same shape as an /api/events row.

The stream starts at the moment you connect: it shows what arrives from here on, not history. It is driven by received_at and not occurred_at, so a redelivered event with an old timestamp still shows up.

A single goroutine polls every 2 seconds and fans out to every client, so open tabs do not multiply the database load, and no query runs when nobody is connected. Each client has a 64-message buffer and a client that cannot keep up loses messages rather than holding up the others. Every 20 seconds an SSE comment (: ping) goes out so no proxy closes an idle connection.

This route is not gzipped: compressing a stream that has to flush on every event only adds buffering between the database and the screen.

Everything else

GET / and anything unmatched serves the embedded UI.

Database

PostgreSQL. The schema is created by the migrations the server applies at startup, so there is no separate migration step and no migration tool to install.

Migrations

The .sql files under migrations/ are embedded in the binary and applied in name order, each in its own transaction, and recorded in schema_migration (name, applied_at). A file that has already been applied is skipped.

Startup takes a PostgreSQL advisory lock (id 8101975031) first, so two instances booting at the same time do not apply the same migration twice.

Migrations run as NINHO_DSN, the owner. The ingest and read roles never apply schema changes.

audit_event

The trail. Partitioned by RANGE (occurred_at), primary key (id, occurred_at).

columntypesource
idtextThe envelope's id, the deduplication key.
hosttextThe envelope's host.
occurred_attimestamptzThe event's @timestamp, or the receive instant when it cannot be parsed.
received_attimestamptzDefaults to now(). What the live tail reads.
formatsmallintThe envelope's v.
category, record_typetext, not null, default ''
result, session, auid, uid, actor, action, exetext, nullableExtracted from event.
args_texttextprocess.args joined by a space. Never process.title.
eventjsonb, not nullThe whole event, with no loss.

The extracted columns exist to be queried and indexed. The jsonb is the source of truth, which is why a new record_type never needs a schema migration: it is already stored, and already queryable through the jsonb.

Partitions

Monthly, named audit_event_YYYY_MM, created at startup and rechecked every hour for the current month plus NINHO_PARTITION_MONTHS - 1 ahead. The hourly recheck is what makes the month rollover independent of anyone restarting the server.

audit_event_default catches everything that falls outside them. Without that escape hatch, an event from a host with a badly wrong clock would fail to insert, the response would be 503, and the agent would retry that batch forever.

If a monthly partition cannot be created because the default partition already holds rows in its range, startup logs a warning and carries on. Moving those rows is a manual operation, and blocking the server over it would be worse.

Indexes

indexwhat it serves
(occurred_at DESC, id DESC)Keyset pagination on /api/events.
(received_at DESC)The live tail.
(host, occurred_at DESC)Filtering by host.
(auid, occurred_at DESC)Filtering by who logged in.
(host, session, occurred_at)The session screen.
gin (args_text gin_trgm_ops)Substring search on the command, via pg_trgm.

agent

One row per host: first_seen, last_seen, last_event_at, agent_version, events_total. Upserted once per batch, in the same transaction as the events, with last_event_at taking the greater of the old and new values and events_total accumulating.

agent_version comes from the User-Agent header and not from the body, since the body is written by the same agent the field describes.

This table exists because a table of events cannot detect silence on its own.

v_command

A convenience view over events that have a process.exe, exposing exe, cwd, and session pulled out of the jsonb. The test is for process.exe being filled in and not for process existing, because process is a struct and shows up in the JSON even when empty.

principal_name(event, which)

003_principal_name.sql adds an immutable SQL function that turns 'auid' or 'uid' into the name a person is held accountable by. It reads summary.actor and user.names first, and when corvo resolved neither (an agent in a container has no host /etc/passwd), falls back to /home/<name> in process.cwd. The read queries call it for the auid_name and uid_name fields; the jsonb still never leaves the server.

Roles

002_roles.sql creates two roles, NOLOGIN and passwordless, because a password in versioned SQL is a leaked password.

roleprivileges
ninho_ingestINSERT on audit_event, INSERT and UPDATE on agent.
ninho_readerSELECT on audit_event, agent, and v_command.

Neither can DELETE or UPDATE the trail. The trail stays append-only even if the endpoint token leaks, which is the scenario this separation exists for.

On a database where the owner cannot create roles, the migration raises a notice and moves on. Create them by hand with a superuser, then follow Split the database roles.

Writes

A batch is one transaction: all events through pgx's SendBatch, plus one upsert per distinct host. The event insert is ON CONFLICT (id, occurred_at) DO NOTHING, because corvo's delivery is at-least-once and a duplicate is routine rather than an edge case.

Make targets

make help lists them at any time. Targets that read .env do so only when the file exists, and never in production.

Building

targetwhat it does
buildRuns web, then go build, producing ./ninho. Needs Go and Node.
webnpm ci and npm run build in web/, writing web/dist, which the binary embeds.
cleanRemoves the binary, the coverage files, web/dist, and dist.

The version is stamped in at link time with -X main.version=$(BUILD), where BUILD is the exact git tag when HEAD is on one, and otherwise the contents of .version plus git describe, for example 0.1.0+8db552f-dirty. It shows up in the startup log, in /api/me, and in the UI footer.

Developing

targetwhat it does
devLoads .env, then runs the Go server and Vite together. Ctrl-C stops both. The UI is on http://localhost:5173 and proxies /api to 127.0.0.1:8080.
pg-upStarts a disposable PostgreSQL 18 in podman, named ninho-pg, reading user, password, port, and database out of the NINHO_DSN in .env. Waits until it accepts connections.
pg-downRemoves that container, data included.
generateRegenerates internal/store/db/ from the SQL in internal/store/query/ with sqlc. The generated code is checked in, so building the binary does not need sqlc.
fmtgofmt -w over the Go sources.

dev injects the version the same way build does, minus the symbol stripping, so the debugger keeps its DWARF.

Checking

targetwhat it does
testgo test ./.... The integration tests spin up an ephemeral PostgreSQL with podman.
raceThe tests under the race detector.
coverCoverage report in cover.html.
vetgo vet ./....
lintvet, plus golangci-lint when it is installed.

Releasing

targetwhat it does
dockerAsks Quay to build and push the image for the current commit. Nothing is built locally.

docker needs NINHO_QUAY_TOKEN and NINHO_QUAY_TRIGGER in .env, and refuses to fire when the commit is not on a remote branch: Quay builds from what it can fetch, so an unpushed commit would build the wrong thing. It sends the exact commit sha, and the ref is the tag when HEAD is on one.

See Build and release.

auditd → https · nothing gets lost corvo (Portuguese for raven) is the agent, ninho (nest) is the server. Both speak the same versioned contract.
docs how it works corvo (agent) ninho (server)
ninho docs tutorials how-to guides reference explanation
corvo docs tutorials how-to guides reference explanation
reference API contract install metrics and queue
code git.rda.run/corvo/corvo git.rda.run/corvo/ninho MIT license