backend / hal-meetings
Hal
A self-hostable meeting agent: a Next.js control plane on Vercel encrypts a join request into a Postgres job queue, and a Dockerised Playwright worker on EC2 claims it, joins Google Meet or Zoom as a disclosed bot, captures tab audio through PulseAudio into Deepgram, and stores an envelope-encrypted transcript and summary.
2026 · live · open source
- [*]LanguagesTypeScript · HCL · Bash · SQL
- [*]FrameworksNext.js · better-auth · Drizzle ORM · Playwright · Bun · commander
- [*]StoresPostgres (Neon)
- [*]AsyncPostgres job queue (FOR UPDATE SKIP LOCKED) · exponential-backoff retries · stuck-claim reaper (60 s interval) · fire-and-forget segment persistence
- [*]InfraVercel · AWS EC2 · Terraform · AWS SSM Parameter Store · Docker · Render (blueprint) · PulseAudio + Xvfb · systemd
- [*]Third partiesDeepgram · Google Gemini · Anthropic · Ollama · Google OAuth · Google Calendar API · Resend · ElevenLabs
Two deployables share 5 workspace packages (db, crypto, media, meeting-links, ui). The web app (Next.js on Vercel, better-auth with Google) handles sign-in, Google Calendar connection, and a bearer-token ingest endpoint that finds a meeting link in any text and enqueues a join. The agent is a long-lived Bun worker in a Docker image that bundles Xvfb, PulseAudio with a null sink, and Playwright Chromium. It runs on an EC2 instance provisioned with Terraform, and the repo also has a Render blueprint.
The queue is a Postgres jobs table claimed with a CTE using FOR UPDATE SKIP LOCKED and ordered by priority then scheduled time. Failed jobs go back to pending with exponential backoff (2^n seconds, capped at 5 minutes) until max_retries. A reaper runs every 60 s and releases claims older than 5 minutes from crashed workers, and SIGTERM drains in-flight jobs for up to 30 s.
Every sensitive payload is envelope-encrypted: AES-256-GCM through WebCrypto, a versioned wire format [version | iv | ct+tag], per-user and per-workspace data keys wrapped by a KMS interface, and DEK bytes zeroed after each use. That covers job payloads, OAuth tokens, transcripts, summaries and each live transcript line. Final STT lines are written as they arrive to transcript_segments, which has a unique (meeting_id, seq) index, so a worker that dies mid-call keeps what it had and a retried write cannot duplicate a line.
Much of the agent code exists because scraping a UI fails in specific ways: lobby text treated as proof the bot was not yet admitted, a 4-hour hard ceiling on a meeting, a DOM inventory dumped to the log when a join fails, a canary that fails audio made only of digital zeros, and removal of Hal's own disclosure text so it does not read its own '/hal stop' as a kill command.
How it holds up
- Auth
- better-auth with Google sign-in (identity scopes only; calendar scopes requested separately with forced consent to get a refresh token) and Drizzle-backed sessions. Middleware is a cookie-presence check and pages remain the authority. Machine ingest uses a bearer token of at least 32 characters compared with timingSafeEqual and bound to one configured user. Meeting reads return not-found for any meeting outside the caller's workspace.
- Data
- 13 tables across 5 SQL migrations: users, workspaces, workspace_members, meetings, transcripts, transcript_segments, oauth_tokens, jobs, audit_log, and 4 better-auth tables. All content columns are bytea ciphertext. Migration 0001 backfilled a personal workspace for every existing user and moved every tenant-scoped table onto workspace_id in one migration.
- Consistency
- Job claims are atomic through a SKIP LOCKED CTE. The claim returns ids only and re-selects through Drizzle, because the raw row's snake_case columns had been mis-mapped to undefined. Transcript segments are idempotent on (meeting_id, seq) via ON CONFLICT DO NOTHING, and live views page by seq rather than timestamp so lines in the same millisecond are not skipped or repeated. Rejoining a meeting reuses the row only if the caller's own workspace owns it.
- Failure
- Failed jobs back off exponentially up to max_retries and then stay failed. Crashed workers' claims are reaped. A failed summary falls back to a placeholder and still persists the transcript. A failed email is logged and does not fail the job. One undecryptable segment renders as a warning line instead of blanking the transcript. On EC2 the env file is rebuilt from SSM atomically on every service start, so a transient SSM outage leaves the previous file in place.
- Testing
- 17 bun:test files with 223 cases: calendar 39, hal-plan 27, meeting-links 23, chat-commands 22, zone 17, conferencing 14, google-scopes 13, canary-checks 13, ingest-links 11, diagnostics 10, envelope 9, redactor 7, transcript 5, google-oauth 5, join-meeting 4, workspace-dek 2, auth-schema 2. infra/aws/test-hal-write-env.sh tests the env writer.
Techniques
- Postgres job queue with SKIP LOCKED, backoff and a reaperA CTE claims pending, due jobs ordered by priority and schedule with FOR UPDATE SKIP LOCKED, so several workers never take the same meeting. Failures requeue with capped exponential backoff, and claims from crashed workers are released every minute.
packages/db/src/repositories/jobs.ts:33packages/db/src/repositories/jobs.ts:111packages/db/src/repositories/jobs.ts:156apps/agent/src/jobs/consumer.ts:52 - Envelope encryption with a versioned wire formatAES-256-GCM through WebCrypto with KMS-wrapped data keys, never cached and zeroed after every call. The output is [version byte | 12-byte IV | ciphertext+tag], and tampering or a short input raises a typed auth error. Job payloads, OAuth tokens, transcripts and live lines all go through it.
packages/crypto/src/envelope.ts:41packages/crypto/src/envelope.ts:86apps/agent/src/workspace.ts:20 - Crash-surviving, idempotent live transcriptEach final STT line is encrypted and appended with an explicit seq under a unique (meeting_id, seq) index, without blocking the audio pump. A dead worker keeps its lines, a retry cannot duplicate one, and readers resume by seq instead of timestamp.
apps/agent/src/pipeline/meeting-session.ts:96packages/db/src/repositories/transcript-segments.ts:19packages/db/migrations/0004_transcript_segments.sql:41 - Single-migration multi-tenancy backfillOne SQL migration creates workspaces and members, copies each user's wrapped DEK into a personal workspace through a temporary owner column, backfills workspace_id on 5 tables, then drops the helper column and enforces NOT NULL plus foreign keys.
packages/db/migrations/0001_workspaces.sql:36packages/db/migrations/0001_workspaces.sql:44 - Headless meeting bot with audio capture and failure ceilingsA container entrypoint starts Xvfb and a PulseAudio null sink. Chromium's output is pinned to the sink and parec reads its monitor as 16 kHz s16le. A 4-hour hard cap ends sessions whose DOM end-detection breaks, and lobby text is treated as proof of not being admitted.
apps/agent/docker/entrypoint.sh:9apps/agent/src/audio/pulse.ts:36apps/agent/src/pipeline/meeting-session.ts:186apps/agent/src/runtime/meet.ts:28 - Silence-aware canary and failure-time DOM inventoryThe canary fails audio whose fraction of non-zero s16le samples is below 1%, which catches a well-formed recording of digital silence. Join failures log a bounded inventory of buttons and inputs, flagging disabled-but-visible ones, so a change in Meet's UI can be diagnosed from the log.
apps/agent/src/canary/checks.ts:47apps/agent/src/runtime/diagnostics.ts:58
Counted
- 81commits
git rev-list --count HEAD - 17008TypeScript LOC (all tracked .ts/.tsx)
git ls-files '*.ts' '*.tsx' | xargs wc -l | tail -1 - 8556backend LOC (agent, db, crypto, media, meeting-links, web server/api/auth)
git ls-files 'apps/agent/src/*.ts' 'packages/{db,crypto,media,meeting-links}/src/*.ts' 'apps/web/src/server/*.ts' 'apps/web/src/app/api/*.ts' apps/web/src/lib/auth.ts | xargs wc -l - 520Terraform LOC
git ls-files 'infra/*.tf' | xargs wc -l | tail -1 - 5SQL migrations
ls packages/db/migrations/*.sql | wc -l - 13tables
grep -h 'CREATE TABLE' packages/db/migrations/*.sql - 6 / 7API route files / handlers
git ls-files 'apps/web/src/app/api/**/route.ts'; auth exports GET+POST - 17 / 223test files / cases
grep -cE '^\s*(it|test)\(' over git ls-files '*.test.ts' - 733Meet runtime LOC
wc -l apps/agent/src/runtime/meet.ts - 4 KMS, 3 LLM, 2 STT, 2 TTSprovider adapters
packages/crypto/src/kms/factory.ts, packages/media/src/{llm,stt,tts}/factory.ts