work / voxtar

Voxtar

Clinical voice AI: a LiveKit voice agent records voice and cough samples, and acoustic measurements stream to a clinician console.

Open voxtar.hallelx2.com How it works Voxtar backendpyzheimer
Years
2026
Status
paused
Live
voxtar.hallelx2.com

What it is

A caller talks to a configurable clinical voice agent. When the script calls for it, the agent records a five-second sustained vowel or a cough, and pyzheimer, a Python service I wrote for acoustic analysis, turns the sample into measurements that stream back to the console: jitter, shimmer, fundamental frequency tracked with pYIN, and maximum phonation time.

Read more

Each person gets a baseline, a 10%-trimmed mean with σ taken as 1.4826 × the median absolute deviation, so the console shows drift from that person's own normal rather than a population average: under 1.5σ is normal, then mild, moderate and severe. A sample is only analysed if it clears the gate: SNR at least 15 dB, at most 1% clipping, at least 2 seconds long.

The browser and the agent never call each other for voice. Both join a LiveKit room. Speech runs Deepgram nova-3 for transcription, GLM 4.5 Air by default for the conversation (GPT-4o mini or Gemini 2.5 Flash optional), Deepgram aura-2, ElevenLabs or OpenAI for the voice, with Silero for voice activity. LiveKit, the agent and pyzheimer run together on one EC2 machine behind one domain.

Stack

  • TypeScript
  • Next.js
  • LiveKit
  • Deepgram
  • ElevenLabs
  • OpenAI
  • Gemini
  • Z.ai GLM
  • Silero VAD
  • Hono
  • tRPC
  • Expo
  • Python
  • FastAPI
  • AWS EC2

Counted

  • 113monorepo commits
    git rev-list, voxtar
  • 6apps
    web, agent, backend, admin, patient, docs
  • Jan → Aug 2026first commit → last
    git log

How it works

The system as a machine on a bench. Hover or use the arrow keys to drive it.

001pyzheimer
Needles and chart pen: jitter, shimmer, F0 and MPT of each accepted sampleBand on the chart: the speaker's baseline, trimmed mean ± 1.5 × 1.4826 · MADBelt: the conversation loop, STT → LLM → TTSVoxtar's voice agent asks for a 5-second sustained “ah” in a LiveKit room (Deepgram STT, GLM 4.5 Air by default or GPT-4o mini or Gemini 2.5 Flash, Deepgram or ElevenLabs TTS, Silero VAD) and posts the recording to pyzheimer, which rejects it unless SNR ≥ 15 dB and clipping ≤ 1%, then measures jitter, shimmer, F0 and MPT and scores each against the speaker's own baseline, flagging drift beyond 1.5 σ. The ten baseline sessions and eight samples on this bench are synthetic; every verdict, band and z-score comes from pyzheimer's own formulas.

The path through it

  1. JoinThe browser gets a LiveKit token and joins a room; an agent worker is dispatched into the same room.
  2. ConfigureThe agent loads its schema and builds its system prompt from the shared package.
  3. ConverseDeepgram nova-3 STT → GLM 4.5 Air (or GPT-4o mini, Gemini 2.5 Flash) → Deepgram aura-2, ElevenLabs or OpenAI TTS, with Silero VAD.
  4. SampleVoice and cough recordings go to pyzheimer's FastAPI endpoints.
  5. Gate and analyseSNR ≥ 15 dB, clipping ≤ 1%, ≥ 2 s; then jitter, shimmer, F0 (pYIN, 60–500 Hz) and maximum phonation time.
  6. Baseline and drift10%-trimmed mean and 1.4826 × MAD per person; z under 1.5 is normal, then mild, moderate, severe. Results stream back over the LiveKit data channel.