backend / aurasense

Aurasense

A FastAPI service where a LangGraph supervisor routes each voice or text turn to onboarding, food or profile agents, with per-user memory kept in a Graphiti knowledge graph on Neo4j.

2025–26 · paused · open source

web → nginx: HTTPS / WSS + Bearer JWTnginx → api: proxy :8000api → sup: ainvoke(thread_id=supervisor:uid)sup → agents: conditional edge by intentagents → gw: get_llm(role)gw → llm: chat completionssup → redis: AsyncRedisSaver checkpointsagents → neo4j: Graphiti add_episode / searchagents → fsq: Places v3 searchapi → redis: blacklist + INCR rate limitNext.js frontendNext.js frontend (client) NextAuth + axios, WS clientNginxNginx (edge) reverse proxy on dropletFastAPI appFastAPI app (service) REST /api/v1 + 2 WebSocketsSupervisor graphSupervisor graph (worker) classify + onboarding gateSpecialist agentsSpecialist agents (worker) onboarding, food (8 nodes), profileLLM gatewayLLM gateway (service) role -> provider:modelRedis StackRedis Stack (store) checkpoints, JWT blacklist, rate limitsNeo4jNeo4j (store) neomodel graph + Graphiti episodesGroq / OpenAI / Gemini/ OpenRouter / OllamaGroq / OpenAI / Gemini / OpenRouter / Ollama (model) chat + WhisperFoursquare PlacesFoursquare Places (third party) restaurant searchHTTPS / WSS + Bearer …proxy :8000ainvoke(thread_id=sup…conditional edge by i…get_llm(role)chat completionsAsyncRedisSaver check…Graphiti add_episode …Places v3 searchblacklist + INCR rate… web → nginx: HTTPS / WSS + Bearer JWTnginx → api: proxy :8000api → sup: ainvoke(thread_id=supervisor:uid)sup → agents: conditional edge by intentagents → gw: get_llm(role)gw → llm: chat completionssup → redis: AsyncRedisSaver checkpointsagents → neo4j: Graphiti add_episode / searchagents → fsq: Places v3 searchapi → redis: blacklist + INCR rate limitNext.js frontendNext.js frontend (client) NextAuth + axios, WS clientNginxNginx (edge) reverse proxy on dropletFastAPI appFastAPI app (service) REST /api/v1 + 2 WebSocketsSupervisor graphSupervisor graph (worker) classify + onboarding gateSpecialist agentsSpecialist agents (worker) onboarding, food (8 nodes), profileLLM gatewayLLM gateway (service) role -> provider:modelRedis StackRedis Stack (store) checkpoints, JWT blacklist, rate limitsNeo4jNeo4j (store) neomodel graph + Graphiti episodesGroq / OpenAI /Gemini / OpenRouter…Groq / OpenAI / Gemini / OpenRouter / Ollama (model) chat + WhisperFoursquare PlacesFoursquare Places (third party) restaurant searchHTTPS / WSS + Bearer …proxy :8000ainvoke(thread_id=sup…conditional edge by i…get_llm(role)chat completionsAsyncRedisSaver check…Graphiti add_episode …Places v3 searchblacklist + INCR rate…
clientedgeserviceworkerstoremodelthird partyasync
  • [*]LanguagesPython
  • [*]FrameworksFastAPI · LangGraph · LangChain · graphiti-core · neomodel · Pydantic Settings
  • [*]StoresNeo4j · Redis Stack (RediSearch + RedisJSON)
  • [*]Asyncasyncio.to_thread offload for sync neomodel · LangGraph AsyncRedisSaver checkpoints · WebSocket streaming
  • [*]InfraDocker · Docker Compose · DigitalOcean droplet · Nginx · GitHub Actions
  • [*]Third partiesGroq (LLM + Whisper STT) · OpenAI · OpenRouter · Gemini (OpenAI-compatible) · Ollama · Foursquare Places v3

Aurasense started in July 2025 as a voice-first lifestyle companion. It had JWT auth, an onboarding agent and a Docker Compose stack of Neo4j, Redis and a separate Graphiti container. Milan (pathfindermilan) wrote 6 of the 34 commits, mostly onboarding-agent and WebSocket fixes in that first week. The rest is the owner's.

On 3 and 4 May 2026 the backend was rebuilt across six merged PRs, phase 0 to phase 4.5. Phase 0 made the defaults secure: production refuses to boot with the placeholder SECRET_KEY, and CI gates deploys on pytest. Later phases added an LLM gateway that every model call goes through, a BaseAgent with a Redis-backed LangGraph checkpointer, Graphiti running in-process as a typed RAG layer, a Profile agent that merges declared and observed facts, and a supervisor graph that sits behind one /ws/agent WebSocket.

The food agent runs as an eight-node graph: intent, context, health_screen, search, recommend, confirm_or_order, place_order, record. A regex allergy filter runs after the LLM recommends dishes, so the model is never the safety check. Restaurant data comes from a provider switch between a 12-dish mock catalogue and the Foursquare Places v3 API.

Measured in the code: 22 route decorators. 11 of them (all voice, social and travel routes) are still stubs whose bodies are pass. 16 test files define 156 test functions. Deploys go to a DigitalOcean droplet over SSH with docker compose up. The domain in the Nginx config, teepon.tech, does not resolve today.

How it holds up

Auth
Email and password with Argon2 hashing and PyJWT bearer tokens. Logout writes the token to a Redis blacklist with TTL equal to its remaining life. WebSockets authenticate with ?token=. The blacklist and the Redis rate limiters (login 5/min, register 3/hour) fail closed in production and fail open in development.
Data
Neo4j holds 6 neomodel StructuredNode types (User, Order, Restaurant, MenuItem, Hotel, Location) and 9 StructuredRel types. Graphiti adds 6 typed Pydantic entity types (Allergy, DietaryRestriction, HealthCondition, FoodPreference, CulturalContext, RestaurantVisit), every episode scoped with group_id = user_id. Conversation state lives in Redis as LangGraph checkpoints under thread_id 'supervisor:{uid}'. There are no migrations; schema constraints are created idempotently at startup.
Consistency
Each user has one checkpointed supervisor thread, so a multi-turn conversation survives restarts and is shared across workers. Sync Neo4j calls run in a thread pool so they do not block the event loop. Nothing is idempotent: place_order creates a new Order node every time it runs, the rate limiter's INCR and EXPIRE are two separate calls rather than one atomic operation, and order persistence is best-effort with no transaction across the provider call and the Neo4j write.
Failure
Graphiti reads and writes catch every error and return an empty bundle or None, so a memory failure never breaks a reply. A failed provider order is recorded with status 'failed'. Audio downloads are streamed with a size cap and a total timeout. Startup fails fast if Neo4j or Redis does not answer a ping.
Testing
16 pytest files, 156 test functions (count of def test_/async def test_), 2,217 lines. They cover the allergy filter, LLM gateway, Graphiti contract, retriever and entity types, Foursquare normalisation, the rate limiter, supervisor nodes and the users route. CI runs uv run pytest -q before the SSH deploy job.

Techniques

  1. Deterministic allergy post-filter after the LLMThe LLM's recommendations go through a whole-word regex built from a reverse synonym map before the user sees them. 'Dairy', 'milk' and 'cheese' all expand to the same equivalence class. Matches are rejected with a recorded reason, so safety never depends on the model following instructions.src/agents/food_agent/allergy_filter.py:141 src/agents/food_agent/allergy_filter.py:153 src/agents/food_agent/allergy_filter.py:209 src/agents/food_agent/nodes.py:302
  2. Supervisor graph with inlined specialist sub-graphsEach specialist exposes compose_workflow(set_entry=False, leaf_target='finish'). The supervisor mounts their nodes onto one StateGraph behind an onboarding gate and an intent classifier, then compiles it with one shared AsyncRedisSaver. A whole conversation is one checkpointed thread per user.src/agents/supervisor/graph.py:75 src/agents/supervisor/graph.py:96 src/agents/supervisor/graph.py:110 src/agents/food_agent/graph.py:22 src/app/api/routes/agent_ws.py:138
  3. Typed, user-scoped Graphiti memory contractAgents never call Graphiti directly. Writes go through one contract module that stamps group_id=user_id, reference_time and the typed entity schema, and swallows failures. Reads return a ContextBundle that sorts facts into kinds (allergies, cuisines and so on) and renders itself into a prompt.src/app/services/graphiti/contract.py:49 src/app/services/graphiti/contract.py:67 src/app/services/graphiti/retriever.py:126 src/app/services/graphiti/retriever.py:187
  4. Role-based LLM gatewayEvery model call asks for a role, not a vendor. The role resolves from env to a provider:model spec across 5 providers, with provider SDKs imported lazily inside each branch. Swapping the food agent's model is a config change, and settings validate the profiles at boot.src/app/services/llm_gateway.py:43 src/app/services/llm_gateway.py:59 src/app/core/config.py:234
  5. Environment-aware fail-closed security dependenciesThe JWT blacklist check and the Redis rate limiter fail closed in production when Redis is unreachable: the token is treated as revoked, or the request gets a 503. In development they fail open. The limiter keys on the authenticated uid when it has one and on the X-Forwarded-For IP otherwise.src/app/core/security.py:53 src/app/core/security.py:77 src/app/api/middleware/rate_limit.py:70 src/app/api/middleware/rate_limit.py:114
  6. Bounded streaming audio downloadAudio arriving by URL is pulled in 64 KB chunks under an aiohttp total timeout and aborted once it passes MAX_AUDIO_FILE_SIZE_MB, so a hostile URL cannot exhaust memory before transcription.src/app/api/routes/agent_ws.py:85

Counted

  • 34Commits
    git rev-list --count HEAD
  • 6Commits by Milan (pathfindermilan)
    git shortlog -sne HEAD
  • 9,789Python LOC (src)
    git ls-files 'src/*.py' | xargs wc -l | tail -1
  • 2,217Python LOC (tests)
    git ls-files 'tests/*.py' | xargs wc -l | tail -1
  • 22 (20 HTTP, 2 WebSocket)Route decorators (HTTP + WS)
    grep @router.(get|post|...|websocket) src/app/api/routes
  • 11 (voice 4, social 4, travel 3)Stubbed routes (body is `pass`)
    grep -c 'pass$' src/app/api/routes/{voice,social,travel}.py
  • 16 / 156Test files / test functions
    ls tests/test_*.py; grep -c 'def test_' tests/*.py
  • 6 / 9Neo4j node / relationship classes
    grep '^class ' src/app/models/*.py (StructuredNode / StructuredRel)
  • 6Graphiti entity types
    src/app/services/graphiti/entity_types.py
  • 5LLM providers in gateway
    src/app/services/llm_gateway.py
  • 8Food agent graph nodes
    src/agents/food_agent/graph.py