backend / aurasense
Aurasense
A FastAPI service where a LangGraph supervisor routes each voice or text turn to onboarding, food or profile agents, with per-user memory kept in a Graphiti knowledge graph on Neo4j.
2025–26 · paused · open source
- [*]LanguagesPython
- [*]FrameworksFastAPI · LangGraph · LangChain · graphiti-core · neomodel · Pydantic Settings
- [*]StoresNeo4j · Redis Stack (RediSearch + RedisJSON)
- [*]Asyncasyncio.to_thread offload for sync neomodel · LangGraph AsyncRedisSaver checkpoints · WebSocket streaming
- [*]InfraDocker · Docker Compose · DigitalOcean droplet · Nginx · GitHub Actions
- [*]Third partiesGroq (LLM + Whisper STT) · OpenAI · OpenRouter · Gemini (OpenAI-compatible) · Ollama · Foursquare Places v3
Aurasense started in July 2025 as a voice-first lifestyle companion. It had JWT auth, an onboarding agent and a Docker Compose stack of Neo4j, Redis and a separate Graphiti container. Milan (pathfindermilan) wrote 6 of the 34 commits, mostly onboarding-agent and WebSocket fixes in that first week. The rest is the owner's.
On 3 and 4 May 2026 the backend was rebuilt across six merged PRs, phase 0 to phase 4.5. Phase 0 made the defaults secure: production refuses to boot with the placeholder SECRET_KEY, and CI gates deploys on pytest. Later phases added an LLM gateway that every model call goes through, a BaseAgent with a Redis-backed LangGraph checkpointer, Graphiti running in-process as a typed RAG layer, a Profile agent that merges declared and observed facts, and a supervisor graph that sits behind one /ws/agent WebSocket.
The food agent runs as an eight-node graph: intent, context, health_screen, search, recommend, confirm_or_order, place_order, record. A regex allergy filter runs after the LLM recommends dishes, so the model is never the safety check. Restaurant data comes from a provider switch between a 12-dish mock catalogue and the Foursquare Places v3 API.
Measured in the code: 22 route decorators. 11 of them (all voice, social and travel routes) are still stubs whose bodies are pass. 16 test files define 156 test functions. Deploys go to a DigitalOcean droplet over SSH with docker compose up. The domain in the Nginx config, teepon.tech, does not resolve today.
How it holds up
- Auth
- Email and password with Argon2 hashing and PyJWT bearer tokens. Logout writes the token to a Redis blacklist with TTL equal to its remaining life. WebSockets authenticate with ?token=. The blacklist and the Redis rate limiters (login 5/min, register 3/hour) fail closed in production and fail open in development.
- Data
- Neo4j holds 6 neomodel StructuredNode types (User, Order, Restaurant, MenuItem, Hotel, Location) and 9 StructuredRel types. Graphiti adds 6 typed Pydantic entity types (Allergy, DietaryRestriction, HealthCondition, FoodPreference, CulturalContext, RestaurantVisit), every episode scoped with group_id = user_id. Conversation state lives in Redis as LangGraph checkpoints under thread_id 'supervisor:{uid}'. There are no migrations; schema constraints are created idempotently at startup.
- Consistency
- Each user has one checkpointed supervisor thread, so a multi-turn conversation survives restarts and is shared across workers. Sync Neo4j calls run in a thread pool so they do not block the event loop. Nothing is idempotent: place_order creates a new Order node every time it runs, the rate limiter's INCR and EXPIRE are two separate calls rather than one atomic operation, and order persistence is best-effort with no transaction across the provider call and the Neo4j write.
- Failure
- Graphiti reads and writes catch every error and return an empty bundle or None, so a memory failure never breaks a reply. A failed provider order is recorded with status 'failed'. Audio downloads are streamed with a size cap and a total timeout. Startup fails fast if Neo4j or Redis does not answer a ping.
- Testing
- 16 pytest files, 156 test functions (count of
def test_/async def test_), 2,217 lines. They cover the allergy filter, LLM gateway, Graphiti contract, retriever and entity types, Foursquare normalisation, the rate limiter, supervisor nodes and the users route. CI runsuv run pytest -qbefore the SSH deploy job.
Techniques
- Deterministic allergy post-filter after the LLMThe LLM's recommendations go through a whole-word regex built from a reverse synonym map before the user sees them. 'Dairy', 'milk' and 'cheese' all expand to the same equivalence class. Matches are rejected with a recorded reason, so safety never depends on the model following instructions.
src/agents/food_agent/allergy_filter.py:141src/agents/food_agent/allergy_filter.py:153src/agents/food_agent/allergy_filter.py:209src/agents/food_agent/nodes.py:302 - Supervisor graph with inlined specialist sub-graphsEach specialist exposes compose_workflow(set_entry=False, leaf_target='finish'). The supervisor mounts their nodes onto one StateGraph behind an onboarding gate and an intent classifier, then compiles it with one shared AsyncRedisSaver. A whole conversation is one checkpointed thread per user.
src/agents/supervisor/graph.py:75src/agents/supervisor/graph.py:96src/agents/supervisor/graph.py:110src/agents/food_agent/graph.py:22src/app/api/routes/agent_ws.py:138 - Typed, user-scoped Graphiti memory contractAgents never call Graphiti directly. Writes go through one contract module that stamps group_id=user_id, reference_time and the typed entity schema, and swallows failures. Reads return a ContextBundle that sorts facts into kinds (allergies, cuisines and so on) and renders itself into a prompt.
src/app/services/graphiti/contract.py:49src/app/services/graphiti/contract.py:67src/app/services/graphiti/retriever.py:126src/app/services/graphiti/retriever.py:187 - Role-based LLM gatewayEvery model call asks for a role, not a vendor. The role resolves from env to a provider:model spec across 5 providers, with provider SDKs imported lazily inside each branch. Swapping the food agent's model is a config change, and settings validate the profiles at boot.
src/app/services/llm_gateway.py:43src/app/services/llm_gateway.py:59src/app/core/config.py:234 - Environment-aware fail-closed security dependenciesThe JWT blacklist check and the Redis rate limiter fail closed in production when Redis is unreachable: the token is treated as revoked, or the request gets a 503. In development they fail open. The limiter keys on the authenticated uid when it has one and on the X-Forwarded-For IP otherwise.
src/app/core/security.py:53src/app/core/security.py:77src/app/api/middleware/rate_limit.py:70src/app/api/middleware/rate_limit.py:114 - Bounded streaming audio downloadAudio arriving by URL is pulled in 64 KB chunks under an aiohttp total timeout and aborted once it passes MAX_AUDIO_FILE_SIZE_MB, so a hostile URL cannot exhaust memory before transcription.
src/app/api/routes/agent_ws.py:85
Counted
- 34Commits
git rev-list --count HEAD - 6Commits by Milan (pathfindermilan)
git shortlog -sne HEAD - 9,789Python LOC (src)
git ls-files 'src/*.py' | xargs wc -l | tail -1 - 2,217Python LOC (tests)
git ls-files 'tests/*.py' | xargs wc -l | tail -1 - 22 (20 HTTP, 2 WebSocket)Route decorators (HTTP + WS)
grep @router.(get|post|...|websocket) src/app/api/routes - 11 (voice 4, social 4, travel 3)Stubbed routes (body is `pass`)
grep -c 'pass$' src/app/api/routes/{voice,social,travel}.py - 16 / 156Test files / test functions
ls tests/test_*.py; grep -c 'def test_' tests/*.py - 6 / 9Neo4j node / relationship classes
grep '^class ' src/app/models/*.py (StructuredNode / StructuredRel) - 6Graphiti entity types
src/app/services/graphiti/entity_types.py - 5LLM providers in gateway
src/app/services/llm_gateway.py - 8Food agent graph nodes
src/agents/food_agent/graph.py