Two views: how it runs today, and where it's headed (fully backed by your servers). The colors tell you where each thing lives.
Almost everything happens on the phone. The phone holds the real database, and it reaches out directly to outside AI services to do the heavy lifting. You have almost no backend yet.
The Expo / React-Native app. This is where you record, and it's where your data actually lives.
The phone calls these directly. They process and reply — they don't store your data.
Supabase — and it does almost nothing yet.
pilot_extraction table gets a copy of each extraction + transcript, so you can review extraction quality. Not your main data store.Review onlyThis is one level below the “Microphone capture” box above — the part you’re debugging. Every recording is a row in the phone’s SQLite recordings table that walks a fixed ladder of statuses. The audio bugs all live at a specific rung.
src/db/recordings.tsThe first six are NON_TERMINAL — anything not complete or failed is treated as unfinished, and is a candidate for automatic recovery. That recovery machinery is what makes a recording survive an app kill — and it’s also where the subtlest bug lives.
Format. Capture is 16 kHz mono AAC (.m4a) via expo-audio (src/lib/audio-recorder.ts). The live mic level (dBFS) is read from getStatus().metering and drives the Record Console’s visualizer.
Interruptions pause, they don’t stop. A phone call, Siri, a Bluetooth route change, or another app grabbing the mic pauses the recorder mid-session. src/hooks/use-session-recorder.ts detects this, flags the session as having a gap, keeps the visualizer honest (it must not look like it’s recording while paused), and auto-resumes once the interruption clears.
Stop always finalizes — even when paused. A paused recorder still holds a half-written .m4a with no moov atom — an unplayable file. expo-audio’s stop() accepts the paused state and writes the moov, producing a playable file. Finalizing is therefore not gated on “is it currently recording”; skipping it on the paused case is what used to strand audio and hang transcription downstream.
src/lib/recording-pipeline.tsrunNativeRecordingPipeline runs the work as resumable stages, each keeping its input until the result is durably saved — so a kill or a flaky network never dead-ends a recording; the next run resumes from the last completed stage. The chain: VAD trims silence to speech segments → those upload to WhisperX on Replicate → the transcript returns and the audio is deleted → the transcript goes to Claude for extraction → map-placement → dedup → journal clustering → observations are written and the row goes complete. Per-stage progress + timing is persisted (S-D-46) so the Log card shows a live stage and a running timer.
Every time the app foregrounds (AppState → 'active'), reconcileAudioRecordings scans for unfinished (NON_TERMINAL) rows and calls decideResumeAction to resume or fail each. It’s fenced by an in-memory in-flight set and a durable attempts cap so it can’t loop forever.
'recording' is itself in NON_TERMINAL — so a recording you are currently making also looks like a “recovery candidate.” The in-flight guard is only populated when you press Stop; lock the phone and unlock it mid-recording and the foreground reconcile can run first, see a live recording row that looks orphaned, and fail it. It’s a race against your own Stop. DEF-125recording — interruptions.Pause not resumed, finalize skipped on a paused recorder (no-moov file), or the visualizer falsely showing “recording” while paused. DEF-119 · DEF-120 · DEF-121recording — reaped by recovery.The live-capture trap above: the foreground reconcile fails an in-progress capture. DEF-125transcribing — a silent hang.A transcription network call that never settles strands the row in transcribing with no in-session retry; only a cold app launch rescues it. DEF-118extracting — a stalled journal pass.The journal-clustering call cancels or idle-times-out (the network-stall family) → no Knowledge / Species / Skills written. DEF-110 · DEF-115Live status of each defect lives in the bug register (Product → bugs) — deliberately not pinned here, so this page doesn’t drift stale.
The heavy lifting and the source-of-truth move off the phone and onto your servers. The phone becomes a thin client. Keys leave the phone. Data becomes multi-user and centrally stored.
Captures and displays. No longer the brain.
Same services — but now reached through your backend, never directly from the phone.
This is where the work and the data now live.
Today: nearly everything is on the phone — the app, the recording, the GPS log, and the entire real database. The cloud only holds your login + a quality-review copy. Future: the phone becomes a thin client; the master database moves to your servers.
Only the transcript text of what was said. Not your GPS/location, not the audio, not who you are, not any past session. Each extraction is a fresh, anonymous text-in / JSON-out call. Your location gets attached afterward, on the phone.
To Whisper (on Replicate) to be turned into text, then it's deleted from the phone. It is not stored anywhere long-term today. (Future: optionally retained in your own storage bucket.)
Today: both keys are baked into the installed app — extractable, a known security debt. Future: keys live only on your backend; the phone never holds them.
Today: on the phone (SQLite). Single device, single user — no cloud copy of the real data. Future: a multi-user cloud database, one secured slice per account, mirrored to each phone.
Today: you do — by stopping a recording or importing files. Nothing is automatic or scheduled. Future: same trigger, but the work runs on your backend instead of the phone.