Eyes through sound — and a university in your ear.
An AI companion for blind and visually-impaired people that describes the world as it happens, guards their safety, gives them back their education — and knows when to stay silent. One screen, three giant buttons, and an agent that speaks only when it should.
Ask what blindness took, and the answer is never just 'sight'
Hundreds of millions live with uncorrectable vision loss. The disability is not blindness — the disability is friction and isolation, and both are information problems software can solve.
I stopped studying.
Because reading a page required another person.
I stopped going out.
Because the world became an obstacle course.
I stopped being part of things.
Because a picnic, a graduation, a smile became events attended beside, never inside.
The founding conviction: a blind person is not intellectually disabled — they are deprived of access to knowledge. Remove the barrier and you unlock thinkers, philosophers, scientists and creators the world would otherwise never receive.
The agent is not chatty. A blind user's hearing is prime real estate; an agent that narrates constantly is a nuisance you cannot blink away. Hence Smart Silence.
The person is a student, not a patient. Education is a first-class feature, equal in priority to safety.
Smart Silence — say less, say it only when it counts
Most AI assistants talk too much. For a blind user that is disqualifying. Speech is governed by an explicit, exhaustively tested policy:
| Level | The agent speaks… |
|---|---|
| silent | Only for danger — nothing else, ever. |
| on_demand | Only when the user asks (button / voice command) — and pressing Describe is an explicit request, so it answers every time. Locked by a regression test born when an early build got it wrong. |
| balanced (default) | Danger + warnings + answers to requests. |
| chatty | Everything — off by default. |
One rule above all: danger pierces every level. And honesty cuts both ways — in on_demand mode, an explicit Describe request is always answered. Silence, by default, is respect.
VoiceHome — the blind user's entire interface
Three buttons fill the screen — capture or select an image and hear the scene; replay the last response; alert the guardian instantly. Keyboard shortcuts D / R / E mirror every button.
Screen-reader-first — full ARIA labels, results announced through an aria-live region, logical focus order, and contrast exceeding WCAG-AA.
Instant EN ⇄ AR — one toggle flips the interface and RTL direction — with a spoken confirmation, because for blind users the language choice is heard, not just displayed.
A guardian dashboard 400 km away — safety events, emergency history, learning progress, memory facts, current silence level: the family sees she is fine — without calling five times a day.
Not features — people
The blind mother who tutors her children
Amina lost her sight at 30; private tutoring was unaffordable and her three children were quietly falling behind. Today she uploads her son's science lesson; the agent reads it aloud chunk by chunk and quizzes her. Because she has studied it too, she drills her son over breakfast — and the dashboard shows his average climbing: 55% to 82%. One blind user — an entire household's education changed.
The student completing his degree
Omar studies history and philosophy — fully capable of the thinking; the barrier was only the reading. His curriculum goes into the library as text; the agent becomes his reader, never tired, never impatient. He listens, asks for a passage again, takes the generated quiz, tracks progress per subject. Blindness removed pages, not thoughts. The agent restores the pages — and the degree follows.
Morning coffee, independently
'Describe the scene.' — 'The kettle is on the counter ahead, the cup is to its left, the counter edge is one step further.' On-demand description turns an unfamiliar kitchen navigable in seconds. Then — silence. Not another word until asked. Silence, by default, is respect.
The walk home after dark
In balanced mode the agent stays quiet until it matters: a step down, a pole, a curb, a turning vehicle — each becomes a short spoken warning and a recorded safety event the family can review. One press of the red SOS button creates an emergency, notifies the guardian, and logs it. At night, the agent is a bodyguard that speaks only when it must.
It remembers you — and encourages
'Your daughter's name is Layla.' 'Your medicine is on the second shelf.' The top memory facts ride into every exchange — an agent that remembers your family's names is no longer a tool; it is a companion. After a weak quiz score the next session starts from where you struggled; after a strong one, it says so. Depression among visually-impaired people is several-fold higher — and the most evidence-backed countermeasures are connection, purpose and learning. This app delivers all three by design.
The graduation, described live
Her daughter crosses the stage — the gown, the dean's handshake, the family jumping in the third row. The agent narrates it live. Years later the moment is still in the conversation history: replayable, searchable. Participation is not a luxury feature; it is the difference between presence and absence.
The waterfall is about twenty meters ahead, wide, falling over dark rocks — spray drifting to your left. Your daughter is standing closest to it, laughing, her hands out in the mist. Your son is filming her, and your husband is setting the picnic mat under the tree behind you — he's smiling at her.The family picnic she can finally see — scenery, movement, facial expressions. The blind parent stops attending beside the family, and starts being inside the moment.
A university in your ear
The founding study called the agent 'a gateway to knowledge and life.' The learning module makes that literal — and treats education as equal in priority to safety, because isolation has two faces, and only one of them is physical.
Shipping todayLive
The blind mother's upload → listen → quiz → tutor cycle is fully supported.
- The audio library — any text becomes a reading resource, split into navigable ~800-word chunks with read-aloud.
- Lesson review — read, replay, and navigate forward/backward through chunks at will.
- Deterministic quizzes — fill-in-the-blank from key terms, four options each. Same text always yields the same quiz — so progress is measurable, not vibes.
- Progress tracking — quizzes taken, rolling average per resource; the student and their family watch competence rise.
Phase 2 — the full visionInterfaces ready
The provider abstraction is the seam — activation, not rewrite.
- Google Drive sync — connect once; curricula, PDFs and lecture notes flow into the library.
- PDF → audiobook — scanned and digital material read aloud, OCR included; summaries and simpler explanations on demand.
- Language learning — pronunciation-checked vocabulary drills — plus spoken internet search with sourced answers.
- Live providers — GPT-4V-class vision, Whisper STT, ElevenLabs TTS — an environment change away.
Where 'silence unless it matters' already saves lives
Underneath the accessibility application is a pattern: perceive an environment, decide whether the moment is worth interrupting someone for, and let exactly one condition — real danger — pierce through. Three researched fields run on it:
ICU alarm fatigue
The closest and most consequential match. ICUs generate a staggering alarm volume — one study found ~85% non-actionable — and the documented consequence is desensitization; the FDA's database records hundreds of alarm-related deaths from exactly this failure. The prescribed clinical fix is precisely Smart Silence's structure. One hospital cut weekly alarms from ~88,000 to under 10,000 without compromising safety.
Industrial & construction safety
Documented since the 1980s: proximity warnings fire so often that workers tune them out precisely when a real hazard appears. The same 'personalize the alert, reserve full intensity for genuine danger' solutions mirror Smart Silence's tiers directly — and the lone-worker 'no one noticed for forty minutes' failure mode is close kin to this project's no-response emergency trigger and guardian notification.
Audio learning for reading disabilities
The education module already has a proven adjacent field: text-to-speech and audiobook tools built for dyslexia and other reading disabilities, with published research showing meaningful comprehension gains — removing the decoding burden frees cognitive resources for actual understanding. The mechanism is the same one the founding study insisted on: remove the barrier, measure comprehension consistently, let progress become visible and motivating. This isn't a chatbot — it's a gateway to knowledge that happens to speak.
Adversarially audited — and honest about it
An independent review caught a critical flaw before release: agent endpoints verified a profile existed but not that the caller owned it — any authenticated user could have read any blind person's conversation history. Fixed with unified authorization (404 before 403), proven closed by live attack scenarios, locked forever by the TestDataIsolation regression suite.
Honest AI, offline by default
Vision, STT, TTS and LLM are interfaces with two implementations each: deterministic mocks and real providers. All 125 tests run with zero API keys and zero internet — swapping in GPT-4V, Whisper, ElevenLabs is an environment change, not a rewrite.
The silence gate is tested
The policy is code, not vibes: exhaustive tests pin each level's behavior — including the regression born the day an early build stayed silent after an explicit Describe press. Danger-pierces-every-level is asserted on every run.
Guardian injection, stripped
Registration is restricted to blind/guardian roles, and guardian_user_id injection from payloads is stripped server-side — a profile's guardian can only be set by the system that owns it.
Synthetic demo data, disclosed
The demo profile is entirely fictional: 8 memory facts, 3 learning resources, 3 safety events, 7 conversations, 1 resolved emergency. No real personal data is stored — the disclosure is part of the page, not a footnote.
125 tests, strict types
pytest 125 passed (silence policy, intent word-boundary matching, isolation) — with strict TypeScript at zero errors and a PWA service worker generated at build.
Spoken language, not displayed
English base, full Arabic RTL — and for blind users the language switch speaks its confirmation. Adding any language is a documented 3-step process; no code changes.
Every interaction, one deterministic path
Intent
Rule-based, EN + AR keywords
Context
5 memory facts + 5 turns
Vision
Scene → safety events
LLM
Language-aware reply
Priority
SOS ⇒ danger, else scene
Silence gate
should_speak()? danger passes
TTS + log
Spoken if OK, always logged
Where it stands, where it goes
Opening the door of knowledge may create thinkers, philosophers, scientists — and creators.
Meet the companion — three buttons, one philosophy: silence unless it matters.
The live demo is pre-populated with clearly-disclosed synthetic data — entirely fictional, no real personal data.