Ember — Multimodal AI Concierge

An intelligent, voice-enabled AI mascot and interactive virtual concierge engineered with Next.js 16, Web Speech API, Neural TTS, and live Headless CMS RAG synchronization for highland tourism.

Overview & Architecture

Ember is a multimodal AI concierge and virtual tour guide built for Dragon Treasure Castle in Baguio City. Featuring real-time voice-to-text recognition, natural speech synthesis, live Baguio atmospheric/seismic radar ingestion, and on-demand Headless CMS knowledge retrieval, Ember provides instant answers on tickets (₱165), commute directions, and rules—while narrating a dynamic 7-step guided audio walkthrough rendered in an interactive 3D claymorphic UI.

1. Multimodal Hands-Free Interaction (Voice & TTS)

• Real-Time Speech Recognition: Native Web Speech API integration with dynamic volume visualizers and automated silence detection.

• Neural Voice Synthesis: Natural vocal delivery using high-fidelity edge neural speech synthesis (Edge Neural TTS).

• Web Audio API Acoustics: Client-side synthesized UI sound effects delivering instant 0ms acoustic tactile feedback without external media asset dependencies.

2. Real-Time Ground-Truth RAG Pipeline

• Zero-Stale CMS Synchronization: Real-time GROQ queries fetch up-to-the-minute ticket pricing (₱165 regular, ₱132 Senior/PWD/Toddler), parking rates, and park guidelines directly from Sanity Headless CMS.

• Atmospheric & Weather Intelligence: Ingests live Baguio mountain rainfall radar and cloud inversion probability to automatically recommend attire or alert users to prime sea-of-clouds photo windows.

• Seismic Safety Monitoring: Connected to live USGS geological telemetry within 600km of Northern Luzon to verify structural safety protocols in real time.

• Strict Anti-Hallucination Grounding: Exact boundary matchers ensure zero factual discrepancies regarding ticketing policies, routes, and operational hours.

3. 7-Step Interactive Guided Audio Walkthrough

Guides visitors through registration, shuttle pickup, castle gates, scenic viewing decks, and exits with Play/Pause controls, step navigation, and voice toggle synchronization.

4. 3D Claymorphic Mascot & Interactive Portal

• High-Contrast Obsidian Portal: Circular badge featuring an obsidian black interior with a soft radial white spotlight halo for maximum 3D contrast.

• Transparent Alpha Video: High-resolution transparent alpha-channel video rendering with hover state transitions (sleeping with Zzz → waking up and waving hello).

• Reactive Lighting Halo: Dynamic glow that pulses blue when Ember is speaking, rose when listening, and warm amber when idle.

• Floating Speech Teaser Balloon: Floating badge on mobile and desktop that auto-cycles through contextual teasers (tickets, tours, weather, commute) with an ambient radar ping.

5. Performance & Battery Optimization

• Eco-Sleep Mode: Pauses transparent video rendering loops and audio decoders during 30s of user inactivity or tab changes to conserve mobile battery.

• 100% Serverless Architecture: Built with client-side caching, ISR, and free cloud edge endpoints, delivering $0.00 ongoing operational costs.