Skip to content
Erfan.
Product

thru.io

A multilingual voice AI drive-through: a camera notices you pull up, an agent takes your order in your language, and the kitchen sees it live.

Devpost GitHub
Role
Team of four · GDG McMaster Mac-a-Thon 2026
Type
Hackathon · multilingual voice AI
Stack
  • React
  • Node.js
  • Express
  • ElevenLabs
  • Gemini
  • TensorFlow.js
  • Socket.IO
Highlight
Built at GDG McMaster Mac-a-Thon 2026 to take language barriers out of drive-through ordering.
Voice ordering kiosk: the order builds live as the customer speaks

The problem

Drive-through lines move fast, but language barriers slow everything down, and it's stressful for customers and staff alike. A misheard order costs time at the window and in the kitchen.

What I built

A drive-through that speaks your language: multilingual voice ordering that knows the menu and reads your order back, a kiosk that updates as you talk, and a kitchen display that receives every order in real time.

How it's built

  1. 01CameraCOCO-SSD
  2. 02Customerspeech
  3. 03Voice agentElevenLabs
  4. 04Order serverNode · Express
  5. 05KioskReact · live totals
  6. 06Kitchen displayKanban
Order path: detection → voice agent → order state → every connected display.
  • ElevenLabs Conversational AI Agent runs the voice pipeline, with Gemini handling the ordering logic.
  • Strict menu grounding and structured JSON output keep orders accurate and stop the model from inventing items.
  • TensorFlow.js person detection starts the conversation when a customer pulls up, with a manual fallback.
  • React + Vite + Tailwind frontend and a Node.js / Express backend, with Socket.IO broadcasting orders to every kitchen display.

Decisions

Ground the model in the menu

Keeping the voice agent and the ordering logic aligned took the most iteration; strict grounding made the JSON orders consistent and menu-accurate.

Zero-touch start

Person detection starts the conversation, so a customer never has to press anything to begin ordering.

Pivot when it's right

The team started in Python and Flutter and moved to Node.js and React mid-hackathon to wire vision, voice and real-time updates together.

What happened

  • A working end-to-end workflow: detection, multilingual voice ordering, order confirmation and a live kitchen display.
  • Next steps the team scoped: a Raspberry Pi kiosk, payments and pilot testing at real locations.
Next projectOrbitour