AI Interviewer

Problem

Technical hiring interviews are expensive and inconsistent. Senior engineers spend 5-10 hours per week interviewing candidates, time that doesn't scale. Different interviewers ask different questions, apply different standards, and introduce unconscious bias. Companies need a way to conduct structured first-round interviews at scale without sacrificing assessment quality.

Solution

A real-time AI interviewer that conducts video interviews with adaptive questioning. The system sees and hears the candidate via WebRTC (LiveKit), processes their responses in real-time, detects facial expressions for engagement signals, and dynamically adjusts question difficulty and follow-ups based on the conversation.

Key Technical Decisions

LiveKit over raw WebRTC — Building a reliable video call with echo cancellation, bandwidth adaptation, and reconnection handling from scratch would take months. LiveKit provides production-grade WebRTC infrastructure with React hooks. The trade-off is vendor dependency, but the time savings justified it — we shipped the video pipeline in 2 weeks instead of 3 months.

face-api.js for lightweight emotion detection — Full facial action coding (FACS) is overkill for interview engagement signals. face-api.js runs in the browser with negligible performance impact and detects basic emotional states (engaged, confused, nervous) that are useful for adjusting question pacing. We explicitly do NOT use this for candidate scoring — only for conversational flow.

Qdrant for question bank matching — The question bank has 500+ questions across skill domains. When the AI needs a follow-up question based on the candidate's response, vector similarity search finds the most relevant next question. This is better than a decision tree (which becomes brittle) or random selection (which loses coherence).

Impact

• Real-time multimodal AI — simultaneous video, audio, LLM, and face detection processing
• Adaptive questioning from a 500+ question bank via semantic vector matching
• Sub-second response latency via Pipecat audio pipeline with streaming TTS
• Langfuse observability for interview quality monitoring and LLM cost tracking
• Built for a healthcare venture studio to scale technical hiring