After the student is allocated to the new Faculty, the Wayfinder Academy provides a hyper care by allowing students to verbally communicate with the voice digital twin of their mentor.

From the technology part, we are using:
1. LiveKit – to handle real-time audio communication. Students join room via the LiveKit SDK embedded in the Next.js frontend.

2. ChatGPT Voice Assistant – to process voice inputs using a pipeline:
– ASR (speech-to-text)

• LLM (GPT-4) to generate intelligent responses
• TTS text-to-speech
• STT to process audio streams and convert them to text

3.Next.js application – serves as the frontend:

• SSR ensures fast loading and API integration
• connects students to LiveKit rooms and the assistant, displaying responses or playing them as audio

Here are more details on the backend part:

• Transcription Handler:

an entrypoint function to connect to LiveKit Room, track subscriptions and initialise the assistant: