Integrating a New AI Feature

Relevant source files

The following files were used as context for generating this wiki page:

• services/geminiService.ts
• components/AiDemo.tsx
• components/Chatbot.tsx
• types.ts
• App.tsx
• vite.config.ts
• metadata.json

This document outlines the architecture and implementation for integrating AI features within the AI for Thai EMS project. The primary goal of the AI integration is to enhance Emergency Medical Service (EMS) processes by providing tools for real-time analysis of emergency calls and an interactive AI assistant. This is achieved by leveraging the Google Gemini API for natural language understanding, structured data extraction, and conversational AI.

The integration is encapsulated in two main features visible in the application: the "AI in Action: Emergency Dispatch" simulation, which demonstrates structured data extraction from a call transcript, and a floating "EMS AI Assistant" chatbot. Both features rely on a central service, geminiService.ts, which acts as a bridge between the frontend components and the Google GenAI backend.

Architecture Overview

The system's architecture is client-centric, with React components directly communicating with the Google Gemini API via a dedicated service module. The Vite build tool is used to manage environment variables, such as the API key.

The following diagram illustrates the high-level component and data flow for the AI features.

This diagram shows that App.tsx renders the main AI components, AiDemo.tsx and Chatbot.tsx. Both components use geminiService.ts to make API calls to the Google GenAI API.

Sources: App.tsx:5,10,60,131, components/AiDemo.tsx, components/Chatbot.tsx

Environment Configuration

The Gemini API key is managed as an environment variable GEMINI_API_KEY. The Vite configuration file makes this variable available to the application frontend under process.env.API_KEY.

Sources: vite.config.ts:9-12, README.md:12

Core AI Service (geminiService.ts)

This service is the central point of interaction with the Google Gemini API. It abstracts the API calls into two main functions.

Sources: services/geminiService.ts

Function	Description
analyzeEmergencyCallWithGemini	Analyzes an emergency call transcript and returns structured JSON data based on a predefined schema.
sendMessageToChatStream	Manages a conversational chat session, sending user messages and history to the Gemini API and returning a streaming response.

Transcript Analysis: analyzeEmergencyCallWithGemini

This function is designed to process a raw text transcript of an emergency call and extract key information in a structured format. It uses the gemini-2.5-flash model with a specific JSON schema to ensure the output is consistent and machine-readable.

The function constructs a prompt that includes the transcript and instructions to return data in the specified JSON format.

Data Flow:

1. Receives a transcript string.
2. Constructs a prompt containing the transcript.
3. Calls the ai.models.generateContent method with the model, prompt, and a configuration specifying responseMimeType: "application/json" and the responseSchema.
4. Parses the JSON response text.
5. Returns the parsed object, typed as AiAnalysisResult.

Sources: services/geminiService.ts:9-55

Response Schema

The analyzeEmergencyCallWithGemini function enforces a strict response schema to structure the AI's output.

This diagram shows the structure of the AiAnalysisResult object, which includes incident details and nested ExtractedData about the patient.

Sources: services/geminiService.ts:11-30, types.ts:12-21

Chat Stream: sendMessageToChatStream

This function powers the Chatbot component. It supports streaming responses, model selection, and the use of external tools like Google Search and Maps.

Key Features:

• Model Selection: It dynamically selects the AI model based on the isThinkingMode flag. If true, it uses the more powerful gemini-2.5-pro; otherwise, it defaults to gemini-2.5-flash.
• Tool Integration: It can enable googleSearch and googleMaps tools based on boolean flags (useSearch, useMaps), allowing the AI to fetch external information.
• Streaming: It uses chat.sendMessageStream to return an asynchronous iterator, allowing the UI to display the response as it's being generated.

Sources: services/geminiService.ts:70-93

Frontend Integration

The AI features are integrated into the frontend through two primary React components.

AI Dispatch Simulation (AiDemo.tsx)

This component simulates an emergency call and visualizes the AI's real-time analysis. It manages the simulation state (idle, running, finished) and progressively reveals a mock transcript.

When a line from the "Caller" is added to the transcript, the runAnalysis function is triggered. This function calls analyzeEmergencyCallWithGemini and updates the component's state with the structured analysis results, which are then displayed in the "AI Analysis Dashboard".

The following sequence diagram illustrates the analysis process.

AI Assistant Chatbot (Chatbot.tsx)

This component provides a floating chat interface for users to interact with an AI assistant. It manages the conversation history and UI state (e.g., isLoading, isRecording).

Text-based Chat

When a user sends a message, the handleSendMessage function is called. It constructs the chat history from the component's state and passes it along with the new message to sendMessageToChatStream. It then iterates through the returned stream to update the currently displayed bot message in real-time.

Audio Transcription

The chatbot also supports voice input. It uses the browser's MediaRecorder API to capture audio. When recording stops, it connects to the Gemini API's live session endpoint (ai.live.connect) for real-time transcription.

1. startRecording is called, requesting microphone access.
2. MediaRecorder captures audio into chunks.
3. stopRecording is called, which finalizes the audio into a Blob.
4. A live session is established with the gemini-2.5-flash-native-audio-preview-09-2025 model.
5. The audio data is sent to the session.
6. The onmessage callback receives transcription text and updates the input field in the UI.

Sources: components/Chatbot.tsx:106-173, metadata.json:4-6

Data Structures

The primary data structure for the AI analysis feature is AiAnalysisResult.

Sources: types.ts:12-21

Field	Type	Description
incidentType	string | null	The type of incident (e.g., "Cardiac Emergency").
location	string | null	The full address of the incident.
severity	'Critical' | 'Urgent' | 'Non-Urgent' | null	The assessed severity level of the incident.
recommendedAction	string | null	The suggested next step (e.g., "Dispatch ALS Ambulance").
extractedData	object	An object containing specific details about the patient.
extractedData.name	string	The patient's name.
extractedData.age	string	The patient's age.
extractedData.condition	string	A summary of the patient's condition.

Conclusion

The integration of AI features in the AI for Thai EMS project demonstrates a practical application of large language models to solve real-world problems in emergency services. By using Google Gemini, the application provides both a powerful tool for structured data extraction from unstructured conversations (AiDemo) and a flexible, interactive assistant (Chatbot). The architecture is modular, with a clear separation between the UI components and the AI service layer, allowing for easier maintenance and future expansion of AI capabilities.