Skip to Content
DocsLoremind SdkVoice Input

Voice Input

Let players speak to NPCs instead of typing. Voice input is optional - LoreMindNPC works without it.

Prerequisites

Voice input requires:

  • whisper.unity package (Unity Asset Store or GitHub)
  • LOREMIND_WHISPER scripting define symbol

Installation

1. Install whisper.unity

Download and import the whisper.unity package.

2. Add Scripting Define

  1. Go to Edit > Project Settings > Player
  2. Expand Other Settings > Script Compilation
  3. Add LOREMIND_WHISPER to Scripting Define Symbols
  4. Click Apply

3. Enable in Project Settings

  1. Open Window > LoreMind > Control Panel
  2. Go to Voice tab
  3. Enable Speech-to-Text
  4. Set STT Provider to Whisper
  5. Set Language (e.g., “en” for English)

Quick Setup

Add the Component

  1. Create a GameObject (or use your Player)
  2. Add Component > LoreMind > Voice > Voice Input

The component auto-adds required dependencies:

  • MicrophoneInputHandler - Captures audio
  • SpeechCaptureController - Manages recording and transcription

Wire to NPC

Option A: Inspector

Set Target NPC to your LoreMindNPC component. Transcriptions automatically trigger NPC responses.

Option B: Code

voiceInput.OnTranscription.AddListener(text =>
{
    npc.Respond(text);
});

Capture Modes

Push-to-Talk (Default)

Player holds a key to record, releases to transcribe:

voiceInput.PushToTalk = true;
voiceInput.PushToTalkKey = KeyCode.V;

Manual Mode

Control capture from code:

voiceInput.PushToTalk = false;
 
// Start/stop manually
voiceInput.StartCapture();
voiceInput.StopCapture();
 
// Or capture for a specific duration (adjust based on expected speech length)
string text = await voiceInput.CaptureAndTranscribeAsync(maxDuration);

Complete Example

using UnityEngine;
using UnityEngine.UI;
using Peek.LoreMind;
using Peek.LoreMind.Voice;
 
public class VoiceNPCDemo : MonoBehaviour
{
    [SerializeField] private LoreMindNPC npc;
    [SerializeField] private LoreMindVoiceInput voiceInput;
 
    [Header("UI")]
    [SerializeField] private GameObject recordingIndicator;
    [SerializeField] private Text subtitleText;
 
    void Start()
    {
        // Configure
        voiceInput.PushToTalk = true;
        voiceInput.PushToTalkKey = KeyCode.V;
 
        // Wire up events
        voiceInput.OnCaptureStarted.AddListener(() =>
        {
            recordingIndicator.SetActive(true);
            subtitleText.text = "Listening...";
        });
 
        voiceInput.OnCaptureStopped.AddListener(() =>
        {
            recordingIndicator.SetActive(false);
            subtitleText.text = "Processing...";
        });
 
        voiceInput.OnTranscription.AddListener(text =>
        {
            subtitleText.text = $"You: {text}";
            npc.Respond(text);
        });
 
        voiceInput.OnError.AddListener(error =>
        {
            subtitleText.text = $"Error: {error}";
            recordingIndicator.SetActive(false);
        });
 
        // Show NPC response
        npc.OnResponseReceived.AddListener(response =>
        {
            subtitleText.text = $"NPC: {response}";
        });
    }
}

Configuration

Capture Settings

SettingDescriptionDefault
Max Recording SecondsAuto-stop after this duration10s
Min Recording SecondsDiscard shorter recordings0.3s

Auto-Response Settings

SettingDescriptionDefault
Target NPCNPC to send transcriptions toNone
Minimum Text LengthDiscard shorter transcriptions3 chars

Events

// Capture lifecycle
voiceInput.OnCaptureStarted.AddListener(() => ShowRecordingUI());
voiceInput.OnCaptureStopped.AddListener(() => HideRecordingUI());
 
// Results
voiceInput.OnTranscription.AddListener(text => HandlePlayerSpeech(text));
voiceInput.OnError.AddListener(error => ShowErrorMessage(error));

API Summary

Properties

bool IsCapturing { get; }
float CaptureDuration { get; }
bool IsTranscribing { get; }
string LastTranscription { get; }
 
bool PushToTalk { get; set; }
KeyCode PushToTalkKey { get; set; }
LoreMindNPC TargetNPC { get; set; }

Methods

void StartCapture()
void StopCapture()
Task<string> CaptureAndTranscribeAsync(float durationSeconds = 0f)
void CancelCapture()

Audio Providers

UnityMicrophone (Default)

Uses Unity’s built-in Microphone class. Works everywhere Unity supports microphone input.

Dissonance (Multiplayer)

For multiplayer games using Dissonance Voice Chat:

// Add DissonanceAudioProvider component
// Captures audio from Dissonance instead of the microphone

Custom Provider

Implement IAudioProvider for custom audio sources:

public class CustomAudioProvider : MonoBehaviour, IAudioProvider
{
    public event AudioCaptureEventHandler OnAudioCaptured;
    public bool IsCapturing { get; private set; }
 
    public void StartCapture(int sampleRate)
    {
        // Start capturing from your audio source
    }
 
    public void StopCapture()
    {
        // Stop and fire OnAudioCaptured with captured data
    }
}

Troubleshooting

No microphone detected

Check microphone permissions:

  • Windows: Settings > Privacy > Microphone
  • macOS: System Preferences > Security & Privacy > Microphone
  • Mobile: App must request microphone permission

Transcription returns empty

  • Verify recording wasn’t too short (below MinRecordingSeconds)
  • Check Whisper model loaded correctly
  • Speak louder or closer to microphone

”Speech-to-text not enabled”

Open Window > LoreMind > Control Panel > Voice and enable Speech-to-Text.

First transcription is slow

Whisper runs locally. First transcription loads the model. Subsequent transcriptions are faster.

Best Practices

Provide Visual Feedback

Always show recording and processing states:

voiceInput.OnCaptureStarted.AddListener(() => recordingIcon.SetActive(true));
voiceInput.OnCaptureStopped.AddListener(() =>
{
    recordingIcon.SetActive(false);
    processingIcon.SetActive(true);
});
voiceInput.OnTranscription.AddListener(text => processingIcon.SetActive(false));

Display Transcriptions

Show players what was heard:

voiceInput.OnTranscription.AddListener(text =>
{
    subtitleText.text = $"You: {text}";
});

Offer Both Voice and Text

// Voice input
voiceInput.OnTranscription.AddListener(SendToNPC);
 
// Text input
textInput.onEndEdit.AddListener(SendToNPC);
 
void SendToNPC(string playerInput)
{
    if (!string.IsNullOrEmpty(playerInput))
        npc.Respond(playerInput);
}

Next Steps

Last updated on
LoreMindNPC ComponentTinySave