Speech not always detected

[Lloydy]

HI,

I’m using OpenEars in my app and occasionally it doesn’t detect speech, it seems to be particularly funny about the word NO. I have to shout quite loud next to the mic for it to detect.

Basically i am playing a video using AVFoundation then i start listening for a response. Then based on the response i play a new clip. COuld this be something to do with AVFoundation and AudioSessions

I’m running the latest distribution from the download link with iOS5

If you need some more information please let me know

Hope you can help

thank you

Lloyd

[Halle Winkler]

Can you turn on OpenEarsLogging and show the log? Is the same issue there if you use the sample app and make “NO” one of the dynamically-created words in the dynamic model without any of your video code?

[Lloydy]

HI Halle,

Thanks so much for the quick response, it seems ok in the sample app, which suggests i’m doing something wrong, i’m creating my language model online as i’m trying to keep the app as small as possible – could this be anything to do with it. Also below is the reading g i get before it dean;t pick up my voice

Thanks again

Lloyd

Audio route has changed for the following reason:
There has been a change of category
The previous audio route was SpeakerAndMicrophone
This is not a case in which OpenEars performs a route change voluntarily. At the close of this function, the audio route is SpeakerAndMicrophone
Audio route has changed for the following reason:
There has been a change of category
The previous audio route was Speaker
This is not a case in which OpenEars performs a route change voluntarily. At the close of this function, the audio route is SpeakerAndMicrophone
Pocketsphinx has resumed recognition.

[Halle Winkler]

It might not be an actual mistake, but possibly some kind of limit to how well OpenEars can override the audio session with respect to the timing of your video if it’s close. What is in your log excerpt isn’t anything bad, but it might be helpful to see the whole thing (minus your own app logging which I don’t need to see). It does sound like the video might be changing the audio session and your recognition loop has quiet input as a result or possibly a wrong sample rate or something similar.

Are you playing the video before starting the recognition loop or during it?

[Lloydy]

Here is the complete log

I’ll do my best to explain – basically i initialise OpenEars then suspend listening whilst i play the first video, when the video gets to the end i resume listening but i also play a silent video (Idle) on loop whilst i wait for the user to respond, once i get the response i load a new video. During the idle stage i use the following code to remove the audio from the video track – could this be affecting anything

NSArray *audioTracks = [self.idleAsset tracksWithMediaType:AVMediaTypeAudio];

// Mute all the audio tracks
NSMutableArray *allAudioParams = [NSMutableArray array];
for (AVAssetTrack *track in audioTracks) {
AVMutableAudioMixInputParameters *audioInputParams =[AVMutableAudioMixInputParameters audioMixInputParameters];
[audioInputParams setVolume:0.0 atTime:kCMTimeZero];
[audioInputParams setTrackID:[track trackID]];
[allAudioParams addObject:audioInputParams];
}
AVMutableAudioMix *audioZeroMix = [AVMutableAudioMix audioMix];
[audioZeroMix setInputParameters:allAudioParams];

The audio session has never been initialized so we will do that now.
Checking and resetting all audio session settings.
audioCategory is incorrect, we will change it.
audioCategory is now on the correct setting of kAudioSessionCategory_PlayAndRecord.
bluetoothInput is incorrect, we will change it.
bluetooth input is now on the correct setting of 1.
categoryDefaultToSpeaker is incorrect, we will change it.
CategoryDefaultToSpeaker is now on the correct setting of 1.
preferredBufferSize is incorrect, we will change it.
PreferredBufferSize is now on the correct setting of 0.128000.
preferredSampleRateCheck is incorrect, we will change it.
preferred hardware sample rate is now on the correct setting of 16000.000000.
AudioSessionManager startAudioSession has reached the end of the initialization.
Exiting startAudioSession.
Recognition loop has started
Starting openAudioDevice on the device.
Audio unit wrapper successfully created.
Set audio route to SpeakerAndMicrophone
Checking and resetting all audio session settings.
audioCategory is correct, we will leave it as it is.
bluetoothInput is correct, we will leave it as it is.
categoryDefaultToSpeaker is correct, we will leave it as it is.
preferredBufferSize is correct, we will leave it as it is.
preferredSampleRateCheck is correct, we will leave it as it is.
Setting the variables for the device and starting it.
Looping through ringbuffer sections and pre-allocating them.
Started audio output unit.
Calibration has started
Calibration has completed
Project has these words in its dictionary:
AHA
ARMS
BLUE
COURSE
DAY
DO

GREEN
HELLO
HELLO(2)
I
INDEED

NASTY
NICE
NICE(2)
NIGHT
NO
NOSE
OF
OK
RED
SILENCE
SORRY
STOP
SURE
THANK
THANKS
YEAH
YELLOW
YEP
YES
YOU

Listening.
Pocketsphinx calibration is complete.

[Halle Winkler]

OK, the issue is that the video player completely changes the audio session, so if you continuously play a video while PocketsphinxController is suspended, it guarantees that its built-in audio session reset behavior won’t work. I think the only option is to find a way to do what you need to do without always running a video.

[Lloydy]

OK thanks for your quick responses – really appreciate it.

Can i stop the video force the audio session and then start the video again?

[Halle Winkler]

I don’t really want to give advice here regarding forcing the audio session to reset because in 99% of cases the issue is not due to the audio session and folks will read the steps here and do stuff to the audio session directly and end up with messed-up apps that are very confusing for me to troubleshoot. That said, in this one case it might be worth your while to go investigate how the shared audio session manager is started by the internal classes and give it a go.

[Lloydy]

thanks

[Author]

Posts