OpenEars & Finch

[benwirz]

Hello,

I’m trying to use Finch with OpenEars. Finch is an OpenAl Sound Engine. I’m able to produce sounds correctly with Finch until I start the pocketsphinxControlle. After I start the controller, it stops producing any sound even if the pocketsphinxController is suspended.

I’m basically just trying to play a short two note sound sequence just prior to capturing a users vocal command similar to the way Siri works. Another tone is played after the capture completes. My users have indicated they are expecting an audible indication that voice capture has started and stopped. I’m not using NeatSpeach.

Any advice would be greatly appreciated.

Thanks,
Ben

https://github.com/zoul/Finch

[benwirz]

Also I should note that I repeated the test using a systemsounds instead of Finch and I get the same result. Starting pocketsphinxController seems to stop both.

-Ben

[Halle Winkler]

Hi Ben,

You can use AVAudioPlayer to play sounds without conflict with OpenEars’ audio session settings.

[benwirz]

Hi Halle,

Thanks for the reply. Everyone seems to recommend using OpenAL for short sound effects where the starting time latency is important and AVAudioPlayer for background music. I see this recommendation repeated time and time again online. Is it the case that it’s not possible to use OpenAL/Finch with OpenEars or you don’t feel that it is required for my application? If there is variable latency between the user pressing the start recording button and the phone playing the start tone, I’m worried it will create a bad user experience.

Also, if I start using the FliteController will it conflict with AVAudioPlayer?

Thanks,
Ben

[Halle Winkler]

Hi Ben,

The issue is that OpenEars requires the playandrecord audio session and an side-effect of this audio session is that it blocks background sounds and system sounds. I think there are a bunch of past threads about this (maybe with possible workarounds IIRC) that would be worth some searching in these forums. But I would never expect OpenAL or a wrapper around it to coexist well with OpenEars because OpenEars needs a great deal of control over audio for its own latency requirements and as a result of that it is nearly an audio framework on its own, with a constant input stream and its own output settings.

I have heard the advice to use OpenAL for game sound design with the expectation that there will be an entire library of tones that need to be close-to-zero latency (although I might personally do this with a remoteio queue instead), but I’m not sure most would necessarily recommend using an OpenAL audio engine simultaneous to another very audio-intensive framework for the purpose of playing back two sound files.

The common approach for lowest latency is remoteio or audio queue services (not available in this case because the input stream is being entirely used by PocketsphinxController), while AVAudioPlayer is preferred for simplicity, and system sounds are good for one-offs (but in this case they are interfered with by playandrecord).

I think you should be fine with a dedicated AVAudioPlayer that you can keep primed for lower-latency playback like so:

[myAVAudioPlayer prepareToPlay];
[myAVAudioPlayer play];
[myAVAudioPlayer stop];

And just have a different one instantiated and available for each of your two tones. I’d recommend making your tones 16k uncompressed mono WAVs.

Also, if I start using the FliteController will it conflict with AVAudioPlayer?

Nope, not if you aren’t trying to play them simultaneously.

[benwirz]

Thanks for the thoughtful reply Halle. You do an excellent job supporting your customers. One suggestion I would offer is that you should add this info to the FAQ. It would have saved me 2-3 days of time to know that OpenAL and System Sounds are incompatible with OpenEars. I’m sure this info is buried in the forums somewhere but I didn’t find during the search I did prior to starting. I’ll give AVAudioPlayer a try. Thanks again. -Ben

[Halle Winkler]

Hi Ben,

Thank you! I think you’re correct that it isn’t self-evident and would be a good addition to the FAQ. I will take a fresh look at the FAQ after the next round of updates is released.

[Halle Winkler]

I’ve added this early in the FAQ. Thank you for the suggestion.

[darasan]

Hi,

Just a follow up to this, as I had similar issues that I managed to work around and thought the solution might be helpful to others.

My app uses Open Ears for speech recognition and I also have a requirement to play very short, low-latency sounds for alerts, button click noises etc.

If I use my own instance of AVAudioPlayer to play these sounds, the latency is still unacceptably high (~ 500ms), even using the [myAVAudioPlayer prepareToPlay] method described by Halle above.

My solution was to use the [AudioServicesPlaySystemSound ] API to play these short sounds. It works with very low latency. That method is described here: http://stackoverflow.com/questions/900461/slow-start-for-avaudioplayer-the-first-time-a-sound-is-played?rq=1

If Open Ears has started listening however, the sound produced by [AudioServicesPlaySystemSound ] will not be audible. After reading around the forums, I set [OEPocketsphinxController sharedInstance].audioMode to @”VoiceChat”, which makes the sound audible, although at a lower volume. Not quite sure why that is, maybe related to http://www.politepix.com/forums/topic/openearspocketsphinx-turns-audio-player-volume-down-and-wont-let-go/ ? Halle?

Anyway if you need very low-latency playback of short sounds in a project using OpenEars, this is what currently works for me. Hope it helps someone!

D

[Halle Winkler]

Thank you for sharing! The lower volume is because VoiceChat uses a different audio unit which has the characteristic of playing system sounds, but reducing their volume by an undocumented extent (i.e. the volume drop changes from OS version to OS version) because it also does some amount of echo cancellation.

I don’t use it as the default because its behavior isn’t contract behavior (it isn’t described in the docs so there is no absolute promise about what it is supposed to do) and my experience over the years is that Apple does less testing of it than the Remote IO unit, meaning that occasionally an OS version will come out that breaks it briefly. However, I think it’s OK for individual projects to use as long as you test it occasionally to make sure big OS updates don’t affect it.

[Author]

Posts