[Resolved] Keep system sounds while listening

Here is the full URL:
https://developer.apple.com/library/ios/codinghowtos/AudioAndVideo/_index.html#//apple_ref/doc/uid/TP40007426-CH1-DontLinkElementID_13

I can already play sound from my app while listening. What I want the built in system sounds.

Yes, all my system sounds and alerts (including vibration) are muted while pocketsphinx is listening. Did I do something to make this happen in my project or is this by OpenEars design? I will look into that link and let you know if it makes any difference.

Halle,

I actually already tried that fix yesterday. It allows for simultaneous playing of sound from within the app while recording, but system sounds are still muted. I can play my own beep fine but all system beeps/vibrations are muted. I would really like email notifications and text alerts to still beep/vibrate the phone while pocketsphinx is listening. Any other ideas?

Thanks for your quick response and help!
Brock

OK, that’s interesting — I will try to check this out tomorrow and get back to you about it. I think it would be generally preferable for this to be configurable but I don’t know offhand how involved that will be before taking a look at it.

Ok, upon more testing it looks like it isn’t actually the audio session. I can start the audio session and maintain system sounds. It is when I call pocketsphinx to start listening that all system sounds are muted. Then as soon as I stop pocketsphinx they start back up again. All without interrupting the audio session. Hope this helps!

Brock

It is when I call pocketsphinx to start listening that all system sounds are muted. Then as soon as I stop pocketsphinx they start back up again.

How strange that AVAudioPlayer plays back while PocketsphinxController is running but the system sounds don’t. I’ll be interested to see what’s up with this when I get a moment. Have you tried this with the sample app to confirm that it isn’t the result of a change you’ve made?

Yes, all testing I am doing is on your default sample app.

Brock

Hmm…. interesting. As I have tried to include this code I find I cannot successfully enable output on the audio unit, while I can disable it:

(Trying to enable)
AudioUnitElement outputBus = 1;
OSStatus setEnableIOStatusOutput = AudioUnitSetProperty (
audioDriver->audioUnit,
kAudioOutputUnitProperty_EnableIO,
kAudioUnitScope_Output,
outputBus,
&enableIO,
sizeof(enableIO)
);
if(setEnableIOStatusOutput != noErr) {
OpenEarsLog(@”Couldn’t enable IO output: %d”,setEnableIOStatusOutput);
audioDriver->unitIsRunning = 0;
return NULL;
}
else
OpenEarsLog(@”Enabled IO Output successfully: %d”,setEnableIOStatusOutput);

2011-10-25 11:42:37.797 OpenEarsSampleProject[1219:1607] OPENEARSLOGGING: Starting openAudioDevice on the device.
2011-10-25 11:42:37.798 OpenEarsSampleProject[1219:1607] OPENEARSLOGGING: Audio unit wrapper successfully created.
2011-10-25 11:42:37.802 OpenEarsSampleProject[1219:1607] OPENEARSLOGGING: Successfully got new audio unit component instance: 0
2011-10-25 11:42:37.803 OpenEarsSampleProject[1219:1607] OPENEARSLOGGING: Success 0: set maximum frames property.
2011-10-25 11:42:37.804 OpenEarsSampleProject[1219:1607] OPENEARSLOGGING: Enabled IO Input successfully: 0
2011-10-25 11:42:37.805 OpenEarsSampleProject[1219:1607] OPENEARSLOGGING: Couldn’t enable IO output: -10877
2011-10-25 11:42:37.806 OpenEarsSampleProject[1219:1607] OPENEARSLOGGING: openAudioDevice failed

(Trying to disable)
UInt32 disableIO = 0;
AudioUnitElement outputBus = 0;
OSStatus setEnableIOStatusOutput = AudioUnitSetProperty (
audioDriver->audioUnit,
kAudioOutputUnitProperty_EnableIO,
kAudioUnitScope_Output,
outputBus,
&disableIO,
sizeof(disableIO)
);
if(setEnableIOStatusOutput != noErr) {
OpenEarsLog(@”Couldn’t disable IO output: %d”,setEnableIOStatusOutput);
audioDriver->unitIsRunning = 0;
return NULL;
}
else
OpenEarsLog(@”Disabled IO Output successfully: %d”,setEnableIOStatusOutput);

2011-10-25 11:47:32.099 OpenEarsSampleProject[1235:1607] OPENEARSLOGGING: Starting openAudioDevice on the device.
2011-10-25 11:47:32.103 OpenEarsSampleProject[1235:1607] OPENEARSLOGGING: Audio unit wrapper successfully created.
2011-10-25 11:47:32.106 OpenEarsSampleProject[1235:1607] OPENEARSLOGGING: Successfully got new audio unit component instance: 0
2011-10-25 11:47:32.109 OpenEarsSampleProject[1235:1607] OPENEARSLOGGING: Success 0: set maximum frames property.
2011-10-25 11:47:32.110 OpenEarsSampleProject[1235:1607] OPENEARSLOGGING: Enabled IO Input successfully: 0
2011-10-25 11:47:32.110 OpenEarsSampleProject[1235:1607] OPENEARSLOGGING: Disabled IO Output successfully: 0

So why can I not enable the audio output on the unit? Am I doing something wrong here?

Brock

Next is I’ll take a look at it when I get some time. I wouldn’t really have expected it to be related to the audio unit since it should only have to do with what goes in and out of the audio unit’s buffer callback, which is just speech going in for analysis. Other outgoing audio in the app doesn’t route through the audio unit which is why it has no output.

Can you revert the sample app to its original code and then try out this code:

UInt32 allowMixing = true;
AudioSessionSetProperty(kAudioSessionProperty_OverrideCategoryMixWithOthers, sizeof(allowMixing), &allowMixing);
AudioSessionSetProperty(kAudioSessionProperty_OtherMixableAudioShouldDuck, sizeof(allowMixing), &allowMixing);

after the audio session chunk as follows:

		UInt32 audioCategory = kAudioSessionCategory_PlayAndRecord; // Set the Audio Session category to kAudioSessionCategory_PlayAndRecord.
		OSStatus audioCategoryStatus = AudioSessionSetProperty(kAudioSessionProperty_AudioCategory, sizeof(audioCategory), &audioCategory);
		if (audioCategoryStatus != 0) {
			OpenEarsLog(@"Error %d: Unable to set audio category.", (int)audioCategoryStatus);
		}

• This reply was modified 206 days ago by  Halle.

This is confirmed on an original copy of your sample app with no changes but the addition you just suggested to make.

OK, I’ve set up a test and done some research on the results and this seems to be a bug (or an undocumented but expected behavior) of the kAudioSessionCategory_PlayAndRecord:

https://devforums.apple.com/message/181931
http://webcache.googleusercontent.com/search?q=cache:qbwbNiXXabQJ:lists.apple.com/archives/coreaudio-api/2011/Jul/msg00055.html+AudioServicesPlaySystemSound+%22audio+unit%22&cd=5&hl=en&ct=clnk
http://stackoverflow.com/questions/3589294/iphone-does-not-vibrate-while-recording
http://stackoverflow.com/questions/1183366/play-alert-sound-vibrate-while-audioqueue-is-recording

(nearly all of these links refer to AudioQueueServices rather than Audio Unit since a lot more people use queues, but I believe it’s the same underlying issue).

I attempted to replicate the behavior by seeing if AudioServicesPlayAlertSound() could be shown to play a sound when Pocketsphinx wasn’t listening and be unable to play a sound when it was, which I think correlates closely enough to the issue you’re reporting to serve as a good test. I was able to replicate the issue with this simulated system sound. Then I looked into reports of AudioServicesPlayAlertSound failing while using a recording category while the recording was started, and there were many reports. Sorry to be the bearer of bad news, really odd that they have AVAudioPlayer working differently in this case.

Well, I guess the question is how you ended up with the specification to pass through alert sounds. If it’s a generalized UX preference (fair enough, it would be better for it to work that way) I don’t have any advice, because as far as I know the only alerts you can listen for within the app are local and remote notifications, which means you don’t have the option of intercepting an alert and playing the sound it ought to make using an AVAudioPlayer. Maybe I’m incorrect about this (I haven’t ever thought about it particularly) and there is a way to listen for alerts or an indirect way of knowing when an alert comes in. It would probably make a good Stack Overflow search.

However, if there is a very particular app-related reason for wanting the alerts (e.g. the alerts and the app somehow have significance to each other), this is where I would question whether there is another way to accomplish your goal besides having the alert play (for instance, whatever the incoming event is that is important, maybe it could be a local or remote notification instead so that you can capture the event and play a sound).

You could also use one of your Apple support questions and ask them why you can’t hear system sounds when using kAudioSessionCategory_PlayAndRecord and recording, and see what they tell you.

Yes it is, it turns out that I have sorted this bug out. The order in which you set the AudioSessionSetProperty is significant. The order (if you want them all to play nice together) HAS TO BE THESE IN THIS ORDER:

-set category to playandrecord

-set property bluetooth

-set property overridecategorydefaulttospeaker

-set property to mixwithothers

If you don’t set these in this order in the AudioSessionManager.m file it won’t layer with bluetooth and music at the same time and either of these will interrupt music playing. I don’t know why Apple didn’t specify that order is significant when setting these flags but much trial and error has shown it does. For the code see: http://pastie.org/2859849. Let me know if you have any questions.

That is great Halle. Only make sure if you are copying and pasting this code folks, I only pasted the top of the method. When it gets down to currentPreferredBufferSize the method continues. Don’t paste this (and only this) into your AudioSessionManager.m file or you’ll kill your OpenEars. You will need the rest of the method also as it appears in AudioSessionManager.m, but since no changes were needed I didn’t see a need to paste it.

Hi,

I’m facing the same problem with 0.912. I added the following code in ContinuousAudioUnit.mm openAudioDevice method

OSStatus setEnableIOOutputStatus = AudioUnitSetProperty(audioDriver->audioUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Output, 0, &enableIO, sizeof(enableIO));
if(setEnableIOOutputStatus != noErr) {
OpenEarsLog(@”Couldn’t enable IO output bus: %d”,setEnableIOStatus);
audioDriver->unitIsRunning = 0;
return NULL;
}

and my AudioSessionManager.StartSession has the right order as mentioned in this thread. Am I missing something?

Thanks in advance.

Thanks Halle for your response. The problem I’m trying to fix is our iPhone app when connected to Car Bluetooth, it blocks the speaker output. If a music is playing, as soon as the app is launched (it startsAudioSession on appLaunch) the music no longer plays. I tried the same with other sounds on the device and noticed the same behavior with a Calendar event notification that the app is blocking the device sound. Appreciate your help.

Thanks

The change in AudioSessionManager that I added is only intended to fix the interaction of PocketsphinxController and media objects such as MPMovieController and AVPlayer while listening is not in progress, its purpose isn’t explicitly for keeping system sounds while listening. I don’t have any insight into this issue beyond what is discussed in this topic.