iOS 7 Compatibility

[nvrtd frst]

I noticed that in iOS 7 they deprecated the AudioSession API for the AudioToolbox framework

“The AudioSession API in the Audio Toolbox framework is deprecated. Apps should use the AVAudioSession class in the AV Foundation framework instead.” (from Apple docs)

If I’m correct, OpenEars uses this and I’m wondering how functionality will be affected for iOS 7 apps.

[Halle Winkler]

Hello,

Good question, but it won’t be affected. Deprecation means that the functionality is not being changed in the version in which the API is first deprecated, but it’s a request to start the process of switching to the preferred functionality so that a later OS release can phase out the deprecated functionality. From the page you linked:

“From time to time, Apple adds deprecation macros to APIs to indicate that those APIs should no longer be used in active development. When a deprecation occurs, it is not an immediate end-of-life to the specified API. Instead, it is the beginning of a grace period for transitioning off that API and onto newer and more modern replacements.”

There are deprecated APIs from iOS 4 which are still supported in iOS 7 without any functional changes. I appreciate the concern and I will make sure that only AVAudioSession is in use before iOS 8.

[nvrtd frst]

Thanks for the quick reply as always Halle!

[Halle Winkler]

No problem! I think I should probably make this change when I drop support for iOS4, since AudioSessionManager also has some switches for iOS4/iOS5 stuff which could be nicely simplified without iOS4 support at the same time that AVAudioSession will simplify the gnarly old AudioSession stuff, so everything in there can be looked at as a whole and the overall class will be much shorter. I am considering dropping iOS4 sooner rather than later based on these charts and others which seem to show iOS4 usage in the 5% range or less:

http://david-smith.org/iosversionstats/

But I’ve always tried to keep the backward OS compatibility for a pretty long time since I’m not crazy about the phenomenon of OS compatibility driving new device purchases from an environmental perspective. But at this point iOS4 usage is starting to look like the margin of error, so it’s probably time.

[nvrtd frst]

I think that would be really great to simplify the code and I would be 100% for that sooner rather than later. If anyone is still using iOS 4, I feel sorry for them but progress must occur!

[nvrtd frst]

Halle, the reason I ask is because I’m finding a weird bug as I move my app to iOS 7. To give some background, I made changes to the OpenEars classes that would allow my app to start pocketSphinxController in RemoteIO audioUnitSubtype or VPIO audioUnitSubtype. In the app, I could dealloc the pocketSphinxController instance and reinstantiate pocketSphinxController in another audioUnitSubtype dynamically.

This would work fine in iOS 6. While a user was in app, they could switch from VPIO to listening in RemoteIO mode and back.

However when the same app now loaded in iOS 7, the user will set the variable to switch from VPIO to remoteIO but it seems to stay in VPIO mode despite all the variables being passed correctly. I know it is being passed correctly because if they app is then terminated and restarted, it will be in RemoteIO mode.

I hope this wasn’t too confusing. Basically what I’m saying is that in iOS6 the same code allows users to switch between VPIO and RemoteIO seemlessly, and yet when loaded on iOS7 it will only switch if you terminate the app and restart it.

Thoughts? Is there any OpenEars classes that remain on the stack whenever a pocketSphinxController instance is dealloced and reinstantiated? Any help at all would be appreciated. I hope that this isn’t just some random iOS 7 bug in Core Audio…

Using OpenEars 1.5.2.

[Halle Winkler]

Sorry, I don’t have a lot of insight about the particulars of this behavior and it’s a bit outside of what I can give code-level support for, but I can share the following observations:

• AudioSession has remained very stable and similar throughout the entire history of the API being exposed in the three years OpenEars has been using it, and AVAudioSession most likely just wraps it in some respect — just a guess, but the underlying audio code used by the high-level API is pretty likely to still be in C or C++ for performance reasons and the wrapper is probably being pushed not only because it results in fewer support requests to Apple but because it lets them make small changes in the underlying implementation, but probably not because the code used by AudioSession has actually gone away or changed massively. So my suspicion would not tend to point towards AudioSession even if it has just been deprecated. It would be very bad form to break something right at the time of deprecating it that all audio code depends on, and really out of character.

• The entire reason OpenEars never used VPIO is because it has broken in one x.0 OS version and one minor version I can think of off the top of my head (it could have happened with more but since I stopped using it, I stopped testing it), while remoteIO has always behaved the same. So my suspicion would tend to gravitate towards VPIO at the time of a known-to-be-high-pressure x.0 release because there seems to be some kind of minor hole in the API’s testbed where VPIO behavior sometimes changes between releases and then gets fixed.

• It may not be necessary for you to use VPIO with OpenEars >=1.5.x because it has a new audio modes API if you’re supporting 5.0 or greater. It’s controlled via PocketsphinxController so it might be worth a look.

[nvrtd frst]

Thanks for the insights Halle. I do use the audio session modes, but what I’m really looking for is noise cancellation because the app plays sounds while it still has to listen for voice commands… If you have any thoughts on noise cancellation other than using VPIO, I’d really appreciate it.

I use the VoiceChat audio session mode with VPIO:

AVAudioSessionModeVoiceChat
Specify this mode if your app is performing two-way voice communication, such as using Voice over Internet Protocol (VoIP).
When this mode is in use, the device’s tonal equalization is optimized for voice and the set of allowable audio routes is reduced to only those appropriate for voice chat. For use with the AVAudioSessionCategoryPlayAndRecord audio session category. Apple recommends using the Voice Processing Audio Unit with this mode (see Audio Unit Hosting Guide for iOS).

[Halle Winkler]

If VPIO works other than switching, what is the requirement to also use remoteio with it?

[nvrtd frst]

The reason for allowing them to switch is that VPIO seems to drastically reduce the output volume. I’m not sure if it’s a bug or if I need to set other audioUnitProperties but it makes it EXTREMELY low volume when using VPIO. I also noticed that in iOS 5 the volume for VPIO was much louder (but still softer than when using remoteIO).

[Halle Winkler]

The AudioSessionManager does a route override so that output doesn’t go to the ear speaker when the main speaker is available (you can check it out by searching AudioSessionManager for the string “override”). Are you sure that when you are setting up VPIO, this override is occurring? It would be the most straightforward explanation for a big output volume drop.

[Halle Winkler]

(And it’s possible that the override has to occur at a different place in the overall order of events in order to make VPIO happy).

[Halle Winkler]

BTW, does VPIO successfully echo cancel all sound played out of the phone when you use it without any further configuration? I have had it in the back of my mind for re-introducing for echo-canceling purposes on the basis that I haven’t heard about any wonky behavior for a couple OS versions, although it sounds like it’s getting weird again.

[nvrtd frst]

The echo cancellation works perfectly. It’s amazing. I have checked and made sure that sound is definitely coming out of the speakers in VPIO mode. That being the case, do you still think the order of the override matters?

[Halle Winkler]

Hmm. I think if this were my thing, I would drop the idea of switching between two IOs because that sounds a bit complex to me as an approach, and put all the energy into debugging why VPIO is quiet, because there is no intrinsic reason that its playback should be quieter than remoteIO.

I might check this out by first writing a _really_ simple app that does nothing but set up a VPIO and plays something back to see if it is always an issue or if it is an issue due to something in AudioSessionManager, and start building up the code until it exhibits the bad behavior. If the simplest possible version exhibits the bad behavior, I’d add a really simple remoteIO version for comparison to show the expected behavior and send it to Apple DTS and ask them why they play back at different volumes and if you need to change any code.

Does it help if you give whatever you are playing back an explicit volume level?

The echo cancellation works perfectly. It’s amazing.

Good to know, this should probably be reexamined.

[Halle Winkler]

Are things quieter with echo cancellation on, or whether it’s on or off?

[nvrtd frst]

I might check this out by first writing a _really_ simple app that does nothing but set up a VPIO

I would actually like nothing more than to do away with remoteIO completely and just use VPIO if not for the volume thing. I would love to do your suggestion, but my core audio is mediocre at best, so it would be hard for me to even build a simple core audio app (but I will try!). Any tips on general steps would be helpful!

Does it help if you give whatever you are playing back an explicit volume level?

I could try to give an explicit volume level, but I’m not sure how to do this in the low level core audio stuff. I only know how to do this with the high level AVAudio APIs.

Are things quieter with echo cancellation on, or whether it’s on or off?

There is a audioUnitProperty kAUVoiceIOProperty_BypassVoiceProcessing that supposedly bypasses echo cancellation while in VPIO, but things are still quiet regardless of this. Is that what you mean?

Good to know, this [VPIO] should probably be reexamined.

Please do soon!!!

[Halle Winkler]

I have a big-deal improvement I’m working on right now, so it’s going to be a bit of time before I could take a look at that. Just wanted to let you know that it could take a while so you can plan/not wait.

Have you seen this and tried its order of events:

http://stackoverflow.com/questions/13502293/setting-kaudiosessionmode-in-apples-speakhere-example-causes-big-drop-in-volume

This looks intriguingly like maybe you can use both simultaneously but for different stages of the IO:

http://stackoverflow.com/a/10827587/119717

I edited this one to attract more answers, put 50 rep as a bounty on it and upvoted it to see if we can scare up any experts — if you have some extra SO rep to spare it might be worth adding more to it and upvoting it:

http://stackoverflow.com/questions/17528057/issue-with-voiceprocessingio

[nvrtd frst]

I just changed the order of events and made sure audio session mode was being set AFTER default speaker was set. It did not increase the volume.

It would be interesting to see if there were 2 audio units, but I’m not sure if even the person who created the answer solved his own volume problem with it.

I upvoted the answer and am getting friends to upvote it as well. I would gladly put more bounty on it, but I couldn’t find the bounty button after I edited the question. I think maybe because there is already a bounty on it?

[Halle Winkler]

Yup, it’s possible that only one person can add a bounty at once. Better than asking people to upvote it who don’t have a direct interest (which might not delight SO) would be asking them to share the link to the question on Twitter since that will often get results.

[nvrtd frst]

On the low volume for VPIO. More people have been having this problem:
http://tokbox.com/forums/ios/speaker-volume-turned-down-on-init-t14592/page10

These seem to be about recording volume with some thoughts on output voluem too. Lot of people are saying it is a bug:
https://devforums.apple.com/thread/162542?tstart=0
http://stackoverflow.com/questions/13801297/recording-volume-drop-switching-between-remoteio-and-vpio

[Halle Winkler]

Yeah, I actually had it in the very earliest version of OpenEars and removed it due to too many issues like this. Well, let’s give the SO question some time and see what turns up.

[nvrtd frst]

Yes, let’s wait on the bounty. Incidentally I did try setting up a very basic VPIO mode app (I just edited the “AurioTouch 2” sample code from Apple and created some buttons to play a sound at a known volume level). Sound volume is definitely lower in VPIO mode. I think this is normal to have it be somewhat lower to allow for optimal echo cancellation. However I think it is way too low.

[Halle Winkler]

Now that you have a sample app you can submit a bug to Apple and/or ask DTS about it. They might have some feedback.

[nvrtd frst]

Hey Halle, I just got my hands on an iPhone 5s. The VPIO handling on the 5s definitely has some bugs and when playing audio, it will play it haltingly. This was not a problem for 4s or 4 on iOS7 (have not been able to test on iPhone 5 or 5c). I can see why you removed VPIO support and maybe that was a good idea! I think I will just have to focus on remoteIO.

[Halle Winkler]

Thanks for following up — I will admit that everything I read in this thread had slowly led me to the conclusion that it probably wasn’t prudent to reconsider supporting VPIO. I will continue to keep the plan of eventually adding software echo canceling if/when it’s feasible and avoiding VPIO.

[Author]

Posts