Flite in background

[mspector]

I am writing a location-based app that runs primarily in the background. In info.plist for background execution I have enabled location and app plays audio, but when testing the application, the location updates worked but the flite controller didn’t speak. Can the flite controller be configured to speak during background execution?

[Halle Winkler]

Hello,

I’m pretty doubtful that you can initiate new audio while in the background. I think you’re limited to continuing an audio stream you’ve already started before going into the background.

[mspector]

And thats true even if the audio session was started before the program entered the background correct? If so, would there be a way to save the speech to an audio file and link it to a local notification?

[Halle Winkler]

To the best of my knowledge, the only consideration is whether there is an open stream from or to a device speaker or mic at the time of backgrounding; audio session is not a factor.

If so, would there be a way to save the speech to an audio file and link it to a local notification?

I can imagine that this _might_ be possible, but it’s probably going to be a complex implementation that isn’t going to make use of many of the conveniences of the original code. I can’t give pointers on how to do it, sorry.

[mspector]

Thanks for the response and OpenEars has been a great addition to my application. Just as an update I think I have started to get it working once in the background by adding the line of code: [[UIApplication sharedApplication] beginReceivingRemoteControlEvents]; to my viewdidload method.

[Halle Winkler]

Thanks! Always happy to hear that it can do things I don’t expect it to be able to do :) .

[gotok]

@mspector,
Have you tried setting your audio session category to “AVAudioSessionCategoryPlayback”?
That works for me. My app also has the location and audio flags for background execution set in plist.
Both GPS and Flite continue to work for me with the screen locked.
The problem I had was that my app was rejected because the “plays background audio” flag was set and my app did not play background audio. Flite doesn’t count as background audio. I had to add background audio to my app in order for it to be approved.

[jimmychou]

I am having a problem with flite running on an app in the background.
As suggested by others, I tried setting AV Category to ‘AVAudioSessionCategoryPlayback’ with categoryOption to ‘AVAudioSessionCategoryOptionMixWithOthers’
However, the Flite controller always reset the audio session to PlayAndRecord.

As a result, it won’t do TTS when entering background thread.
This is running on an iPhone 5 with IOS6.
I also added the background modes ‘app play audio’ in plist.

[log output below]

Please help.
Thanks,

the log output

// checking av session category
myAV session category in BG: AVAudioSessionCategoryPlayback
myAV session categoryOption in BG: 1

// flite logging
I’m running flite
Checking and resetting all audio session settings.
audioCategory is incorrect, we will change it.
audioCategory is now on the correct setting of kAudioSessionCategory_PlayAndRecord.
bluetoothInput is incorrect, we will change it.
bluetooth input is now on the correct setting of 1.
categoryDefaultToSpeaker is incorrect, we will change it.
CategoryDefaultToSpeaker is now on the correct setting of 1.
preferredBufferSize is correct, we will leave it as it is.
preferredSampleRateCheck is correct, we will leave it as it is.
I’m done running flite and it took 0.275146 seconds
Flite audio player was nil when referenced so attempting to allocate a new audio player.
Loading speech data for Flite concluded successfully.

// checking av session category after flite
myAV session category in BG after flite: AVAudioSessionCategoryPlayAndRecord
myAV session categoryOption in BG after flite: 12

[Halle Winkler]

I think that the suggestion is that you change AudioSessionManager’s audio session category and then recompile the framework so that when it overrides, it overrides to your chosen session category instead of its own.

[jimmychou]

Thanks for the quick response.

Not sure if I am doing it right, I was able to make Flite speak in the background after making two changes in AudioSessionManager.m.

1. self.soundMixing = TRUE;
2. audioCategoryClassification = kAudioSessionCategory_MediaPlayback;

The weird part is that the Flite can only speak one phrase after entering the background mode. It was working fine if the app is running in the foreground.

Below are my findings after many hours of debugging:

1. the AVAudioPlayer delegates did not get called after the [self.audioPlayer play] in the background mode.

2. since the audioPlay did not get destroyed and neither speaking, the lazy accessory of AVAudioPlayer bypassed the initWithData when another phrase was sent to FliteController.

3. When the app is running in the foreground, the AVAudioPlayer delegates were triggered on the main thread.

4. However, since my app was running in another background queue (NSOperationQueue), the Application Delegate was the only application thread running on the main thread. I don’t see any log message coming out of main thread any more.

5. I found a work around by checking to see audioPlay is playing in the lazy accessor code to destroy the audioPlayer and execute the play command again. I kind of suspect this may cause problem some where else.

I am not clear what happen to Flite Controller, as a delegate, running in the back ground mode.

Let me know what you think.

Will do more investigation later.

Thanks,

==============================
logs – (907 is the main thread):
===============================
4784:1803 I’m running flite
4784:1803 Checking and resetting all audio session settings.
4784:1803 audioCategory is correct, we will leave it as it is.
4784:1803 bluetoothInput is correct, we will leave it as it is.
4784:1803 categoryDefaultToSpeaker is correct, we will leave it as it is.
4784:1803 OverrideCategoryMixWithOthers is correct, we will leave it as it is.
4784:1803 preferredBufferSize is correct, we will leave it as it is.
4784:1803 preferredSampleRateCheck is correct, we will leave it as it is.
4784:1803 I’m done running flite and it took 0.306104 seconds
4784:1803 Flite posting suspend notification
4784:907 Flite sending suspend recognition notification.
4784:1803 Flite audio player was nil when referenced so attempting to allocate a new audio player.
4784:1803 Loading speech data for Flite concluded successfully.
4784:907 AVAudioPlayer did finish playing with success flag of 1
4784:907 Flite sending resume recognition notification.
4784:907

Running in the background!

4784:390b I’m running flite
4784:390b Checking and resetting all audio session settings.
4784:390b audioCategory is correct, we will leave it as it is.
4784:390b bluetoothInput is correct, we will leave it as it is.
4784:390b categoryDefaultToSpeaker is correct, we will leave it as it is.
4784:390b OverrideCategoryMixWithOthers is correct, we will leave it as it is.
4784:390b preferredBufferSize is correct, we will leave it as it is.
4784:390b preferredSampleRateCheck is correct, we will leave it as it is.
4784:390b I’m done running flite and it took 0.216623 seconds
4784:390b Flite posting suspend notification
4784:390b Flite audio player was nil when referenced so attempting to allocate a new audio player.
4784:390b Loading speech data for Flite concluded successfully.
(no voice)
4784:390f I’m running flite
4784:390f Checking and resetting all audio session settings.
4784:390f audioCategory is correct, we will leave it as it is.
4784:390f bluetoothInput is correct, we will leave it as it is.
4784:390f categoryDefaultToSpeaker is correct, we will leave it as it is.
4784:390f OverrideCategoryMixWithOthers is correct, we will leave it as it is.
4784:390f preferredBufferSize is correct, we will leave it as it is.
4784:390f preferredSampleRateCheck is correct, we will leave it as it is.
4784:390f I’m done running flite and it took 0.208594 seconds
4784:390f Flite posting suspend notification
4784:390f Loading speech data for Flite concluded successfully.
(no voice)

[jimmychou]

I made a mistake in my previous post.
There should only be one instance of (no voice).
The first instance of (no voice) should read (voice speaking).
The Flite went silent after that.

Sorry about that.

[Halle Winkler]

Hi Jimmy,

I’m sorry I can’t give you more help with this — I haven’t done it myself and it isn’t a feature I actively support because I don’t think it’s exactly what Apple has in mind with their background modes, although I find it interesting. All of the forum discussions on this have been driven by other people’s investigation/discoveries so maybe someone from earlier in this thread will pipe up.

[Author]

Posts