OE Sample App Mic Permissions Error

[Rabshakeh]

A quick disclaimer: While I have substantial experience with programming, I am relatively new to Objective C and Swift.

I’m having an issue with the OE Sample App fresh out of the box. I haven’t changed any code, but it crashes when listening begins. See below the logging with both OE Logging and Pocketsphinx verbose logging set to TRUE.

OpenEars 2.503
Device: iPhone 6 model MG632LL/A. iOS version 10.0.2

Error Log:

2016-10-22 15:44:35.315016 OpenEarsSampleApp[4279:1411254] Starting OpenEars logging for OpenEars version 2.503 on 64-bit device (or build): iPhone running iOS version: 10.000000
2016-10-22 15:44:35.315508 OpenEarsSampleApp[4279:1411254] Creating shared instance of OEPocketsphinxController
2016-10-22 15:44:35.422363 OpenEarsSampleApp[4279:1411254] Starting dynamic language model generation

INFO: ngram_model_arpa_legacy.c(504): ngrams 1=10, 2=16, 3=8
INFO: ngram_model_arpa_legacy.c(136): Reading unigrams
INFO: ngram_model_arpa_legacy.c(543): 10 = #unigrams created
INFO: ngram_model_arpa_legacy.c(196): Reading bigrams
INFO: ngram_model_arpa_legacy.c(561): 16 = #bigrams created
INFO: ngram_model_arpa_legacy.c(562): 3 = #prob2 entries
INFO: ngram_model_arpa_legacy.c(570): 3 = #bo_wt2 entries
INFO: ngram_model_arpa_legacy.c(293): Reading trigrams
INFO: ngram_model_arpa_legacy.c(583): 8 = #trigrams created
INFO: ngram_model_arpa_legacy.c(584): 2 = #prob3 entries
INFO: ngram_model_dmp_legacy.c(521): Building DMP model…
INFO: ngram_model_dmp_legacy.c(551): 10 = #unigrams created
INFO: ngram_model_dmp_legacy.c(652): 16 = #bigrams created
INFO: ngram_model_dmp_legacy.c(653): 3 = #prob2 entries
INFO: ngram_model_dmp_legacy.c(660): 3 = #bo_wt2 entries
INFO: ngram_model_dmp_legacy.c(664): 8 = #trigrams created
INFO: ngram_model_dmp_legacy.c(665): 2 = #prob3 entries
2016-10-22 15:44:35.500876 OpenEarsSampleApp[4279:1411254] Done creating language model with CMUCLMTK in 0.078328 seconds.
2016-10-22 15:44:35.501656 OpenEarsSampleApp[4279:1411254] Since there is no cached version, loading the language model lookup list for the acoustic model called AcousticModelEnglish
2016-10-22 15:44:35.562256 OpenEarsSampleApp[4279:1411254] I’m done running performDictionaryLookup and it took 0.046349 seconds
2016-10-22 15:44:35.569026 OpenEarsSampleApp[4279:1411254] I’m done running dynamic language model generation and it took 0.227962 seconds
2016-10-22 15:44:35.575858 OpenEarsSampleApp[4279:1411254] Starting dynamic language model generation

INFO: ngram_model_arpa_legacy.c(504): ngrams 1=12, 2=19, 3=10
INFO: ngram_model_arpa_legacy.c(136): Reading unigrams
INFO: ngram_model_arpa_legacy.c(543): 12 = #unigrams created
INFO: ngram_model_arpa_legacy.c(196): Reading bigrams
INFO: ngram_model_arpa_legacy.c(561): 19 = #bigrams created
INFO: ngram_model_arpa_legacy.c(562): 3 = #prob2 entries
INFO: ngram_model_arpa_legacy.c(570): 3 = #bo_wt2 entries
INFO: ngram_model_arpa_legacy.c(293): Reading trigrams
INFO: ngram_model_arpa_legacy.c(583): 10 = #trigrams created
INFO: ngram_model_arpa_legacy.c(584): 2 = #prob3 entries
INFO: ngram_model_dmp_legacy.c(521): Building DMP model…
INFO: ngram_model_dmp_legacy.c(551): 12 = #unigrams created
INFO: ngram_model_dmp_legacy.c(652): 19 = #bigrams created
INFO: ngram_model_dmp_legacy.c(653): 3 = #prob2 entries
INFO: ngram_model_dmp_legacy.c(660): 3 = #bo_wt2 entries
INFO: ngram_model_dmp_legacy.c(664): 10 = #trigrams created
INFO: ngram_model_dmp_legacy.c(665): 2 = #prob3 entries
2016-10-22 15:44:35.642681 OpenEarsSampleApp[4279:1411254] Done creating language model with CMUCLMTK in 0.066751 seconds.
2016-10-22 15:44:35.642887 OpenEarsSampleApp[4279:1411254] Returning a cached version of LanguageModelGeneratorLookupList.text
2016-10-22 15:44:35.674206 OpenEarsSampleApp[4279:1411254] The word QUIDNUNC was not found in the dictionary of the acoustic model /var/containers/Bundle/Application/0CEB9D63-06E6-4EE9-9F7D-CB1F481AB278/OpenEarsSampleApp.app/AcousticModelEnglish.bundle. Now using the fallback method to look it up. If this is happening more frequently than you would expect, likely causes can be that you are entering words in another language from the one you are recognizing, or that there are symbols (including numbers) that need to be spelled out or cleaned up, or you are using your own acoustic model and there is an issue with either its phonetic dictionary or it lacks a g2p file. Please get in touch at the forums for assistance with the last two possible issues.
2016-10-22 15:44:35.674460 OpenEarsSampleApp[4279:1411254] Using convertGraphemes for the word or phrase quidnunc which doesn’t appear in the dictionary
2016-10-22 15:44:35.676221 OpenEarsSampleApp[4279:1411254] Elapsed time to generate unknown word phonemes in English is 0.001844
2016-10-22 15:44:35.676384 OpenEarsSampleApp[4279:1411254] the graphemes “K W IH D N AH NG K” were created for the word QUIDNUNC using the fallback method.
2016-10-22 15:44:35.688082 OpenEarsSampleApp[4279:1411254] I’m done running performDictionaryLookup and it took 0.045066 seconds
2016-10-22 15:44:35.699945 OpenEarsSampleApp[4279:1411254] I’m done running dynamic language model generation and it took 0.130461 seconds
2016-10-22 15:44:35.700390 OpenEarsSampleApp[4279:1411254]

Welcome to the OpenEars sample project. This project understands the words:
BACKWARD,
CHANGE,
FORWARD,
GO,
LEFT,
MODEL,
RIGHT,
TURN,
and if you say “CHANGE MODEL” it will switch to its dynamically-generated model which understands the words:
CHANGE,
MODEL,
MONDAY,
TUESDAY,
WEDNESDAY,
THURSDAY,
FRIDAY,
SATURDAY,
SUNDAY,
QUIDNUNC
2016-10-22 15:44:35.700618 OpenEarsSampleApp[4279:1411254] Attempting to start listening session from startListeningWithLanguageModelAtPath:
2016-10-22 15:44:35.704501 OpenEarsSampleApp[4279:1411254] User has not granted mic permission so recognition won’t be possible, stopping. To receive this information as a callback add the OEEventsObserver delegate method -(void)pocketsphinxFailedNoMicPermissions; to your app. To obtain mic permissions before attempting recognition for the first time, call Pocketsphinx’s method -(void) requestMicPermission; and wait for a result callback in the OEEventsObserver delegate method – (void) micPermissionCheckCompleted:(BOOL)result to decide on what to do next. To programmatically check the mic permission status any time after you have first obtained permission or had permission denied, call OEPocketsphinxController’s method micPermissionIsGranted which will return TRUE or FALSE. OEPocketsphinxController will be able to start the listening loop the first time you call its listening method after the user has been asked for and granted mic permission, or if the user grants permission as a result of the dialog that will appear when this method is first attempted to be run. This will behave differently on the Simulator and a real device because the Simulator does not protect mic permissions, so Simulator behavior shouldn’t be reported as a bug. You can silence this warning by setting self.pocketsphinxController.doNotWarnAboutPermissions = TRUE. None of this applies to iOS versions previous to iOS7, which will not perform a permissions check.
2016-10-22 15:44:35.711672 OpenEarsSampleApp[4279:1411312] [access] This app has crashed because it attempted to access privacy-sensitive data without a usage description. The app’s Info.plist must contain an NSMicrophoneUsageDescription key with a string value explaining to the user how the app uses this data.
(lldb)

Backtrace:

* thread #5: tid = 0xc59ee, 0x00000001908ccd74 libsystem_kernel.dylib`__abort_with_payload + 8, queue = ‘com.apple.root.default-qos’, stop reason = signal SIGABRT
* frame #0: 0x00000001908ccd74 libsystem_kernel.dylib`__abort_with_payload + 8
frame #1: 0x00000001908c9480 libsystem_kernel.dylib`abort_with_payload_wrapper_internal + 100
frame #2: 0x00000001908c94c8 libsystem_kernel.dylib`abort_with_payload + 12
frame #3: 0x0000000193b0a560 TCC`__CRASHING_DUE_TO_PRIVACY_VIOLATION__ + 260
frame #4: 0x0000000193b0a45c TCC`__TCCAccessRequest_block_invoke.73 + 704
frame #5: 0x0000000193b0d52c TCC`__tccd_send_block_invoke + 288
frame #6: 0x00000001909d401c libxpc.dylib`_xpc_connection_reply_callout + 80
frame #7: 0x00000001909d3f8c libxpc.dylib`_xpc_connection_call_reply + 40
frame #8: 0x000000010079921c libdispatch.dylib`_dispatch_client_callout + 16
frame #9: 0x00000001007a72d4 libdispatch.dylib`_dispatch_queue_override_invoke + 980
frame #10: 0x00000001007a8e6c libdispatch.dylib`_dispatch_root_queue_drain + 584
frame #11: 0x00000001007a8bb8 libdispatch.dylib`_dispatch_worker_thread3 + 140
frame #12: 0x00000001909912c8 libsystem_pthread.dylib`_pthread_wqthread + 1288
frame #13: 0x0000000190990db4 libsystem_pthread.dylib`start_wqthread + 4

After encountering this error, I attempted to follow the instructions given in the verbose logging. However, when I added code to run the OEPocketsphinxController’s method -(void)requestMicPermission, it acted as though that class were not imported by default in the OE Sample App.

I would appreciate any guidance as to how to solve this problem. If more details are needed, I will do my best to provide them.

[Halle Winkler]

Welcome,

Sorry, that looks like an iOS 10 requirement that I overlooked in the sample app; I’ll add the key to the info.plist and push a new version next week. In the meantime you should be able to add an NSMicrophoneUsageDescription key and the corresponding string value “Microphone access is used for performing offline speech recognition” to the sample app’s info.plist and it ought to work.

[Halle Winkler]

OK, I’ve added this to today’s update 2.504, should now run fine on iOS 10 devices.

[Author]

Posts