Crash when Generating Language Model

[omgbobbyg]

I am using OpenEars with Rejecto/RapidEars, and for some text artifacts we pass into the language model generator, the app will crash with no warning or exception. For example the following input JFK Innauguration will cause the app to crash roughly 50% of the time after I call generateRejectingLanguageModelFromArray

This only happens for certain text artifacts, others work just fine. Is there something wrong with the input which might be causing this?

Here is a dump of the logging information generated by openears:Crash Log

[Halle Winkler]

OK, I think you just need to do some cleanup on your text before submitting it as an array – it isn’t clear from the logging which character is causing the crash or if the crash is necessarily from the language model because the logging doesn’t have the verbosePocketsphinx on (that’s where you would be getting the warning followed by the exception), but I would strip punctuation, special characters and line breaks from your text before handing it off to LanguageModelGenerator in any case, because the presence of those items which no utterance corresponds to can have a detrimental effect on recognition:

https://www.politepix.com/forums/topic/custom-dictionary-speech-recognition-only-kind-of-working/

It’s also important to upgrade to the current versions of OpenEars and the Rejecto demo for best results with these kinds of issues, since the latest versions of both fix important language model generation and recognition issues.

If you still have these issues after upgrading to the current versions, you can show me the full log with verbosePocketsphinx on and I’ll take a look at it and we can see what the cause of the crash is.

[omgbobbyg]

Halle, thanks for the quick response. I will try to do some cleanup of the file.

In terms of generating hypothesis, right now I notice that OpenEars will generate a hypothesis with an apostrophe in it for words like “it’s” and “i’d”, which is what I would like it do.

Will the generation of hypothesis that do have punctuation in it be effected if I remove all punctuation from the text passed to the language model generator?

Are apostrophes the only kind of punctuation that OpenEars will generate as part of a hypothesis?

Thanks for your help
Bobby

[Halle Winkler]

I’ll give you some background so you can make informed choices about the input text for best results.

In order to perform recognition, it’s necessary to have an acoustic model, a phonetic dictionary, and either a language model or a grammar. The acoustic model is a bundle in OpenEars and under most circumstances you don’t do anything special with it, and the language model or grammar can be made from any kind of arbitrary input (punctuation is fine) but it has to match what it in the phonetic dictionary in order for recognition to work. The language model/grammar are about constraining the search space for language, and the phonetic dictionary is about transcribing the actual word sounds which a word is comprised of.

In order to make these phonetic transcriptions of word sounds for the phonetic dictionary, OpenEars first just tries to look them up in the file LanguageModelGeneratorLookupList.text in the acoustic model bundle, and it may get one or several alternate pronunciations. That is the best case scenario because it means that a word with a few different pronunciations will tend to be recognized correctly. If it can’t find any pronunciations in that file, it will go to the fallback method, which will take a single best guess. This is not as good because a) the guess might be wrong, or b) it might be right but it’s only one pronunciation and there might be more than one correct way to say the word (e.g. the word “orange” which has different regional pronunciations), so the other pronunciations will not be recognized accurately.

So if words are submitted to the first method with extraneous punctuation, they won’t be found in the preferred lookup method (as an example, “it’s” is not in the main lookup dictionary for some reason). But if it’s very obvious how it would be pronounced, a successful phonetic transcription might be made by the fallback method anyway. That is what is happening when you submit “it’s” and it appears to be working. For “I’d”, it is actually in the main lookup dictionary. I think this probably (narrowly) means you should keep apostrophes in, since “id” is pronounced different from “I’d”, but keep an eye on things and see if you end up with any words which can never be recognized by the engine, which is a potential sign of a bad entry from the fallback method.

Remember to put your text in all caps!

[omgbobbyg]

Halle, thanks for the great explanation, that helps a lot to understand what’s going on inside of OpenEars that I didn’t know.

I followed your suggestion and stripped all of my input of punctuation/special characters except for apostrophes. However, I am still getting crashes intermittently.

I’ve turned on OpenEars logging and have set verbosePocketSphinx = YES before the language model generation begins.

This is the log generated when there is no crash:JFK Innauguration – NO CRASH

This is the log generated when there is a crash:JFK Innauguration – CRASH

This is the dump of my input NSArray being bassed to the language model generator: Language model generator input text

I’d appreciate any further insight you can give me into what might be happening here.

Thanks again

[Halle Winkler]

Hello,

OK, just go ahead and upgrade your frameworks (OpenEars, Rejecto and RapidEars) to the current versions before we troubleshoot more, since the upgrade versions fix relevant issues. The current version of OpenEars can be downloaded from here:

https://politepix.com/openears/

and you can get the current versions of the plugins at the same link you were emailed when you requested the demos originally via this site.

[Halle Winkler]

Super, thank you for letting me know (I removed your previous post to protect the email). So, in that case it’s very easy to upgrade your licenses – just go to the licensee site link that was sent in the original license mail and log in with the licensee login credentials that were set up at that time. The latest framework versions will be on that site and you can get the latest OpenEars version at the link above.

Let me know if upgrading doesn’t fix your issues and I’ll be happy to help.

[Author]

Posts