How to use JSGF?

Thanks Halle, I will check out the page try to figure out how to get JSGF working.

Another questions I have is for the PocketsphinxController (or OpenEarsEventsObserver), is there a way I can limit time allowed for user input? For example, in my app to allow user 10 seconds to say something before prompting them again?

I suppose I can setup a timer loop and let it timeout if no pocketsphinxDidDetectSpeech is called, just wondering if there’s a cleaner way of doing it.

ok, I tried the JSGF with a simple grammar :
public = YES | NO | MAYBE;

and specified the .gram and .dic file in the method “startListeningWithLanguageModelAtPath” and it worked like a charm!

However, couple of issues :
- Looking at the code it seems dynamically switching gram file is not supported, is this a future enhancement? I sort of used a work around by calling “stopListening” and “startListeningWithLanguageModelAtPath” for the new gram file but there seems to be some long delay (around 7 seconds). I think dynamically switching gram file would be nice since I plan to have some grammar of “YES | NO | MAYBE” for one menu and just “YES | NO” for others. I’m not sure if there’s any other way to achieve this?

- The score that is returned in “pocketsphinxDidReceiveHypothesis” seems very wierd. Sometimes when it correctly detected something it would get a score of 0, and at other times when I just say some gibberish and it return a very high score. Is there a good way of obtaining how certain is the detection results?

Hello,

I’ve been doing some experimentation with a JSGF grammar and have had much success. I’ve dynamically created a JSGF grammar just by building up an NSString and writing this to a file which is then referenced instead of the file provided.

However the main issue I’m having is responsiveness – I’m getting around 2-3 seconds response time which isn’t bad, but not ideal for a real-time recognition system that updates the UI. I’ve been doing user research around our prototype and every user has commented on the length of time it takes to get feedback. We’re using an iPad 2.

Of these 2-3 seconds I think the break down is as follows:

1 second silence detection.
1-2 seconds processing time.

Halle, have you any suggestions on how to reduce this processing time?

From my work using the other model supported, its possible to reduce the silence detection time, however is this supported with JSGF? It doesn’t seem to be at the moment.

Thank you so much again for all your responsiveness and support for this project, we are all greatly indebted to you.

Hi Halle,

I’m using the latest version (0.912) and my grammar only has around 50 words.

Thanks again.

Halle/Dave,

This is an interesting discussion, I just noticed that I’m seeing about the same response time for version 0.912 (around 2 sec processing time) as well. I’m using iPhone 3GS and the ARPA language model only has two choices : YES and NO.

I will try out the KDS parameter to see if it helps the processing time. Thanks for the info!

Halle,

I’m only using ARPA files (no gram file). The files I’m using are the “.dic” and the “.languagemodel” file. The corpus is composed of just “YES” and “NO”.

I did try the options you suggested. For my iPhone I didn’t notice any significant change in the compile for thumb option. However changing the downsample ratio seems to decrease recognition time by about 1 second when set to 2. Not sure if it changes the accuracy much, I suppose for the simple model I’m using it’s probably okay.

Halle,

Actually, by coincidence couple of days ago I was experimenting with different things and what I noticed was that in the sample project if I commented out the TTS processing in pocketsphinxDidReceiveHypothesis (the call to [flitecontroller say:...]), then the response time is also significantly reduced to well less than a sec. Here’s the response time I measured is how long between it says detectFinishSpeech to the display of “YOU said”.

So I think you are right on that the time between pocketsphinxDidDetectFinishedSpeech to pocketsphinxDidReceiveHypothesis may actually be very small. I think I was wrong to base the response on how long does it take to hear the TTS response back, which can involve some TTS processing time. Nontheless I think I will try it out tonight with the default down sampling ratio to see if that makes any difference.

On a side note, for the IVR application I’m building I plan to be using pre-recorded prompt (either wav or mp3) so I suspect the processing time for this TTS will be eliminated. Do you have any recommendation on how should I go about implement the (prompt-recognition) loop? I’m deciding whether to user AVaudioplayer or OpenAL, the reason is that some of the prompt are concatenated together and I’m not sure if there are delay issues with using AVaudioplayer this way.

Halle,

Sorry to take up so much bandwidth on this thread, but I did some more experimentation and it seems that using “prepareToPlay” does not really improve the delay. This is especially for the case when I’m dynamically playing the prerecorded mp3 files together : for example “three” + “hundred” + “and” + “seventy” + “four”. The gaps between each file is around 1 sec no matter when I call the “prepareToPlay” beforehand or just when each of them is about to be played.

I did some goggling and it looks like quite a lot of people have similar problem as well so some people just resort to using OpenAL. Some of them suggested using wav format instead but it seems to make little difference. However, I think there are some workaround I can explore :

1. Concatenate mp3 files together dynamically. and load the big file during run time. I’m not sure how to even approach this since I’m new to Objetive-C, I suppose a temporary file
2. Create the mp3 data manually by reading the data in from each of the short mp3 file to create a long mp3 data stream (similar to what you have done for flitecontroller?). However I’m not sure the complexity involved, do you have any suggestion or examples on how do I go about this?

Actually, maybe an easier approach for you would be to use Extended Audio File Services and just try to write out your entire audio file to disk and then play it back. That will also be a drop-in approach for AVAudioPlayer that should cause no conflicts.

There’s some audio file code in version 0.902 of OpenEars in the audio driver that you can take a look at.

Very cool! I’m glad that was a good approach for you.