Reducing the calibration time

On my iPhone 4 I start the pocket sphinx initialization immediately at the top of awakeFromNib in may main ViewController. I don’t get the notification that calibration has begun for another three seconds. Granted, some other things are being initialized in parallel, and I haven’t done profiling to confirm it, but it seems like most of the time is spent in the pocket sphinx initialization.

Hi James,

I’ve rechecked here and on recent devices I’m seeing <1.3 seconds from starting listening to starting calibration. How large of a vocabulary are you using, what device, what OS, etc?

Yup, you’re supposed to heavily test apps running with this setup with single recognition sessions running over long time periods (10-30 minutes) and see that it will almost always crash in the neighborhood of the 15 minute mark with a calibration that is only shortened via that method, and that it will not always be able to correctly detect silence in a session depending on what happened during the short calibration in the real world (it’s very easy for a blast of level-pinning noise to last a second). If neither of these possible issues applies to your app, go to town.

Reducing the calibration time without side-effects will mean making some small changes in other classes involved with the driver and then doing a large round of QA since this is a known crash danger.

It’s just the VAD being calibrated. Why not do it in parallel with the rest of your startup?

It’s a low computation load. Theoretically, it can run in parallel with models loading, grammars or language models being built or compiled, and even the UI being assembled.

Then make the VAD parameters (there’s only 4 of them) persistent from session to session, and set CONT_AD_THRESH_UPDATE to something like 50 (1.6 seconds) instead of the current 100 (3.2 seconds). Then you can pump the calibration for 2 seconds instead of 4 (In the background, while the models load).

The VAD literally only uses one parameter, “noise_level”, the peak of a power histogram, as it’s estimate of the background noise. That’s what it’s spending 3.2 seconds trying to figure out. Adaptation always starts from CONT_AD_DEFAULT_NOISE, which is hardcoded to 30.

After the allotted cal time, noise_level typically ramps up to around 34 in a noisy environment, because the adaptation time constants are way too low for anything that doesn’t involve starting in the same environment over and over again. In our environment, it takes another 14 seconds to hit something near 40, which is more typical of driving conditions.

Previous experience has taught me that this sort of value is very “domain specific”, an IVR app might settle over a range of 35-40, an automotive app maybe 40-45, but within that domain, using the last values as the starting point for the next session really gets things up and running faster.

Heck, for our app, 30 is such an epically bad starting point that simply manually specifying 40 instead and totally skipping adaptation gets better results faster. 30 is way low for an iPhone in a car bracket even when the car is parked.

Of course, the Sphinx VAD is on the wrong side of “current”, and definitely not anything we should be doing in 2011 with a 3 gigaflop DSP available to us. I’ve done better in the past with a cepstral VAD that doesn’t need any cal, at all, from office environment to highway speeds. Time to get into vDSP.

The whole VAD is like that. People have been checking in changes to CONT_AD_THRESH_UPDATE, CONT_AD_ADAPT_RATE, CONT_AD_DEFAULT_NOISE, CONT_AD_DELTA_SPEECH, and CONT_AD_DELTA_SIL for 15 years. They’re not “fixing” the system, they’re “locking” it to one task and environment or another.

I can see a simple way to make delta_speech and delta_sil adapt (they don’t, now. They just stay on what they get from CONT_AD_DELTA_SPEECH and CONT_AD_DELTA_SIL for forever, and start them from a table based on a persistent noise_level.

The thing you have to keep in mind about Sphinx (2, 3, 4, or Pocket) is that it isn’t actually designed to be used by anyone, for anything. Seriously, it’s designed to process mounds of precut utterance files and deliver scores in batch mode, so grad students and grant seekers can make changes and say “running this batch proves our change improves performance 14%”.

So, the developers don’t care that the VAD is decades out of date, or that no_search destabilizes the main recognition loop. That’s “user” stuff. You and I, my friend, are “untouchables”. We’re actually doing that filthy “work” stuff and soiling the ivory towers and marble halls.

I’ve been getting into that kind of trouble for decades. ;)