Table of Contents:

Welcome to OpenEars!

Download Politepix’s OpenEars

OpenEars is an shared-source iOS framework for implementing round-trip English language speech recognition and text-to-speech on the iPhone and iPad, which uses the CMU Pocketsphinx, CMU Flite, and CMUCLMTK libraries. Highly-accurate large-vocabulary recognition (that is, trying to recognize any word the user speaks out of many thousands of known words) is not yet a reality for local in-app processing on the iPhone given the hardware limitations of the platform; even Siri does its large-vocabulary recognition on the server side. However, Pocketsphinx is capable of local recognition on the iPhone of vocabularies with hundreds of words depending on the environment and other factors, and performs very well with command-and-control language models.

The current version of OpenEars is 1.01.

OpenEars can:

  • Listen continuously for speech on a background thread, while suspending or resuming speech processing on demand, all while using less than 8% CPU on average on a first-generation iPhone (decoding speech, text-to-speech, updating the UI and other intermittent functions use more CPU),
  • Use any of 9 voices for speech, including male and female voices with a range of speed/quality level, and switch between them on the fly,
  • Change the pitch, speed and variance of any text-to-speech voice,
  • Know whether headphones are plugged in and continue voice recognition during text-to-speech only when they are plugged in,
  • Support bluetooth audio devices (experimental),
  • Dispatch information to any part of your app about the results of speech recognition and speech, or changes in the state of the audio session (such as an incoming phone call or headphones being plugged in),
  • Deliver level metering for both speech input and speech output so you can design visual feedback for both states.
  • Support JSGF grammars,
  • Dynamically generate new ARPA language models in-app based on input from an NSArray of NSStrings,
  • Switch between ARPA language models or JSGF grammars on the fly,
  • Be easily interacted with via standard and simple Objective-C methods,
  • Control all audio functions with text-to-speech and speech recognition in memory instead of writing audio files to disk and then reading them,
  • Drive speech recognition with a low-latency Audio Unit driver for highest responsiveness,
  • Be installed in a Cocoa-standard fashion using an easy-peasy already-compiled framework.

In addition to its various new features and faster recognition/text-to-speech responsiveness, OpenEars now has improved recognition accuracy.

Before using OpenEars, please note that its low-latency Audio Unit driver is not compatible with the Simulator, so it has a fallback Audio Queue driver for the Simulator provided as a convenience so you can debug recognition logic. This means is that recognition is better on the device, and that I’d appreciate it if bug reports are limited to issues which affect the device.

To use OpenEars:

1. If you have installed a pre-1.0 version of OpenEars previously, you can follow the uninstall steps here.
2. Download the distribution and unpack it somewhere that you’d like it to remain. We’ll refer to this file path as [OpenEarsDistributionLocation]. At this point you can open and run the sample app in the distribution folder. Note: the sample app is written using ARC since it is the recommended memory management technique now, so if you are using a pre-ARC version of Xcode you will need to change the ARC code into memory-managed code in order to run the sample app (this is not a restriction of the OpenEars framework itself, just the sample application).

3. Create your own app, and add the iOS frameworks AudioToolbox and AVFoundation to it.

4. Drag the folder [OpenEarsDistributionLocation]/OpenEars/framework into your app project, making sure that the checkbox next to “Copy items into destination group’s folder (if needed)” is NOT CHECKED. Do not check. No.

OK, now that you’ve finished laying the groundwork, you have to…wait, that’s everything. You’re ready to start using OpenEars.

Before shipping your app, you will want to remove unused voices from it so that the app size won’t be too big, as explained here. In order to do this, you need to keep the original distribution files around, so don’t throw them out. You will also need them if you have an issue and need to turn on the verbose debugging output. Otherwise you can basically ignore them.

If the steps on this page didn’t work for you, you can get free support at the forums, read the FAQ, or open a private email support incident at the Politepix shop. Otherwise, carry on to the next part: using OpenEars in your app.

OpenEars isOpenEars uses the Carnegie Mellon University Pocketsphinx library