RuleORama

Introducing RuleORama!

RuleORama creates rules-based grammars that let you define precise rules for which words and phrases can be recognized in what order. It's fast enough to use with RapidEars and will also increase the speed of grammar-based recognition when used with OpenEars.

Introduction and Installation

Introduction

RuleORama is a plugin for the generation of rules-based grammars for OpenEars® (in this case grammar doesn't mean a grammar in the sense of a collection of definitions for parts of speech and their usage, but a grammar in the sense of a finite state grammar, i.e. a simplified ruleset for speech).

Rulesets are useful for cases in which you know, for instance, that you will always wish to recognize the phrase "go forward" with both of those words together, instead of using a probability-based model such as OpenEars' ARPA models which could deliver the recognized results "go" by itself, "forward" by itself, "forward go", and "go forward", depending on confidence.

However, OpenEars' ruleset tools go much deeper than just the ability to set complete recognition phrases, since they use a human-language specification for defining many different features of a ruleset, and in a form which can be implemented dynamically based on as-yet-unknown input to be turned into rules at runtime.

RuleORama adds a second output format to OpenEars' JSGF output format for its grammar generator, which is many times faster than the stock OpenEars JSGF output format during recognition and is compatible with RapidEars as well as OpenEars. RuleORama is compatible with the English acoustic model AcousticModelEnglish.bundle and the other compatible acoustic models found at the <a/ href="http://www.politepix.com/openears/languagedownloads">acoustic model download page.

RuleORama can be purchased at the Politepix shop here and we recommend thoroughly evaluating the demo version before purchasing, which can be downloaded here.

The installation and usage process is the same for both the demo and licensed version, but the demo times out after 3 minutes of use and can't be submitted to the App Store. The best way to get started using RuleORama is to get a tutorial from the Politepix interactive tutorial tool. You can also read more about grammars with OpenEars in this blog post. Steps for getting started and more in-depth documentation are provided on this page, below.

Installation

How to install and use RuleORama:

RuleORama is a plugin for OpenEars, so it is added to an already-working OpenEars project in order to enable new OpenEars features. In these instructions we are using the OpenEars sample app as an example for adding the plugin and new features, but the steps are basically the same for any app that already has a working OpenEars installation. Please note that RuleORama requires OpenEars 2.5 or greater. RuleORama also works with RapidEars versions 2.5 or greater. RuleORama does not yet work with Rejecto although if there are many requests for this functionality it may be considered, if it is possible.

  1. Download and try out the OpenEars distribution and try the OpenEars sample app out. RuleORama is a plug-in for OpenEars that is added to an OpenEars project so you first need a known-working OpenEars app to work with. The OpenEars sample app is fine for this to get started. You can also get a complete tutorial on both creating an OpenEars app and adding RuleORama to it using the automatic customized tutorial.

  2. Open up the OpenEars Sample App in Xcode. Drag your downloaded RuleORamaDemo.framework into the OpenEars sample app project file navigator.

  3. Open up the Build Settings tab of your app or OpenEarsSampleApp and find the entry "Other Linker Flags" and add the linker flag "-ObjC". Do this for debug and release builds. More explanation of this step can be seen in the tutorial by selecting the "Offline (does not use the network) speech recognition which uses a rules-based grammar fast enough for RapidEars (paid plugin)" tutorial, which will also show exactly how to use the new methods added by RuleORama.
  4. Next, navigate to Xcode's Build Settings for your target and find the setting "Framework Search Paths".

  5. If adding the framework in previous step did not automatically add it to "Framework Search Paths", add it manually. You can find the path by going into the Project Navigator (the main Xcode project file view), finding and then selecting your just-added RuleORamaDemo.framework, and typing ⌘⌥-1 to open the File Inspector for it (it may already be open – it is the narrow window pane on the far right of the main Xcode interface). The full path to your added framework is shown under "Identity and Type"->"Full Path". The "Framework Search Path" is this path minus the last path element, so if it says /Users/you/Documents/YourApp/Resources/Framework/RuleORamaDemo.framework, the path to add to "Framework Search Paths" is /Users/yourname/Documents/YourApp/Resources/Framework/ and you should keep the "Recursive" checkbox unchecked.

  6. While we're here, take a moment to look at your Framework Search Paths build setting and verify that it doesn't contain any peculiar entries (for instance, entries with many extra quotes and/or backslashed quote marks) and that each search path is on its own line and hasn't been concatenated to another entry, and that the setting isn't pointing to old versions of the frameworks you're installing that are in other locations.

Support

With your demo download you can receive support via the forums, according to its rules of conduct. In order to receive forum support, it is necessary to have used accurate information with your initial demo download such as a valid email address and your name.

Once you have completed licensing of the framework for your app, forum support will continue to be available to you, and if you need private support via email it is possible to purchase a support contract or individual support incidents at the shop.

Licensing the framework requires giving the exact application name that the framework will be linked to, so don't purchase the license until you know the app name, and again, please try the demo first. It is not possible to change bundle IDs after a purchase, and there are no refunds post-purchase, due to the ability to completely test the comprehensive demo over a full development period.

Please read on for the RuleORama documentation.

BACK TO TOP

OELanguageModelGenerator+RuleORama Category Reference

Detailed Description

A plugin which adds the ability to create rules-based grammars.

Usage examples

What to add to your OpenEars implementation:

First, find the line
#import <OpenEars/OELanguageModelGenerator.h>
in your app and add the following line right underneath it:
#import <RuleORamaDemo/OELanguageModelGenerator+RuleORama.h>
Next, change this line where you create a language model:
NSError *err = [lmGenerator generateLanguageModelFromArray:words withFilesNamed:name forAcousticModelAtPath:[OEAcousticModel pathToModel:@"AcousticModelEnglish"]];
to use this grammar and method instead:
NSDictionary *grammar = @{
     ThisWillBeSaidOnce : @[
         @{ OneOfTheseCanBeSaidOnce : @[@"HELLO COMPUTER", @"GREETINGS ROBOT"]},
         @{ OneOfTheseWillBeSaidOnce : @[@"DO THE FOLLOWING", @"INSTRUCTION"]},
         @{ OneOfTheseWillBeSaidOnce : @[@"GO", @"MOVE"]},
         @{ThisWillBeSaidOnce : @[
             @{ OneOfTheseWillBeSaidOnce : @[@"10", @"20",@"30"]}, 
             @{ OneOfTheseWillBeSaidOnce : @[@"LEFT", @"RIGHT", @"FORWARD"]}
         ]},
         @{ ThisCanBeSaidOnce : @[@"THANK YOU"]}
     ]
 };
 
    NSError *err = [lmGenerator generateFastGrammarFromDictionary:grammar withFilesNamed:name forAcousticModelAtPath:[OEAcousticModel pathToModel:@"AcousticModelEnglish"]];


and change this line:
	lmPath = [lmGenerator pathToSuccessfullyGeneratedLanguageModelWithRequestedName:@"NameIWantForMyLanguageModelFiles"];

to this:
	lmPath = [lmGenerator pathToSuccessfullyGeneratedRuleORamaRulesetWithRequestedName:@"NameIWantForMyLanguageModelFiles"];

This will allow you to recognize statements in accordance with this grammar, such as: HELLO COMPUTER DO THE FOLLOWING MOVE 10 LEFT THANK YOU or GREETINGS ROBOT INSTRUCTION MOVE 20 RIGHT but it will not recognize individual words or words in orders outside of the grammar. Please note that unlike the JSGF output type in stock OpenEars, RuleORama doesn't support the rule types with optional repetitions. Rules defined with repetitions will be composed into a rule with a single repetition. You can learn much more about how grammars work in OpenEars and RuleORama here.

Method Documentation

- (NSError *) generateFastGrammarFromDictionary: (NSDictionary *)  grammarDictionary
withFilesNamed: (NSString *)  fileName
forAcousticModelAtPath: (NSString *)  acousticModelPath 

Swift 3

generateFastGrammar(from: [AnyHashable : Any]!, withFilesNamed: String!, forAcousticModelAtPath: String!)

Dynamically create a fast grammar using OpenEars' natural language system for defining a speech recognition ruleset, that can be used with OEPocketsphinxController or RapidEars. The NSDictionary you submit to the argument generateGrammarFromDictionary: is a key-value pair consisting of an NSArray of words stored in NSStrings indicating the vocabulary to be listened for, and an NSString key which is one of the following #defines from GrammarDefinitions.h, indicating the rule for the vocabulary in the NSArray:

ThisWillBeSaidOnce
ThisCanBeSaidOnce
ThisWillBeSaidWithOptionalRepetitions
ThisCanBeSaidWithOptionalRepetitions
OneOfTheseWillBeSaidOnce
OneOfTheseCanBeSaidOnce
OneOfTheseWillBeSaidWithOptionalRepetitions // NOTE: RULEORAMA ONLY SUPPORTS A SINGLE REPETITION
OneOfTheseCanBeSaidWithOptionalRepetitions // NOTE: RULEORAMA ONLY SUPPORTS A SINGLE REPETITION

Breaking them down one at a time for their specific meaning in defining a rule:

ThisWillBeSaidOnce // This indicates that the word or words in the array must be said (in sequence, in the case of multiple words), one time.
ThisCanBeSaidOnce // This indicates that the word or words in the array can be said (in sequence, in the case of multiple words), one time, but can also be omitted as a whole from the utterance.
ThisWillBeSaidWithOptionalRepetitions // This indicates that the word or words in the array must be said (in sequence, in the case of multiple words), one time or more.
ThisCanBeSaidWithOptionalRepetitions // This indicates that the word or words in the array can be said (in sequence, in the case of multiple words), one time or more, but can also be omitted as a whole from the utterance.
OneOfTheseWillBeSaidOnce // This indicates that exactly one selection from the words in the array must be said one time.
OneOfTheseCanBeSaidOnce // This indicates that exactly one selection from the words in the array can be said one time, but that all of the words can also be omitted from the utterance.
OneOfTheseWillBeSaidWithOptionalRepetitions // This indicates that exactly one selection from the words in the array must be said, one time or more. NOTE: RULEORAMA ONLY SUPPORTS A SINGLE REPETITION
OneOfTheseCanBeSaidWithOptionalRepetitions // This indicates that exactly one selection from the words in the array can be said, one time or more, but that all of the words can also be omitted from the utterance. NOTE: RULEORAMA ONLY SUPPORTS A SINGLE REPETITION

Since an NSString in these NSArrays can also be a phrase, references to words above should also be understood to apply to complete phrases when they are contained in a single NSString.

A key-value pair can also have NSDictionaries in the NSArray instead of NSStrings, or a mix of NSStrings and NSDictionaries, meaning that you can nest rules in other rules.

Here is an example of a complex rule which can be submitted to the generateGrammarFromDictionary: argument followed by an explanation of what it means:

@{
    ThisWillBeSaidOnce : @[
        @{ OneOfTheseCanBeSaidOnce : @[@"HELLO COMPUTER", @"GREETINGS ROBOT"]},
        @{ OneOfTheseWillBeSaidOnce : @[@"DO THE FOLLOWING", @"INSTRUCTION"]},
        @{ OneOfTheseWillBeSaidOnce : @[@"GO", @"MOVE"]},
        @{ThisWillBeSaidWithOptionalRepetitions : @[
            @{ OneOfTheseWillBeSaidOnce : @[@"10", @"20",@"30"]}, 
            @{ OneOfTheseWillBeSaidOnce : @[@"LEFT", @"RIGHT", @"FORWARD"]}
        ]},
        @{ OneOfTheseWillBeSaidOnce : @[@"EXECUTE", @"DO IT"]},
        @{ ThisCanBeSaidOnce : @[@"THANK YOU"]}
    ]
};
 
 or in Swift 3:
 
 
 let grammar = [
    ThisWillBeSaidOnce : [
 [ OneOfTheseCanBeSaidOnce : ["HELLO COMPUTER", "GREETINGS ROBOT"]],
 [ OneOfTheseWillBeSaidOnce : ["DO THE FOLLOWING", "INSTRUCTION"]],
 [ OneOfTheseWillBeSaidOnce : ["GO", "MOVE"]],
 [ThisWillBeSaidOnce : [
 [ OneOfTheseWillBeSaidOnce : ["10", "20","30"]], 
 [ OneOfTheseWillBeSaidOnce : ["LEFT", "RIGHT", "FORWARD"]]
 ]],
 [ ThisCanBeSaidOnce : ["THANK YOU"]]
    ]
 ]

Breaking it down step by step to explain exactly what the contents mean:

@{
    ThisWillBeSaidOnce : @[ // This means that a valid utterance for this ruleset will obey all of the following rules in sequence in a single complete utterance:
            @{ OneOfTheseCanBeSaidOnce : @[@"HELLO COMPUTER", @"GREETINGS ROBOT"]}, // At the beginning of the utterance there is an optional statement. The optional statement can be either "HELLO COMPUTER" or "GREETINGS ROBOT" or it can be omitted.
            @{ OneOfTheseWillBeSaidOnce : @[@"DO THE FOLLOWING", @"INSTRUCTION"]}, // Next, an utterance will have exactly one of the following required statements: "DO THE FOLLOWING" or "INSTRUCTION".
            @{ OneOfTheseWillBeSaidOnce : @[@"GO", @"MOVE"]}, // Next, an utterance will have exactly one of the following required statements: "GO" or "MOVE"
            @{ThisWillBeSaidWithOptionalRepetitions : @[ // Next, an utterance will have a minimum of one statement of the following nested instructions, but can also accept multiple valid versions of the nested instructions:
                @{ OneOfTheseWillBeSaidOnce : @[@"10", @"20",@"30"]}, // Exactly one utterance of either the number "10", "20" or "30",
                @{ OneOfTheseWillBeSaidOnce : @[@"LEFT", @"RIGHT", @"FORWARD"]} // Followed by exactly one utterance of either the word "LEFT", "RIGHT", or "FORWARD".
            ]},
        @{ OneOfTheseWillBeSaidOnce : @[@"EXECUTE", @"DO IT"]}, // Next, an utterance must contain either the word "EXECUTE" or the phrase "DO IT",
        @{ ThisCanBeSaidOnce : @[@"THANK YOU"]} and there can be an optional single statement of the phrase "THANK YOU" at the end.
    ]
};

So as examples, here are some sentences that this ruleset will report as hypotheses from user utterances:

"HELLO COMPUTER DO THE FOLLOWING GO 20 LEFT FORWARD EXECUTE THANK YOU"
"GREETINGS ROBOT DO THE FOLLOWING MOVE 10 FORWARD DO IT"
"INSTRUCTION 20 LEFT LEFT EXECUTE"

But it will not report hypotheses for sentences such as the following which are not allowed by the rules:

"HELLO COMPUTER HELLO COMPUTER"
"MOVE 10"
"GO RIGHT"

Since you as the developer are the designer of the ruleset, you can extract the behavioral triggers from your app from hypotheses which observe your rules.

The words and phrases in languageModelArray must be written with capital letters exclusively, for instance "word" must appear in the array as "WORD".

The last two arguments of the method work identically to the equivalent language model method. The withFilesNamed: argument takes an NSString which is the naming you would like for the files output by this method. Please give your grammars unique names within your session if you want to switch between them, so there is no danger of the engine getting confused between new and old grammars and dictionaries at the time of switching. The argument acousticModelPath takes the path to the relevant acoustic model.

If this method is successful it will return nil. If it returns nil, you can use the methods pathToSuccessfullyGeneratedDictionaryWithRequestedName: and pathToSuccessfullyGeneratedRuleORamaRulesetWithRequestedName: to get your paths to your newly-generated RuleORama fast grammar and dictionaries for use with OEPocketsphinxController or RapidEars. If it doesn't return nil, it will return an error which you can check for debugging purposes. Keep in mind when passing a RuleORama ruleset to OEPocketsphinxController than languageModelISJSGF should be set to FALSE or NO since RuleORama is not a JSGF format.

- (NSError *) generateFastSentenceRuleFromString: (NSString *)  sentenceString
withFilesNamed: (NSString *)  fileName
forAcousticModelAtPath: (NSString *)  acousticModelPath 

Swift 3:

generateFastSentenceRule(from: String!, withFilesNamed: String!, forAcousticModelAtPath: String!)

This is a convenience method for when you want to create a single sentence where the rule is that it is recognized progressively until it is complete, for instance if you are having the user read out a line from a book and want it to emerge in real time as they say it. The advantage to doing this with RuleORama versus a default language model is that RuleORama will not reorder the sentence words.

If this method is successful it will return nil. If it returns nil, you can use the methods pathToSuccessfullyGeneratedDictionaryWithRequestedName: and pathToSuccessfullyGeneratedRuleORamaRulesetWithRequestedName: to get your paths to your newly-generated RuleORama fast grammar and dictionaries for use with OEPocketsphinxController or RapidEars. If it doesn't return nil, it will return an error which you can check for debugging purposes. Keep in mind when passing a RuleORama ruleset to OEPocketsphinxController than languageModelISJSGF should be set to FALSE or NO since RuleORama is not a JSGF format.

- (NSString *) pathToSuccessfullyGeneratedRuleORamaRulesetWithRequestedName: (NSString *)  name

Swift 3:

pathToSuccessfullyGeneratedRuleORamaRuleset(withRequestedName: String!)

The method returns the path you can pass to RapidEars or to OEPocketsphinxController's startListening method. Keep in mind that with RuleORama's format, OEPocketsphinxController's parameter languageModelIsJSGF should be set to FALSE or NO. RapidEars does not have to be set to not use JSGF. Swift 3 pathToSuccessfullyGeneratedRuleORamaRuleset(withRequestedName: String!)

Property Documentation

- (BOOL) noWarningsAboutPureOptionalRulesets

Swift 3:

noWarningsAboutPureOptionalRulesets

Rule-O-Rama will warn you if you have rulesets with nothing in them but optionals, since this creates an overly-large and/or impossible logic for the grammar. If you get tired of hearing the warning and want to use those rulesets despite the mentioned problems, you can turn the warning off, but please don't report bugs that occur when you are receiving these warnings, supressed or otherwise.

- (BOOL) getAlternativePronunciations

Swift 3:

getAlternativePronunciations

Rule-O-Rama can look for every alternative pronunciation of a term, however, this can result is extremely large grammars and phonetic dictionaries, so it defaults to not doing so. If you are developing for a fast device and you want to receive more alternate pronunciations, set this to TRUE. This was previously "noAlternativePronunciations" and defaulted to FALSE but this was leading to slowdowns on older devices in cases where the developer didn't know this was a property so it is now an active option that has to be set.

ruleorama_icon_reflection

Demo is identical to licensed version, but times out after a few minutes. Changelog »

Download The RuleORama Demo
Go to the quickstart tutorial Buy RuleORama

OpenEars™ Plugins


RapidEars

RapidEars is a paid plugin for OpenEars™ that lets you perform live recognition on in-progress speech for times that you can't wait for the user to pause! Try out the RapidEars demo free of charge.

Rejecto

Rejecto is a paid plugin for OpenEars™ that improves accuracy and UX by letting OpenEars™ ignore utterances of words that aren't in its vocabulary. Try out the Rejecto demo free of charge.

RuleORama

Did you know that the free version of OpenEars™ can perform recognition of fixed phrases using rules-based grammars? And RuleORama is a paid plugin that lets you use the same grammar format as stock OpenEars™, but the grammars are fast enough to work with RapidEars. Try out the RuleORama demo free of charge.

NeatSpeech

NeatSpeech is a plugin for OpenEars™ that lets it do fast, high-quality offline speech synthesis which is compatible with iOS6.1, and even lets you edit the pronunciations of words! Try out the NeatSpeech demo free of charge.


Learn more about the OpenEars™ Plugin Platform

 

Help with OpenEars™

There is free public support for OpenEars™ in the OpenEars Forums, and you can also purchase private email support at the Politepix Shop. Most OpenEars™ questions are answered in the OpenEars support FAQ.