Always getting null hypothesis

[sudohalt]

Hello I have followed the 5-minute tutorial, but have been unsuccessful in getting the hypothesis to display. The app begins when the user clicks on the listen button, and from there is starts to listen. I have turned on debugging, and added a counter to count all the times speech was heard. Below is the logging info, and below that is the code.

2015-02-27 02:05:31.449 Stutr[20214:1643986] Starting OpenEars logging for OpenEars version 2.03 on 64-bit device (or build): iPhone Simulator running iOS version: 8.100000
2015-02-27 02:05:31.449 Stutr[20214:1643986] Creating shared instance of OEPocketsphinxController
2015-02-27 02:05:31.451 Stutr[20214:1643986] Starting dynamic language model generation

2015-02-27 02:05:31.459 Stutr[20214:1643986] Done creating language model with CMUCLMTK in 0.008354 seconds.
2015-02-27 02:05:31.478 Stutr[20214:1643986] I'm done running performDictionaryLookup and it took 0.015163 seconds
2015-02-27 02:05:31.478 Stutr[20214:1643986] I'm done running dynamic language model generation and it took 0.028175 seconds
2015-02-27 02:05:32.488 Stutr[20214:1643986] Attempting to start listening session from startListeningWithLanguageModelAtPath:
2015-02-27 02:05:32.488 Stutr[20214:1643986] User gave mic permission for this app.
2015-02-27 02:05:32.488 Stutr[20214:1643986] setSecondsOfSilence wasn't set, using default of 0.700000.
2015-02-27 02:05:32.489 Stutr[20214:1643986] Successfully started listening session from startListeningWithLanguageModelAtPath:
2015-02-27 02:05:32.489 Stutr[20214:1644045] Starting listening.
2015-02-27 02:05:32.489 Stutr[20214:1644045] about to set up audio session
2015-02-27 02:05:32.528 Stutr[20214:1644045] done starting audio unit
2015-02-27 02:05:32.541 Stutr[20214:1644045] Restoring SmartCMN value of 26.988281
2015-02-27 02:05:32.541 Stutr[20214:1644045] Listening.
2015-02-27 02:05:32.541 Stutr[20214:1644045] Project has these words or phrases in its dictionary:
2015-02-27 02:05:32.542 Stutr[20214:1644045] Recognition loop has started
2015-02-27 02:05:32.542 Stutr[20214:1643986] Pocketsphinx is now listening.
2015-02-27 02:05:32.922 Stutr[20214:1644045] Speech detected...
2015-02-27 02:05:32.922 Stutr[20214:1643986] Pocketsphinx has detected speech.
2015-02-27 02:05:32.922 Stutr[20214:1643986] 0
2015-02-27 02:05:34.722 Stutr[20214:1644045] End of speech detected...
2015-02-27 02:05:34.722 Stutr[20214:1643986] Pocketsphinx has detected a period of silence, concluding an utterance.
2015-02-27 02:05:34.735 Stutr[20214:1644045] Pocketsphinx heard "" with a score of (1) and an utterance ID of 0.
2015-02-27 02:05:34.735 Stutr[20214:1644045] Hypothesis was null so we aren't returning it. If you want null hypotheses to also be returned, set OEPocketsphinxController's property returnNullHypotheses to TRUE before starting OEPocketsphinxController.
2015-02-27 02:05:34.856 Stutr[20214:1644045] Speech detected...
2015-02-27 02:05:34.856 Stutr[20214:1643986] Pocketsphinx has detected speech.
2015-02-27 02:05:34.857 Stutr[20214:1643986] 1
2015-02-27 02:05:38.567 Stutr[20214:1644045] End of speech detected...
2015-02-27 02:05:38.567 Stutr[20214:1643986] Pocketsphinx has detected a period of silence, concluding an utterance.
2015-02-27 02:05:38.591 Stutr[20214:1644045] Pocketsphinx heard "" with a score of (0) and an utterance ID of 1.
2015-02-27 02:05:38.591 Stutr[20214:1644045] Hypothesis was null so we aren't returning it. If you want null hypotheses to also be returned, set OEPocketsphinxController's property returnNullHypotheses to TRUE before starting OEPocketsphinxController.
2015-02-27 02:05:39.591 Stutr[20214:1644045] Speech detected...
2015-02-27 02:05:39.591 Stutr[20214:1643986] Pocketsphinx has detected speech.
2015-02-27 02:05:39.591 Stutr[20214:1643986] 2
2015-02-27 02:05:40.740 Stutr[20214:1644045] End of speech detected...
2015-02-27 02:05:40.741 Stutr[20214:1643986] Pocketsphinx has detected a period of silence, concluding an utterance.
2015-02-27 02:05:40.748 Stutr[20214:1644045] Pocketsphinx heard "" with a score of (0) and an utterance ID of 2.
2015-02-27 02:05:40.748 Stutr[20214:1644045] Hypothesis was null so we aren't returning it. If you want null hypotheses to also be returned, set OEPocketsphinxController's property returnNullHypotheses to TRUE before starting OEPocketsphinxController.
2015-02-27 02:05:42.272 Stutr[20214:1644045] Speech detected...
2015-02-27 02:05:42.272 Stutr[20214:1643986] Pocketsphinx has detected speech.
2015-02-27 02:05:42.272 Stutr[20214:1643986] 3
2015-02-27 02:05:43.679 Stutr[20214:1644045] End of speech detected...
2015-02-27 02:05:43.679 Stutr[20214:1643986] Pocketsphinx has detected a period of silence, concluding an utterance.
2015-02-27 02:05:43.688 Stutr[20214:1644045] Pocketsphinx heard "" with a score of (1) and an utterance ID of 3.
2015-02-27 02:05:43.688 Stutr[20214:1644045] Hypothesis was null so we aren't returning it. If you want null hypotheses to also be returned, set OEPocketsphinxController's property returnNullHypotheses to TRUE before starting OEPocketsphinxController.
2015-02-27 02:05:45.479 Stutr[20214:1644045] Speech detected...
2015-02-27 02:05:45.479 Stutr[20214:1643986] Pocketsphinx has detected speech.
2015-02-27 02:05:45.479 Stutr[20214:1643986] 4
2015-02-27 02:05:45.783 Stutr[20214:1643986] Stopping listening.
2015-02-27 02:05:45.798 Stutr[20214:1643986] No longer listening.
2015-02-27 02:05:45.798 Stutr[20214:1643986] Pocketsphinx has stopped listening.

//
//  FirstViewController.m
//  Stutr
//
//  Created by Umayah Abdennabi on 2/19/15.
//  Copyright (c) 2015 stutter. All rights reserved.
//

#import "FirstViewController.h"
#import <OpenEars/OELanguageModelGenerator.h>
#import <OpenEars/OEAcousticModel.h>
#import <OpenEars/OEPocketsphinxController.h>
#import <OpenEars/OEEventsObserver.h>
#import <OpenEars/OELogging.h>;

@interface FirstViewController()

@property BOOL isListening;                                                     // Used to indicate whether the application is currently listening to your speech
@property (weak, nonatomic) IBOutlet UIButton *button;                          // Button to begin listening or to turn off listening
@property (weak, nonatomic) IBOutlet UILabel *stutterCountLabel;                // Counts the amount of times stuttered during talk
@property (weak, nonatomic) IBOutlet UILabel *pauseCountLabel;                  // Counts the amount of long pauses
@property (weak, nonatomic) IBOutlet UILabel *repetitionCountLabel;             // Count the amount of times you repeated a word
@property (weak, nonatomic) IBOutlet UILabel *statusLabel;                      // Lights up when you stutter, pause, or repeat
@property (weak, nonatomic) NSString *prevString;                               // Previous string uttered, this is used to check for repetition
@property (strong, nonatomic) OEEventsObserver *openEarsEventsObserver;
@property (strong, nonatomic) OELanguageModelGenerator *lmGenerator;
@property (strong, nonatomic) NSString *lmPath;
@property (strong, nonatomic) NSString *dicPath;
@property (strong, nonatomic) NSArray *words;
@property (strong, nonatomic) NSString *name;
@property (strong, nonatomic) NSError *err;

@property (nonatomic, assign) int restartAttemptsDueToPermissionRequests;
@property (nonatomic, assign) BOOL startupFailedDueToLackOfPermissions;
@property int count;
@end

@implementation FirstViewController

- (void)viewDidLoad {
    [super viewDidLoad];
    [OELogging startOpenEarsLogging];
    [[OEPocketsphinxController sharedInstance] verbosePocketSphinx]; // verbosePocketsphinx = TRUE;
    [[OEPocketsphinxController sharedInstance] returnNullHypotheses];
    self.isListening = false;
    self.openEarsEventsObserver = [[OEEventsObserver alloc] init];              // Create an event observer which constantly keeps us updated about the
                                                                                // status of your listening session
    self.openEarsEventsObserver.delegate = self;                                // The even observer works via delegates
    self.lmGenerator =                                                          // Create a offline langauage model
        [[OELanguageModelGenerator alloc] init];
    self.words = @[@"HELLO", @"WORLD", @"FORWARD", @"BACKWARD"];       // Array of words that our model will understand
    self.name = @"MyModelFile";
    self.err = [self.lmGenerator generateLanguageModelFromArray:self.words
                                            withFilesNamed:self.name
                                    forAcousticModelAtPath:[OEAcousticModel pathToModel:@"AcousticModelEnglish"]];
    self.lmPath = nil;
    self.dicPath = nil;
    if (self.err == nil) {                                                      // If successfully we use the generated language model with the event observer
        self.lmPath =                                                         // to detect speech
            [self.lmGenerator pathToSuccessfullyGeneratedLanguageModelWithRequestedName:self.name];
        self.dicPath =
            [self.lmGenerator pathToSuccessfullyGeneratedLanguageModelWithRequestedName:self.name];
    } else {
        NSLog(@"Error: %@", [self.err localizedDescription]);
    }
}

- (void)viewWillDisappear:(BOOL)animated {
    // Make sure to turn of listening when switching views
    self.isListening = false;
    self.statusLabel.backgroundColor = [UIColor whiteColor];
    NSError *error = [[OEPocketsphinxController sharedInstance] stopListening];
    if(error)NSLog(@"Error stopping listening in stopButtonAction: %@", error);
}

- (void)didReceiveMemoryWarning {
    [super didReceiveMemoryWarning];
    // Dispose of any resources that can be recreated.
}

- (IBAction)buttonListenter:(id)sender {
    if (self.isListening) {
        self.isListening = false;
        self.statusLabel.backgroundColor = [UIColor whiteColor];
        NSError *
        error = [[OEPocketsphinxController sharedInstance] stopListening];
        if(error)NSLog(@"Error stopping listening in stopButtonAction: %@", error);
        [self.button setTitle:@"Listen" forState:nil];
    } else {
        self.isListening = true;
        [self.button setTitle:@"Listening" forState:nil];
        self.statusLabel.backgroundColor = [UIColor grayColor];
        [[OEPocketsphinxController sharedInstance] setActive:true error:nil];
        [[OEPocketsphinxController sharedInstance]
         startListeningWithLanguageModelAtPath:self.lmPath
         dictionaryAtPath:self.dicPath acousticModelAtPath:
         [OEAcousticModel pathToModel:@"AcousticModelEnglish"] languageModelIsJSGF:FALSE];
    }
}

- (void) pocketsphinxDidReceiveHypothesis:(NSString *)hypothesis
                         recognitionScore:(NSString *)recognitionScore
                              utteranceID:(NSString *)utteranceID {
    NSLog(@"The received hypothesis is %@ with a score of %@ and an ID of %@",
          hypothesis, recognitionScore, utteranceID);
    self.statusLabel.text = [NSString stringWithFormat:@"\"%@\"", hypothesis];
    if ([self.words containsObject:hypothesis]) {
        if ([hypothesis  isEqualToString: @"UM"])self.stutterCountLabel = 0;
        else if ([hypothesis isEqualToString:self.prevString])
        self.repetitionCountLabel.text = 0;
    }
}

- (void) micPermissionCheckCompleted:(BOOL)result {
    if(result) {
        self.restartAttemptsDueToPermissionRequests++;
        if(self.restartAttemptsDueToPermissionRequests == 1 && self.startupFailedDueToLackOfPermissions) { // If we get here because there was an attempt to start which failed due to lack of permissions, and now permissions have been requested and they returned true, we restart exactly once with the new permissions.
            NSError *error = nil;
            if([OEPocketsphinxController sharedInstance].isListening){
                error = [[OEPocketsphinxController sharedInstance] stopListening]; // Stop listening if we are listening.
                if(error) NSLog(@"Error while stopping listening in micPermissionCheckCompleted: %@", error);
            }
            if(!error && ![OEPocketsphinxController sharedInstance].isListening) { // If there was no error and we aren't listening, start listening.
                [[OEPocketsphinxController sharedInstance] startListeningWithLanguageModelAtPath:self.lmPath dictionaryAtPath:self.dicPath acousticModelAtPath:[OEAcousticModel pathToModel:@"AcousticModelEnglish"] languageModelIsJSGF:FALSE]; // Start speech recognition.
                self.startupFailedDueToLackOfPermissions = FALSE;
            }
        }
    }
}

- (void) pocketsphinxDidStartListening {
    NSLog(@"Pocketsphinx is now listening.");
}

- (void) pocketsphinxDidDetectSpeech {
    NSLog(@"Pocketsphinx has detected speech.");
    self.statusLabel.text = [NSString stringWithFormat:@"%d", self.count];
    NSLog(@"%d", self.count++);
}

- (void) pocketsphinxDidDetectFinishedSpeech {
    NSLog(@"Pocketsphinx has detected a period of silence, concluding an utterance.");
}

- (void) pocketsphinxDidStopListening {
    NSLog(@"Pocketsphinx has stopped listening.");
}

- (void) pocketsphinxDidSuspendRecognition {
    NSLog(@"Pocketsphinx has suspended recognition.");
}

- (void) pocketsphinxDidResumeRecognition {
    NSLog(@"Pocketsphinx has resumed recognition.");
}

- (void) pocketsphinxDidChangeLanguageModelToFile:(NSString *)newLanguageModelPathAsString
                                    andDictionary:(NSString *)newDictionaryPathAsString {
    NSLog(@"Pocketsphinx is now using the following language model: \n%@ and \
          the following dictionary: %@",newLanguageModelPathAsString,
          newDictionaryPathAsString);
}

- (void) pocketSphinxContinuousSetupDidFailWithReason:(NSString *)reasonForFailure {
    NSLog(@"Listening setup wasn't successful and returned the failure reason: \
          %@", reasonForFailure);
}

- (void) pocketSphinxContinuousTeardownDidFailWithReason:
    (NSString *)reasonForFailure {
    NSLog(@"Listening teardown wasn't successful and returned the failure \
          reason: %@", reasonForFailure);
}

- (void) testRecognitionCompleted {
    NSLog(@"A test file that was submitted for recognition is now complete.");
}

@end

[Halle Winkler]

Welcome,

You can’t test accuracy on the Simulator, sorry. Try it with a real device and make sure that the words being spoken are present in your language model (I noticed that you check for the word “UM” but it isn’t present in your model).

Also, there will be some or possibly many null hypotheses due to noises, so there is no advantage to returning null hypotheses and/or worrying about getting null hypotheses – the important question is only whether matching speech is not detected. I would not test things out with null hypotheses being returned, since getting lots of correct negatives is likely to be confusing if what you are looking for is false negatives.

[sudohalt]

Hi, my old model had the word “UM”, but I simplified the dictionary before posting the post. The thing is that the open ears demo application works fine on the simulator, and I don’t see what it is doing differently. My application is based on the 5 minute tutorial. I have gotten the demo application to correctly work, but have never gotten my application to work.

[Halle Winkler]

That’s fine, but in order to troubleshoot it more we need to look at a setup where normal recognition is expected to work, while the simulator is a setup where normal recognition is not expected to work. Can you start with taking either the tutorial output or the sample app and just replacing the vocabulary with your own vocabulary, verifying that it works, and then one step at a time adding your customizations? That should point up any issue very quickly. Make sure not to set OEPocketsphinxController to return null hyps since the important part is only whether your app-oriented speech is not being heard, rather than the presence of null hyps in and of themselves, which is expected.

[Author]

Posts