Score Array in RapidEars returns zero

Home Forums OpenEars plugins Score Array in RapidEars returns zero

Topic Resolution: Resolved
Viewing 16 posts - 1 through 16 (of 16 total)

  • Author
    Posts
  • #1026643

    billylo
    Participant

    I managed to retrieve the words through rapidEarsDidDetectLiveSpeechAsWordArray; but the associated scoreArray is always returning zeroes (the same number of zero elements in the word array.)

    OpenEars 2.0.4.1, RapidEars 2.0.4

    Am I missing some configuration steps in order to receive scores?

    thanks,

    #1026647

    Halle Winkler
    Politepix

    Welcome,

    Could you show me your code for reading your score array?

    #1026649

    billylo
    Participant
             NSNumber *scoreNumber = scores[i];
             int score = [scoreNumber intValue];

    I also used the debugger to view the scoreArray and it shows an array of zeroes.

    any hints?

    #1026653

    Halle Winkler
    Politepix

    Most likely this is a bug, but to confirm, are the hypotheses real words from your model/grammar or null hypotheses or rejecto tokens?

    #1026654

    billylo
    Participant

    yes. the hypothesis are the correct ones.

    While I have you, I am getting a lot of duplicated calls to the rapidEars delegate method for the same audio trigger (just one word). Sometimes, the delegate method is called 20+ times. I have set

        [[OEPocketsphinxController sharedInstance] setReturnDuplicatePartials:NO];
    

    In fact, after a while, RapidEars would stop picking up words. (this is probably a different thread.)

    #1026655

    Halle Winkler
    Politepix

    Hmm, that I haven’t seen in any testing and I’d be surprised if it were a bug. Would it be possible for you to create a replication case to rule out or locate any local issues in the app?

    https://www.politepix.com/forums/topic/how-to-create-a-minimal-case-for-replication/

    This is done by using the SaveThatWave demo to capture your speech input and then add it to the sample app with the minimal code to replicate the result.

    #1026683

    billylo
    Participant

    Hi, Here is a minimal test case that demonstrates my two issues (duplicated calls to delegate methods and zero scores). It’s basically the sampleApp modified to support RapidEars and these three lines added to initialization.

            [[OEPocketsphinxController sharedInstance] setFinalizeHypothesis:NO];
            [[OEPocketsphinxController sharedInstance] setReturnSegments:YES];
            [[OEPocketsphinxController sharedInstance] setReturnSegmentTimes:YES];
    

    and I used this to capture the score and start/end times.

    - (void) rapidEarsDidDetectLiveSpeechAsWordArray:		(NSArray *) 	words
                                          scoreArray:		(NSArray *) 	scores
                                      startTimeArray:       (NSArray *)     startTimes
                                        endTimeArray:       (NSArray *)     endTimes
    {
        NSNumber *scoreNumber, *startTime, *endTime;
        
        for (int i = 0; i< words.count; i++) {
            
            scoreNumber = scores[i];
            startTime = startTimes[i];
            endTime = endTimes[i];
            
            NSLog(@"word = %@, score = %.1f, startTime = %.1f, endTime = %.1f", words[i], [scoreNumber floatValue], [startTime floatValue], [endTime floatValue]);
            
        }
        
        //output score
    }

    The console output looks like this:

    2015-08-31 19:26:27.019 OpenEarsSampleApp[681:135806] Local callback: Pocketsphinx has detected speech.
    2015-08-31 19:26:29.141 OpenEarsSampleApp[681:135806] word = LEFT, score = 0.0, startTime = 1.9, endTime = 2.2
    2015-08-31 19:26:29.268 OpenEarsSampleApp[681:135806] word = LEFT, score = 0.0, startTime = 1.9, endTime = 2.3
    2015-08-31 19:26:29.404 OpenEarsSampleApp[681:135806] word = LEFT, score = 0.0, startTime = 1.9, endTime = 2.3
    2015-08-31 19:26:29.529 OpenEarsSampleApp[681:135806] word = LEFT, score = 0.0, startTime = 1.9, endTime = 2.3
    2015-08-31 19:26:29.657 OpenEarsSampleApp[681:135806] word = LEFT, score = 0.0, startTime = 1.9, endTime = 2.3
    2015-08-31 19:26:29.785 OpenEarsSampleApp[681:135806] Local callback: Pocketsphinx has detected a second of silence, concluding an utterance.
    2015-08-31 19:26:30.296 OpenEarsSampleApp[681:135806] Local callback: Pocketsphinx has detected speech.
    2015-08-31 19:26:30.432 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
    2015-08-31 19:26:30.552 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
    2015-08-31 19:26:30.684 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.3
    2015-08-31 19:26:30.814 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
    2015-08-31 19:26:30.936 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
    2015-08-31 19:26:31.066 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
    2015-08-31 19:26:31.197 OpenEarsSampleApp[681:135806] word = RIGHT, score = 0.0, startTime = 0.0, endTime = 0.2
    2015-08-31 19:

    thanks.

    #1026686

    Halle Winkler
    Politepix

    Thanks, I will enter this in the buglist and take a look at it this week.

    #1026687

    billylo
    Participant

    Great. I did have trouble getting the wavWasSavedAtLocation delegate method triggered. SaveThatWave worked, just the delegate method not triggered somehow. Could use your guidance there. Billy.

     - (void) wavWasSavedAtLocation:(NSString *)location {
        NSLog(@"WAV was saved at the path %@", location);
     }
    #1026688

    Halle Winkler
    Politepix

    startSessionDebugRecord doesn’t call the delegate since it records the entire session until it is complete.

    #1026690

    billylo
    Participant

    I see. Does [stop] actually stop it? I am thinking of adding a support feature to my app so that they can upload sample audio to the cloud if my app does not behave correctly.

    #1026691

    Halle Winkler
    Politepix

    startSessionDebugRecord isn’t intended for any purpose other than outputting a debug dump of the complete app session audio after the app has been terminated, for developer testing and test user testing only. It is not intended to record enduser app sessions.

    #1026764

    billylo
    Participant

    Any luck with tracking this one down, Halle?

    #1026769

    Halle Winkler
    Politepix

    Hi,

    Sorry, this might take some time to get to – there is a large update underway and it will most likely have to be fixed as part of it.

    #1027638

    Halle Winkler
    Politepix

    Hello,

    My apologies again for not being able to give you any feedback on this until now. I have run your case, confirmed the bug, and it will be fixed in the upcoming update. I appreciate your giving me a replication case so that I could locate the problem.

    #1027639

    Halle Winkler
    Politepix

    To clarify, there were two bugs – the scores were not returning with live segmented hypotheses, and live segmented hypotheses could return duplicate hypotheses without there being a request to return duplicates. The update is complete and in testing so it should be out in the first couple of weeks of the new year.

Viewing 16 posts - 1 through 16 (of 16 total)
  • You must be logged in to reply to this topic.