- This topic has 8 replies, 2 voices, and was last updated 9 years, 9 months ago by Halle Winkler.
June 5, 2013 at 3:55 am #1017381
PocketSphinxController provides a useful function called
runRecognitionOnWavFileAtPath:. I would like to execute this as a part of my
Kiwi test suite.
I have a unit test target set up called “MyProjectTests” but it cannot seem to find any headers associated with the framework, nor can the OpenEars.framework be added to the list of frameworks and libraries to link with in
Build Phrasesfor that target.
How may I accomplish this?
HokJune 5, 2013 at 10:41 am #1017395
OpenEars.framework is a totally normal compiled framework with public headers (but it is not a static library project), so I would just continue troubleshooting your issue of not being able to find the headers. Maybe you need to add the headers to Header Search Paths, or maybe you are trying to integrate the framework project as if it were a static library project instead of integrating the compiled framework as if it were something like AVFoundation.framework.June 5, 2013 at 3:39 pm #1017398
Thanks for your feedback; I managed to re-organise my test target completely and added the OpenEars and Slt frameworks to the project separately. Now it’s possible to reference OpenEars in test targets.
HokJune 5, 2013 at 3:42 pm #1017399
That is great news, I’m glad it’s working for you.June 5, 2013 at 7:05 pm #1017407
Another question for you, Halle.
It would be very useful to be able to write automatic tests that verify any one particular language model recognises pre-recorded segments of speech. However, the only function that is able to perform this task,
runRecognitionOnWavFileAtPath:, DOES NOT RETURN A HYPOTHESIS!
In order for a unit test to verify that a wav file is correctly interpreted as a the expected sentence, the only way I see it being possible is to capture NSLog into a file first (see http://parmanoir.com/Redirecting_NSLog_to_a_file) and then opening that file to make assertions afterwards.
The problem with that approach is that while I can stomach the horribleness, XCode relies on NSLog to be written to the console to provide accurate test reporting so even redirecting NSLog is not an attractive option.
Please return a hypothesis or something so that automated testing can be done. Thanks!June 5, 2013 at 9:03 pm #1017412
Why can’t you use the delegate method where the hypothesis is returned? How do you test other results that are returned in delegate methods?June 6, 2013 at 9:03 am #1017416
Okay, so it occurred to me that the
OpenEarsEventsObserverclass registers itself for
NSNotificationson the main thread when it’s initialised, which is how it hears about events. This is asynchronous whereas unit test execution is just synchronous.
The solution was to use so that we assert that the value _eventually_ becomes the one we are looking for. The timeout needs to be set to at least 1s, so that the delegate has a chance to receive the message: in the delegate method I assign onto a property that I’m asserting over.
You can still use
runRecognitionOnWavFileAtPath:: the documentation claims that it’s ‘synchronous’, which may have been the thing that misled me.
So good stuff that that’s working.
Halle, I noticed that OpenEars can’t be tested in a Logic Test with this method (see the definitions of Application and Logic tests here.
Is there a way to make OpenEars recognition work without having the application host our tests (i.e. setting
TEST_HOSTto the application binary)? In principle I think checking whether a WAV test resource is recognised correctly is pure enough to be a ‘Logic’ test rather than an ‘Application’ test.
I’ll be happy to provide some example code if it helps you debug this.
Cheers.June 6, 2013 at 9:09 am #1017417
Nope, not currently possible. It isn’t a bug but I’ll enter it as a feature request.June 6, 2013 at 9:27 am #1017418
BTW, please be so kind as to read these posts on the pocketsphinx board about what you can expect when enabling an automated WAV test with pocketsphinx and what you need to take into account in order to get deterministic results, so we can avoid the non-constructive tone that this request for assistance took on:
- You must be logged in to reply to this topic.