How to create a minimal case for replication

Home Forums OpenEars How to create a minimal case for replication

Tagged: 

Viewing 1 post (of 1 total)

  • Author
    Posts
  • #1024364

    Halle Winkler
    Politepix

    If you have an issue which doesn’t replicate for me and you’d like me to investigate it further, the next step is to create a minimal case for replication. Here is how to do it:

    1. Open the OpenEars sample app (it is found in the folder OpenEarsSampleApp in the main distribution) and make the smallest number of changes to its view controller necessary in order to cause your issue to happen for you. All changes should be made as text, so if you need to show something happening with a button, add the button programmatically (http://stackoverflow.com/questions/1378765/how-do-i-create-a-basic-uibutton-programmatically) so that all changes are only made to the main view controller of the sample app. Changes here must be minimal or it isn’t a simple replication case and may not be possible to take as a support case, due to the reality that issues can be interactions between other complex areas of app architectures unrelated to this framework, which would require the kind of line-by-line app debugging that it unfortunately isn’t possible to offer as support.

    Note: it’s absolutely fine to add the plugin frameworks to this test case in order to show me something happening with a plugin – just add the relevant plugin code to the view controller, and when you share your changes with me, I will add the plugin frameworks to my own local version for replication. Please make sure to use the demo versions of the plugins even if you have registered versions, so you don’t have to make any naming changes to the sample app.

    2. If the issue involves speech recognition, get the SaveThatWave demo here: https://www.politepix.com/savethatwave and install it, and use it to capture the audio from a session in which the issue occurs for you using its method startSessionDebugRecord which records the complete usage session. It’s allowed to use the demo for this purpose (its 3 minute timeout should be sufficient for capturing replicable audio) so you don’t need to purchase it just to create a replication example for me. The tutorial has a SaveThatWave section to walk you through framework integration quickly and easily (skip the parts about OEPocketsphinxController, OELanguageModelGenerator, and OEEventsObserver): https://www.politepix.com/openears/tutorial/. It is very important to make sure you give me the result of the startSessionDebugRecord method, since it is one WAV recording of your entire usage session, rather than using SaveThatWave’s ‘start’ method and giving me several individual speech utterance recordings which lack the non-speech audio from the session, because I can’t replicate an issue you are experiencing using the recorded speech audio alone. Thanks!

    3. If the issue involves speech recognition, add the captured WAV to the sample app project and to the sample app’s OEPocketsphinxController’s property pathToTestFile (more on pathToTestFile can be found in the docs but there is also a commented-out example of using it right in the sample app view controller).

    4. Remove any SaveThatWave and the SaveThatWave debug-related code from your replication case, unless the replication case you are reporting is an issue with SaveThatWave or in some way requires an interaction with SaveThatWave in order to demonstrate.

    5. Very important: run your now-completed replication case and verify that it definitely replicates the issue you are reporting. If it does, you are ready to share it.

    6. Share with me the following here in your original question topic in the forums:

    • Your minimal changes to the view controller of the sample app as pasted plain text in your topic (no links to complete apps please – just the changes for the view controller of the sample app),
    • If your issue involves speech recognition, your WAV file that replicates the issue when added to pathToTestFile (put the audio file up somewhere for me to download and share the link with me in your forum topic – a link to DropBox or your own site is fine)
    • Your logging which includes OELogging (and verbosePocketsphinx in the case that it’s about speech recognition). This logging must be the output from the final replication case where you verified that the case replicates your issue and you’ve removed SaveThatWave. i.e. it needs to match the circumstances under which I will run the case.
    • If replication involves any physical interaction steps (i.e. “first press button one, wait until the word “GO” is recognized, then press button two”) please give me a very clear description of the steps,
    • Your exact hardware device and iOS version on which it replicated. No replication cases showing behavior on the Simulator are accepted.

    The reason to do this process is threefold:

    1) I can see exactly what you’re seeing without any differences of accent, environment, devices, app architecture and/or interaction with other frameworks you may be using, etc.

    2) We have a simple shared baseline for comparing progress on the issue without the same variables going in the opposite direction (i.e. a fix works for me but not for you because of even more differences in the items above).

    3) The issue can then be added to the testbed to prevent a regression.

    This is the fastest and most effective possible way to get bugs looked at and fixed for your app, or to find out whether it is due to something about the implementation or something else about the app that isn’t related.

    Thank you!

Viewing 1 post (of 1 total)
  • The topic ‘How to create a minimal case for replication’ is closed to new replies.