Reply To: RapidEars startRealtimeListeningWithLanguageModelAtPath memory leaks

Home Forums OpenEars plugins RapidEars startRealtimeListeningWithLanguageModelAtPath memory leaks Reply To: RapidEars startRealtimeListeningWithLanguageModelAtPath memory leaks

Halle Winkler


No, sorry, I don’t see that behavior when stopping, that is why I have asked for a replication case. I see the behavior I described at the beginning of the discussion, that both the buffer and the hypothesis search can grow to the size needed (which can get pretty large when there is a long utterance that has noise and the presence of words is unclear) and then is eventually released (the hypothesis search memory releases sometime after the search finishes, and the buffer memory releases sometime after stopping listening). It can take a fair amount of time before Instruments shows it as being released, and of course there are also other things happening in the sample app that can make their own use of memory (such as TTS and language model generation file caching).

I also don’t see leaks in Instruments (other than the very tiny leaks we discussed at the beginning of the discussion, adding up to less than 2k).

When I look at your Instruments example you sent, it doesn’t have leaks other than the tiny <2k leaks (leaks are orphaned memory). But it has live memory that continues to be live, at a time in which it can be expected to be possible to release, and it is a lot of memory. The memory usage is from a 3rd-pass search that doesn't complete successfully when you stop listening (this is shown in the log when you look at the log timing and compare it to the memory usage).

What is happening in that session is that the 3rd-pass search gets very large (probably due to intermittent background noise that builds up in a long utterance without any easily-found words) and it keeps searching even while the stopListening message is in progress and trying to shut down, and after too much time passes, the stopListening method has to stop attempting to release the search because it will lead to an exception. The reason that I've wanted to debug your RapidEars version install is that this behavior used to be possible with RapidEars during a big unclear 3rd-pass search, but this issue was fixed, so it is unexpected that it is happening in your install, and it is also not possible for me to replicate with the current versions of the frameworks using my own audio files or audio input.

It also seems technically impossible that this happens with 3rd-pass searches turned off, so it's a confusing issue. So these are the reasons I'm now hoping for a replication case from you where it replicates using an audio file from your environment. I would _really_ like to fix it if it is a current problem since I have already worked on this and thought it was fixed, but I can't cause it to happen in my own system. My local install demonstrates the fixes that were made, and also doesn't run 3rd-pass searches like the one shown in the Instruments file when I turn 3rd-pass searches off.

If this is an issue:

before it is 13MB there is an increase up to 70MB and falling down at 19Mb

It is a different issue – the one I’ve been trying to replicate is the one that was shown in your Instruments file you sent, where there is a far larger allocation with none of it ever being released when stopping listening fails (this is the old issue that I expected to be fixed). An intermittent usage of a large amount of memory is something that can happen briefly in many kinds of apps that have a temporary need for it without it being a problem, and the search data structures can temporarily get pretty big on a 64-bit platform, although this is something that occurs for a matter of seconds.

I’m happy to investigate a new issue like “after a large search is successfully released, there is still 6MB more memory used than expected at the time of restarting listening and it isn’t clear whether it is due to a new memory need or a bug”, but first we need to wrap up the reported issue in the file you sent, which is a huge allocation where none of it is released because stopListening is not successful, which is a different scale of problem.

Is it possible that the reported issue doesn’t replicate for you now because you have done some troubleshooting on your RapidEars install version and you are now linking to RapidEars 2.5, or do you still not see RapidEars 2.5 logging when you try to set up your replication case?