Hi Liam,
It’s fine to ask questions here; they are useful for other potential askers who can then read the discussion. The vadThreshold property is for situations like this where you want a globally more- or less-sensitive noise threshold that you can switch between logically at runtime. The Sphinx VAD will also make some alterations to noise levels over the life of the session, that’s correct. What to do with vadThreshold is dependent on the nature of the “badness” with arms-length listening.
I always recommend first starting by turning off Rejecto (you’ll turn it back on later) and then set up a test and get the most-accurate result you can by tuning the vadThreshold. This is so that you don’t make attribution errors to results caused by Rejecto (i.e. thinking the threshold is too high and is omitting too much speech when actually it isn’t, but Rejecto is rejecting the speech farther down the line). This way you don’t need to take my advice but you can just see firsthand what works best.
Once you have a best-results vadThreshold it’s time to add Rejecto back in, paying special attention to its tweakable settings such as weight and ignoring phonemes and vowels-only.