My instinct is that nothing should really be objecting to sample rate changes per se, since the whole deal with playing back a formatted file rather than a buffer stream is that the required data is encapsulated by the header so that a qualified player can deal with differences in file details such as sample rate, maybe bitrate, endianness, codec, whatever. This will also be true of other mediaplayer type objects such as videos. So I might be suspicious of other implementation details besides the sample rates of the files played.
I don’t really have insight into what this particular issue is and you might be 100% right that the most direct fix is to change the sample rate, I’m just sharing what my thought process on it would be if it were my implementation to debug.