I'm starting to think this is never going to get fully resolved... I'm also getting more & more convinced this is not a b&o but an Google issue.
Google seems be doing an extremely sh*t job with it's chromecast, even my nvidia shield is starting to have issue with chromecast now (until I reverted it back to the factory version of CC)... hell, since this year even my google chromecast audio devices, so 100% pure google devices, are having issue connecting (and I had many of them for my multiroom)
Luckily there is one extremely fine working solution, get an Ipad and stream using airplay 2, no hickups, never a failure to stop/start, perfect volume control, select what speakers should be playing your music if you have multiple airplay 2 speakers, all nicely in sync. For me personally this was a bit of a pain as I was quite anti-apple and all my devices are android but after 1 month now with the ipad I'm starting tho think I might actually replace my android phone too with an iphone.
Long story short, don't blame B&O for the failures of Google.