Yep it's strange. I tried connecting a headset to the extractor and there was no sound. Might be that it's broken.
I had the 'no sound' problem when trying to use an Atlona AT-HD-M2C as an audio extractor. As in your case, the sound worked when the hdmi output went to other non-B&O TVs in the house, but not when feeding the output to my BV6 via DVI.
I chatted to Atlona helpline (which was very good, especially as I had bought the device second hand) and eventually they suggested it may be an EDID problem. They suggested adding an EDID emulator between the Atlona and the TV. This didn't work initially but then I tried moving the EDID emulator to a position between the Apple TV and the audio extractor (Atlona) - this worked!
In your situation I would probably try a different audio extractor first, but if you want to try an EDID emulator then this is the one that worked for me:
Location: Warwickshire, UK
My B&O Icons: