BL5 can be integrated in a lot of ways with Non B&O hardware. More than that you can have them connected to different sources at the same time, like Powerlink to a Beovision and to an external streamer, room correcter, or whatever with SPDIF. You can also use the Line IN to get third party analog device to connect to it, which I don't recommend because the Lab5 will do a A/N conversion before N/A it in the internal DAC. That's a pity.
The old internal DAC is still excellent, even in 2019. It is limited to take signals at maximum 24/96, but who needs more ?
Regarding your question especially: No, you definitely don't need a sub. Regarding the ceiling speakers, there is the Celestial in the B&O portofolio ( https://www.bang-olufsen.com/fr/speakers/celestial ), which are passive speakers. You can also use any other speaker brand, but the thing is to get all the stuff synced with a specific hardware. B&O is not providing this AFAIK, even with a Beosystem 4, but maybe I am wrong being not expert of the capabilities that a Beosystem 4 has. Otherwise you will have to look after a PreAmp/Amp with such capabilities to feed the Beolab 5 with a SPDIF signal, and some passive speakers.
I have a room correcter Lyngdorf TDAI 2170 which has such capabilities, and you can adjust almost everything in user friendly way and in addition it has bi-amplification capabilities (initially with their Lyngdorf Power Amp to add, but in case of active speakers like Beolab5 I am pretty sure you can get rid of it). It's a danish brand, and the user experience with menus and calibration simplicity reminds me the B&O spirit. Some other non B&O amps can do that too of course, maybe your AV receiver has already this capability.