Is it not possible to pull out the outer race of the bearing and replace the complete bearing?
I just disassembled an M5 this weekend. The bearing in the volume control was just a standard closed bearing, which yours also seems to be (at least until it disintegrated..).