Hah, yeah you're not supposed to kill bees
We had some kids destroy our neighbours 2 beehives (60,000 bees) once, and all the bees decide it'd be fun to have a party in our garden, right before we were going to have a BBQ...! It's quite unnerving seeing that many bees flying around. A pest control guy came out and smoked them for a while, then the one surviving queen flew off and the bees followed, God knows where they went - maybe your chimney