I used to have an E92 M3 and I've now got an F82 M4. The E92 M3 definitely sounds much better as you say(especially the induction noise for me), however because I use them as daily drivers, the M4 wipes the floor with the M3 for me. I'd probably consider the E9x M3 as a weekend car though I reckon.
Everything seems to count against the M3 as a daily to me. They weigh 1650kg, but only have 295 Ib/ft of torque, and max power is at 8300rpm as well. So unless you're absolutely thrashing the backside off them, they can feel quite unrewarding and frustrating to me. Plus the brakes are garbage in them too. Not to mention 17/18 mpg too!
The M4 is lighter/faster, and has far better brakes, plus it has 406 Ib/ft of torque. In my hands doing the same journeys it is also just over 10 mpg better on fuel as well. I'm not sure I'd want the F8x M3/4 as a weekend car though, because they arguably aren't special enough for that I reckon.