The thing is as soon as a sponge makes contact the dirt has nowhere to go being a flat surface, and stays between the surface and the paint. Rinsing after is too late.
With a mitt, being big and fluffy(mmmmm) the dirty has somewhere to go away from the paint, hopefully no swirls.
But hey thats just MHO
I tried your clay procedure btw, just seem to have a problem getting it off the wheel, and also have creepy images of Patrick Swayze
Bradders.