The odd time I do clay, I also use a proper lube, Dodo Juice, born slippy is my current choice.
If youre inflicting swirls with the clay it would suggest that the clean phase wasn't detailed enough, the clay itself cannot cause swirls, but it can inflict them if it gets dirty, ive seen plenty of people drop them on the floor, fold it over and carry on !