You'll get no lubrication with just water, so you will inflict marring. A good shampoo will also clean more efficiently/effectively too so yes, you will notice a difference between using a shampoo and just water, but you might also notice differences between shampoos too; some clean better, some lubricate better, some add gloss enhancers, some add protection, some gives lots of suds, some just smell nice.
For me, the two main things a shampoo must do is firstly remove dirt effectively, and secondly be well lubricated whilst doing that - I like a shampoo that feels nice and slick on the paint over one that gives loads of suds in the bucket/on the panel. On the odd occasion I may also want something that leaves a decent gloss but most of the time I don't, as such my choice of shampoo is usually dictated by the LSP on the car. Which shampoo does what best though is quite subjective, as shown in this thread.
hmm where's this going