bottom line? There are always install horror stories for every update. Usually they happen because people forget to repair the disks or whatnot, but for the most part installations go well, and as a result people don't post about those positive experiences.
In general it's just a better idea to have your system completely up to date, so I'd do the whole 10.2.4 thing. (fwiw, I had no problems upgrading from 10.1.5 to 10.2 and then through each of the releases to 10.2.4)