If you like your music, it's definitely worthwhile upgrading the speakers - and as already pointed out in previous post, change the crossovers as well with the ones that come with new speakers.
Up to you if you go 2 way or 3way - as also mentioned above, swapping the mids can be tight, but some do fit without need for modifying anything.
If you already have the 3 way, even upgrading to a half decent 2 way set up will drastically improve the sound! I know others who have simply fitted new speaker wire behind the door card, leaving the original wiring unused and in tact? Easy enough to do.
I would also recommend adding a little sound deadening around the bass driver as well, if you have the door card off anyway. This alone will greatly improve the sound