I had a worn patch a bit like this, but not quite as bad. I found some specialist colouring/repair stuff by googling around. It is apparently quite difficult to use - bet there is someone out there who has found the easy way though! Can't find the link to it but will post if I do.
So I bought some grey shoe polish from a shoe shop and tried that. It took 2 or 3 applications, leaving it overnight to dry, then buffing off so as not to get over my clothes. Prob not as good as the proper stuff but a big improvement though.