I used to have a 1994 XR650L until a few years ago, and it was a very good bike. I downsized to the KLX250S a few years ago, as I wanted the lighter weight for riding trails and such. The 650 was a much better on road bike for me, but I have plenty of those. As such I traded the highway/interstate ability for a much lighter/easier to ride smaller bike offroad.
Its really going to depend on what you want from the bike. Mostly road riding with the occasional trail, the 650's will serve you well. But if you find yourself using it off road mostly, you will want a lighter bike.
Regards,
R