Joomla website weird problem of wrong URLs

I have one joomla site which was working fine for last 2 years. Suddenly I noticed that the most of the links on my site are weird. Some of the links content actual IP address of the website, some link contained ftp user name ( xx.com/~xx/index.php). I was sure that nothing has been changed no the website and had no clue why its happening.

Upon careful observation, I noticed that someone accessed the website using adding IP address and user name (~xx) as context root (it was same as website name..so he must have guessed it). And since the joomla caching was enabled, all the pages were cached that contained these links. any new hits were served using this pages and even newly cached pages were following same context root !

Once the problem was identified, solution was simple. Simply clear the cache/ (or disable it for time being).