You are right that the job of the intercooler is to cool the pressurised intake air/boost but you will also get a small amount of oil in the system from the turbo and the recirculated oil vapours from the crankcase/camcover pressure (in general, not a case with all engines) with it being a liquid this will always collect at the lowest point in the system at the intercooler union.
A new clip isnt a must, but its worth getting a seal for the sake of 32p. The oil is a leak, if oil can get out, so can boost pressure. I clean mine every time its having a service, but its not a concern, its 'normal' on most VAG turbo'd engines, especially diesels. An oil catch can/breather mods can help lower the amount of oil in the boost system. (look at the BSH catch can kit on
www.dubtek.co.uk)