Yes cold engine, disappears when warmed up. Yes next Cam belt change, get the tensioner changed and that'll cure it.
Removing the dipstick will make it run lumpy, a faulty pcv causes lumpy idling , and if by pulling out the dipstick wouldn't make any difference
Your just de stabilising crankcase pressure by removing the filler cap or pulling the dipstick, which will upset the manor in which the engine stabilises it's idle.