If the issue periodically goes away, I would be suspicious of a valved system connected to the inlet manifold; having a sticky valve. The EVAP, PCV, and brake booster all have one-way valves to separate those systems from the inlet manifold pressure; when boost is present. Those valves are open and allow those systems to utilize the vacuum, when in vacuum. But those valves will be shut, by the pressure of boost, to isolate boost pressure (when present) to the manifold. Perhaps, from how you described it, a valve is hesitating to shut. And when it finally does, it goes back to normal. Someone doing the upcoming smoke test, if they are familiar with it all, should be able to thoroughly test it for you.
