the last time i had the same problem, it was bad, till i open up the flap n seal it on my e34..
all models has this self ventilating flap that the air-con ecu will operate when your car is on or off, during off mode the flap opens up to dispell all hot air in the cabin pressure, hence the self heating timer work in colder countries.it also acts as freah air intake when u are moving.
sometimes, when u start or off the engine, you will hear some thing runing inside the dashboard. as for the flap, it's for the stupid parents that lock their children in the car , so no air to breathe,hence suffocation, bmw has this feature.
remember the police man and the woman that died in the e36 car ? the flap was open and it had a leak in the exhaust branch connection has what vllad has mentioned.
during long period of time, the motor worns out, dirt gets caught in the flap and opening apreature and air gets in.
solution ? remove the flap pin holder rod that hooks to the motor, clean and remove all dirt from the opening, secure it with silicone and duck tape. and it will be nice, but no fresh air intake.. when long distance, remember to wind down your window to let new air in..
regards.