There is a relief valve and a stop valve in the oil pump,
The relief valve regulates the oil pressure at RPM and temp, the Stop valve shuts off oil flow when the engine stops, I suspect the stop valve may be stuck partialy open.
Personally I would strip the pump and check, since a failure here would be catastophic.