Bear in mind that it will be difficult or impossible to keep the press-fit metal insert from turning when the drill bit contacts it in trying to drill it out. Nor can the metal insert be tapped out if it continues to turn in the plastic housing when any torque is put upon it.
Unless there is something I may have missed here, my previous description and assessment of the problem stands to reason.