He immediately recognised the Fifth Doctor. He thought the Doctor intended to stop the Xyz, and was surprised to learn that the Doctor was addicted to the programme. When the Doctor's addiction wore off, the Producer decided that the programme needed a villain, and cast an actor who resembled the Doctor, down to the cricket clothing.
When the Doctor and Nyssa stopped broadcast of the programme, the Producer and the rest of the Xyz sneaked off Earth before the humans had completely recovered. (PROSE: Not So Much a Programme, More a Way of Life)