When the first Bond film, Dr. No, was in production, Ian Fleming singled Baker out to play Bond, but he had other commitments. He did, however, play characters in the Bond films The Spy Who Loved Me, On Her Majesty's Secret Service and You Only Live Twice.
In 1976, he played Tiberius in the BBC's I, Claudius. From 1987 until 2000, Baker starred as Inspector Wexford in ITV's The Ruth Rendell Mysteries, which was perhaps his best known role. He also played the New Number Two in the first episode of The Prisoner and Sir Richard of Leaford in Robin of Sherwood.