Hemmings had an affair with Marjory Kennedy, and she became pregnant with his child. When she was murdered by her boyfriend, Walter Seacombe, he blamed her for the baby's death and refused to give her Last Rites when she died.
Later, his guilt at refusing Marjory Last Rites consumed him. When he found aliens washed up on the beach, he thought they were fallen angels, and brought them back to his church so he could administer Last Rites to them. He didn't realise that they let off harmful radiation when they died.
Hemmings wouldn't listen when the Fifth Doctor tried to convince him to stop taking the dead aliens, but when Peri forgave him for refusing Marjory's Last Rites, he let the Doctor take the alien bodies away. (PROSE: The Canvey Angels)