Toby Avery was the son of the pirate captain Henry Avery.
Toby Avery's mother brought him up to believe that his father was an officer with the British Navy. After she died of a fever, he then went to find his father. In 1699, Toby managed to stow away on his father's ship, Fancy, and went unnoticed until the ship was attacked by the Siren. He was discovered, and learned the truth: his father was a pirate. Toby had a fever which caused the Siren to mark his hand with the black spot and he ended up in the Siren's hospital with Rory. Toby took the co-pilot seat alongside his father while flying the spaceship the Siren came from. (TV: The Curse of the Black Spot)
He later accompanied his father in taking over Madame Kovarian's ship in the 52nd century. By this time, it was apparent that he had somehow been cured of the fever judging by the fact that the tube that he used onboard the spaceship was no longer connected to his throat. (TV: A Good Man Goes to War)