The Cloisters were defended by Cloister Wraiths and alien invaders that had been neutralised and "filed" by the Matrix; these forces prevented anyone from leaving the Cloisters once they entered. However, when the Doctor was at the Time Lord Academy, he entered the Cloisters and was shown a secret passage out by the Wraiths.
After escaping his confession dial onto Gallifrey, the Twelfth Doctor took Clara Oswald into the Cloisters to prevent the Time Lords from returning her to the moment before her death. The Doctor then stole a TARDIS and took him and Clara towards the end of the universe. There, in the now-ruined Cloisters, he found Me waiting for him in a bubble of reality. (TV: Hell Bent)