- You may be looking for the in-universe concept.
The term "not we" (sometimes rendered Not We and not-we) is used by fandom to indicate the casual viewer of Doctor Who. Sometimes, it simply means "anyone who is not an obsessive Doctor Who fan", and does not assume that the people described actually watch Doctor Who at all. The term is often used on internet forums and podcasts in arguments where distinguishing between "hardcore" and casual fans is useful. Popular Doctor Who forum Gallifrey Base has for years featured a thread in each of its episode discussion areas called, "What did the kids and the "Not We" think of <insert name>?" — which undoubtedly helps to reinforce the term in the minds of its users.
It derives from the Not-We, a group of outsiders seen in Kinda. Unlike the original term — which is hyphenated and fully capitalised in the novelisation and had a mildly pejorative connotation — the fan term is inconsistently capitalised and hyphenated, and it's rarely, if ever, meant as an insult.