Before going to college and becoming Greek myself, the only GLOs I had heard of were Phi Beta Kappa (my dad is one), Beta Theta Pi (one of my best friends from high school was a year older than me and the first of my friends to go greek, so that's how I heard of them), and Tri Delt. Other than that, I had heard nothing, but that's not surprising considering I grew up in Omaha, NE. Not exactly a hotbed of greek activity.
I would have to agree with others though who've said that Phi Beta Kappa is the most well-known nationally.