I'm currently working in the same building as some of the most brilliant mathematicians of our generation. Earlier today, John Nash walked by my classroom and poked his head in the window. I tried really hard not to squeal. 

Fine Hall changed location in the 60's, so unfortunately this isn't the same building that Church, Turing, Godel, and mathematicians at Princeton in the early 20th century used. Still, the Institute for Advanced Study is right down the road, just a few blocks from Einstein's old home. 

You can read more about the Golden Age of Fine Hall here:

http://www.princeton.edu/~mudd/finding_aids/mathoral/pmcxrota.htm