Eh, I don't agree with the others. I met my husband at work.
We worked in the same department and while he was not MY supervisor, he was a superior. However, we didn't "officially" start dating until the last week I was there, but we had our share flirting and lunch dates before I left. Also, given your environment (I would assume a library to be a it more laid back than an investment bank), I think it would be fine. And if you don't work with him constantly, if it doesn't work out I doubt it would get that awkward.
All that said, I'd give it a little more time before you push a date. I'd go with the sitting by him at lunch, or casually tell him your heading to lunch "wanna come", that kind of stuff. Do a little more casual, getting to know you stuff at work. Give him some time to see what he's thinking, too. Remember, he's brand new, he might be really nervous to ask someone out at work for fear of getting in trouble. In a month or so, if nothing progesses, then I'd do the "wanna go see an anime movie" thing.