Is his work situation a long-term or temporary thing? I don't think you are wrong in feeling the way that you do. It is difficult being in a relationship when you both work opposite hours. I am assuming that you have already spoken to him about this. What was his response? I know what you are going thru, I think we have all been there at one time or another. Let us know how things are going 
