Sounds like he has some tender feelings for you and maybe also gets a vibe from you that if you weren't with hubby you might be interested in dating him.
Nothing necessarily wrong with all that.
Anyway guys are like that, we tend not to talk about girls with other girls, especially girls we might cherish a little feeling towards.
His protestations of no girl ever coming between you two blah blah blah is a little extreme . . . thats definitely a declaration of like.
And honestly your rambling post gives a little away also
If you didn't cherish some tender thoughts none of this stuff would bother you enough to think about, then to ask your friends about, and then to ask the people on GC about.
You might want to be really careful. Unless you are just utterly bored with your marriage?