OP, this happened at my school. It was a little before my time, but this is what happened, as far as I can tell:

1. Big-Name Fraternity (BNF) got their charter pulled for hazing.

2. A bunch of BNF members (possibly just the uninitiated new members, but maybe the whole fraternity -I'm unclear on this point) began a new fraternity (NF). It might have been a small national or they may have started their own local - again, unclear on what exactly went down.

3. NF fell apart somehow, I think interference from the school, possibly weren't recongnized by IFC.

4. Former BNF brothers all joined Small Struggling Fraternity (SSF). This actually worked out really well for SSF, as they gained a lot of notoriety. SSF was on the verge of closing, but by the time I got to the school (a few years later), it was considered a very respectable fraternity to join ("mid-tier" if you want to classify that way).

5. Years later, BNF recolonized and are now considered hippy-dippy musician fraternity. They aren't even called by their old nickname anymore. The former BNF members were really rude to colonizing BNF members, which was unfortunate.

All of which is to say, yes, this can and does happen. And far from making the OP's Fraternity B look desperate, it actually really helped a struggling group, and prevented having two chapters close instead of just one.

Just because it helped a struggling group doesn't mean it didn't make them look desperate at the time. I'll assume enough time passed that everyone that knew about the old BNF graduated and forgot about the situation, porbably why SSF is no longer small and struggling.

I don't know much of your campus or greek life comment, so I won't speculate any further.

Well, by the time I got to the school, I had no idea SSF had ever not been a good fraternity. I'm sure they did seem desperate, but at the same time, they were close enough to closing that I think they had the "desperate" reputation already. I don't think taking the NBF members hurt them in any way, and it definitely helped in the long run.

Desperate or not the answer is YES. It can be done. As long as the right paperwork and channels are used to release the member from his previous Fraternity.

I would make sure that Fraternity B actually is willing to accept member from Fraternity A before he even does this move. It just looks real bad ... no matter what happens.
Loyalty to a certain Organization is of the utmost importance unless they do something that messes with your core belief's.