No, it shouldn't matter ;but it still happens to me to this day. My daughter is considerably lighter than I.
Now being from the South, I have found that what Northerners (West and East included) describe as light is not the same where I am from. For instance, Jada Pinkett would be considered as brown to some but light where I am from.
Although, most people I knew didn't care what color you were as long as you were pretty.
When I was younger, a guy I was dating (a jerk) once said in a conversation I overheard that he would marry a light-skinned girl and only f*** dark ones. Well I guess you know I told him where to get off!
But it used to really bother me and I would pray often that my daughter didn't experience those same prejudices.
My husband and I just had a discussion very similar to this (black men marrying white women and the reasons why--another issue another thread)--my argument was and still is that we, as an Afro-American community, have a social responsibility to enlighten, require, and demand that the face of the A-A female community is not just some chick with "slightly" dark skin with very "thin" features or some BBW with a hearty voice selling Pine-Sol. My color, or ANY woman's color should not set the standard for what's beautiful but should expose our blackness as merely existentialism.