If you're familiar with the Myers-Briggs/Carl Jung personality type system, I believe it can apply to this discussion, in connecting certain personality types with society's views of "intelligence."
For example, I am an INTP (a "Thinker"), and consider myself to be fairly intelligent. Most who are familiar with the system would also agree that INTPs tend to exhibit traits typically viewed as "intelligent." However, I am constantly reminding my girlfriend, who is an ISFJ (a "Nurturer") that she is a very intelligent person. She seems to have an ingrained notion that she is less intelligent than others around her, and a general lack of self-confidence when it comes to intellectual pursuits. She is actually much better than I am at various mental tasks, specifically factual memory, recalling dates, multitasking, numbers, etc.
My understanding is that the same societal and environmental influences which contribute to the development of one's personality are the same that give that person their sense of self-understanding, and transitively their sense of self-worth and intelligence.
I may have a more developed "intuition" trait, which allows me to make conceptual connections very easily (ie. inventing, engineering, programming), but I will never be quite as good at recalling dates, or at being very aware of my immediate surroundings as she is.
