Memory is complex.
I'd rank myself at the top of the spectrum in what Paul Ekman / Daniel Goleman term "cognitive empathy" (also called perspective-taking).
Yet I can draw the most amazing blanks when encountering someone I haven't seen recently—for the first five minutes. Gradually it all comes back. I tend to recognize people far quicker by their physical mannerisms than by face, dress, or other static aspects of appearance. I recall ambiguities I've detected in people far better than their outward, declared identity.
My sister, by contrast, never fails to recognize someone on first glance, complete with name, recent concerns, context of last meeting, and last fragment of personal information exchanged. "Stephanie, how's it going with new job assignment? Oh, I like this necklace. Is it new? Hey, you know what? I saw Bob downtown just last week. Are you still in touch?" and so on. Meanwhile, I would still be at "you know, that walk looks familiar, and there's that ironic tilt of the head—there's something lurking behind that I haven't figured out yet—I so know this chick, whatsherfuckingname".
It's not just a profound difference in social orientation. My sister's social skills are, well ... social. Whereas my social skills are cognitive. I pick up many shades of a person's internal self-image vs their external self-image and projected self-image. My sister socializes first, connects second. I connect first, socialize second. And both our memories reflect this.
A long time ago, when we were growing up, I used to grief her about watching the same TV shows. "Don't you remember? You watched this show last summer. That guy is going to do this, and this girl is going to do that. The dialogue was so bad, I could hardly concentrate on my calculus assignment from the next room." She never seemed to recall watching the same show twice.
Now, on basic personal dossier, she'd beat me 99 times out of 100. Until it gets deep.
What can you figure out from a face anyway? Whether someone is Jewish? How useless. What you can tell from microscopic hitches in eye movement or verbal delivery will tell you whether a person is pretending to be someone they're really not, or not. Fixed or flexible? Now we're getting somewhere.
I can only conclude that memory is deeply conditioned on the purposes in life we choose to regard as most important.
When I was roughly thirty, I looked at a photo of my grade two classroom, and remembered nearly every face and name (pretty much a fixed cohort through grade five, which certainly helped).
On a parallel note, I suck at basic plot analysis. Never had any gift with the conventions of genre fiction. I've sat beside people steeped in genre who seem to process fiction (whether a book or a movie) the way the Terminator itemizes conversational gambits, whereas I anticipate nothing immediate, but somewhere during the second act I'll start to growl, "this director is going to panic and fuck the ending, I just know it already" (by the time a film is edited, the director's success/failure in bringing the movie home is front and center in everyone's mind—director, editor, producer—so these early hints are not incidental). Yet again, conventions of form are slow for me, while deeper traces knock early.
In particular, movies that go for the fake suspense scene never work for me. He's dead! No, he's not really dead! LOTR pulls this stunt quite a few times (Frodo, Aragorn, Samwise; all the hobbits snug in their feathery Bree beds; meanwhile while Gandalf struts around as the meme personified). The form never hooks me enough to take me along on these silly ruses. After a certain amount of time wandering around in the labyrinths of tvtropes (a quantity of time that shall remain nameless), I now pick up much of this on an analytical level. But still not gut.
Part of the reason I think I classify slowly on form is that I tend to maintain a portfolio of possibility anyway. "This could be five different things, let's leave it at that and see where it goes." I notice this all the time whenever I compare social notes after an unusual interaction. The other person's account is always more definite in nature. And then I'll go, "but that's not exactly how it was said; the way it was said could also have been this or that."
On the other hand, if it's a long term thing, the analysis goes the other way. The other person will be going, "what's driving this behaviour, could it be this or that" and I'll go "it certainly isn't that, because that's not how the psychological feedback loops work in this person; have you ever seen a time where X lead to Y? For this dough ass?"
The space of options begins to collapse for me once the loops connect full circle.
It's just a seriously fascinating thing how much people differ in their cognitive reflexes and strategies. I could never tire of observing the human cognitive circus up close and personal. It's seven thousand shades of amazing, entirely unlike ice cream with fudge and marshmallows and random cookie bits and weird sprinkle topping. Who the fuck remembers their last random Wheel of Fortune selection at Baskin-Robbins? But no. Wrong. That person is one of the 7000 shades.