"So really the article is bogus as they are two different things"
Nope, your understanding is bogus.
It's not important whether the test is technically math or arithmetic: what's important is whether *those being studied* consider it to be math. Follow it through. The test demonstrates quite strongly that both women and men acting as 'hiring managers' tend to believe men will be better than women at this arithmetical task, even though it is known that in fact the two perform equally. (This is why the test designers chose the simple arithmetical task: because we know that *in truth*, women and men do it equally well).
The key question is this: do you really think they'd be biased when they assess the ability of men and women to perform a simple arithmetical task, but *not* be biased when assessing their ability to perform a complex mathematical one? Unless you truly think the answer to that question is 'yes', which seems unlikely, then no, you haven't invalidated the test. If anything, I'd have expected that perhaps people wouldn't be biased against women when it came to something this simple (i.e. that if anything, the test had a chance of coming up 'negative' even though the bias does exist). The fact that people displayed such a strong bias *even when the task in question is very simple arithmetic which we know the sexes actually perform equally well*, if anything, makes the study more compelling.