Loss of neurons due to aging has been shown to be a measurement error before.
Haug, H. (1985). Are neurons of the human cerebral cortex really lost during aging? A morphometric examination. In Senile dementia of the Alzheimer type (pp. 150-163). Springer Berlin Heidelberg.
Unfortunately the article is paywalled. However it says that there is no statistical evidence of neuron loss in healthy people due to aging. Previously widely reported loss was shown to be a systematic bias in measurement.