Same thing here.
Coffee and milk both affect your
Acid-base homeostasis negatively.
Negatively in that case means they tip the scale towards acids, and modern nutrition is already pretty heavy on things which, after digestion, produce surplus acids.
After a diagnosis with Chromic Fatigue Syndrome and a plethora of doctors that don't know what to do and resort to anti-depressiva, a smart health professional pointed this out to me and we analyzed my nutrition habits.
Turns out, basically it's all acids, no bases. Lots of tea, lots of coffee, lots of meat, lots of grain, lots of sugar, very little fresh fruits and vegetables. Just the regular canteen food software developer diet.
Already after three days of changed diet, my skin felt less dry, the chronic pain was almost gone completely, and I felt 10 years younger.