When stuff falls into a black hole, it gets measurably heavier. If a charged particle falls into one, the black hole retains a measurable electric field. If a black hole picks up angular momentum from gas circling in sideways, the hole spins faster, and the gas fired from the jets comes out at a higher speed.
Your argument that mass or energy exists that isn't measurable since it isn't observable sounds a little illogical... how would you even know there was such a thing if nobody had measured it for you in the first place?
Actually Stephen Hawking would have agreed with you in 1997, but by 2004 he decided he had lost the bet with John Preskill of Caltech.