Where I work, electricity is 0,25ct/kWh and a specialist in IT or law costs 1.000,- EUR/d or more.
Assuming a server we're planning to shut down is rather old, they usually are, so it will probably fail on its own within 3 years, if not much sooner. It is not doing much anymore, so it's sitting at idle, drawing only idle loads. Assuming the idle load of an old server is 100W, how much specialist's time can we allocate to shutting it down?
100W * 8760 h/y * 5y = 2.628 kWh. This will cost us about 657,- EUR or much less if the server fails earlier. So electricity savings alone buy us 5,5 person hours for the specialists. What do we need: 2x1h for two IT guys to check what server are possibly unused, 2h for inquiries and talking to probable (ex-)users and the team that was once responsible for that particular project, 1h for the IT management and 1h for the legal team to give the go-ahead. Costs 750,- = 100,- EUR more than it will ever save.
We still save on cooling, right? Removing 1kWh of input (= heat) requires less than 1kWh for the cooling system. I am have no idea how efficient these things are, but a cheap electric heat pump for heating a small house has an efficiency factor of at least 3, so it moves 3kWh of heat for every 1kWh electricity consumed. Larger and more professional installations will probably be more efficient. So to remove 100W of heat, the cooling system consumes 30W more. So with less cooling, we save another ~200,- EUR over the course of 3 years, which isn't even enough to cover the costs to actually remove the server from the rack, reroute cabling, disassemble the case, destroy the hard drive and dispose of the rest. Costing 250,- EUR, that is 50,- EUR more than it will ever save.
The real savings are in the rack space, depending on the contract and the actual savings of HUs. Assume the price is 40,- EUR/HU/month and we have a 2 HU server. Over 3 years, this saves us 960,- EUR per year or 2.880,- EUR under the most ideal conditions imaginable. (If you rent data center space by the rack, it's ZERO savings, since you're saving nothing unless you get permission to clear out an entire rack, which is not going to happen until the servers burn out by themselves)
So you're Head of IT management for a minute: do you give the order to decommission the server, expending 1.000,- EUR and 1 day of your team today, risking angry users and maybe in one way or another violating a data retention obligation by an obscure law or contract that we just forgot about to save maybe 1.300,- EUR per year or less for the next 3 years or shorter? I wouldn't. I would rather allocate my team and resources on a) making absolutely sure our accounting system keeps running perfectly, since every day of outage there would cost us more than 20.000 EUR in interest and b) that big project X has all the resources it needs to finish on time so the 10 expensive consultants working on it cannot bill more hours and upper management does not need to find a person responsible for that.