Yeah, what the worlds needs is some disgruntled employee putting a computer in their office that will dump client data out a particular port without IT knowing what is going on.
Besides, it shouldn't kill them to white list your server on one freaking port.
No... It can kill them. You're running an application that isn't approved, and they haven't weighed the vulnerabilities. An open port is always a target for exploitation, which is why the IT department needs to be able to audit the machine and ensure what software is installed, so they can mitigate those vulnerabilities.
I'm going to guess that if this person set up a server just say, in their office, this machine is on a network segment that may not be as firewalled-off as a data center may be. That means if something malicious does happen to this server, there's a greater chance of infection elsewhere, as well as some risk of productivity loss. Besides, the machine itself doesn't have to be the target of attack-- it can just be the jumping-off point for something bigger, once they've installed tools to probe the network.
Especially when you're in a healthcare setting, privacy is a big issue. You could conceivably have someone post patient data in a calendar appointment, even. If that connection isn't TLS encrypted, and the devices not properly managed, it just takes one theft of a device sitting in a coffee shop to result in a serious breach of privacy and patient trust, even if the thief doesn't access the data that might be contained on the device.