Port addresses are not part of the ISP's business, either. That is an internal matter for the networks/routers at source and destination, not in between.
If the ISP is doing NAT for customer traffic, they need to know the port addresses. The source and destination routers are not the only routers in the network. And if they are providing outbound connections only, they need to know what outbound ports are in use so they can allow the responses to get back while blocking the rest.
My ISP, for example, has absolutely no business knowing (or caring) whether I am doing SSH over port 22 or port 23456.
Knowing what ports are in use does not tell the ISP what kind of traffic is going over the connection. You would have a valid point had you said "whether I'm doing SSH or FTP" and stopped there.
It isn't essential for their services and it's private information.
How is what port you are using "private information"? It is in every outgoing packet you send. And it is essential for some kinds of service.
Further, if they intentionally redirect my packets, in any way that wasn't essential for internet routing, they're interfering with a private communication, which is illegal for a common carrier to do.
Ummm, so if they use a longer route for your packets because it is cheaper for them, it's illegal?