Linux IP Tables has all kinds of cool stuff to overcome that issue.

With lots of configuration, you can overcome some of the problems. But you'll never get it working as smooth as it would without NAT.

B/c neither device can host a server

And the reason for that is NAT. Also, the definition of a server is unclear in the first place. Not all protocols are client-server protocols. Do you call every listening end of a TCP connection a server? Historically some protocols including FTP and X11 have been listening for connections on the client device. And peer-to-peer protocols need to transfer data between two devices, neither of which would be considered a server. In many cases UDP has been used in scenarios, where TCP would have been better suited. The only reason being, that it is easier to punch holes in a NAT, if you are using UDP than if you are using TCP.

You obviously do not understand NAT very well.

I have written a NAT implementation from scratch. What background do you have, which gives you more knowledge about NAT than writing a NAT implementation from scratch?

A correct design would be to use a service that you can query quickly using disparate network connections

Such a design is never going to work. What are you going to do once you face the requirement of notifying the user about an incoming message within a five second deadline, while your power budget does not allow you to send packets more frequently than once every 30 seconds during idle time?

that is all handled transparently by the network, and the software should be able to manage keeping itself going even as its IP address changes

With current network design, changing the IP address is not going to be transparent to the application. Change of IP address will cause ongoing communication to break (or even worse - stall). The best you can do is for the application to notice the IP change and establish a new connection. That will however only work reliably in the absence of NAT. You can do it more transparently to the application, if you are using MPTCP. But that doesn't change any of my points, it just moves some of the logic one layer down the stack.

NAT also has a lot of benefits - including privacy.

Another common misconception. All of the perceived privacy you would get from using NAT can be achieved from IPv6 privacy addresses. You get additional privacy that way, because moving from IPv4 to IPv6 eliminates the leaks you'd otherwise get from the IPID field in the IPv4 header.

You seem to believe that everyone shares your opinion about the "good"ness of the results of the current extremely widespread adoption of NAT44.

No. I am perfectly aware that lots of people disagree. However so far none of the people who disagree with my view have shown me a technically sound reasoning for their position. Generally you can do better by simply using a firewall, that passes or rejects packets without mangling them, combined with utilizing multiple IPv6 addresses per interface inside your LAN. Those two methods cover all of the perceived advantages of NAT44, which I remember having heard about.

Except of course for the problems of getting legacy IPv4 hosts to talk to each

I was explaining why leaving NAT out of the IPv6 specification was a sensible decision. The question was whether native IPv6 without NAT or NAT66 was a better choice. Now you bring up the challenge of getting two legacy IPv4 hosts to talk to communicate with each other. Which technology is better for that purpose NAT66 or IPv6 without NAT? The answer is, neither will do that particular job. So that is by no means an argument in favor of NAT66.

Except of course for the problems of getting legacy IPv4 hosts to talk to each other in the presence of site renumbering or deliberate site-local topology information hiding.

As a matter of fact, I have developed a system, which does all of that. On the edge between the LAN and the internet backbone, there is one or more gateways translating between IPv4 and IPv6. Whether the other end of the connection is using translation as well is transparent to the hosts. With this system it is possible to communicate between two LANs using IPv4 only through an IPv6 backbone. Topology hiding is achieved through an optional encryption of addresses on the edge. Using IPv6 across the backbone eliminates a bunch of the scaling issues a traditional NAT44 suffers from. Finally I support automatic fallback to tunnelling in case the native IPv6 link goes down.

I really do not get your point and suspect you are merely looking for an argument.

You are the one who hasn't made any constructive suggestions on how to solve the problem.

The polling interval or other way of checking if the connection is still alive

First of all, keepalive checks is not the goal. It is merely a means to workaround obstacles getting in the way of solving the real problem. The goal is to get a notification to the phone, when it needs attention. The deadline to get the notification there is on the order of five seconds, possibly less. At the same time there is no power budget to send packets every five seconds, even sending a packet every minute may be too power consuming.

How do you propose solving that problem? It is quite clear that it can only be achieve by having the phone passively listening for traffic.

should obviously depend on what the application needs instead of some arbitrary number imposed from elsewhere.

And which polling interval do you suggest using if the needs are: 1: Application must get notified no later than five seconds after a certain event happening in the cloud. 2: While the application is idle, the radio transmitter on the phone must not be activated more frequently than once every five minutes, to preserve battery power.

If you choose to use polling to solve that problem, you must choose an interval which is shorter than five seconds and longer than five minutes. That is impossible. Opening a TCP connection and waiting for a reply will however work.

Open connections without a way to check if the other end is still there are a bad idea on networks that frequently drop out.

That is not a problem. If the network connection disappears temporarily, then notifications cannot be received while the connection is gone. Users can understand, that they cannot receive notifications while the connection is gone, and that no protocol changes can make that happen. Once the network connection has been re-established, there are two possibilities. Either the device still has the same IP address, in which case the TCP connection is still open. Or the device got a new IP address, in which case the application can get notified about the IP change and take necessary actions to establish a new connection.

The only problem left to consider in case of temporary loss of network connection is the following: The phone may lose network connection temporarily and not realize this, while the connection is gone the server decides the TCP connection needs to be closed and sends a FIN. Due to the FIN getting lost, the FIN gets retransmitted a few times. Eventually the server gives up discards state and stops sending further FIN packets.

This is the situation TCP keepalive is mainly designed to deal with. Additionally there is the possibility that the server crashed and got rebooted and thereby lost all TCP connections without having a chance to send a FIN. Use TCP keepalive to ensure both of those situations are noticed by the application. If you just use keepalives for this, you don't need frequent keepalives, because this is not in the critical path for receiving notifications about events in the cloud.

The IPng selection process that the IETF used ultimately *deliberately* chose to be incompatible with NAT for architectural reasons

That is utter bullshit. If you look at the actual IPv4 protocol versus the actual IPv6 protocol, you'll find IPv4 to be slightly more hostile towards NAT than IPv6 is.

The mandatory IPID field in the IPv4 header is almost impossible to handle in an IPv4 NAT without violating the IPv4 spec. And RFC1918 addresses are prone to have collisions.

It was deliberately chosen not to make NAT part of the IPv6 standard. But NAT was not part of the IPv4 standard either. Some later RFCs standardize various aspects of NAT44, but as far as I can tell actual implementations of NAT44 predate the standards, and implementations made after the standards don't follow the standards anyway.

Nothing in the RFCs was stopping IPv6 implementations from including NAT66. What stopped them from including it was common sense. People who actually know how the networks work realized that nothing good came from NAT44, and that's why it wasn't implemented for IPv6. For every problem that people have applied NAT44 to, IPv6 offers a better solution, which does not involve NAT.

Polling at time intervals, short if necessary

Polling is even worse than keepalives. Polling is a waste of resources. Let's say I have a chat client, and need to get notified about incoming chat messages in a timely fashion. A reasonable polling frequency for that would probably be around once every five seconds. For each poll I will exchange three packets for a TCP handshake, two packets for application level query, and four packets to tear down the TCP connection. That is a total of nine packets every five seconds.

There could be hours between the time where I receive a chat message. But that does not mean I can wait hours to be notified about one arriving.

If instead the client open a connection and tell the server to notify me, once a message arrives, the connection could exchange the following packets. Three packets for TCP handshake, three packets for application level request, application level ACK and TCP level ACK of the application level ACK. A total of six packets after which the connection goes idle. Once a chat message arrives there could be an application level notification followed by four packets to tear down the connection. That is a total of eleven packets for the entire duration of this connection, which could last for hours.

Eleven packets in the span of hours seem a lot more efficient than nine packets every five seconds. And the open connection can even give me a notification in less than a second compared to the average delay of 2½ second I'd get from polling every five seconds.

Apps that keep live connections on your cell phone are usually doing a bad thing.

How do you expect to get notified once an email or a chat message arrives?

makes me think migrating to IPv6 will require legislative mandate.

I have had that feeling for more than a decade. Unfortunately, legislators don't understand how the internet works. I guess the majority of legislators have never seen an internet connection without NAT.

There's plenty of ipv4 addresses around, maybe time for operator change?

No other operator could deliver connectivity at my address. Besides I have no guarantee another operator would do any better. And none of them are offering IPv6, which would otherwise be sufficient reason for me to switch right away.

Can you shorten more than one block in an address?

Nope. That is not permitted (for exactly the reason you mentioned).

For example, "fe80:0000:0000:0000:0025:0000:0000:0001". Would that become "fe80::0025::0001"? Wouldn't this be a problem then if you try to shorten "fe80:0000:0000:0025:0000:0000:0000:0001" and get the same result "fe80::0025::0001"?

That's why you can't shorten two blocks. One might argue that as long as each shortened group was shortened by the same amount, it could be expanded unambiguously. But that would have added complexity, for very little gain. So instead the standard says you can only shorten one block.

Some interpretations go further and says you must shorten the longest run of zeros, and only if it contains at least two zero blocks. And if there are two blocks of identical length you must shorten a particular one. Those more strict rules are relevant if you absolutely need a canonical representation of an IPv6 address. But some implementations have taken this a step too far and reject IP addresses, if the shorting does not follow these rules exactly. For example IPv6 addresses has been rejected due to having used shorting when there was only a single block of zeros.

What was the rationale for making the IPV6 address space so huge in the first place? Seems like simply going to 40 or 48 bits would have been sufficient for decades if not longer.

The rationale was roughly this. We want to get rid of the complexity by having variable length network prefixes. So the address would consist of a fixed length field to address a network and another fixed length field to address a device on that network.

For the first part we knew 32 bits was insufficient, and we don't want the administrative overhead of an extremely large HD-ratio, so go for a more relaxed HD-ratio of 80%. If we have 45 bits and assume an HD-ratio of 80% we get the equivalent of 36 bits efficiently used.

For the second part we wanted to be able to embed a full MAC address, so we need at least 48 bits.

To keep things simple, each part was rounded up to 64 bits.

We already now see situations, where the size of IPv6 addresses is a shortcoming. First case is in tunnel protocols. First there was 6to4, which embed an IPv4 address right in the middle of the upper 64 bits (the network part). This worked great from an addressing perspective. A /16 of IPv6 address space allocated for a transition mechanism isn't bad. And 16 bits for subnetting is sufficient for the size of network you typically have behind an IPv4 address. But then people realized that somebody has been deploying NAT on the IPv4 network. So another tunnelling protocol is needed. Along comes Teredo. Teredo embeds two IPv4 addresses and a port number into the IPv6 address. That's 80 bits. There is no way you are going to embed all of that into the top 64 bits of an IPv6 address and still have room to spare. In fact Teredo would probably have benefited from having the server port number embedded in the IPv6 address as well, but there just wasn't room for that. If you would have wanted to combine the benefits of 6to4 with those of Teredo into a single protocol, you would have needed 64 bits more in the IPv6 address, that is 192 bits total.

There is also the desire to put cryptographic information inside an IP address. With IPv4 this idea was more or less unthinkable, because 32 bits of security is just not enough for anything. With IPv6 it starts to become possible. But you can't use the entire IPv6 address for only cryptographic data, you absolutely need some bits at the start to identify the class of address. And some bits left over to identify network and device would be nice. You can meaningfully embed cryptographic information inside an IPv6 address. But you don't get to do much else. If you wanted to embed say a SHA1 hash plus some other information in an IP address, you are soon going to need 256 bits.

Consider all the things I mentioned and you might find a use for even 512 address bits. But at this time it is too late to change the size of the IPv6 address. Anybody who want to propose an alternative to IPv6 have to remember that in order to prove that a solution does solve all the shortcomings of IPv6, you have to get it standardized, implemented and deployed before February 2011.

But actually, the users of those services pay more to avoid converting the infrastructure. (Although, rather than a direct monetary cost, it often comes in the form of things like bugs or missing features that could've been developed/fixed if developer time wasn't being spent dealing with NAT.)

Plus the developer has to spend time fixing bugs in the code working around the NAT induced problems. I think the Skype outage a few years back is the most spectacular example of how bad it can go.

(proper) CGNAT uses 100.64.0.0/10, so it doesn't collide with RFC1918 reserved addresses.

That introduces other problems. Lots of software will look at the assigned address to figure out if it has a public address or is located behind a NAT. If it is behind a NAT it will make certain workarounds, if it has a public address it can ignore most of that and do things in a simpler way, which works more reliably as long as no NAT is involved.

Any such software written before the introduction of RFC6598 will of course consider addresses from 100.64.0.0/10 to be public addresses and use them as such. This is going to cause breakage.

If you know for sure that there is going to be another layer of NAT behind the addresses you hand out, you might avoid some application layer problems, and thus think assigning RFC6598 addresses won't cause such problems. However if those NAT devices are some of those that enable 6to4 by default, if they get a public IPv4 address, you are going to see addresses from 2002:6440::/26 being assigned, which is not going to work.

The intention of RFC6598 may be good. But the problems you avoid may very well be replaced by some new and poorly understood problems.

My understanding was that cell phones all already use IPv6 via 6-to-4 NAT on the backend.

I'm pretty sure they are not using 6to4. Some carriers use DNS64+NAT64 for cell phones, and thus the phones themselves are IPv6-only. But NAT64 is still a NAT with connection tracking and all that. Additionally since the phone doesn't even know, that it is really IPv4 on the other side of the NAT, it may actually cause some software to have more problems with that.

An advantage of DNS64+NAT64 is that you can migrate clients between NAT64 devices without affecting established connections. The address seen by the client has plenty of bits that it can contain both an identification of the NAT64 device in use and the IPv4 address of the server on the other side of the internet. And by handing the client multiple IPv6 addresses using different NAT64 devices, you can let the client perform failover between them.

We need to get the ground work done so that IPv8 can be introduced smoothly

I think most people gave up on PIP already.