From that description it does sound like the string is still valid. However if the display is crashing on a certain sequence containing an ellipsis, I am not clear why you can't construct that string directly, rather than rely on the insertion of the ellipsis.

It does sound like they maybe rely on "sanitizing" but of a far more complex scheme that I was aware of. This is still wrong, maybe far worse, as they are detecting and rejecting patterns containing ellipsis and some other character, that is INSANE!!!. Any such work should be delayed until the VERY LAST moment. In this case their glyph layout should simply not crash on any possible arrangement of bytes or words in the incoming string. This is very much the same stupidity that I was ranting about for UTF-8. Nobody used to crash because you put mis-spelled words in your text and tried to print it. Apply the same logic to UTF-8 and Unicode. It is not hard and it seems really obvious, but for some reason Unicode turns some otherwise really smart programmers into total idiots.

I like that idea. You're right, it should be pretty efficient to implement, regardless of the string's backend encoding. And the value represented by the iterator will, by nature of being implemented as a pointer to a certain part in the string, be able to point to a glyph of arbitrary length (unlike a getter function with a fixed-length return type). Being an iterator it'll fit into all standard c++ libraries that take iterators.

It would be nice to have it be a random-access iterator so that you can jump to an arbitrary offset. There's a lot of optimizations they could do internally to help facilitate that. But obviously you still want to let programmers choose - by some means or another - whether they want such unicode optimizations (or unicode iteration, or so forth). Because while the overhead they'd impose wouldn't be huge, there still would be overhead.

We have to have something, though. Not everybody writes in English.

Except wait - we've got a phase change from gas to plasma in there, which almost certainly breaks their calculations badly.

Again, no, you don't. All of the particles are moving in the same direction. They're not hot. They're not slamming into each other and kicking electrons off.

Do you think if you had a spacecraft moving at 25.4 kilometers per second it would be plasma too?

If this works, the monied and in-power will make this as illegal as LSD and heroin.

Not necessarily.

If the anti-aging drug(s) make people healthier, reducing the drain on the government pensions and enabling the government to push the retirement age out over the horizon, so the people will be working and taxed, they might prefer to have the drugs put into use.

Heck, they'd probably add them to the water.

Or as just "text" on non-broken compilers. Compilers that translate the input into UTF-16 as part of the parsing are broken however.

Actually I think "Unicode strings" should be avoided completely.

They do not help at all in doing text manipulation, because Unicode code points are *not* "characters" or any other unit that users think about. This is due to combining characters and invisible characters such as bidi indicators. There is a prefix code unit that eats the next 2 letters and turns it into a country flag! It is a huge mess.

Far more important is they all lack the ability to store errors that are in a UTF-8 string in a lossless way. This means you cannot trust arbitrary 8-bit data to survive translation to "Unicode" and back. This has been the source of endless bugs and is the reason people can't use Python 3.0.

My recommendation is special interators on std::string. Something like this:

    for (utf16_interator i = string.begin(); i != string.end(); ++i) {
          int x = *i;
          if (x < 0) error_byte_found();
          else utf16_found(x);
    }

There would also be interators for UTF-32 (probably what you were thinking of as "Unicode" but a lot of Microsoft programmers think "Unicode" means UTF-16). And iterators for other normalization forms. In all cases these would return negative numbers or some value that cannot be confused with a code point for UTF-8 error bytes.

This would be very fast because you can find the next Unicode code unit or whatever in constant time. Any api where you can arbitrarily index a unit using an integer is not going to be constant, it will be linear with that integer. Iterators avoid this.

No you don't. You are demonstrating the typical moronic attempts to deal with UTF-8.

Here is how you do it:

Go X bytes into the string. If that byte is a continuation byte, back up. Back up a maximum of 3 times. This will find a truncation point that will not introduce more errors into the string than are already there.

BUT BUT BUT I'm sure you are sputtering about how this won't give you exactly X "characters". NOBODY F**KING CARES!!!! If you want the string to "fit" you should be *measuring* it, not saying stuff that has not been true on computers since the 1950's about "N characters fit". I bet you think a combining letter and accent should count as 2, huh?

And your display function should not crash because it was given a string with an error in it! Even if you stupidly inserted the ellipsis all it should do is draw a few error indicators before the ellipsis.

No, the problem is code that pretends that illegal UTF-8 sequences magically don't exist!

For some reason UTF-8 turns otherwise intelligent programmers into complete morons. Here is another example from Apple. Let me state some rules about how to deal with UTF-8:

1. Stop thinking about "characters"!!!! This is a byte stream. The ONLY reason to think about a "character" is because you are DRAWING it on a display designed for a human to read, and humans do think about "characters". All other software either does not care, or is concerned with far more complex patterns (such as regexp and editors that deal with words and sentences), these second ones are not helped at all by an intermediate translation.

2. It is TRIVIAL to detect that the byte sequence you are looking at is not a valid UTF-8 character. In this case draw a replacement for exactly ONE byte and then try the next byte to see if it is a valid sequence. Do not skip more. There must be one error per byte so that the maximum number of good characters is preserved and so that a sequence with errors can be parsed bidirectionally without looking more than a few bytes ahead, and so that it is possible to search for error patterns. It also means there are only 128 different errors, not millions.

3. NEVER "translate to Unicode" (ie UTF-16) because this will be a lossy conversion of these invalid sequences and thus you have not preserved the original data. I'm sorry but Microsoft really screwed us here. Best recommendation is to write a wrapper around the filesystem calls and translate from UTF-8 to UTF-16 at the last moment, using U+DCxx as a translation for the error bytes (this is lossy but filenames already are, due to case independence, Apple's normalization, and even on Unix where "./foo" and "foo" are the same file).

This is blatantly obvious if you substitute "words" for "characters" and imagine how you would write a program to deal with text strings. Words are also composed of multiple bytes in a row. For some reason nobody seems to crash on misspelled words, and they manage to concatenate and split strings and make whole file systems and diff programs and all kinds of other fancy text manipulation without having to translate the text so that each word is a fixed-sized integer. Amazing!

First off, you're misusing temperature. You don't call it heat if all of the particles are moving in the same direction and unionized, you just call it "wind". It only becomes heat if that windstream suddenly slams into a non-moving solid surface and becomes instantly thermalized (but of course even then that would be a very short-lived event as it would correspond with a pressure rise and the deflection of the stream behind the high-pressure zone). Additionally, nor would that be the windspeed touching the surface as, obviously, wind forms boundary layers.

Secondly, hundreds of km/s from Venus escape to Mars intercept? That doesn't at all correspond to any delta-V chart I've ever seen.