Comment It's not a new idea (Score 1) 195
For example, code indentation can be done by walking the AST (you need to be careful to preserve hidden tokens, such as comments).
You can also allow code completion by changing the compiler to accept a "COMPLETION" token in some places in the grammar. Then, from the editor, when someone presses "Ctrl+SPACE" (or whatever) you mark the location in the lexer and send the code to the compiler. When you build the ast, you insert a completion node in the AST, and you have now contextual information about what can go in there and produce a list of potential things that can go in there.
Also, syntax highlighting can use the lexer for basic coloring and some type information to then add more information (such as what are field, or functions, etc.)
What's new is exposing these phases in a standardized manner in the language. That's a bold move, since backward compatibility will be tricky to maintain. Maybe they're thinking in finally stabilizing C#.