Line terminators in programming languages

[alexdima]

Extracted from microsoft/TypeScript#38078

VS Code currently recognizes CRLF, CR and LF as end-of-line sequences.

But programming languages have different definitions for what constitutes a line terminator. Here is a summary from some of the specifications I found:

char/seq	Unicode	JS	C#	HTML	Python	PHP	Java	YAML
CRLF	✅	✅	✅	✅	✅	✅	✅	✅
CR	✅	✅	✅	✅	✅	✅	✅	✅
LF	✅	✅	✅	✅	✅	✅	✅	✅
LS	✅	✅	✅					✅
PS	✅	✅	✅					✅
NEL	✅							✅
FF	✅
VT	✅

Specifications:

• JS spec Table 33
• C# spec A.1.1 Line terminators
• HTML
• Python
• PHP
• Java
• YAML section 3.1.4 Line Breaks

Some language servers might use their specification variant for what a line is, so the (line,char) coordinates used for document change events, reference positions, diagnostics etc. might be off for files containing LS, PS, or NEL.

Since the characters LS, PS, or NEL are very rarely used in practice, and most likely they appear unintentionally in source code (through copy-pasting), I suggest we prompt users when such a file is opened and ask them to "fix" the line terminators for the file. Fixing would mean to replace LS, PS, or NEL with the current configured EOL sequence.

[alexdima]

Here is some text with such a character:

var Input;
var lastInput;
var position =0;
var ftext = [
"Und", "dann", "eines"];
var opacityTitle=1;


document.addEventListener("keydown", function(event) {
 if (event.which >= 32 && event.which <= 122 && event.which != lastInput) {
    lastInput = event.which;
     advance();
 }
else if (event.which == 8) {
    lastInput = event.which;
    strikeOut();
}
});

document.addEventListener("keydown", function(event) {
    document.getElementById("fade").style.opacity = opacityTitle;
    opacityTitle -= 0.06;
   });

function advance(){
    document.getElementById("butt").innerHTML += "<p id=\"" + position + "\" class=\"word\">" + ftext[position] + "</p>";
   
    var nextWord = document.createElement("P");
    var textnode = document.createTextNode(ftext[position]);
    nextWord.appendChild(textnode);
    nextWord.classList.add("word");
    document.getElementById("butt").appendChild(nextWord);
    position++;
    var foo=3;
    var bar=4;

}

function strikeOut(){
    document.getElementById("butt").innerHTML = lastInput;

}