Escape U+007F and C1 controls in the JavaScript encoders (#163)

In non-ASCII mode, all four JavaScript encoder modes passed U+007F
(DEL) and the C1 control characters U+0080 to U+009F through raw.
This is not a string-literal breakout (ECMAScript line terminators
are only LF, CR, U+2028 and U+2029), but every other encoder already
refuses to emit these characters raw, and encoded output often
reaches logs, templates and other consumers that treat U+0085 as a
line break or U+009B as a terminal control.

Hex encode them as \xNN (lowercase, like the C0 controls) in every
mode: DEL is no longer marked valid in the mask, and firstEncodedOffset
and encodeArrays send U+0080 to U+009F to the existing \xNN writer.
U+00A0 and above still pass through except U+2028/U+2029, and the
ASCII-only mode is unchanged.

JavaScriptEncoderTest now starts the valid range at U+00A0, marks
U+007F to U+009F as encoded, and has exact cases for DEL, U+0080,
U+0085 (NEL), U+009B and U+009F in all four modes. Document the new
row in the forJavaScript table, the other three methods and the JSP
and Jakarta TLDs, and add a README News entry.

Author: VolodymyrLinuxovich

README.md

@@ -156,6 +156,7 @@ News
 ### Unreleased - 1.4.2
 * fix: `forHtmlUnquotedAttribute` now replaces U+0085 (NEL) with a hyphen like the other C1 control characters, instead of emitting `…`, which HTML5 parsers decode as U+2026 [#136](https://github.com/OWASP/owasp-java-encoder/issues/136).
 * fix: the XML 1.1 encoders (`forXml11`, `forXml11Content`, `forXml11Attribute`) now encode U+0085 (NEL) as `…` and U+2028 (line separator) as `
`, so they are not normalized to a line feed [#136](https://github.com/OWASP/owasp-java-encoder/issues/136).
+* change: the JavaScript encoders (`forJavaScript`, `forJavaScriptAttribute`, `forJavaScriptBlock`, `forJavaScriptSource`) now hex encode U+007F (DEL) and the C1 control characters U+0080 to U+009F as `\xNN` instead of passing them through, for consistency with the other encoders. This changes output for inputs containing these characters but not the decoded string value; it is not a security fix, since JavaScript does not treat them as line terminators [#163](https://github.com/OWASP/owasp-java-encoder/issues/163).
 
 ### 1.4.1 Security Release
 Upgrade all OWASP Java Encoder dependencies to 1.4.1. This release fixes three

core/src/main/java/org/owasp/encoder/Encode.java

@@ -1174,6 +1174,13 @@ public static void forJava(Writer out, String input)
      *       <td class="colLast">Hyphen character</td>
      *     </tr>
      *     <tr class="altColor">
+     *       <td class="colFirst" colspan="2">U+007F&nbsp;to&nbsp;U+009F</td>
+     *       <td class="colLast"><code>\x##</code></td>
+     *       <td class="colLast">DEL and the C1 control characters, including
+     *       U+0085 (next line), are hexadecimal encoded so they never appear
+     *       raw in the output.</td>
+     *     </tr>
+     *     <tr class="rowColor">
      *       <td class="colFirst" colspan="2">U+2028, U+2029</td>
      *       <td class="colLast"><code>&#92;u2028</code>, <code>&#92;u2029</code></td>
      *       <td class="colLast">Line and paragraph separators</td>
@@ -1218,6 +1225,10 @@ public static void forJavaScript(Writer out, String input)
      * same encode as {@link #forJavaScript(String)} with the
      * exception that <code>/</code> and <code>-</code> are not escaped.</p>
      *
+     * <p>Like {@link #forJavaScript(String)}, this method hex encodes
+     * U+007F and the C1 control characters U+0080 to U+009F, for example
+     * U+0085 as <code>\x85</code>.</p>
+     *
      * <p><strong>Unless you are interested in saving a few bytes of
      * output or are writing a framework on top of this library, it is
      * recommend that you use {@link #forJavaScript(String)} over this
@@ -1267,6 +1278,10 @@ public static void forJavaScriptAttribute(Writer out, String input)
      * encoded as <code>\"</code> and <code>\'</code>
      * respectively.</p>
      *
+     * <p>Like {@link #forJavaScript(String)}, this method hex encodes
+     * U+007F and the C1 control characters U+0080 to U+009F, for example
+     * U+0085 as <code>\x85</code>.</p>
+     *
      * <p><strong>Unless you are interested in saving a few bytes of
      * output or are writing a framework on top of this library, it is
      * recommend that you use {@link #forJavaScript(String)} over this
@@ -1318,6 +1333,10 @@ public static void forJavaScriptBlock(Writer out, String input)
      * escaped and <code>"</code> and <code>'</code> are encoded as
      * <code>\"</code> and <code>\'</code> respectively.</p>
      *
+     * <p>Like {@link #forJavaScript(String)}, this method hex encodes
+     * U+007F and the C1 control characters U+0080 to U+009F, for example
+     * U+0085 as <code>\x85</code>.</p>
+     *
      * <p><strong>Unless you are interested in saving a few bytes of
      * output or are writing a framework on top of this library, it is
      * recommend that you use {@link #forJavaScript(String)} over this

core/src/main/java/org/owasp/encoder/JavaScriptEncoder.java

@@ -123,7 +123,8 @@ enum Mode {
             0,
             -1 & ~((1 << '\'') | (1 << '\"')),
             -1 & ~((1 << '\\')),
-            asciiOnly ? ~(1 << Unicode.DEL) : -1,};
+            // DEL is hex encoded in every mode, like the C1 controls
+            ~(1 << Unicode.DEL),};
 
         if (mode == Mode.BLOCK || mode == Mode.HTML) {
             // in <script> blocks, we need to prevent the browser from seeing
@@ -160,7 +161,8 @@ int firstEncodedOffset(String input, int off, int len) {
                 if ((validMasks[ch >>> 5] & (1 << ch)) == 0) {
                     return i;
                 }
-            } else if (_asciiOnly || ch == Unicode.LINE_SEPARATOR || ch == Unicode.PARAGRAPH_SEPARATOR) {
+            } else if (_asciiOnly || ch <= Unicode.MAX_C1_CTRL_CHAR
+                    || ch == Unicode.LINE_SEPARATOR || ch == Unicode.PARAGRAPH_SEPARATOR) {
                 return i;
             }
         }
@@ -189,7 +191,8 @@ protected CoderResult encodeArrays(CharBuffer input, CharBuffer output, boolean
                         if ((validMasks[ch >>> 5] & (1 << ch)) == 0) {
                             break encoded;
                         }
-                    } else if (_asciiOnly || ch == Unicode.LINE_SEPARATOR || ch == Unicode.PARAGRAPH_SEPARATOR) {
+                    } else if (_asciiOnly || ch <= Unicode.MAX_C1_CTRL_CHAR
+                        || ch == Unicode.LINE_SEPARATOR || ch == Unicode.PARAGRAPH_SEPARATOR) {
                         if (ch <= 0xff) {
                             break hexEncoded;
                         }

core/src/test/java/org/owasp/encoder/JavaScriptEncoderTest.java

@@ -100,13 +100,22 @@ public static Test suite() {
                     .encode("Line Separator", "\\u2028", "\u2028")
                     .encode("Paragraph Separator", "\\u2029", "\u2029")
                     .encode("abc", "abc")
-                    .encode("ABC", "ABC");
+                    .encode("ABC", "ABC")
+                    // DEL and the C1 controls are hex encoded in every mode
+                    .encode("DEL", "\\x7f", "\u007f")
+                    .encode("U+0080", "\\x80", "\u0080")
+                    .encode("NEL", "\\x85", "\u0085")
+                    .encode("CSI", "\\x9b", "\u009b")
+                    .encode("U+009F", "\\x9f", "\u009f")
+                    .encode("NEL in text", "a\\x85b", "a\u0085b");
 
                 if (asciiOnly == 0) {
                     builder
                         .encode("unicode", "\u1234", "\u1234")
                         .encode("high-ascii", "\u00ff", "\u00ff")
-                        .valid(0x7f, Character.MAX_CODE_POINT)
+                        .encode("NBSP", "\u00a0", "\u00a0")
+                        .valid(0xa0, Character.MAX_CODE_POINT)
+                        .encoded(0x7f, Unicode.MAX_C1_CTRL_CHAR)
                         .encoded("\u2028\u2029");
                 } else {
                     builder

jakarta/src/main/resources/META-INF/java-encoder-advanced.tld

@@ -77,6 +77,7 @@
             quotation marks around the JavaScript string. Template literals (backticks) and JSON are
             unsupported; use a JSON serializer for JSON. Do not use in javascript: or data: URLs,
             where URL decoding introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScript</display-name>
         <name>forJavaScript</name>
@@ -98,6 +99,7 @@
             JavaScript string. Template literals (backticks) and JSON are unsupported; use a JSON
             serializer for JSON. Do not use in javascript: or data: URLs, where URL decoding
             introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptAttribute</display-name>
         <name>forJavaScriptAttribute</name>
@@ -119,6 +121,7 @@
             the JavaScript string. Template literals (backticks) and JSON are unsupported; use a
             JSON serializer for JSON. Do not use in javascript: or data: URLs, where URL decoding
             introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptBlock</display-name>
         <name>forJavaScriptBlock</name>
@@ -141,6 +144,7 @@
             Template literals (backticks) and JSON are unsupported; use a JSON serializer for JSON.
             Do not use in javascript: or data: URLs, where URL decoding introduces a separate
             parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptSource</display-name>
         <name>forJavaScriptSource</name>
@@ -489,6 +493,7 @@
             quotation marks around the JavaScript string. Template literals (backticks) and JSON are
             unsupported; use a JSON serializer for JSON. Do not use in javascript: or data: URLs,
             where URL decoding introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScript</display-name>
         <name>forJavaScript</name>
@@ -504,6 +509,7 @@
             JavaScript string. Template literals (backticks) and JSON are unsupported; use a JSON
             serializer for JSON. Do not use in javascript: or data: URLs, where URL decoding
             introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptAttribute</display-name>
         <name>forJavaScriptAttribute</name>
@@ -519,6 +525,7 @@
             the JavaScript string. Template literals (backticks) and JSON are unsupported; use a
             JSON serializer for JSON. Do not use in javascript: or data: URLs, where URL decoding
             introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptBlock</display-name>
         <name>forJavaScriptBlock</name>
@@ -535,6 +542,7 @@
             Template literals (backticks) and JSON are unsupported; use a JSON serializer for JSON.
             Do not use in javascript: or data: URLs, where URL decoding introduces a separate
             parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptSource</display-name>
         <name>forJavaScriptSource</name>

jakarta/src/main/resources/META-INF/java-encoder.tld

@@ -80,6 +80,7 @@
             quotation marks around the JavaScript string. Template literals (backticks) and JSON are
             unsupported; use a JSON serializer for JSON. Do not use in javascript: or data: URLs,
             where URL decoding introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScript</display-name>
         <name>forJavaScript</name>
@@ -392,6 +393,7 @@
             quotation marks around the JavaScript string. Template literals (backticks) and JSON are
             unsupported; use a JSON serializer for JSON. Do not use in javascript: or data: URLs,
             where URL decoding introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScript</display-name>
         <name>forJavaScript</name>

jsp/src/main/resources/META-INF/java-encoder-advanced.tld

@@ -77,6 +77,7 @@
             quotation marks around the JavaScript string. Template literals (backticks) and JSON are
             unsupported; use a JSON serializer for JSON. Do not use in javascript: or data: URLs,
             where URL decoding introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScript</display-name>
         <name>forJavaScript</name>
@@ -98,6 +99,7 @@
             JavaScript string. Template literals (backticks) and JSON are unsupported; use a JSON
             serializer for JSON. Do not use in javascript: or data: URLs, where URL decoding
             introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptAttribute</display-name>
         <name>forJavaScriptAttribute</name>
@@ -119,6 +121,7 @@
             the JavaScript string. Template literals (backticks) and JSON are unsupported; use a
             JSON serializer for JSON. Do not use in javascript: or data: URLs, where URL decoding
             introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptBlock</display-name>
         <name>forJavaScriptBlock</name>
@@ -141,6 +144,7 @@
             Template literals (backticks) and JSON are unsupported; use a JSON serializer for JSON.
             Do not use in javascript: or data: URLs, where URL decoding introduces a separate
             parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptSource</display-name>
         <name>forJavaScriptSource</name>
@@ -489,6 +493,7 @@
             quotation marks around the JavaScript string. Template literals (backticks) and JSON are
             unsupported; use a JSON serializer for JSON. Do not use in javascript: or data: URLs,
             where URL decoding introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScript</display-name>
         <name>forJavaScript</name>
@@ -504,6 +509,7 @@
             JavaScript string. Template literals (backticks) and JSON are unsupported; use a JSON
             serializer for JSON. Do not use in javascript: or data: URLs, where URL decoding
             introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptAttribute</display-name>
         <name>forJavaScriptAttribute</name>
@@ -519,6 +525,7 @@
             the JavaScript string. Template literals (backticks) and JSON are unsupported; use a
             JSON serializer for JSON. Do not use in javascript: or data: URLs, where URL decoding
             introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptBlock</display-name>
         <name>forJavaScriptBlock</name>
@@ -535,6 +542,7 @@
             Template literals (backticks) and JSON are unsupported; use a JSON serializer for JSON.
             Do not use in javascript: or data: URLs, where URL decoding introduces a separate
             parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScriptSource</display-name>
         <name>forJavaScriptSource</name>

jsp/src/main/resources/META-INF/java-encoder.tld

@@ -77,6 +77,7 @@
             quotation marks around the JavaScript string. Template literals (backticks) and JSON are
             unsupported; use a JSON serializer for JSON. Do not use in javascript: or data: URLs,
             where URL decoding introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScript</display-name>
         <name>forJavaScript</name>
@@ -389,6 +390,7 @@
             quotation marks around the JavaScript string. Template literals (backticks) and JSON are
             unsupported; use a JSON serializer for JSON. Do not use in javascript: or data: URLs,
             where URL decoding introduces a separate parsing context.
+            U+007F and the C1 controls U+0080 to U+009F are hex encoded, e.g. U+0085 as \x85.
         </description>
         <display-name>forJavaScript</display-name>
         <name>forJavaScript</name>