Different implementation of toLowerCase() and toUpperCase() compared to Firefox

[fschuetz04]

Description

CouchDB apparently has a different implementation of toLowerCase() and toUpperCase() for some Unicode characters compared to Firefox 60.9.0esr. I suspect that this also occurs with other versions and browsers, but I couldn't test this yet. All characters for which I found different results can be found here:

• toLowerCase(): https://gist.github.com/fschuetz04/54cca79199f11205d662410ed453d2e0#tolowercase
• toUpperCase(): https://gist.github.com/fschuetz04/54cca79199f11205d662410ed453d2e0#touppercase

Steps to Reproduce

I created a database containing documents with IDs from 0 to 2^16 - 1. I then created the following view to get CouchDB's implementation of toLowerCase().

function (doc) {
  emit([doc._id, String.fromCharCode(doc._id), String.fromCharCode(doc._id).toLowerCase()]);
}

I then iterated through all characters in the view and checked whether the implementation of toLowerCase() in my browser is different from CouchDB's implementation for a particular character:

fetch("http://couchdb_host:5984/test/_design/to_lower_case/_view/to_lower_case").then(async response => {
  const result = await response.json();
  let count = 0;
  for (const row of result.rows) {
    const [id, char, lower] = row.key;
    if (char.toLowerCase() != lower) {
      console.log(id, char, lower, char.toLowerCase());
      count += 1;
    }
  }
  console.log(count);
});

I did the same thing for toUpperCase(), only replacing the relevant function calls.

Expected Behaviour

I expect CouchDB's implementation of toLowerCase() and toUpperCase() to match the browser implementation and didn't find any notes about differences. This is desirable to implement something like case-insensitive search on Unicode strings.

Your Environment

CouchDB information:

{
  "couchdb": "Welcome",
  "version": "2.3.1",
  "git_sha": "c298091a4",
  "uuid": "5f7250de36b64a810fb4013d8e20c2f6",
  "features": ["pluggable-storage-engines", "scheduler"],
  "vendor": { "name": "The Apache Software Foundation" }
}

• Browser name and version: Firefox, 60.9.0esr (32-bit)
• Operating System and version: Windows Server 2016 Standard, 10.0.14393

[rnewson]

CouchDB uses spidermonkey, the same javascript interpreter that Firefox uses, but only up to version 1.8.5, which is quite old.

Work is afoot to incorporate a newer library that will reflect the latest ECMAScript spec.

[rnewson]

see #2170 in fact.

[fschuetz04]

Okay, thank you. I hope this is brought forward.