Try to implement native sorting algorithm using luajit on Windows failed

[dyphire]

mpvnet-player/mpv.net#575 (comment) mentioned a method to implement native sorting algorithms on Windows for Lua.
But an error occurred when I applied the following patch to file-browser.lua, causing it to fail.

---
 scripts/file-browser.lua | 58 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 57 insertions(+), 1 deletion(-)

diff --git a/scripts/file-browser.lua b/scripts/file-browser.lua
index c59e7c8..23f2eea 100644
--- a/scripts/file-browser.lua
+++ b/scripts/file-browser.lua
@@ -474,7 +474,55 @@ end
 --sorts the table lexicographically ignoring case and accounting for leading/non-leading zeroes
 --the number format functionality was proposed by github user twophyro, and was presumably taken
 --from here: http://notebook.kulchenko.com/algorithms/alphanumeric-natural-sorting-for-humans-in-lua
-function API.sort(t)
+--this function was taken from https://github.com/mpvnet-player/mpv.net/issues/575#issuecomment-1817413401
+local winapi = {}
+local is_windows = package.config:sub(1,1) == "\\"
+
+if is_windows then
+    is_ffi_loaded, ffi = pcall(require, "ffi")
+
+    if is_ffi_loaded then
+        winapi = {
+            ffi = ffi,
+            C = ffi.C,
+            CP_UTF8 = 65001,
+            shlwapi = ffi.load("shlwapi"),
+        }
+
+        -- ffi code from https://github.com/po5/thumbfast, Mozilla Public License Version 2.0
+        ffi.cdef[[
+            int __stdcall MultiByteToWideChar(unsigned int CodePage, unsigned long dwFlags, const char *lpMultiByteStr,
+            int cbMultiByte, wchar_t *lpWideCharStr, int cchWideChar);
+            int __stdcall StrCmpLogicalW(wchar_t *psz1, wchar_t *psz2);
+        ]]
+
+        winapi.utf8_to_wide = function(utf8_str)
+            if utf8_str then
+                local utf16_len = winapi.C.MultiByteToWideChar(winapi.CP_UTF8, 0, utf8_str, -1, nil, 0)
+
+                if utf16_len > 0 then
+                    local utf16_str = winapi.ffi.new("wchar_t[?]", utf16_len)
+
+                    if winapi.C.MultiByteToWideChar(winapi.CP_UTF8, 0, utf8_str, -1, utf16_str, utf16_len) > 0 then
+                        return utf16_str
+                    end
+                end
+            end
+
+            return ""
+        end
+    end
+end
+
+function API.sort_windows(t)
+    table.sort(t, function(a, b)
+        local a_wide = winapi.utf8_to_wide(a.type:sub(1, 1) .. (a.label or a.name))
+        local b_wide = winapi.utf8_to_wide(b.type:sub(1, 1) .. (b.label or b.name))
+        return winapi.shlwapi.StrCmpLogicalW(a_wide, b_wide) == -1
+    end)
+end
+
+function API.sort_lua(t)
     local function padnum(n, d)
         return #d > 0 and ("%03d%s%.12f"):format(#n, n, tonumber(d) / (10 ^ #d))
             or ("%03d%s"):format(#n, n)
@@ -492,6 +540,14 @@ function API.sort(t)
     return t
 end
 
+function API.sort(t)
+    if is_windows and is_ffi_loaded then
+        API.sort_windows(t)
+    else
+        API.sort_lua(t)
+    end
+end
+
 function API.valid_dir(dir)
     if o.filter_dot_dirs and string.sub(dir, 1, 1) == "." then return false end
     return true
-- 

This is the error log:

[   5.537][v][file_browser] opening directory: G:/ 
[   5.537][v][file_browser] scanning files in G:/ 
[   5.537][d][file_browser] finding parser for G:/ 
[   5.537][d][file_browser] attempting parser: filter 
[   5.537][d][file_browser] finding parser for G:/ 
[   5.537][d][file_browser] attempting parser: sort 
[   5.537][d][file_browser] attempting parser: cmd-dir 
[   5.537][d][cplayer] Run command: subprocess, flags=64, args=[args="cmd,/U,/c,dir,/b,/ad,G:\\", playback_only="no", capture_size="67108864", capture_stdout="yes", capture_stderr="yes", detach="no", env="", stdin_data="", passthrough_stdin="no"]
[   5.558][d][cplayer] Run command: subprocess, flags=64, args=[args="cmd,/U,/c,dir,/b,/a-d,G:\\", playback_only="no", capture_size="67108864", capture_stdout="yes", capture_stderr="yes", detach="no", env="", stdin_data="", passthrough_stdin="no"]
[   5.576][e][file_browser] [windir] 找不到文件
[   5.576][e][file_browser]  
[   5.576][d][file_browser] attempting parser: file 
[   5.576][d][file_browser] no successful parsers found 
[   5.576][w][file_browser] could not read directory G:/ 

[CogentRedTester]

It's likely because you're not returning the results of API.sort() and API.sort_windows. The documentation specifies that the table being sorted is also returned by the function.

[CogentRedTester]

I created an addon that replaces the default sorting algorithm with this algorithm. Perhaps you could test it (I'm not sure what to look for):

local fb_utils = require 'modules.utils'

--this code is based on https://github.com/mpvnet-player/mpv.net/issues/575#issuecomment-1817413401
local ffi = require "ffi"
local winapi = {
        ffi = ffi,
        C = ffi.C,
        CP_UTF8 = 65001,
        shlwapi = ffi.load("shlwapi"),
    }

-- ffi code from https://github.com/po5/thumbfast, Mozilla Public License Version 2.0
ffi.cdef[[
    int __stdcall MultiByteToWideChar(unsigned int CodePage, unsigned long dwFlags, const char *lpMultiByteStr,
    int cbMultiByte, wchar_t *lpWideCharStr, int cchWideChar);
    int __stdcall StrCmpLogicalW(wchar_t *psz1, wchar_t *psz2);
]]

winapi.utf8_to_wide = function(utf8_str)
    if utf8_str then
        local utf16_len = winapi.C.MultiByteToWideChar(winapi.CP_UTF8, 0, utf8_str, -1, nil, 0)

        if utf16_len > 0 then
            local utf16_str = winapi.ffi.new("wchar_t[?]", utf16_len)

            if winapi.C.MultiByteToWideChar(winapi.CP_UTF8, 0, utf8_str, -1, utf16_str, utf16_len) > 0 then
                return utf16_str
            end
        end
    end

    return ""
end

fb_utils.sort = function (t)
    table.sort(t, function(a, b)
        local a_wide = winapi.utf8_to_wide(a.type:sub(1, 1) .. (a.label or a.name))
        local b_wide = winapi.utf8_to_wide(b.type:sub(1, 1) .. (b.label or b.name))
        return winapi.shlwapi.StrCmpLogicalW(a_wide, b_wide) == -1
    end)

    return t
end

return { version = '1.1.0', api_version = '1.1.0 }

Edit: updated on 2024-01-08 to work with new file-browser internal changes (#100)
Edit: updated on 2025-03-31 to add new api_version field.

[dyphire]

It's likely because you're not returning the results of API.sort() and API.sort_windows. The documentation specifies that the table being sorted is also returned by the function.

Oh, I missed it.

I created an addon that replaces the default sorting algorithm with this algorithm. Perhaps you could test it (I'm not sure what to look for):

Yes, it works.

[CogentRedTester]

Good to hear, I am going to add this addon to the wiki.