Update to mupdf 1.25

[itsjunetime]

This updates to mupdf 1.25, mainly so that we can use the new fz_search_stext_page_cb function (since the old searching interface was quite clunky). There's also a lot of other changes in this release (see here) but since we haven't added bindings for the other new stuff yet, we won't really get any of these benefits.

I feel pretty good about the ffi fuckery I had to do to get this new search interface working, but obviously some second eyes would still be appreciated :)

Also, it seems that mupdf 1.25 does some computations about search result quads slightly differently, resulting in some different results that I adjusted to here. Most of these result in a change of ~0.1, but some resulted in a change of ~7, which is probably not a big issue but just maybe something to be aware of. I'd be happy to chalk it up to some slight differences in text rendering or whatever.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[messense]

Looks like the build failure is caused by ArtifexSoftware/mupdf@0e21bb9

[itsjunetime]

Hmm, yeah this may be quite difficult. I don't have any windows computers, so I'm not able to try out msys2 builds on my local machine. And the fact that everything besides msys2 is working correctly is somewhat concerning - maybe there's some documentation somewhere about building on msys2 that I just need to read? I'm not sure. I'll take another look at this later.

[messense]

cc @vinxv in case you know how to resolve this.

[vinxv]

cc @vinxv in case you know how to resolve this.

After investigating the issue, I've identified two necessary adjustments for successful compilation of MuPDF 1.25.x under MSYS2:

Required Changes:

1. Compilation Flag:
   Include the SSE4.1 instruction set support by adding the following compiler flag:

   XCFLAGS="-msse4.1"
   

2. Source Code Modification:
   A correction is needed in the MuPDF source code, specifically in source/fitz/time.c at line 37:

   Current code (Microsoft-specific):

   #define DELTA_EPOCH_IN_MICROSECS 11644473600000000Ui64

   replacement (C99 standard):

   #define DELTA_EPOCH_IN_MICROSECS 11644473600000000ULL

This change replaces the Microsoft-specific integer suffix Ui64 with the standard C99 ULL suffix, ensuring proper recognition as a 64-bit unsigned integer when compiling with GCC/MinGW.

I anticipate addressing this issue this Saturday. Of course, I would greatly appreciate if someone else implements these fixes beforehand.

Thank you.

[itsjunetime]

1. Compilation Flag:
   Include the SSE4.1 instruction set support by adding the following compiler flag:

   XCFLAGS="-msse4.1"
   

It seems we were thinking about the same thing at the same time - I actually just added this in d37faf4, just a few minutes before your comment. I tested that my commit did, in fact, produce the expected features flags (such as -mneon on my machine). It seems to not have fixed compilation on CI, though, unfortunately.

I also feel like this should have worked, but maybe these flags are somehow being lost somewhere in the various layers of build scripts and makefiles? I'm not certain.

2. Source Code Modification:
   A correction is needed in the MuPDF source code, specifically in source/fitz/time.c at line 37:

Current code (Microsoft-specific):

#define DELTA_EPOCH_IN_MICROSECS 11644473600000000Ui64

replacement (C99 standard):

#define DELTA_EPOCH_IN_MICROSECS 11644473600000000ULL

According to the comment associated with the commit that made this change, this suffix works with mingw, but I think you are right that this wouldn't work on clang on windows. I can file a revert PR for upstream mupdf and see what they think.

[itsjunetime]

Ah, looks like I was confused - the commit I linked actually fixes the issue you mentioned; we just need to pull it in. I'll upgrade this branch to get that commit in.

[vinxv]

The MuPDF makefile does not appear to support HAVE_SSE4_1.

In mupdf/include/mupdf/fitz/system.h, the relevant code is defined as:

/* We assume that pretty much any X86 or X64 machine has SSE these days. */
#ifndef ARCH_HAS_SSE
#if defined(_M_IX86) || defined(_M_AMD64) || defined(_M_X64)
#define ARCH_HAS_SSE 1
#endif
#endif

#ifndef ARCH_HAS_SSE
#define ARCH_HAS_SSE 0
#endif

One approach to customize SSE support is to pass "-DARCH_HAS_SSE=0" in the make parameters.

We can compile MuPDF in the MSYS2 environment using:

make HAVE_X11=no HAVE_GLUT=no XCFLAGS="-DARCH_HAS_SSE=0"

However, using XCFLAGS='-msse4.1' seems clearer.

Additionally, during compilation, we may encounter an "argument list too long" issue:

/bin/sh: line 1: /ucrt64/bin/ar: Argument list too long

The MuPDF community has already addressed this issue: ArtifexSoftware/mupdf@a0b3ae1

We can resolve this by adding the USE_ARGUMENT_FILE=yes parameter, though it may require further MuPDF dependency upgrades.

In summary, here are the specific changes to address these issues:

@@ -114,6 +114,7 @@ fn build_libmupdf() {
         "HAVE_X11=no".to_owned(),
         "HAVE_GLUT=no".to_owned(),
         "HAVE_CURL=no".to_owned(),
+        "USE_ARGUMENT_FILE=yes".to_owned(),
         "verbose=yes".to_owned(),
     ];
 
@@ -176,6 +177,9 @@ fn build_libmupdf() {
     make_flags.push(format!("XCFLAGS={}", c_flags.to_string_lossy()));
     make_flags.push(format!("XCXXFLAGS={}", cxx_flags.to_string_lossy()));
 
+    #[cfg(all(target_os="windows", any(target_arch="x86", target_arch="x86_64")))]
+    make_flags.push("XCFLAGS='-msse4.1'".to_string());
+
     // Enable parallel compilation
     if let Ok(n) = std::thread::available_parallelism() {
         make_flags.push(format!("-j{}", n));