Skip to content

Commit

Permalink
refactor: replace re2 with re2-wasm (DIYgod#12998)
Browse files Browse the repository at this point in the history
* refactor: replace re2 with re2-wasm

* refactor: remove migrated routes

* ci: remove foresight

https://github.com/runforesight
Foresight is no longer in service

* fix: add back removed wasm

chore: bump base OS image
  • Loading branch information
TonyRL authored Aug 11, 2023
1 parent fa27a27 commit 79eae1e
Show file tree
Hide file tree
Showing 7 changed files with 60 additions and 1,726 deletions.
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
// https://github.com/devcontainers/images/blob/v0.2.24/src/javascript-node/.devcontainer/devcontainer.json
{
"name": "Node.js",
"image": "mcr.microsoft.com/devcontainers/javascript-node:18-bullseye",
"image": "mcr.microsoft.com/devcontainers/javascript-node:18-bookworm",

// Configure tool-specific properties.
"customizations": {
Expand Down
7 changes: 1 addition & 6 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,6 @@ jobs:
node-version: [ 18, 20 ]
name: Jest on Node ${{ matrix.node-version }}
steps:
- name: Collect Workflow Telemetry
uses: runforesight/foresight-workflow-kit-action@v1
if: success() || failure()
with:
api_key: ${{ secrets.FORESIGHT_KEY }}
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2
with:
Expand Down Expand Up @@ -107,7 +102,7 @@ jobs:
if: ${{ matrix.chromium.dependency != '' }}
# 'chromium-browser' from Ubuntu APT repo is a dummy package. Its version (85.0.4183.83) means
# nothing since it calls Snap (disgusting!) to install Chromium, which should be up-to-date.
# That's not really a problem since the Chromium-bundled Docker image is based on Debian bullseye,
# That's not really a problem since the Chromium-bundled Docker image is based on Debian bookworm,
# which provides up-to-date native packages.
run: |
set -ex
Expand Down
18 changes: 11 additions & 7 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:18-bullseye as dep-builder
FROM node:18-bookworm AS dep-builder
# Here we use the non-slim image to provide build-time deps (compilers and python), thus no need to install later.
# This effectively speeds up qemu-based cross-build.

Expand Down Expand Up @@ -28,11 +28,11 @@ RUN \

# ---------------------------------------------------------------------------------------------------------------------

FROM debian:bullseye-slim as dep-version-parser
FROM debian:bookworm-slim AS dep-version-parser
# This stage is necessary to limit the cache miss scope.
# With this stage, any modification to package.json won't break the build cache of the next two stages as long as the
# version unchanged.
# node:18-bullseye-slim is based on debian:bullseye-slim so this stage would not cause any additional download.
# node:18-bookworm-slim is based on debian:bookworm-slim so this stage would not cause any additional download.

WORKDIR /ver
COPY ./package.json /app/
Expand All @@ -44,7 +44,7 @@ RUN \

# ---------------------------------------------------------------------------------------------------------------------

FROM node:18-bullseye-slim as docker-minifier
FROM node:18-bookworm-slim AS docker-minifier
# The stage is used to further reduce the image size by removing unused files.

WORKDIR /minifier
Expand All @@ -66,9 +66,13 @@ COPY --from=dep-builder /app /app

RUN \
set -ex && \
# Minify Docker image
cp /app/scripts/docker/minify-docker.js /minifier/ && \
export PROJECT_ROOT=/app && \
node /minifier/minify-docker.js && \
# Add back anything that are required but removed by by @vercel/nft
cp /app/node_modules/re2-wasm/build/wasm/re2.wasm /app/app-minimal/node_modules/re2-wasm/build/wasm/re2.wasm && \
# Use the minified node_modules
rm -rf /app/node_modules /app/scripts && \
mv /app/app-minimal/node_modules /app/ && \
rm -rf /app/app-minimal && \
Expand All @@ -77,7 +81,7 @@ RUN \

# ---------------------------------------------------------------------------------------------------------------------

FROM node:18-bullseye-slim as chromium-downloader
FROM node:18-bookworm-slim AS chromium-downloader
# This stage is necessary to improve build concurrency and minimize the image size.
# Yeah, downloading Chromium never needs those dependencies below.

Expand Down Expand Up @@ -109,7 +113,7 @@ RUN \

# ---------------------------------------------------------------------------------------------------------------------

FROM node:18-bullseye-slim as app
FROM node:18-bookworm-slim AS app

LABEL org.opencontainers.image.authors="https://github.com/DIYgod/RSSHub"

Expand All @@ -123,7 +127,7 @@ ARG TARGETPLATFORM
ARG PUPPETEER_SKIP_DOWNLOAD=1
# https://pptr.dev/troubleshooting#chrome-headless-doesnt-launch-on-unix
# https://github.com/puppeteer/puppeteer/issues/7822
# https://www.debian.org/releases/bullseye/amd64/release-notes/ch-information.en.html#noteworthy-obsolete-packages
# https://www.debian.org/releases/bookworm/amd64/release-notes/ch-information.en.html#noteworthy-obsolete-packages
# The official recommended way to use Puppeteer on arm/arm64 is to install Chromium from the distribution repositories:
# https://github.com/puppeteer/puppeteer/blob/07391bbf5feaf85c191e1aa8aa78138dce84008d/packages/puppeteer-core/src/node/BrowserFetcher.ts#L128-L131
RUN \
Expand Down
6 changes: 3 additions & 3 deletions lib/middleware/parameter.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ const cheerio = require('cheerio');
const { simplecc } = require('simplecc-wasm');
const got = require('@/utils/got');
const config = require('@/config').value;
const RE2 = require('re2');
const { RE2 } = require('re2-wasm');

let mercury_parser;

Expand Down Expand Up @@ -167,7 +167,7 @@ module.exports = async (ctx, next) => {
case 'regexp':
return new RegExp(string, 'i');
case 're2':
return new RE2(string, 'i');
return new RE2(string, 'iu');
default:
throw Error(`Invalid Engine Value: ${engine}, please check your config.`);
}
Expand All @@ -176,7 +176,7 @@ module.exports = async (ctx, next) => {
case 'regexp':
return new RegExp(string);
case 're2':
return new RE2(string);
return new RE2(string, 'u');
default:
throw Error(`Invalid Engine Value: ${engine}, please check your config.`);
}
Expand Down
Loading

0 comments on commit 79eae1e

Please sign in to comment.