Skip to content

feat: add i18n, multilingual ML, and OpenAI-like API support#134

Open
reg2005 wants to merge 5 commits into
IliasHad:mainfrom
reg2005:main
Open

feat: add i18n, multilingual ML, and OpenAI-like API support#134
reg2005 wants to merge 5 commits into
IliasHad:mainfrom
reg2005:main

Conversation

@reg2005

@reg2005 reg2005 commented May 16, 2026

Copy link
Copy Markdown

Add first-class English/Russian internationalization across the web app and wire the selected language through the backend/ML pipeline.

This introduces an i18n layer with EN/RU resource bundles, a language preference setting, localized UI copy across the main product surfaces, and shared language typing so app routes, jobs, settings, and services can consistently resolve the active locale.

The ML/search pipeline is updated for multilingual workflows as well. Text embeddings now use the multilingual Xenova/paraphrase-multilingual-mpnet-base-v2 model, Russian search queries are normalized into canonical metadata filters without losing the original semantic query, suggestions can display localized text while preserving canonical backend values, and scene/vector descriptions can be generated in Russian from detected metadata. Transcription and reverse-geocoding also receive the selected language so language-specific results are produced closer to the source.

This also adds OpenAI-like API support for chat/model providers, including diagnostics that make provider configuration, routing, and compatibility issues easier to debug.

Reviewer notes:

  • Existing text vector collections need to be rebuilt/reindexed after the text embedding model change.
  • Canonical filter values remain English internally; Russian is used for UI display and query normalization.
  • Russian scene descriptions are metadata-based rather than direct translations of BLIP captions, avoiding brittle caption translation while keeping generated descriptions localized.

@coderabbitai

coderabbitai Bot commented May 16, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 277 files, which is 127 over the limit of 150.

To get a review, narrow the scope:
• coderabbit review --type committed # exclude uncommitted changes
• coderabbit review --dir # limit to a subdirectory
• coderabbit review --base # compare against a closer base

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c4a1b653-8357-4074-aa08-541ea9c81ce0

📥 Commits

Reviewing files that changed from the base of the PR and between 28b58bf and 2b0108c.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (277)
  • .env.example
  • apps/background-jobs/src/jobs/chat.ts
  • apps/background-jobs/src/jobs/faceDeletion.ts
  • apps/background-jobs/src/jobs/faceLabelling.ts
  • apps/background-jobs/src/jobs/faceRenaming.ts
  • apps/background-jobs/src/jobs/frameAnalysis.ts
  • apps/background-jobs/src/jobs/sceneCreation.ts
  • apps/background-jobs/src/jobs/textEmbedding.ts
  • apps/background-jobs/src/jobs/transcription.ts
  • apps/background-jobs/src/jobs/updateVideo.ts
  • apps/background-jobs/src/routes/chats.ts
  • apps/background-jobs/src/routes/folders.ts
  • apps/background-jobs/src/routes/indexer.ts
  • apps/background-jobs/src/routes/jobs.ts
  • apps/background-jobs/src/schemas/chat.ts
  • apps/background-jobs/src/schemas/indexer.ts
  • apps/background-jobs/src/services/videoIndexer.ts
  • apps/background-jobs/src/watcher.ts
  • apps/background-jobs/tests/routes/chats.test.ts
  • apps/background-jobs/tests/routes/folders.test.ts
  • apps/background-jobs/tests/routes/indexer.test.ts
  • apps/web/app/features/auth/components/ForgotPasswordLink.tsx
  • apps/web/app/features/auth/components/MobileLogo.tsx
  • apps/web/app/features/auth/components/Sidebar.tsx
  • apps/web/app/features/auth/components/TermsOfService.tsx
  • apps/web/app/features/chats/components/ChatCard.tsx
  • apps/web/app/features/chats/components/ChatHistory.tsx
  • apps/web/app/features/chats/components/ChatInput.tsx
  • apps/web/app/features/chats/components/ChatItem.tsx
  • apps/web/app/features/chats/components/Compiler.tsx
  • apps/web/app/features/chats/components/ExportedScenes.tsx
  • apps/web/app/features/chats/components/LoadingIndicator.tsx
  • apps/web/app/features/chats/components/Message.tsx
  • apps/web/app/features/chats/components/PreviewModal.tsx
  • apps/web/app/features/chats/components/ProjectSelector.tsx
  • apps/web/app/features/chats/components/SceneCard.tsx
  • apps/web/app/features/chats/components/StitchedVideo.tsx
  • apps/web/app/features/chats/components/VideoResults.tsx
  • apps/web/app/features/chats/components/Welcome.tsx
  • apps/web/app/features/chats/plugins/FacesMention.tsx
  • apps/web/app/features/chats/schemas/index.ts
  • apps/web/app/features/chats/stores/index.ts
  • apps/web/app/features/collections/components/CollectionCard.tsx
  • apps/web/app/features/collections/constants/index.tsx
  • apps/web/app/features/collections/schemas/index.ts
  • apps/web/app/features/collections/stores/index.ts
  • apps/web/app/features/customVideoPlayer/components/AIVisionBadge.tsx
  • apps/web/app/features/customVideoPlayer/components/CaptionsButton.tsx
  • apps/web/app/features/customVideoPlayer/components/FitButton.tsx
  • apps/web/app/features/customVideoPlayer/components/FullscreenButton.tsx
  • apps/web/app/features/customVideoPlayer/components/LoadingOverlay.tsx
  • apps/web/app/features/customVideoPlayer/components/OverlayControls.tsx
  • apps/web/app/features/customVideoPlayer/components/PlaybackControls.tsx
  • apps/web/app/features/customVideoPlayer/components/TranscodingOverlay.tsx
  • apps/web/app/features/customVideoPlayer/components/VolumeControl.tsx
  • apps/web/app/features/customVideoPlayer/components/index.tsx
  • apps/web/app/features/faces/components/FaceCard.tsx
  • apps/web/app/features/faces/components/KnownFacesGrid.tsx
  • apps/web/app/features/faces/components/LabelingForm.tsx
  • apps/web/app/features/faces/components/Pagination.tsx
  • apps/web/app/features/faces/components/RenameDialog.tsx
  • apps/web/app/features/faces/components/UnknownFacesGrid.tsx
  • apps/web/app/features/folders/components/AddFolder.tsx
  • apps/web/app/features/folders/components/FolderCard.tsx
  • apps/web/app/features/folders/utils/index.tsx
  • apps/web/app/features/immich/components/ActiveIntegration.tsx
  • apps/web/app/features/immich/components/Documentation.tsx
  • apps/web/app/features/immich/components/ImportProgress.tsx
  • apps/web/app/features/immich/components/IntegrationForm.tsx
  • apps/web/app/features/immich/hooks/useImmichForm.ts
  • apps/web/app/features/immich/hooks/useImmichImportStatus.ts
  • apps/web/app/features/immich/stores/index.ts
  • apps/web/app/features/jobs/components/JobCard.tsx
  • apps/web/app/features/onboarding/components/OnboardingNavigation.tsx
  • apps/web/app/features/onboarding/components/OnboardingStep.tsx
  • apps/web/app/features/onboarding/components/SkipButton.tsx
  • apps/web/app/features/onboarding/constants/onboarding.ts
  • apps/web/app/features/projects/components/ProjectCard.tsx
  • apps/web/app/features/projects/components/ProjectDetails.tsx
  • apps/web/app/features/projects/components/SelectedVideoChip.tsx
  • apps/web/app/features/projects/schemas/index.ts
  • apps/web/app/features/projects/stores/index.ts
  • apps/web/app/features/search/components/FilterChips.tsx
  • apps/web/app/features/search/components/ImageUpload.tsx
  • apps/web/app/features/search/components/ProgressBar.tsx
  • apps/web/app/features/search/components/SearchInput.tsx
  • apps/web/app/features/search/components/SearchModeTabs.tsx
  • apps/web/app/features/search/components/SearchResults.tsx
  • apps/web/app/features/search/components/SearchSuggestions.tsx
  • apps/web/app/features/search/components/SuggestionsDropdown.tsx
  • apps/web/app/features/search/components/VideoCard.tsx
  • apps/web/app/features/search/constants/index.ts
  • apps/web/app/features/search/stores/index.ts
  • apps/web/app/features/services/constants/index.test.ts
  • apps/web/app/features/services/constants/index.ts
  • apps/web/app/features/settings/components/AdvancedSettings.tsx
  • apps/web/app/features/settings/components/FolderSettings.tsx
  • apps/web/app/features/settings/components/LanguageSelector.tsx
  • apps/web/app/features/settings/components/PreferencesSettings.tsx
  • apps/web/app/features/settings/components/Tabs.tsx
  • apps/web/app/features/setup/components/FolderPanel.tsx
  • apps/web/app/features/setup/components/ServicesPanel.tsx
  • apps/web/app/features/setup/components/SetupHeader.tsx
  • apps/web/app/features/setup/components/SetupNavigation.tsx
  • apps/web/app/features/setup/components/SetupStep.tsx
  • apps/web/app/features/setup/constants/index.ts
  • apps/web/app/features/setup/types/index.ts
  • apps/web/app/features/shared/components/Pagination.tsx
  • apps/web/app/features/shared/components/ServicesStatus.tsx
  • apps/web/app/features/shared/components/Sidebar.tsx
  • apps/web/app/features/shared/components/Toggle.tsx
  • apps/web/app/features/shared/components/Version.tsx
  • apps/web/app/features/shared/components/VideoCard.tsx
  • apps/web/app/features/videos/components/ActiveSceneCard.tsx
  • apps/web/app/features/videos/components/AddVideoLabels.tsx
  • apps/web/app/features/videos/components/ProcessingJobDetails.tsx
  • apps/web/app/features/videos/components/RelinkVideo.tsx
  • apps/web/app/features/videos/components/ScenesList.tsx
  • apps/web/app/features/videos/components/ScenesSidebar.tsx
  • apps/web/app/features/videos/components/SortButton.tsx
  • apps/web/app/features/videos/components/UpdateLocation.tsx
  • apps/web/app/features/videos/components/VideoHeader.tsx
  • apps/web/app/i18n/I18nProvider.tsx
  • apps/web/app/i18n/config.ts
  • apps/web/app/i18n/index.ts
  • apps/web/app/i18n/resources/en.ts
  • apps/web/app/i18n/resources/en/auth.ts
  • apps/web/app/i18n/resources/en/chats.ts
  • apps/web/app/i18n/resources/en/collections.ts
  • apps/web/app/i18n/resources/en/common.ts
  • apps/web/app/i18n/resources/en/faces.ts
  • apps/web/app/i18n/resources/en/folders.ts
  • apps/web/app/i18n/resources/en/home.ts
  • apps/web/app/i18n/resources/en/immich.ts
  • apps/web/app/i18n/resources/en/jobs.ts
  • apps/web/app/i18n/resources/en/onboarding.ts
  • apps/web/app/i18n/resources/en/player.ts
  • apps/web/app/i18n/resources/en/projects.ts
  • apps/web/app/i18n/resources/en/root.ts
  • apps/web/app/i18n/resources/en/search.ts
  • apps/web/app/i18n/resources/en/settings.ts
  • apps/web/app/i18n/resources/en/setup.ts
  • apps/web/app/i18n/resources/en/shell.ts
  • apps/web/app/i18n/resources/en/sidebar.ts
  • apps/web/app/i18n/resources/en/ui.ts
  • apps/web/app/i18n/resources/en/videos.ts
  • apps/web/app/i18n/resources/resources.test.ts
  • apps/web/app/i18n/resources/ru.ts
  • apps/web/app/i18n/resources/ru/auth.ts
  • apps/web/app/i18n/resources/ru/chats.ts
  • apps/web/app/i18n/resources/ru/collections.ts
  • apps/web/app/i18n/resources/ru/common.ts
  • apps/web/app/i18n/resources/ru/faces.ts
  • apps/web/app/i18n/resources/ru/folders.ts
  • apps/web/app/i18n/resources/ru/home.ts
  • apps/web/app/i18n/resources/ru/immich.ts
  • apps/web/app/i18n/resources/ru/jobs.ts
  • apps/web/app/i18n/resources/ru/onboarding.ts
  • apps/web/app/i18n/resources/ru/player.ts
  • apps/web/app/i18n/resources/ru/projects.ts
  • apps/web/app/i18n/resources/ru/root.ts
  • apps/web/app/i18n/resources/ru/search.ts
  • apps/web/app/i18n/resources/ru/settings.ts
  • apps/web/app/i18n/resources/ru/setup.ts
  • apps/web/app/i18n/resources/ru/shell.ts
  • apps/web/app/i18n/resources/ru/sidebar.ts
  • apps/web/app/i18n/resources/ru/ui.ts
  • apps/web/app/i18n/resources/ru/videos.ts
  • apps/web/app/i18n/translate.ts
  • apps/web/app/root.tsx
  • apps/web/app/routes/api.chats.$id.messages.ts
  • apps/web/app/routes/api.chats.ts
  • apps/web/app/routes/api.import-videos.ts
  • apps/web/app/routes/api.search._index.ts
  • apps/web/app/routes/api.search.image.ts
  • apps/web/app/routes/api.search.suggestions.ts
  • apps/web/app/routes/api.settings.ts
  • apps/web/app/routes/api.videos.$id.ts
  • apps/web/app/routes/app.chats.$id.tsx
  • apps/web/app/routes/app.chats._index.tsx
  • apps/web/app/routes/app.chats.new.tsx
  • apps/web/app/routes/app.collections.$id._index.tsx
  • apps/web/app/routes/app.collections.$id.scenes.tsx
  • apps/web/app/routes/app.collections._index.tsx
  • apps/web/app/routes/app.faces.$name.tsx
  • apps/web/app/routes/app.faces._index.tsx
  • apps/web/app/routes/app.folders.$id.tsx
  • apps/web/app/routes/app.home.tsx
  • apps/web/app/routes/app.immich-import.tsx
  • apps/web/app/routes/app.jobs._index.tsx
  • apps/web/app/routes/app.projects.$id.tsx
  • apps/web/app/routes/app.projects._index.tsx
  • apps/web/app/routes/app.projects.new.tsx
  • apps/web/app/routes/app.search.tsx
  • apps/web/app/routes/app.settings._index.tsx
  • apps/web/app/routes/app.videos.$id.tsx
  • apps/web/app/routes/auth.login.tsx
  • apps/web/app/routes/auth.register.tsx
  • apps/web/app/routes/onboarding.tsx
  • apps/web/app/services/user.server.ts
  • apps/web/package.json
  • apps/web/server.js
  • apps/web/vite.config.ts
  • docker/docker-compose.dev.yml
  • packages/ai/package.json
  • packages/ai/src/constants/index.ts
  • packages/ai/src/constants/prompts.ts
  • packages/ai/src/services/gemini.ts
  • packages/ai/src/services/modelRouter.ts
  • packages/ai/src/services/ollama.ts
  • packages/ai/src/services/openaiLike.ts
  • packages/ai/src/types/ai.ts
  • packages/ai/tests/constants/prompts.test.ts
  • packages/ai/tests/services/modelRouter.test.ts
  • packages/ai/tests/services/openaiLike.test.ts
  • packages/chat/package.json
  • packages/chat/src/handlers/analytics.ts
  • packages/chat/src/handlers/compilation.ts
  • packages/chat/src/handlers/refinement.ts
  • packages/chat/src/handlers/similarity.ts
  • packages/chat/src/services/processor.ts
  • packages/chat/src/types/index.ts
  • packages/chat/tests/processor.test.ts
  • packages/chat/tsconfig.json
  • packages/chat/vitest.config.ts
  • packages/db/src/index.ts
  • packages/db/src/models/AppSettings.ts
  • packages/embedding-core/src/utils/textEmbedding.ts
  • packages/media-utils/package.json
  • packages/media-utils/src/types/video.ts
  • packages/media-utils/src/utils/scenes.ts
  • packages/media-utils/tests/scenes.test.ts
  • packages/media-utils/vitest.config.ts
  • packages/prisma/migrations/20260516000000_add_app_settings_language/migration.sql
  • packages/prisma/schema.prisma
  • packages/search/package.json
  • packages/search/src/services/suggestion.ts
  • packages/search/src/utils/localizedQuery.ts
  • packages/search/src/utils/query.ts
  • packages/search/tests/localizedQuery.test.ts
  • packages/search/tests/query.test.ts
  • packages/search/tests/suggestion.test.ts
  • packages/search/vitest.config.ts
  • packages/shared/src/constants/embedding.ts
  • packages/shared/src/schemas/search.ts
  • packages/shared/src/services/pythonService.ts
  • packages/shared/src/types/index.ts
  • packages/shared/src/types/language.ts
  • packages/shared/src/types/search.ts
  • packages/shared/src/types/video.ts
  • packages/shared/src/utils/frameAnalyze.ts
  • packages/shared/src/utils/location.ts
  • packages/shared/src/utils/transcribe.ts
  • packages/shared/tests/embedding.test.ts
  • packages/shared/tests/language.test.ts
  • packages/shared/tests/location.test.ts
  • packages/ui/src/components/AlertModal.tsx
  • packages/ui/src/components/ConfirmationModal.tsx
  • packages/ui/src/components/DeleteModal.tsx
  • packages/ui/src/components/Modal.tsx
  • packages/vector/package.json
  • packages/vector/src/utils/shared.ts
  • packages/vector/tests/shared.test.ts
  • packages/vector/vitest.config.ts
  • python/core/types.py
  • python/plugins/base.py
  • python/plugins/descriptor.py
  • python/plugins/dominant_color.py
  • python/plugins/face_recognition.py
  • python/plugins/object_detection.py
  • python/plugins/shot_type.py
  • python/plugins/text_detection.py
  • python/services/analysis/plugins.py
  • python/services/analysis/service.py
  • python/services/transcription/service.py
  • python/services/websocket/messages.py
  • python/tests/test_multilingual_ocr.py

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@reg2005 reg2005 changed the title feat: add i18n and multilingual ML support feat: add i18n, multilingual ML, and OpenAI-like API support May 16, 2026
@IliasHad

Copy link
Copy Markdown
Owner

@reg2005 Thank you so much for your contribution. I'll review it next week and get back to you!

Evgeniy and others added 5 commits May 19, 2026 03:14
Log model routing, OpenAI-like request/response metadata, and chat fallback details so empty browser responses can be traced safely. Also preserve dev container node_modules mounts and update exiftool-vendored past the high-severity advisory.

Co-Authored-By: Claude <noreply@anthropic.com>
Add English and Russian i18n resources, language settings, and localized UI text across the app.

Wire the selected app language through settings, background jobs, transcription, location lookup, search suggestions, and scene/vector text generation.

Switch text embeddings and search normalization toward multilingual Russian/English workflows, including localized suggestions, Russian metadata-based scene descriptions, and canonical filter handling.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants