Resources with hash to be used in static web apps

[ThomasVandenbon]

🚀 Feature Proposal

I've been struggling for a few months to find the right approach to caching using i18next in combination with react.
Using a Create React App template, most examples I've found suggest to place resource files in public/locales/en/translations.json and to then use a plugin like i18next-http-backend to load them.

However, this seems to have the flaw that if a new version of the project is published with modified translations these won't be picked up, unless you configure your server in such a way to add no-cache, no-store headers to all resources files in this path.
This does mean that:

• These resource files will never be cached, which is a step too far.
• You need to configure your server to add these headers.

Looking at the output of a Create React App build, you'll normally have a bunch of generated .js files with hashes assigned to them. Wouldn't it make a lot more sense if there was a way to also add hashes to these translations?

I know that an alternative is to use cache-busting like this:

      backend: {
        loadPath: "/locales/{{lng}}/{{ns}}.json?version={{currentVersion}}",
      },

But then you'd need to add some infrastructure to provide that currentVersion, which feels like setting up a work-around for a very common case.

Motivation

I think i18next configuration should be kept in one place and not require configuration on the server end or exposing version numbers in the building step.

Example

I'd hope that aside from maybe turning on a configuration flag, the usage of i18next or react-i18next would remain the same.
Only the build output would change and the calls to fetch the resouces.

Note

I'm open to other suggestions for handling caching, as I might be missing something obvious.

[jamuhl]

No idea how todo so...PR welcome...

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[RogierdeRuijter]

Would love to see the ability to detect a hash in the loadPath variable.

loadPath: "/locales/{{lng}}/{{ns}}.{{some-random-hash}}.json"

Now I have "Cache-Control: no-cache", which has the downside every time the page is requested a call has to be made to the webserver to check if the content changed. With a couple of namespaces, this can have an impact on the initial load time.

Cache-busting would be the ideal solution here.

[jamuhl]

why not just

loadPath: `/locales/{{lng}}/{{ns}}.${some-random-hash}.json`

[RogierdeRuijter]

Because you don't know the ${random-hash} upfront, this gets determined at build time and is different every time the files change.

When using webpack to build your application you can use it to convert your asset (e.g. image, js or json file) to an immutable asset by adding a hash to it, therefore if you have e.g. the login namespace, login.json can be converted to login."some-hash-based-on-the-content".json. Knowing the hashes upfront is therefore difficult.

[jamuhl]

and how should i18next know about that hash?

[RogierdeRuijter]

I think using the interpolation notation {{}} was a mistake on my part.

I18next also can't know about the hash. It has to 'blindly' accept the characters that are after the dot (.) and before the .json. For example, login.1cb671935af07e6272652b8c5194d7e3.json. The hash in bold can be anything.

Maybe some restrictions could be put into place like the number of characters in the hash. This could be defined by the consumer of the library.

[RogierdeRuijter]

Ahh never mind this doesn't work 😅. I18next has to know the hash otherwise it can't fetch it from the webser. I'll come back when I have a proper answer

[RogierdeRuijter]

They seem to have a solution here, where they calculate the hashes in javascript and match them with the hash given by webpack.

I don't know how suitable this solution would be for i18next since it also requires some setup from webpack's side... Nevertheless, I am going to give this a try and see I can make it work.

[shaominfeng]

I've also encountered this issue and developed a pragmatic solution. While not optimal, it's a practical compromise that has proven effective and straightforward.

My approach involves using a script to calculate a hash of the translation files before starting or building the project. This hash is then saved to a temporary file and can be easily incorporated where needed. The hash is recalculated upon each startup following any modifications.

Here's an overview of the implementation:

First, we have a script generate-locales-hash.js that calculates the hash:

const fs = require('fs');
const path = require('path');
const crypto = require('crypto');

function calculateHash(directory) {
  let hash = crypto.createHash('md5');
  const files = fs.readdirSync(directory, { recursive: true });

  files.forEach((file) => {
    const filePath = path.join(directory, file);
    if (fs.statSync(filePath).isFile()) {
      const content = fs.readFileSync(filePath);
      hash.update(content);
    }
  });

  return hash.digest('hex').substring(0, 5);
}

const localesDir = path.join(__dirname, '..', '..', 'public', 'locales');
const hash = calculateHash(localesDir);

const hashFile = path.join(__dirname, '..', '..', 'src', 'locales-hash.json');
fs.writeFileSync(hashFile, JSON.stringify({ hash }));

console.log(`Locales hash generated: ${hash}`);

Running npm run generate-locales-hash.js produces a locales-hash.json file:

{"hash":"97f63"}

The hash is then utilized in the i18n configuration (i18n.ts):

import localesHash from './locales-hash.json';
....
backend: {
          loadPath: `/locales/{{lng}}/{{ns}}.json?v=${localesHash.hash}`,
}

Finally, we integrate the hash generation into our build and start scripts in package.json:

  "start": "npm run generate-locales-hash && vite",
   "build": "npm run  generate-locales-hash && vite build",

This solution ensures that the translation files are properly versioned, mitigating caching issues while maintaining simplicity in implementation.