Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for listing redundant whitelist entries #649

Merged
merged 1 commit into from
Feb 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,19 @@ To prevent that from happening you can configure a `whitelist`, which accepts an
array of regular expressions that will be checked when looking for unused
translations.

### `errorOnUnusedWhitelistEntries`
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's nice to make the exit code dependent on unused whitelist entries as well. However, I chose to leave this as a config option, so that current users aren't affected by this change. Alternative is just not having this config option and still error, but this might block applications after upgrading. Also, if this feature contains a bug that I overlooked, it's easy to turn it off.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we will keep it opt in for now, and see how it goes and that it is working well, then i think for the next Major version i think i will have this on by default.


When using a whitelist to ignore dynamic translation keys, it can be easy to forget
to clean up the whitelist when an entry is not used anymore. You can opt-in to make
this analyzer error when this occurs, by setting the `errorOnUnusedWhitelistEntries`
flag in the configuration file:

```js
export default {
errorOnUnusedWhitelistEntries: true,
};
```

### `analyzeConcatExpression`

If your template contains translations like this:
Expand Down
18 changes: 18 additions & 0 deletions __snapshots__/test.js.snap
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,24 @@ exports[`Test Fixtures unused-translations 1`] = `

exports[`Test Fixtures unused-translations 2`] = `Map {}`;

exports[`Test Fixtures unused-whitelist 1`] = `
"[1/4] 🔍 Finding JS and HBS files...
[2/4] 🔍 Searching for translations keys in JS and HBS files...
[3/4] ⚙️ Checking for unused translations...
[4/4] ⚙️ Checking for missing translations...

👏 No unused translations were found!

👏 No missing translations were found!

⚠️ Found 1 unused whitelist entry! Please remove it from the whitelist:

- /some\\\\.unused\\\\.whitelisted\\\\.translation-(a|b)/
"
`;

exports[`Test Fixtures unused-whitelist 2`] = `Map {}`;

exports[`Test Fixtures yaml-translations 1`] = `
"[1/4] 🔍 Finding JS and HBS files...
[2/4] 🔍 Searching for translations keys in JS and HBS files...
Expand Down
3 changes: 3 additions & 0 deletions fixtures/unused-whitelist/app/templates/application.hbs
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{{t "some.present.translation"}}
{{t "some.whitelisted.translation-a"}}
{{t "some.whitelisted.translation-b"}}
3 changes: 3 additions & 0 deletions fixtures/unused-whitelist/translations/en.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
some:
present:
translation: This translation exists
43 changes: 37 additions & 6 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -67,21 +67,30 @@ async function run(rootDir, options = {}) {
existingExternalTranslationKeys
);
let whitelist = config.whitelist || [];
let usedWhitelistEntries = new Set();
let errorOnUnusedWhitelistEntries = config.errorOnUnusedWhitelistEntries || false;

let unusedTranslations = findDifferenceInTranslations(
existingOwnTranslationKeys,
usedTranslationKeys,
whitelist
whitelist,
usedWhitelistEntries
);

log(`${step(4)} ⚙️ Checking for missing translations...`);
log();
let missingTranslations = findDifferenceInTranslations(
usedTranslationKeys,
existingTranslationKeys,
whitelist
whitelist,
usedWhitelistEntries
);

let unusedWhitelistEntries = new Set(whitelist);
for (let entry of usedWhitelistEntries) {
unusedWhitelistEntries.delete(entry);
}

if (unusedTranslations.size === 0) {
log(' 👏 No unused translations were found!');
} else {
Expand All @@ -106,7 +115,23 @@ async function run(rootDir, options = {}) {
}
}

let totalErrors = missingTranslations.size + unusedTranslations.size;
if (unusedWhitelistEntries.size > 0) {
log();
log(
` ⚠️ Found ${chalk.bold.yellow(unusedWhitelistEntries.size)} unused whitelist ${
unusedWhitelistEntries.size === 1 ? 'entry' : 'entries'
}! Please remove ${unusedWhitelistEntries.size === 1 ? 'it' : 'them'} from the whitelist:`
);
log();
for (let entry of unusedWhitelistEntries) {
log(` - ${entry}`);
}
}

let totalErrors =
missingTranslations.size +
unusedTranslations.size +
(errorOnUnusedWhitelistEntries ? unusedWhitelistEntries.size : 0);

if (shouldFix) {
removeUnusedTranslations(writeToFile, rootDir, ownTranslationFiles, unusedTranslations);
Expand Down Expand Up @@ -421,15 +446,21 @@ function forEachTranslation(json, callback, prefix = '') {

// Find all translation keys that appear in translation map A, but not
// in translation map B
function findDifferenceInTranslations(mapA, mapB, whitelist) {
function findDifferenceInTranslations(mapA, mapB, whitelist, usedWhitelistEntries) {
let missingTranslations = new Map();

for (let [key, files] of mapA) {
const keyTrimmed = key.trim();
const isKeyMissing = !mapB.has(keyTrimmed);
const isKeyAllowed = !whitelist.some(regex => regex.test(keyTrimmed));

if (isKeyMissing && isKeyAllowed) {
if (!isKeyMissing) continue;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Little performance improvement here; if the key wasn't missing, it would still be looked up in the whitelist, which is useless.


const whitelistKey = whitelist.find(regex => regex.test(keyTrimmed));
const isKeyWhitelisted = whitelistKey != null;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it easier to follow when called isKeyWhitelisted as opposed to isKeyAllowed


if (isKeyWhitelisted) {
usedWhitelistEntries.add(whitelistKey);
} else {
missingTranslations.set(keyTrimmed, files);
}
}
Expand Down
8 changes: 8 additions & 0 deletions test.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ describe('Test Fixtures', () => {
'external-addon-translations',
'concat-expression',
'chosen-translations',
'unused-whitelist',
];
let fixturesWithFix = ['remove-unused-translations', 'remove-unused-translations-nested'];
let fixturesWithConcat = ['concat-expression'];
Expand All @@ -32,6 +33,13 @@ describe('Test Fixtures', () => {
babelParserPlugins: ['typescript'],
extensions: ['.gts'],
},
'unused-whitelist': {
errorOnUnusedWhitelistEntries: true,
whitelist: [
/some\.whitelisted\.translation-(a|b)/,
/some\.unused\.whitelisted\.translation-(a|b)/,
],
},
};

beforeEach(() => {
Expand Down
Loading