extract translations from source code automatically

[samuelstroschein]

Following idea:

1. Developer codes website (without i18n in mind)
2. Run cli inlang extract command
3. The CLI extracts hardcoded messages / does the i18n for the dev

Theoretically, a whole app could be i18n in a minute or so. Currently, i18n an existing app involves manually checking every file for hardcoded strings and copy & paste them into translation files. I just had to go through that process with the inlang dashboard. It took 5 hours, even though I was using the vscode extension which eliminates the copy & paste step.

Workflow

Option 1: Part of CI/CD

Developers run the inlang extract CLI command as part of their CI/CD pipeline.

Pros

• extracting keys/i18n on autopilot

Cons

• thousands of edge cases
  • relying on the extract command will lead to errors/bugs/missing translations down the road

Option 2: One time process

The value of automatic extraction is extremely high when i18n an existing, previously not internationalized app (correct me if I am wrong). Subsequent extractions can be performed with the VS Code extension, avoiding ambiguity issues.

I am thinking of a tool like the Dart migration tool that exists(ed) for null-safety migration. Running the inlang extract command starts a process that

1. analyzes the source code based on the heuristic explained above
2. starts a UI interface where developers can go through each match and decide whether the match is correct or not (and add manual matches)

See the Dart migrate tool in action https://youtu.be/eBr5tlumwlg?t=271

Pros

• should substantially speed up the i18n process by avoiding manual checking of files
• avoids ambiguity by forcing developers to verify the matches

Cons

• no full autopilot as compared to option 1
• likely misses some hardcoded text
  • developers would need to add those manually over time
  • for new text, the VS Code extension is used.

[blindfish3]

How would the cli tool identify text that needs to be translated? In step 1 the dev would have to at least tag the text that needs to be translated. I've worked on multiple projects where we had no immediate need for translation but did this so that translations could be implemented at a later date. Using an attribute and dedicated function in scripts this requires very little effort. Assuming that has been done, then the tool could extract all strings to a file for hand-off to a translator.

pros: very little friction in terms of development
cons: extra step to generate translation source file

[samuelstroschein]

@blindfish3 I updated the description containing thoughts about an implementation.

[samuelstroschein]

LLMs are likely a crucial element to extract strings. Using ChatGPT to ask "what is hardcoded and what not?" should get us close

[NilsJacobsen]

I vote for a lint rule. There is no need for extracting, but for showing where there is something to do and letting them handle edge cases by themselves.

[samuelstroschein]

This is now possible with LLMs opral/inlang-paraglide-js#334 🔥