Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JavaScript String Extraction #26

Merged
merged 20 commits into from
Jul 4, 2018
Merged

JavaScript String Extraction #26

merged 20 commits into from
Jul 4, 2018

Conversation

swissspidy
Copy link
Member

@swissspidy swissspidy commented Apr 27, 2018

More and more plugins rely heavily on JavaScript and the JS I18N library established by Gutenberg.

Since neither makepot.php nor this WP-CLI command support JavaScript, they had to build their own Babel plugin to extract JavaScript files.

However, one cannot expect users to a) use Babel b) use two tools to generate a simple POT file.

So, why not support this in this little WP-CLI command here? :-)

Even though the Gettext library we use for the PHP part has a JS functions scanner, it is very limited: no support for comments, no clear indication which ECMAScript standard is supported, no usage of an AST, which is really important with such a quickly evolving language.

I decided to use Peast to traverse a script's AST, which was incredibly easy. We might even consider suggesting this for the Gettext library.

Still needs some tests, but so far it works fine.

Fixes #30.

@swissspidy swissspidy requested a review from a team April 30, 2018 17:45
@aduth
Copy link

aduth commented Apr 30, 2018

However, one cannot expect users to a) use Babel

While it's not implied, just to note that a common point of confusion is that its implementation as a Babel plugin has something to do with an opt-in decision to use newer language syntax, when in fact it was largely a combination of (a) taking advantage of the Babylon parser which, being at the core of Babel's language transpilation, is assumed to be well-maintained and (b) to combine into the same transpilation pipeline if one decides to use newer language features (as Gutenberg does, avoiding two separate parse steps).

@aduth
Copy link

aduth commented Apr 30, 2018

b) use two tools to generate a simple POT file.

While it may be implemented as such in its current form, there's nothing which requires this to be a two-step process.

@swissspidy
Copy link
Member Author

@aduth Totally get the point. There's nothing wrong with that and I don't think we can get around that anyway.

Although JSX support is planned for Peast (the parser used in this PR), it makes more sense to use something like the Babel plugin for core and Gutenberg. That's why I've been working on things like #31 to make it easier to use both hand in hand.

As mentioned in yesterday's meeting, we'll likely test such a setup with Gutenberg and some other plugin(s) on WordPress.org soon. That way one at least won't have to generate these PHP files anymore during a build.

@swissspidy
Copy link
Member Author

Did some updates today:

@@ -140,7 +153,7 @@ protected function retrieve_main_file_data() {

$files = new IteratorIterator( new DirectoryIterator( $this->source ) );

/** @var DirectoryIterator $file */
/* @var DirectoryIterator $file */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a real docblock /** */, but inside of the foreach block as the first line, as $file only exists within the foreach block.
This will then allow an IDE to know how to handle $file in there.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@schlessera What IDE do you use? PhpStorm for example handles this just fine outside the foreach.

In WordPress core both /** and /* is used for these annotations. Not sure if WPCS would think the former is a badly formatted doc block.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use PHPStorm as well, and for a few versions now, it interprets this correctly. But technically, the variable does not exist within the scope where you qualify it, and some other editors and static analysis tools will flag this.

I usually do one of these two instead:

foreach ( $files as $file ) {
	/** @var DirectoryIterator $file */
	if ( $file->isFile() && $file->isReadable() && 'php' === $file->getExtension()) {
		$plugin_files[] = $file->getRealPath();
	}
}
foreach ( $files as /** @var DirectoryIterator $file */ $file ) {
	if ( $file->isFile() && $file->isReadable() && 'php' === $file->getExtension()) {
		$plugin_files[] = $file->getRealPath();
	}
}

Regarding the docblock:
Although PHPStorm is generally very lax in this regard, using @var inside of a regular multiline comment does not have any semantic meaning whatsoever. Tags only have a proper meaning within docblocks. Again, some tools are more strict about this and will ignore the @var if it is not within a proper docblock.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, nevermind my comments about moving the docblock inside of the foreach. This is apparently handled as an exception in the proposed PSR: https://github.com/php-fig/fig-standards/blob/master/proposed/phpdoc.md

image 2018-07-04 at 9 41 08 am

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, the above standard proposal also states that tags need to be within a docblock, not a regular comment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know, thanks.

I moved it into the foreach now and use a proper docblock and everything still works :-)

$this->slug = Utils\get_flag_value( $assoc_args, 'slug', Utils\basename( $this->source ) );
$this->source = realpath( $args[0] );
$this->slug = Utils\get_flag_value( $assoc_args, 'slug', Utils\basename( $this->source ) );
$this->scan_js = ! Utils\get_flag_value( $assoc_args, 'skip-js', ! $this->scan_js );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer the property to be called $skip_js, to avoid the weird double negation here. As it is only used once, just directly mapping the parameter seems clearer to me.

use Peast\Syntax\Node\Node;
use Peast\Syntax\Node\VariableDeclaration;
use Peast\Traverser;
use Peast\Syntax\Node\CallExpression;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these Node\Something imports are a case where I would prefer the use of relative namespaces.
So, have only 1 import Peast\Syntax\Node here, and then use relative namespaces like Node\Identifier or Node\Literal in the code.
This makes the import list less daunting, and it gives more context to the consuming code, where it is clear that these are all different types of nodes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I now see that the library has a Node\Node, which is unfortunate and might make my above suggestion weird.

I usually build my code so that the interface is outside of the namespace with the implementations. This way, you can use Node as well as Node\Literal, ...

I'll let you decide what you find cleaner, but having a Node\Node inside the code is certainly not really elegant.

use Peast\Syntax\Node\Identifier;
use Peast\Syntax\Node\Literal;

class JsFunctionsScanner extends GettextJsFunctionsScanner {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Class should be final and protected methods/properties should be private (unless they were already protected in the superclass).

@@ -11,7 +11,7 @@
use SplFileInfo;
use WP_CLI;

class WordPressCodeExtractor extends PhpCode {
class PhpCodeExtractor extends PhpCode {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Class should be final and protected methods/properties should be private (unless they were already protected in the superclass).

$functions = $options['functions'];
$file = $options['file'];

/* @var Node $node */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a real docblock inside of the foreach block.

RecursiveIteratorIterator::CHILD_FIRST
);

/* @var \DirectoryIterator $file */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a real docblock inside of the foreach block.

RecursiveIteratorIterator::CHILD_FIRST
);

/* @var \DirectoryIterator $file */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a real docblock inside of the foreach block.

RecursiveIteratorIterator::CHILD_FIRST
);

/* @var \DirectoryIterator $file */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be imported instead of using a FQCN.

RecursiveIteratorIterator::CHILD_FIRST
);

/* @var \DirectoryIterator $file */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be imported instead of using a FQCN.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One could also argue that this would result in an unused import statement as it's only used for the code comment.

Is this defined in the coding standards somewhere?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The coding standards don't deal with namespaces, so there's no imports either.

It is not an unused import, as you're referencing it through the docblock. Note that PHP annotations like @var are considered parsable and executable code. So the annotation is something like new var( DirectorIterator::class, 'file' ).

@schlessera schlessera added this to the 0.1.0 milestone Jul 4, 2018
@swissspidy swissspidy requested a review from schlessera July 4, 2018 08:11
@schlessera schlessera merged commit 7e11d8c into master Jul 4, 2018
@schlessera schlessera deleted the js-code-extraction branch July 4, 2018 08:15
@schlessera
Copy link
Member

🎉

We will probably still need to iterate on this in the future, but having it merged now means we can better test it in actual use cases.

schlessera added a commit that referenced this pull request Jan 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants