Testing multilingual sites

[jurgenhaas]

Using 3.0 of the Drupal Extension I recently ran into trouble with a client's site which is multi-lingual. It happens that basic tests are failing because e.g. content or error messages no longer match the tests when the tests are written in English and the site defaults to German.

I was trying to change the base URL in behat.yml of the project to http://www.exampl.com?language=en which is a valid URL that forces the site to switch to the English context, but that breaks the tests as well as they then seem to be appending the path like "/user" to the base URL which then makes that http://www.exampl.com?language=en/user which is no longer valid.

I wonder what's the best approach here. We could either support queries in the base URL, then the approach like desribed above would already work, or there might be a new setting in the behat.yml that forces the site to a certain context or sends HTTP headers that help to set the language context. Maybe that's already possible and I just haven't found it?

[eliza411]

I haven't done a lot of multilinugual work, but on the few sites with that requirement, I've always switched languages with clean URLs, so I've not run into the problem you're experiencing. It was my understanding that the taking the clean url approach was better for search engine indexing, and it was better for communicating URLs to others outloud.

That aside, the behavior you're describing is part of Mink, and I think it's probably best addressed there. https://github.com/Behat/Mink/issues/566 is related, although it has not been updated since May, it looks like.

[jurgenhaas]

Thanks for your feedback but I wasn't quite clear in building the URL. We are of course using clean URLs, so the destination we want to check is http://www.example.com/user to test the login form. This page is only available as /user and not like other content as /en/page/title-1 or /de/seite/titel-1

So, system pages like /user follow other hints to find out which page should be used. By default it check for domain, sub-path, query parameter, browser settings in that given order. As domain and sub-path are not different for the /user page, the next two options should be tried. I used the language=en query which is causing problems with Mink. The fourth one would be looking into the browser language settings and I wonder if we had a chance to provide those settings via the behat.yml file somehow.

[eliza411]

Thank you for clarifying! (It's still early here.) I see what you're saying now.

[jurgenhaas]

After some digging in Drupal 7 I found locale_language_from_browser() in /includes/locale.inc and there they are negotiating the current language from the $_SERVER['HTTP_ACCEPT_LANGUAGE'] variable and if we could set this variable in the Behat context, that would provide much more control.

[jurgenhaas]

Here is how it can be modified to work as expected. In my FeatureContext I added the following function:

  /**
   * @BeforeStep
   */
  public function beforeEachStep($event) {
    /* @var $event Behat\Behat\Hook\Scope\BeforeStepScope */
    /* @var $environment Behat\Behat\Context\Environment\InitializedContextEnvironment */
    /* @var $mink Drupal\DrupalExtension\Context\MinkContext */
    $environment = $event->getEnvironment();
    if ($environment->hasContextClass('Drupal\DrupalExtension\Context\MinkContext')) {
      $mink = $environment->getContext('Drupal\DrupalExtension\Context\MinkContext');
      $mink->getSession()->setRequestHeader('ACCEPT_LANGUAGE', 'de,en-US;q=0.7,en;q=0.3');
    }
  }

This is forcing the content to be in German. Of course this can now be forced to English by simply changing the value of "Accept Language".

May I suggest that this could be added to the Drupal Extension as a feature so that a language property (defaulting to en-US) can be added to the behat.yml file and then be used globally?

[jurgenhaas]

Thinking about it even more, it would be great if we could extend the idea of testing multilingual sites with multiple contexts in a way that we could use placeholders in our feature files for all the strings we're going to test and replace them with the proper strings matching the language that we are about to test.

I'd be happy to contribute to such a framework if that's something you would like to get into the drupal extension.

[eliza411]

This is definitely Jonathan's decision, but especially with Multilingual in Drupal 8 core, a coherent approach to multilingual testing seems very much in order.

[jurgenhaas]

Good point well made. I'd like to offer my help for this, i.e. I'm happy to contribute code to address this in case Jonathan is up for it too. But I'd require someone to discuss the best approach then.

[jhedstrom]

I'm certainly up for helping this along. The solution you posted above looks like a good start. Similar url manipulation was needed for dealing with sites behind basic auth, so that might also be an approach to consider.

[jurgenhaas]

From your code sample I can certainly improve my approach from above. However, as mentioned afterwards there is more to multilingual site testing. Let's assume the feature files are all written in English and the tests are using strings for button labels, messages and other content they are testing for. This makes sense of course and nobody wants to write all those tests over and over again if they had to test the same site in multiple languages. Instead, they might want to maintain the feature files in English only and then in their behat.yml file they want to set a language parameter to define which language should be tested.

If that's reasonable, the following extensions are required:

• a new Drupal (or Mink) parameter for the language, defaulting to "en"
• additional translation maps where the translated strings from the feature files can be found

If that's a good approach, I would like to chat about a good starting point on how to get started.

What do you think?

[jhedstrom]

You should be able to use the @transform tag for the method that actually does the mapping. I bet this could go into a new context, say TranslationContext. The new option defaulting to en makes sense. In the docs we should be careful to clarify the difference between using this, and the already existing xliff translation of steps.

Feel free to ping me on #behat irc if you want to talk through any of this.

[jhedstrom]

Closing this out for now. Let's open new issues to further this along. Thanks!