Initial French grammar rules to insert articles to way names #252

yuryleb · 2018-06-19T08:32:38Z

Issue

Initial French grammar rules prototype to insert articles to way names (#251).

Currently only Rue status street name is supported:

Tasklist

Add other status street names handling (?)
Add changelog entry
Test with osrm-frontend
Review

Requirements / Relations

New languages/overrides/fr.js script adds article option to most way_name keywords during translations export from Transifex, languages/grammar/fr.json contains regular expressions (currently one) to process street names, changed languages.js loads and registers that French expressions file. test/grammar_test.js is extended with test for French.

…stinations

yuryleb · 2018-06-19T21:08:03Z

The next step:

Penegal

Grammatical notes, but nothing grave.

Penegal · 2018-06-20T06:09:34Z

languages/grammar/fr.json

+            ["^ All[ée]e ", " l’allée "],
+            ["^ (L['’])?Autoroute ", " l’autoroute "],
+            ["^ Avenue ", " l’avenue "],
+            ["^ Boulevard ", " la boulevard "],


We say « le boulevard ».

Penegal · 2018-06-20T06:09:46Z

languages/grammar/fr.json

+            ["^ (L['’])?Autoroute ", " l’autoroute "],
+            ["^ Avenue ", " l’avenue "],
+            ["^ Boulevard ", " la boulevard "],
+            ["^ Chemin ", " la chemin "],


We say « le chemin ».

Penegal · 2018-06-20T06:09:57Z

languages/grammar/fr.json

+            ["^ Cours ", " la cours "],
+            ["^ Entr[ée]e ", " l’entrée "],
+            ["^ Impasse ", " l’impasse "],
+            ["^ Parvis ", " la parvis "],


Same remark here.

Penegal · 2018-06-20T06:10:04Z

languages/grammar/fr.json

+            ["^ Entr[ée]e ", " l’entrée "],
+            ["^ Impasse ", " l’impasse "],
+            ["^ Parvis ", " la parvis "],
+            ["^ Passage ", " la passage "],


Penegal · 2018-06-20T06:10:11Z

languages/grammar/fr.json

+            ["^ Parvis ", " la parvis "],
+            ["^ Passage ", " la passage "],
+            ["^ Place ", " la place "],
+            ["^ Petit[\\- ]Pont ", " la petit-pont "],


Penegal · 2018-06-20T06:12:36Z

languages/grammar/fr.json

+            ["^ Petit[\\- ]Pont ", "  de la petit-pont "],
+            ["^ Pont ", "  de la pont "],
+            ["^ Promenade ", "  de la promenade "],
+            ["^ Quai ", "  de la quai "],


to be more precise, it would be "du quai"

Penegal · 2018-06-20T06:12:45Z

languages/grammar/fr.json

+            ["^ Route ", "  de la route "],
+            ["^ Rue ", "  de la rue "],
+            ["^ Sortie ", "  de la sortie "],
+            ["^ Souterrain ", "  de la souterrain "],


"du souterrain"

Penegal · 2018-06-20T06:12:52Z

languages/grammar/fr.json

+            ["^ Rue ", "  de la rue "],
+            ["^ Sortie ", "  de la sortie "],
+            ["^ Souterrain ", "  de la souterrain "],
+            ["^ Square ", "  de la square "],


"du square"

Penegal · 2018-06-20T06:12:57Z

languages/grammar/fr.json

+            ["^ Sortie ", "  de la sortie "],
+            ["^ Souterrain ", "  de la souterrain "],
+            ["^ Square ", "  de la square "],
+            ["^ Tunnel ", "  de la tunnel "],


"du tunnel"

Penegal · 2018-06-20T06:16:52Z

languages/grammar/fr.json

+            ["^ Tunnel ", "  de la tunnel "],
+            ["^ Voie ", "  de la voie "],
+
+            ["^ ([AÂÀEÈÉÊËHIÎÏOÔUÙÛÜYŸÆŒ])", "  d’$1"],


It seems that there is no real rule for elision before an h, so it can't be easily done apart with a list of all concerned words. I think you can remove it from this list and the other one. If you want to do it, here is a list of all h-beginning words which don't provoke an elision: https://fr.wikipedia.org/wiki/H_aspir%C3%A9

This list looks too big to be inserted into this humble expressions array 😕 So it's really simpler just to exclude H from there

benjamintd

Nice work!

There are a few instances of masculine vs. feminine nouns to correct that @Penegal already noted, otherwise that's good.

@Penegal is that test case correct? I'm not sure whether the capital letter should go or not:

['La Rochelle', 'elision', 'de la Rochelle']
If it is I think that's a test case that we should add.

Penegal · 2018-06-20T09:29:32Z

@benjamintd: I would say the correct string would be de La Rochelle: La is an article which is not subject to elision, so it would have to stay, and, as it is part of the name, it should keep its capital first letter.

This would have been different if the name was, for instance, Les Marches, as the elision would form des Marches, without capital since de has none, but, since the La article is not subject to this kind of elision, it should keep its capital first letter.

Penegal · 2018-06-20T09:38:52Z

I also notice that grammar rules should be applied on two other cases:

when a name is placed after at: for instance, the correct translation for You have arrived at Les Marches is Vous êtes arrivé aux Marches, with elision, and not Vous êtes arrivé à Les Marches;
when a name is placed after onto: the correct translation of Merge left onto Avenue des Champs-Élysées is S’insérer à gauche sur l’avenue des Champs-Élysées, not S’insérer à gauche sur Avenue des Champs-Élysées: the second case have a faulty capital and misses the required article la, which is elided in l’ in this case.

yuryleb · 2018-06-20T14:07:23Z

@Penegal, I need more detailed cases for replacing à to aux in Vous êtes arrivés à {waypoint_name} strings 😉 Or just aux should be used everywhere with {waypoint_name}?

Penegal · 2018-06-20T14:28:33Z

@yuryleb: à le is to be replaced by au, and à les by aux. This is not related to the first letter, as with l’, but to the first word; it is to be applied even if le or les is in the variable instead of the OSRM string, and is case insensitive: à Le is also to be replaced by au, and à Les by aux. You can see an example in my last comment.

Penegal · 2018-06-20T14:43:49Z

languages/translations/fr.json

-                "name": "Tourner {modifier} pour rester {way_name}",
-                "destination": "Tourner {modifier} en direction de {destination}",
-                "exit": "Tourner {modifier} sur {way_name}"
+                "name": "Tourner {modifier} pour rester {way_name:article}",


The correct translation would be Tourner {modifier} pour rester sur {way_name:article}.

Penegal · 2018-06-20T14:45:00Z

languages/translations/fr.json

            },
            "uturn": {
                "default": "Faire demi-tour à la fin de la route",
-                "name": "Faire demi-tour à la fin de la route {way_name}",
-                "destination": "Faire demi-tour à la fin de la route en direction de {destination}"
+                "name": "Faire demi-tour à la fin de la route {way_name:article}",


The correct translation would be Faire demi-tour à la fin {way_name:elision}.

Correction: Faire demi-tour à la fin de {way_name:article}.

So Faire demi-tour à la fin de le boulevard Saint-Germain not Faire demi-tour à la fin du boulevard Saint-Germain?

Penegal

Maybe the name of the classes should be changed for a more descriptive version? Both classes elide, but one for an article, le/la/les, and the other for a preposition, de/des/du; naming the second case preposition could do it.

There could also be a redundancy in the computed instructions using the current rotary strings, but this should be tested, as I don't know enough real-world examples of roundabout names to be sure of it.

yuryleb · 2018-06-20T21:46:42Z

Example of rotary:

Penegal · 2018-06-21T06:15:26Z

Here is what I expected: the correct second instruction would be Prendre le rond-point de la place Charles de Gaulle. It should change according to the name of the roundabout; if it is named like Rond-point x, then the correct sentence would be Prendre le rond-point x, but, if it is another name, it should use the new elision rules: Place x would give Prendre le rond-point de la place x.

yuryleb · 2018-06-21T13:24:33Z

@Penegal, now Transifex's recently changed uturn/name phrase becomes Faire demi-tour à la fin de la route {way_name:article} not Faire demi-tour à la fin {way_name:preposition} as early, that's no de/du/des preposition will be inserted before {way_name} after grammar processing - is it correct or override script should be also updated?

Penegal · 2018-06-21T14:05:04Z

@yuryleb: you should go with Faire demi-tour à la fin {way_name:preposition}, it's the correct sentence.

Penegal · 2018-06-21T15:27:19Z

@yuryleb: I just checked, and, according to an orthographic reform, rond-point can also be written rondpoint.

Also note that I recently updated some Transifex strings; in case of conflict, you can drop the new ones. Just warn me to allow me to restore them.

yuryleb · 2018-06-21T18:29:35Z

Yes, I can support both but what should be in final text - rondpoint or rond-point?

And actually I could also merge your changes from Transifex if you completed - anyway I download them every time I check how override script works 😉

Penegal · 2018-06-22T06:07:04Z

The main orthograph is rond-point; the other one is controversial and most people would see it as an error.

If you can merge the Transifex changes with the one you did, perfect! I have a backup of the first ones, if needed, and I finished my edits, so you can go and merge.

Penegal · 2018-06-22T11:56:22Z

I noticed you added way types in your regex list; there could also be le sentier, which is something like a feminine form of sente.

yuryleb · 2018-06-22T19:54:28Z

It seems all is done. @Penegal, @benjamintd, do you have anything to add?

Penegal · 2018-06-25T06:09:58Z

@yuryleb: de/du/des/le/la/l’ should not use a bold typeface unless they are part of the name/destination, as, in this case, they are technically not part of it.

Edit: also, village has 2 L.

yuryleb · 2018-06-25T07:05:38Z

@Penegal, village has two L in expressions, it was just misprint in commit comment.

Formatting with bold typeface is part of external formatToken() implementation inside OSRM sample frontend (see https://github.com/Project-OSRM/osrm-frontend/blob/gh-pages/src/itinerary_builder.js#L11). Perhaps I could extend my last formatToken() fix with your de/du/des/le/la/l’ formatting proposal.

yuryleb · 2018-06-25T20:34:46Z

@Penegal, so an expression to exclude appended articles/prepositions should look like ^(à )|(au )|(aux )|(d’)|(de )|(des )|(du )|(l’)|(la )|(le )|(les ) (case-sensitive to distinct appended words from existing in name)?

This expression will help to format as below:

la cour d’Honneur
de la cour d’Honneur
le rond-point de la cour d’Honneur

Is it enough or all added words should be also non-bold to keep only cour d’Honneur bolded?

Penegal · 2018-06-26T06:20:35Z

@yuryleb: I would say the correct typography would be de la cour d’Honneur.

yuryleb · 2018-06-26T08:11:31Z

@Penegal, please look how this is now implemented.

Penegal · 2018-06-28T06:28:06Z

@yuryleb: very good work. I'm OK with this.

Penegal · 2018-07-05T12:47:19Z

Is there a problem preventing an approving review?

benjamintd

Looks great! Thanks @yuryleb and @Penegal.

I left a small comment to understand one regexp better, but that can be left as-is or fixed in another PR.

Looks good to merge!

benjamintd · 2018-07-05T13:14:20Z

languages/grammar/fr.json

+            ["^ Traverse ", " la traverse "],
+            ["^ Tunnel ", " le tunnel "],
+            ["^ Viaduc ", " le viaduc "],
+            ["^ Villa(ge)? ", " le villa$1 "],


What are we looking to match other than Village? Unfortunately Villa is feminine in French (la villa)

Thank you, I add other rules for Villa then

1ec5

Awesome, thank you all for your attention to detail!

Penegal · 2018-07-06T14:34:03Z

How often is the code of map.project-osrm.org updated? Just wanted to see it live… 😁

1ec5 · 2018-07-09T23:31:26Z

map.project-osrm.org is powered by OSRM-frontend. That repository will need to be updated, but first I have to publish a new version of OSRMTI.

- Updated French localization with articles and prepositions insertion using grammar rules. #252 - Added a Burmese localization. #247 - Added a Finnish localization. #239 - Added a Korean localization. #243 - Updated translations in Simplified Chinese. #233 #248 - Updated Russian grammar with 'chord' status street name. #245 - Added Russian church names abbreviations. #237 - Corrected Lithuanian streets abbreviations. #238

yuryleb added 2 commits June 19, 2018 11:20

Initial French grammar rules to insert articles to way names

93fbd99

More status street names for 'article' and new 'elision' rules for de…

6d4c2a9

…stinations

yuryleb mentioned this pull request Jun 20, 2018

Grammar rules: limits? #251

Open

Penegal reviewed Jun 20, 2018

View reviewed changes

benjamintd suggested changes Jun 20, 2018

View reviewed changes

Gender fixes in grammar

7f70414

Penegal reviewed Jun 20, 2018

View reviewed changes

Renamed 'elision' grammar group to 'preposition' and added 'arrival'

d1899b7

Added 'rotary' grammar group based on 'preposition'

d08b7f6

Improved {way_name:preposition} insertion for u-turns

407d904

More status street names and tests; 'rondpoint' handling

3ebd172

yuryleb added 3 commits June 22, 2018 18:55

'Aire', 'Sentier', 'Villa' and more tests

ee47628

French translation update from Transifex

56dc686

Changelog update

a4c4e5f

yuryleb added 5 commits June 24, 2018 01:00

'Déviation' status street name

fc2e0bb

'Viaduc' status street name

17c7686

'Anse', 'Corniche', 'Esplanade', 'Montée', 'Traverse', 'Vilage'

1c5fd7c

'Petit Chemin', 'Clos', 'Lotissement', 'Rocade', 'Grande Rue'

524d777

Fix apostrophes in all names

c043c0c

yuryleb mentioned this pull request Jun 26, 2018

Fix formatToken() callback to process only non-empty tokens Project-OSRM/osrm-frontend#271

Merged

benjamintd approved these changes Jul 5, 2018

View reviewed changes

Different rules for 'Villa' and 'Village'

2c83d41

1ec5 approved these changes Jul 6, 2018

View reviewed changes

1ec5 merged commit 4e2c051 into Project-OSRM:master Jul 6, 2018

yuryleb deleted the fr-grammar-proto branch July 6, 2018 05:24

1ec5 mentioned this pull request Dec 4, 2019

Automatically choose preposition-article contraction in Portuguese #294

Open

6 tasks

Initial French grammar rules to insert articles to way names #252

Initial French grammar rules to insert articles to way names #252

Conversation

yuryleb commented Jun 19, 2018 • edited Loading

Issue

Tasklist

Requirements / Relations

yuryleb commented Jun 19, 2018

Penegal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjamintd left a comment

Choose a reason for hiding this comment

Penegal commented Jun 20, 2018

Penegal commented Jun 20, 2018

yuryleb commented Jun 20, 2018 • edited Loading

Penegal commented Jun 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Penegal left a comment

Choose a reason for hiding this comment

yuryleb commented Jun 20, 2018

Penegal commented Jun 21, 2018

yuryleb commented Jun 21, 2018 • edited Loading

Penegal commented Jun 21, 2018 • edited Loading

Penegal commented Jun 21, 2018

yuryleb commented Jun 21, 2018

Penegal commented Jun 22, 2018 • edited Loading

Penegal commented Jun 22, 2018

yuryleb commented Jun 22, 2018

Penegal commented Jun 25, 2018 • edited Loading

yuryleb commented Jun 25, 2018

yuryleb commented Jun 25, 2018 • edited Loading

Penegal commented Jun 26, 2018

yuryleb commented Jun 26, 2018

Penegal commented Jun 28, 2018

Penegal commented Jul 5, 2018

benjamintd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

1ec5 left a comment

Choose a reason for hiding this comment

Penegal commented Jul 6, 2018

1ec5 commented Jul 9, 2018 • edited Loading

yuryleb commented Jun 19, 2018 •

edited

Loading

yuryleb commented Jun 20, 2018 •

edited

Loading

yuryleb commented Jun 21, 2018 •

edited

Loading

Penegal commented Jun 21, 2018 •

edited

Loading

Penegal commented Jun 22, 2018 •

edited

Loading

Penegal commented Jun 25, 2018 •

edited

Loading

yuryleb commented Jun 25, 2018 •

edited

Loading

1ec5 commented Jul 9, 2018 •

edited

Loading