Add support for font-specific formats #3142

Crissov · 2016-08-04T14:13:13Z

Text: ← filenames: FONTLOG http://scripts.sil.org/cms/scripts/page.php?item_id=OFL-FAQ_web#43cecb44
OpenType: ← extensions: .fea https://www.adobe.com/devnet/opentype/afdko/topic_feature_file_syntax.html
Spline Font: ← extensions: .sfd http://fontforge.github.io/en-US/documentation/developers/sfdformat/

- Markdown: ← extensions: .md.txt - Text: ← extensions: .text - Text: ← filenames: FONTLOG http://scripts.sil.org/cms/scripts/page.php?item_id=OFL-FAQ_web#43cecb44 - OpenType: ← extensions: .fea https://www.adobe.com/devnet/opentype/afdko/topic_feature_file_syntax.html - Spline Font: ← extensions: .sfd http://fontforge.github.io/en-US/documentation/developers/sfdformat/

Alhadis · 2016-08-04T14:16:16Z

lib/linguist/languages.yml

@@ -2668,6 +2669,16 @@ OpenSCAD:
  tm_scope: none
  ace_mode: scad

+OpenType Font Feature:
+  type: programming


If this PR is accepted, this should be data, not programming. Same with Spline Font Data.

I’ll agree that data is appropriate for .sfd, but not so much for .fea

What's convinced you that OpenType features are a "programming language"...?

Well, maybe it’s markup (like Postscript and CSS) but OT features are used to implement substitution algorithms etc. That sounds like programming to me. It’s certainly not just data like an image or config file.

PostScript? Uh, no, PostScript is a fully Turing-complete programming language. CSS, though... yes, that's markup.

I'm well aware of what OpenType features are, but I'd argue they're no closer to programming than SVG source is. They describe coordinates, geometry and hinting for the rendering environment...

I know that PS is TC and so is TeX, but like CSS they both have type: markup in languages.yml. CSS is a declarative language (and doesn’t mark up text)), but so are functional programming languages … CSS is certainly not a markup language like HTML is.

The hell. I didn't even notice PostScript was classed as markup, and I was even editing that part the other night.

Right, time to amend.

EDIT: Done.

`type: data` for SFD

OpenType feature ← type: markup

Alhadis · 2016-08-04T15:13:34Z

You'll need to fix the order of file extensions. From the build's feedback:

TestPedantic#test_nonprimary_extensions_are_sorted [/home/travis/build/github/linguist/test/test_pedantic.rb:15]:
.mkdown should come after .md.txt

alphabetic order

pchaigno · 2016-08-05T20:31:30Z

@Crissov Thanks for the pull request! Please see the guidelines to add support for a new extension to Linguist to complete the additional steps required.

Alhadis · 2016-10-27T07:31:07Z

@Crissov You can use this package to provide syntax highlighting for OpenType feature files and spline-font databases. The relevant scopes are:

OpenType feature file: source.opentype
SFD: text.sfd

Also, it appears the "proper" name for the former is "OpenType feature file", not "OpenType font feature". That's going by how the spec refers to the format.

You'll also need to change OpenType's language type to data, as previous suggested. It's not markup.

incorporated suggestions: - “OpenType Font Feature” → “Opentype feature” (no “file” at the end, because that’s left out almost everywhere else, too) - `tm_scope` according to https://github.com/Alhadis/language-fontforge

- `.md.txt` Markdown - `.text` Plain Text

Crissov · 2016-10-27T09:06:05Z

lib/linguist/languages.yml

+  - .fea
+  #- .otl (rare)
+  #interpreters:
+  #- makeotf


I’m not sure I correctly understand the purpose of interpreters

It's used for identifying languages based on an interpreter directive. For example, we have node listed as an interpreter for JavaScript, which tells Linguist that #!/usr/bin/env node identifies the file as being that language.

Since OpenType feature files aren't executed, it's safe to leave this out. ;)

Okay, thanks

remove comment

Alhadis · 2016-11-13T10:28:32Z

Sorry about the long wait! Didn't mean to take this long to get back to you. 😞

Alright, great news: all three additions check out.

`.sfd`

Bit surprised by the abundance of these files; I hadn't expected FontForge's .sfd format was so well-distributed, heh. 11,018 files on GitHub, hosted among 880 users among 1,042 unique repositories. Definitely worth inclusion.

Each of the .sfd files harvested from search results turned out to be a Spline Font Database, so I don't predict there'll be any files misclassified here.

`.fea`

These are slightly less distributed than .sfd files, but still too abundant to ignore: 14,236 indexed results, and among the 2,695 files I downloaded, half of which were duplicates, but distributed between 278 users among 404 unique repositories.

`FONTLOG`

Can't really go wrong here... 17,730 results, each with consistently similar content and structure.

Now, you'll need to fix a few things before we can merge this:

Pull/merge the latest changes from master into your branch
Rerun script/set-language-ids --update to fix missing/conflicting language IDs
Tweak each entry's name in languages.yml

/cc @pchaigno, @larsbrinkhoff

Alhadis · 2016-11-13T10:29:14Z

lib/linguist/languages.yml

@@ -2668,6 +2668,15 @@ OpenSCAD:
  tm_scope: none
  ace_mode: scad

+Opentype feature:


This needs to be changed to OpenType Feature File (case-sensitive, please).

Alhadis · 2016-11-13T10:30:16Z

lib/linguist/languages.yml

@@ -3635,6 +3644,13 @@ SourcePawn:
  tm_scope: source.sp
  ace_mode: text

+Spline Font Data:


The canonical name for the format is Spline Font Database, which is what this needs to be. Again, please make sure it's capitalised correctly (the way I've written it here).

Alhadis

LGTM. 👍

# Conflicts: # lib/linguist/languages.yml

Alhadis · 2016-11-13T15:07:28Z

@Crissov Uhm, you've... kind of committed the conflicts you were supposed to fix. The affected regions are lines 2956-2969 and 4041-4052. Just erase the the blank lines and <<<<====>>>> junk, and rerun script/set-language-ids --update.

The diff for languages.yml is too large to open on GitHub, so I can't comment on the conflicted areas directly.

Alhadis · 2016-11-15T13:03:40Z

@Crissov Just paging you with a reminder we're still waiting for the conflicts to be fixed before we can finish our review. =)

brndnblck · 2016-11-16T23:30:19Z

@Crissov looks like this is near the finish line, but you have some additional conflicts to resolve.

This reverts commit 18e4256.

Alhadis · 2016-11-18T22:10:26Z

@Crissov I sense you may be having difficulties, so I sent you a pull request to resolve these conflicts.

Resolve conflicts for #3142

Alhadis · 2016-11-19T10:11:36Z

Alright, there're two build failures, but only one of them we can fix (the second failure isn't your fault; it's currently a known issue).

This is the one we still have to fix:

  1) Failure:
TestLanguage#test_all_languages_have_grammars [/home/travis/build/github/linguist/test/test_language.rb:409]:
The following languages' scopes are not listed in grammars.yml. Please add grammars for all new languages.
If no grammar exists for a language, mark the language with `tm_scope: none` in lib/linguist/languages.yml.
OpenType Feature File source.opentype
Spline Font Database  text.sfd

I admit the error here was probably mine, since I hastily deleted my fork before seeing what the build result was. Silly me.

@Crissov Try these steps:

Clone your fork to a new directory on your system.
Checkout your branch patch-2
Run script/bootstrap to install the necessary submodules
Run the following:

perl -pi -e 's|script\K (?=list-grammars)|/|;' script/add-grammar
script/add-grammar Alhadis/language-fontforge
git add --all && git commit -m "Update submodules and fix grammar-list script"

Then push to your fork as you would normally. Note this should be done in a clean repository to eliminate the risk of unregistered submodules complicating things. There've been quite a few replaced over the course of this year, and since your fork is a little dated, it's quite likely you have stray submodules floating around...

Using script/add-grammar is the simplest fix here, but the current state of the script isn't working due an error in 43fa563. I've added an extra step that fixes it in order to allow the script to run.

Probably not a clean way to do it, but hey.

Alhadis · 2016-11-23T18:27:24Z

@Crissov I've sent you a second pull request in the hopes of closing the book on this.

@pchaigno Tests were still failing locally for the Ant submodule (even after merging master into the fork's branch). Could this be related to licensee somehow?

pchaigno · 2016-11-24T12:34:38Z

@Alhadis You'll need to update dependencies: bundle update.

Alhadis · 2016-11-24T12:35:49Z

Ah, right. My understanding of what bundle even is is still really shaky.

Is it because I'm from the land of JavaScript, not Ruby? ;)

Actually, serious question: shouldn't there be a version check or something to ensure dependencies are up to date? Because I had no idea what was going on.

pchaigno · 2016-11-26T13:09:42Z

Is it because I'm from the land of JavaScript, not Ruby? ;)

I'm not sure I get it myself, but then I'm not sure either of which land I'm from :p

Actually, serious question: shouldn't there be a version check or something to ensure dependencies are up to date? Because I had no idea what was going on.

We could increase the version of Licensee needed in github-linguist.gemspec. I'm a little hesitant to do that because Licensee 8.7.0 isn't actually needed for anything else than tests.

Alhadis · 2016-11-26T14:13:39Z

@Crissov Could you please merge the PR so we can get this wrapped up?

Add missing grammar to submodules list

Alhadis · 2016-12-02T12:53:15Z

@brandonblack This looks merge-ready to me. ?

Please also remember to squash-merge; the PR's commit history is extremely convoluted and would pollute the codebase's revision history.

brndnblck · 2016-12-06T02:50:21Z

Thanks @Crissov!

Crissov · 2016-12-06T20:26:29Z

It was really @Alhadis who did all the nasty work.

Alhadis reviewed Aug 4, 2016
View reviewed changes

Crissov added 2 commits August 4, 2016 16:23

Update languages.yml

4687bb0

`type: data` for SFD

Update languages.yml

0d94983

OpenType feature ← type: markup

Update languages.yml

56cbdd5

alphabetic order

Update languages.yml

55eca87

incorporated suggestions: - “OpenType Font Feature” → “Opentype feature” (no “file” at the end, because that’s left out almost everywhere else, too) - `tm_scope` according to https://github.com/Alhadis/language-fontforge

Crissov changed the title ~~markdown and font-specific updates languages.yml~~ font-specific updates languages.yml Oct 27, 2016

remove non-font related additions

6dae9c1

- `.md.txt` Markdown - `.text` Plain Text

Crissov commented Oct 27, 2016

View reviewed changes

Update languages.yml

188fa76

remove comment

Alhadis self-assigned this Nov 13, 2016

Alhadis requested changes Nov 13, 2016

View reviewed changes

changed names as requested

01d7cf6

Alhadis approved these changes Nov 13, 2016

View reviewed changes

Crissov added 2 commits November 13, 2016 15:27

Merge remote-tracking branch 'github/master' into patch-2

18e4256

# Conflicts: # lib/linguist/languages.yml

quote marks

0cafc49

Crissov and others added 3 commits November 18, 2016 21:46

Revert "Merge remote-tracking branch 'github/master' into patch-2"

1427974

This reverts commit 18e4256.

Resolve conflicts

bbb5dff

Update language IDs

8382e6a

Merge pull request #1 from Alhadis/patch-2

78beacf

Resolve conflicts for #3142

Alhadis changed the title ~~font-specific updates languages.yml~~ Add support for font-specific formats Nov 19, 2016

Alhadis added 2 commits November 24, 2016 04:11

Merge branch 'master' into patch-2

898ca90

Add missing submodule to grammars list

1199a56

Merge pull request #2 from Alhadis/patch-2

57780d8

Add missing grammar to submodules list

brndnblck merged commit ecdae83 into github-linguist:master Dec 6, 2016

Crissov deleted the patch-2 branch December 6, 2016 20:26

pchaigno referenced this pull request Dec 18, 2016

reverting accidental changes to language ids cc @TwP

a1165b7

github-linguist locked as resolved and limited conversation to collaborators Jun 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for font-specific formats #3142

Add support for font-specific formats #3142

Crissov commented Aug 4, 2016 •

edited by Alhadis

Loading

Alhadis Aug 4, 2016

Crissov Aug 4, 2016

Alhadis Aug 4, 2016

Crissov Aug 4, 2016

Alhadis Aug 4, 2016 •

edited

Loading

Crissov Aug 4, 2016 •

edited

Loading

Alhadis Aug 4, 2016 •

edited

Loading

Alhadis commented Aug 4, 2016

pchaigno commented Aug 5, 2016

Alhadis commented Oct 27, 2016

Crissov Oct 27, 2016

Alhadis Oct 27, 2016

Crissov Oct 27, 2016

Alhadis commented Nov 13, 2016 •

edited

Loading

Alhadis Nov 13, 2016

Alhadis Nov 13, 2016

Alhadis left a comment

Alhadis commented Nov 13, 2016 •

edited

Loading

Alhadis commented Nov 15, 2016

brndnblck commented Nov 16, 2016

Alhadis commented Nov 18, 2016

Alhadis commented Nov 19, 2016 •

edited

Loading

Alhadis commented Nov 23, 2016

pchaigno commented Nov 24, 2016

Alhadis commented Nov 24, 2016 •

edited

Loading

pchaigno commented Nov 26, 2016

Alhadis commented Nov 26, 2016

Alhadis commented Dec 2, 2016

brndnblck commented Dec 6, 2016

Crissov commented Dec 6, 2016

Add support for font-specific formats #3142

Add support for font-specific formats #3142

Conversation

Crissov commented Aug 4, 2016 • edited by Alhadis Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Alhadis Aug 4, 2016 • edited Loading

Choose a reason for hiding this comment

Crissov Aug 4, 2016 • edited Loading

Choose a reason for hiding this comment

Alhadis Aug 4, 2016 • edited Loading

Choose a reason for hiding this comment

Alhadis commented Aug 4, 2016

pchaigno commented Aug 5, 2016

Alhadis commented Oct 27, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Alhadis commented Nov 13, 2016 • edited Loading

FONTLOG

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Alhadis left a comment

Choose a reason for hiding this comment

Alhadis commented Nov 13, 2016 • edited Loading

Alhadis commented Nov 15, 2016

brndnblck commented Nov 16, 2016

Alhadis commented Nov 18, 2016

Alhadis commented Nov 19, 2016 • edited Loading

Alhadis commented Nov 23, 2016

pchaigno commented Nov 24, 2016

Alhadis commented Nov 24, 2016 • edited Loading

pchaigno commented Nov 26, 2016

Alhadis commented Nov 26, 2016

Alhadis commented Dec 2, 2016

brndnblck commented Dec 6, 2016

Crissov commented Dec 6, 2016

Crissov commented Aug 4, 2016 •

edited by Alhadis

Loading

Alhadis Aug 4, 2016 •

edited

Loading

Crissov Aug 4, 2016 •

edited

Loading

Alhadis Aug 4, 2016 •

edited

Loading

Alhadis commented Nov 13, 2016 •

edited

Loading

`FONTLOG`

Alhadis commented Nov 13, 2016 •

edited

Loading

Alhadis commented Nov 19, 2016 •

edited

Loading

Alhadis commented Nov 24, 2016 •

edited

Loading