Code

More bug fixes and features#3

Code

Open

rhdunn wants to merge 149 commits intocmusphinx:mastercmusphinx/cmudict-tools:masterfrom rhdunn:masterrhdunn/cmudict-tools:masterCopy head branch name to clipboard

+4,172-1,374

Commits

Commits on Feb 23, 2015

• vim: support syllable breaks in the arpabet phoneset

  rhdunn

  committed
  f7063c2

Commits on Feb 25, 2015

• vim: use windows-1252 as the default encoding

  rhdunn

  committed
  89be853

• vim: support highlighting acronym files

  rhdunn

  committed
  3876893

• vim: try to detect the format from the filename

  Show description for b160a62
  rhdunn

  committed
  b160a62

• vim: support file-based metadata detection

  rhdunn

  committed
  b9eaf40

• README: Move the usage option value descriptions.

  Show description for 6786210
  rhdunn

  committed
  6786210

• vim: use FORMAT values for the file format

  Show description for eb0f043
  rhdunn

  committed
  eb0f043

• README: use the FORMAT docs for the format metadata values

  rhdunn

  committed
  5c90cf9

• README: Fix the table of contents

  rhdunn

  committed
  393fb95

Commits on Mar 1, 2015

• cmudict-tools: support Windows line endings

  rhdunn

  committed
  e01abfe

• vim: don't specify cmudict content for new buffers

  Show description for 903ffc8
  rhdunn

  committed
  903ffc8

• vim: reworking filetype detection -- 0.1-0.6d

  rhdunn

  committed
  da642ad

• vim: pick up cmudict_format from the ftdetect file

  rhdunn

  committed
  89ba90e

• vim: detect air-based cmudict files

  rhdunn

  committed
  2e81107

• vim: acronym file detection

  rhdunn

  committed
  ba37d56

• vim: cmudict-new dictionary detection

  rhdunn

  committed
  105330f

• vim: support other cmudict formatted dictionaries

  rhdunn

  committed
  419f835

Commits on Mar 15, 2015

• README: document the new format detection logic

  rhdunn

  committed
  b6630f9

• Support type formatted metadata.

  rhdunn

  committed
  4b54a61

• Implement a 'select' command.

  rhdunn

  committed
  ac29654

Commits on Mar 17, 2015

• cmudict.py: fix relative paths for metadata

  rhdunn

  committed
  2733002

Commits on Mar 27, 2015

• arpabet: support the X phoneme (for Scottish lock, etc.)

  rhdunn

  committed
  e6aaec3

Commits on Apr 9, 2015

• Fix formatting metadata without a comment.

  rhdunn

  committed
  fb8ea16

Commits on Jun 2, 2015

• README: fix a typo

  rhdunn

  committed
  ba733c8

Commits on Jun 9, 2015

• Generate an error on metadata when no mapping is provided.

  rhdunn

  committed
  24a6e0b

Commits on Jun 24, 2015

• Extend the consonant phoneme type classification.

  Show description for 9b5b4de
  rhdunn

  committed
  9b5b4de

• Annotate syllabic consonants.

  rhdunn

  committed
  6e4ff43

• Support classifying a phoneme's stress type.

  rhdunn

  committed
  cb30c47

• Check for vowel stress assignment errors.

  Show description for 3e98df9
  rhdunn

  committed
  3e98df9

• Allow single fricative phonemes.

  Show description for 7dca698
  rhdunn

  committed
  7dca698

• Support syllabic consonants in monosyllables.

  Show description for ed2bb9f
  rhdunn

  committed
  ed2bb9f

Commits on Jun 25, 2015

• Use accent and phoneset metadata.

  rhdunn

  committed
  11aeeeb

• Support dictionary formats without comments.

  Show description for fbf6325
  rhdunn

  committed
  fbf6325

• Support formatting to the sphinx lexicon format.

  rhdunn

  committed
  b04eb1f

• README: Move the stemmer example to a new section.

  rhdunn

  committed
  efdec9c

• Support removing context entries.

  rhdunn

  committed
  c83ba12

• Support removing syllable breaks.

  rhdunn

  committed
  6724edd

• Support removing stresses on vowels.

  rhdunn

  committed
  0d63781

• Fix remove_stress on non-word parsed events.

  rhdunn

  committed
  4de299f

• Improve --remove-stress performance.

  Show description for 2b49e7e
  rhdunn

  committed
  2b49e7e

Commits on Jun 26, 2015

• Remove unused order_from in the core file parsers.

  rhdunn

  committed
  c325b1e

• Recognize order-from file-based metadata.

  rhdunn

  committed
  498628d

Commits on Jun 29, 2015

• Add encoding tests.

  rhdunn

  committed
  4a50e98

• Make the output encoding default to the input encoding.

  rhdunn

  committed
  75ae1f3

• Respect the encoding metadata value.

  rhdunn

  committed
  b30476c

• Support passing a comparison key to sort and parse.

  rhdunn

  committed
  94154d7

• Support Unicode Collation Algorithm (UCA) sorting.

  rhdunn

  committed
  6c0eae3

• Pick up sorting metadata for -Wunsorted validation.

  rhdunn

  committed
  653e846

Commits on Jun 30, 2015

• Fix a typo (lock => loch)

  rhdunn

  committed
  4af544a

Commits on Jul 1, 2015

• Fixes for Unicode characters in metadata values.

  rhdunn

  committed
  925fffd

• Factor out the key-value metadata logic.

  rhdunn

  committed
  ab2448a

• Allow custom metadata parsers.

  rhdunn

  committed
  098138c

• Support JSON formatted metadata.

  rhdunn

  committed
  dcbfe73

Commits on Jul 2, 2015

• Relax the word matching (for non-English names).

  rhdunn

  committed
  a304f17

Commits on Jul 4, 2015

• Expand out the entry to dict logic in format_json.

  Show description for 0bfe1e3
  rhdunn

  committed
  0bfe1e3

Commits on Jul 5, 2015

• Create a cmudicttools package.

  rhdunn

  committed
  2d3bc32

• Move the accents files into cmudicttools.

  Show description for 35ff126
  rhdunn

  committed
  35ff126

• Provide a setup.py script.

  rhdunn

  committed
  61a8530

Commits on Oct 16, 2015

• Make parse_festlex use the encoding-neutral line reader algorithm.

  rhdunn

  committed
  c9fcaf3

• Move the line splitting out of the file-based parsers.

  rhdunn

  committed
  bee0c8c

• diff: process a diff on a file with conflict markers

  rhdunn

  committed
  5711e9a

• merge: intelligently merge dictionary entries

  rhdunn

  committed
  907f8ac

Commits on Oct 17, 2015

• README: document the diff and merge commands

  rhdunn

  committed
  347a627

• simplify the usage message

  rhdunn

  committed
  465a8c6

• Improve the help text.

  rhdunn

  committed
  a02038f

• Factor out a helper function for setting up a dictionary file parser.

  rhdunn

  committed
  dd745d5

• Support 'YOURS THEIRS' arguments for diff/merge commands.

  rhdunn

  committed
  fcd57e8

• Use labels for conflict marker types.

  rhdunn

  committed
  2699212

• Use labels for difference types.

  rhdunn

  committed
  8f15721

• Move the diff comparison from the diff/merge commands to the diff helper.

  rhdunn

  committed
  38373d6

• Factor out the line diffing into a helper function.

  rhdunn

  committed
  c7787f8

• Fix the merge command signature.

  rhdunn

  committed
  da80c13

• Use the correct DiffType labels in diff_dict.

  rhdunn

  committed
  5cfaa1b

• Support 3-way diff and merge operations.

  rhdunn

  committed
  9a131b2

• README: Add a git mergetool example.

  rhdunn

  committed
  d857bde

• README: Document the 3-way diff/merge usage.

  rhdunn

  committed
  44af6b3

Commits on Oct 18, 2015

• Fix unreferenced word3 when performing a 2-way diff/merge.

  rhdunn

  committed
  3962c11

• Fixes to the diff/merge algorithm.

  rhdunn

  committed
  e10a572

• More fixes to the diff/merge algorithm.

  rhdunn

  committed
  189d2e9

Commits on Nov 2, 2015

• Fix to the diff logic at the end of the sequences.

  rhdunn

  committed
  a8cc945

Commits on Mar 19, 2016

• Support locally disabling specific warnings.

  rhdunn

  committed
  0438cdd

• Support a 'stats' command to display dictionary statistics.

  rhdunn

  committed
  2a2d4c6

• Fix parsing entries that start with '('.

  rhdunn

  committed
  601cc69

Commits on Apr 2, 2016

• Created a skeleton project.

  rhdunn

  committed
  734276c

• Define the Penn Treebank POS tagset as a SKOS vocabulary.

  rhdunn

  committed
  13f9fe2

• Define the WP39 POS tagset as a SKOS vocabulary.

  rhdunn

  committed
  60819ef

• Map between Penn Treebank and WP39 concepts.

  rhdunn

  committed
  62db1a6

• Define the WP20 POS tagset as a SKOS vocabulary.

  rhdunn

  committed
  b40759e

• Map between WP20 and WP39 concepts.

  rhdunn

  committed
  9b83021

• Map between Penn Treebank and WP20 concepts.

  rhdunn

  committed
  b00f7e2

• Update the description in the README.md file.

  rhdunn

  committed
  c556fdd

• Part of Speech model with mappings to the WP20 tagset.

  rhdunn

  committed
  0c76c35

• Use skos:notation instead of skos:prefLabel to get the key/value labels.

  rhdunn

  committed
  648f835

• Add skos:notation triples to the concept schemes.

  rhdunn

  committed
  6f8f45f

• Use the Set/TypeValidator classes for the context parsers.

  rhdunn

  committed
  0901dfd

• Use skos:scopeNote instead of the non-existant skos:comment.

  rhdunn

  committed
  566110f

• Support specifying a custom context format.

  rhdunn

  committed
  361f20c

Commits on Apr 3, 2016

• stats: don't make assumptions about the entries, words and variants

  rhdunn

  committed
  0e0815c

Commits on Apr 14, 2016

• Create a festlex-specific POS tagset.

  rhdunn

  committed
  8b46384

Commits on Apr 18, 2016

• Add tests for printing Cepstral phonemes.

  rhdunn

  committed
  39b7afa

Commits on Apr 19, 2016

• festvox phonemes: map /ah0/ to /ax/ to match cmu2ft

  rhdunn

  committed
  807dc69

• Map the /axr/ phoneme to /er0/ in festvox as festvox does not have explicit support for axr.

  rhdunn

  committed
  9a2a9f6

• festvox: map the secondary stress phonemes to primary stress phonemes to match cmu2ft.

  rhdunn

  committed
  239055b

Commits on Apr 20, 2016

• Update the test data to match the festlex phone output.

  rhdunn

  committed
  35b48a1

• Create a CMU pronunciation dictionary context tagset.

  rhdunn

  committed
  7db254f

• Add 'cmudicttools/pos-tags/' from commit '7db254f7d43ef5f84c105634aa69bf2a06edb7bd'

  Show description for 3be774f
  rhdunn

  committed
  3be774f

• Rework the context validators to use the RDF/Turtle cmu and festlex pos-tags.

  rhdunn

  committed
  cfe627a

• Fix the festlex prefix declaration.

  rhdunn

  committed
  93d6563

• Merge commit '93d65633406f849bc73e13598a80a929d142e344'

  rhdunn

  committed
  0f00d58

• Add a description of the supported tagsets.

  rhdunn

  committed
  07f65e5

• Add tests for parsing the context tagsets.

  rhdunn

  committed
  5105a7d

• Use skos:note in the cmu tagset and add a note to cmu:1.

  rhdunn

  committed
  7609df6

• Use rdflib instead of rapper to parse RDF metadata.

  rhdunn

  committed
  772fce8

• Fix the comment description line in festlex.ttl.

  rhdunn

  committed
  32e5dd0

Commits on Apr 21, 2016

• Add a Cainteoir part of speech tagset that is compatible with the AmEPD tagset.

  rhdunn

  committed
  c105444

• Add ord (ordinal) to the Cainteoir tagset.

  rhdunn

  committed
  dcd8c6d

• Cainteoir 'verb' is an exact match of 'verb' in the part-of-speech data model.

  rhdunn

  committed
  44f940c

• Add the cainteoir tagset to the readme file.

  rhdunn

  committed
  f19449c

Commits on Apr 23, 2016

• Merge commit 'f19449c13706751ad1921f69a8fbe5716522c2b8'

  rhdunn

  committed
  fafa33a

• Add the pos-tags files to the install set.

  rhdunn

  committed
  8db0fbc

• Support using named tagsets for context-format.

  rhdunn

  committed
  3f32b61

• Rename SkosValidator to TagsetValidator.

  rhdunn

  committed
  2c5102b

• Document the context-format file-based metadata.

  rhdunn

  committed
  e3fba43

Commits on Apr 27, 2016

• README: Fix a typo.

  rhdunn

  committed
  71e3490

Commits on May 2, 2016

• Correct the various mapping relationships.

  rhdunn

  committed
  7464492

• festlex: update the v_p concept label

  rhdunn

  committed
  6b608f6

• cainteoir: correct the Cainteoir Part-of-Speech concept namespace

  rhdunn

  committed
  fa68990

• Map between Cainteoir and festlex concepts.

  rhdunn

  committed
  6e69b12

• WP20: fix mapping conjections to the Part-of-Speech data model.

  rhdunn

  committed
  1cccf0b

• WP20: classify existential there as a pronoun.

  rhdunn

  committed
  b9cf46f

• WP20: classify of as a preposition.

  rhdunn

  committed
  751664c

• WP20: make the verb concept a broad match to the POS verb as it is disctinct to the modal verb concept.

  rhdunn

  committed
  b6d23d8

• WP20: use skos:relatedMatch instead of skos:related for mapping wp20:in to the POS concepts.

  rhdunn

  committed
  7c4e6d5

• upenn: fix the xsd namespace

  rhdunn

  committed
  9f653cb

• Remove unused xsd namespace references.

  rhdunn

  committed
  36a91a9

• festlex: split out 'vl' and 'y' into separate concepts, so each concept only has one skos:notation

  rhdunn

  committed
  0205540

• festlex: better model the concepts that are typos using related and relatedMatch

  rhdunn

  committed
  c594155

• Support loading concept mappings from SKOS metadata.

  rhdunn

  committed
  9db2027

• Merge commit 'c59415506b55b1362388ffa7664ed47b65516f79'

  rhdunn

  committed
  0b5b20c

Commits on May 3, 2016

• Support converting context tagsets (e.g. cainteoir to festlex).

  rhdunn

  committed
  91a604c

• Factor out context mapping into a helper function.

  rhdunn

  committed
  36041ac

• Implement a remove duplicate contexts option.

  rhdunn

  committed
  a1707ea

• Fix printing key/value pairs when the value is a string.

  rhdunn

  committed
  ea83b80

Commits on Dec 21, 2016

• Add standard GNU targets to the Makefile and building documentation.

  rhdunn

  committed
  419297b

• Ignore *~ temporary files.

  rhdunn

  committed
  9524335

Commits on May 13, 2017

• Don't count errors as words in the statistics.

  rhdunn

  committed
  7f3795c

Commits on Jul 1, 2017

• Add adj@attr and adj@pred concepts to the cainteoir pos tagset.

  rhdunn

  committed
  0d5115c

• Merge commit '0d5115c6bb246e6346bfacd21d650dde9228f711'

  rhdunn

  committed
  fae3f42

Commits on Sep 5, 2017

• Let the user provide environment variables for custom prefix installation and python version

  zeehio

  committed
  fe97e68