More bug fixes and features #3
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This attempts to detect the dictionary format based on the filename. It supports the following filenames: | cmudict.0.1 .. cmudict.0.6e | `weide` format | | cmudict.dict | `new` format | | cmudict.vp | `new` format | | cmudict* | `air` format | This matches the usage in the various cmudict releases and the cmusphynx svn repository.
This makes it easier to share the descriptions with the VIM syntax file and file-based metadata options.
This uses the `cmudict`, `cmudict-new` and `cmudict-weide` values for `b:cmudict_format`. This also updates the VIM syntax documentation in the README file to use the Configuration Options documentation.
It is not possible to determine if a new buffer is intended to be a cmudict file.
This uses the phone classifications from `cmudict.0.7a.phones` for the consonant phones.
This checks for common stress patterns to identify errors:
1. A word with only one vowel which is a weak (schwa) vowel.
This is for weak forms like 'a', 'the' and 'had'.
2. A non-weak form without a primary vowel.
3. A non-weak form with more than one primary vowel.
NOTE: Weak form detection requires 'AX' to be used instead of 'AH0'.
In the missing-primary-stress check, allow entries like: SHHH SH ZZZZ Z These should really be syllabic consonants, but that would introduce non-standard phones that would only be used in isolated cases.
…sctinct to the modal verb concept.
…in to the POS concepts.
…pt only has one skos:notation
…tion and python version
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This makes the following additional improvements to
cmudict-tools:README.mdfilesphinxdictionary formatunicodesorting (asairsorting, but using the Unicode Collation Algorithm for sensible ordering of accented characters)--remove-stress-- remove stress markers from vowel phones (collapsing duplicate pronunciations)--remove-context-entries-- removeWORD(CONTEXT)based entries--remove-syllable-breaks-- remove syllable break (-) markers-Wmissing-primary-stress-- warn on non-weak entries without a primary stress-Wmultiple-primary-stress-- warn on entries with more than one primary stressphonesets
cmudict.0.7a.phonesclassification into the accents CSV filesXphone (for Scottishloch)metadata
metadata=@T:namemetadata declarations (whereTissfor strings,ifor integers andffor reals)select=QUERYcommand to extract data from the dictionary (e.g. for porter stemmer tests)format=,accent=,phoneset=,order-from=,encoding=andsorting=file-based metadata when parsing cmudict filesvim syntax files
-as a syllable break in the arpabet phonesetacronymfilesftdetectfile)accent=,phoneset=andformat=file-based metadata