Language model files

A language model or knowledgebase (KB) consists of seven or eight csv files:

Contents	Filename	Description
abbreviations	acro.csv	a list of abbreviations that should not be treated as sentence endings, and if needed also words that in contrary mark a sentence ending
filter	filter.csv	transcription rules that are applied on the concept clusters at the end of the Smart Indexing process in order to optimize the clusters
grammatical labels	labels.csv	a list of all labels that are used in the lexrep file
lexical representations	lexreps.csv	a list of words and word groups with (grammatical) labels
metadata	metadata.csv	language-specific settings for the language model
pre-processor	prepro.csv	transcription rules that are applied on the input text before the actual indexing starts
rules	rules.csv	a series of rules to disambiguate elements that can be a Concept or a Relation depending on their context and to detect attributes and their scope
regular expressions (optional)	regex.csv	extra lexical representations with counterparts in the lexreps file

A full description of the contents for these files can be found in /docs/KB-file-formats.doc.

More on how these files get translated into runnable code in the corresponding section on the Build Process