Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new option allowing for multiline cards with empty lines #1012

Merged
merged 29 commits into from
Aug 25, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
e2f6750
Updated to upstream. Included all changes.
alberti42 Aug 15, 2024
41f90c1
Forgot to include peggy in the dependencies
alberti42 Aug 15, 2024
0e35b5b
Automatically generate parser
alberti42 Aug 15, 2024
f33cfeb
Trying to improve the structure.
alberti42 Aug 16, 2024
7e5529b
Trying to improve the structure.
alberti42 Aug 16, 2024
b9098ee
Trying to understand the structure of tests
alberti42 Aug 16, 2024
1940e7b
Updated rules
alberti42 Aug 16, 2024
7b77d23
Fixing testing units
alberti42 Aug 16, 2024
84523bc
Testing a different approach
alberti42 Aug 16, 2024
f9e3303
Finished with the dynamic parser
alberti42 Aug 16, 2024
460d71b
All tests now pass
alberti42 Aug 16, 2024
1465e9d
Remove json
alberti42 Aug 16, 2024
6297fe5
Parser error case test
4Source Aug 16, 2024
8a63ed2
Add PaserDebugSetting + test case for messages
4Source Aug 16, 2024
6976005
- Minor changes to the language translations
alberti42 Aug 17, 2024
2be3444
- Added a few test cases for testing `multilineCardEndMarker`
alberti42 Aug 17, 2024
ca0f980
- Added more comments to the code
alberti42 Aug 17, 2024
9514c97
- Added tests for multiline clozes with `multilineCardEndMarker`
alberti42 Aug 17, 2024
b2d6d18
Minor cleanup of the parser grammar
alberti42 Aug 17, 2024
d3c0c2e
Squashed commit of the following:
ronzulu Aug 20, 2024
3ed07ee
Added some comments, minor stylistic changes to the parser grammar, u…
ronzulu Aug 20, 2024
7030325
Tiny change.
alberti42 Aug 20, 2024
1320993
Added some comments, minor stylistic changes to the parser grammar, u…
ronzulu Aug 22, 2024
a73601c
Trying to fix but not there yet
alberti42 Aug 24, 2024
d71d3ba
Seems to work again
alberti42 Aug 24, 2024
f700346
One more fix and added one more test case. All tests pass.
alberti42 Aug 24, 2024
04c2ca5
Partial update of the user documentation
ronzulu Aug 25, 2024
3b95700
Merge remote-tracking branch 'alberti42/master' into rz-alberti42-v2
ronzulu Aug 25, 2024
38d15df
Completed changes to user documentation
ronzulu Aug 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@
},
"dependencies": {
"chart.js": "^4.4.2",
"pagerank.js": "^1.0.2"
"pagerank.js": "^1.0.2",
"peggy": "^4.0.3"
},
"packageManager": "^[email protected]"
}
12 changes: 1 addition & 11 deletions src/NoteQuestionParser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,17 +136,7 @@ export class NoteQuestionParser {

private parseQuestions(): ParsedQuestionInfo[] {
// We pass contentText which has the frontmatter blanked out; see extractFrontmatter for reasoning
const settings: SRSettings = this.settings;
const result: ParsedQuestionInfo[] = parseEx(
this.contentText,
settings.singleLineCardSeparator,
settings.singleLineReversedCardSeparator,
settings.multilineCardSeparator,
settings.multilineReversedCardSeparator,
settings.convertHighlightsToClozes,
settings.convertBoldTextToClozes,
settings.convertCurlyBracketsToClozes,
);
const result: ParsedQuestionInfo[] = parseEx(this.contentText);
return result;
}

Expand Down
226 changes: 226 additions & 0 deletions src/generateParser.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
// generateParser.ts

import { SRSettings } from "./settings";
import { generate, Parser } from "peggy";

export function generateParser(settings: SRSettings): Parser {

const close_rules_list: string[] = [];

if(settings.convertHighlightsToClozes) close_rules_list.push("close_equal");
if(settings.convertBoldTextToClozes) close_rules_list.push("close_star");
if(settings.convertCurlyBracketsToClozes) close_rules_list.push("close_bracket");

const close_rules = close_rules_list.join(" / ");

const grammar = `
{
// The fallback case is important if we want to test the rules with https://peggyjs.org/online.html
const CardTypeFallBack = {
SingleLineBasic: 0,
SingleLineReversed: 1,
MultiLineBasic: 2,
MultiLineReversed: 3,
Cloze: 4,
};

// The fallback case is important if we want to test the rules with https://peggyjs.org/online.html
const createParsedQuestionInfoFallBack = (cardType, text, firstLineNum, lastLineNum) => {
return {cardType, text, firstLineNum, lastLineNum};
};

const CardType = options.CardType ? options.CardType : CardTypeFallBack;
CardType.Ignore=null;
const createParsedQuestionInfo = options.createParsedQuestionInfo ? options.createParsedQuestionInfo : createParsedQuestionInfoFallBack;

function filterBlocks(b) {
return b.filter( (d) => d.cardType === CardType.Ignore ? false : true )
}
}

main
= blocks:block* { return filterBlocks(blocks); }

block
= html_comment / inline_rev_card / inline_card / multiline_rev_card / multiline_card / close_card / loose_line

html_comment
= $("<!--" (!"-->" (html_comment / .))* "-->" newline?) {
return createParsedQuestionInfo(CardType.Ignore,"",0,0);
}

tag
= $("#" + name:([a-zA-Z/\\-_] { return 1; } / [0-9]{ return 0;})+ &{
// check if it is a valid Obsidian tag
return name.includes(1);
})

tag_line
= tag:tag _* nl:newline { return tag + nl; }

inline_card
= e:inline newline? { return e; }

inline
= left:(!inline_mark [^\\n\\r])+ inline_mark right:not_newline (newline annotation)? {
return createParsedQuestionInfo(CardType.SingleLineBasic,text(),location().start.line-1,location().end.line-1);
}

inline_rev_card
= e:inline_rev newline? { return e; }

inline_rev
= left:(!inline_rev_mark [^\\n\\r])+ inline_rev_mark right:not_newline (newline annotation)? {
return createParsedQuestionInfo(CardType.SingleLineReversed,text(),location().start.line-1,location().end.line-1);
}

multiline_card
= c:multiline separator_line {
return c;
}

multiline
= tag:tag_line? arg1:multiline_before multiline_mark arg2:multiline_after {
return createParsedQuestionInfo(CardType.MultiLineBasic,((tag ?? "") + arg1+"${settings.multilineCardSeparator}"+"\\n"+arg2.trim()),location().start.line-1,location().end.line-2);
}

multiline_before
= $(!multiline_mark nonempty_text_line)+

multiline_after
= $(!separator_line (tilde_code / backprime_code / text_line))+

tilde_code
= $(left:$tilde_marker text_line t:$(!(middle:$tilde_marker &{ return left.length===middle.length;}) (tilde_code / text_line))* (right:$tilde_marker &{ return left.length===right.length; }) newline)

tilde_marker
= "~~~" "~"*

backprime_code
= $(left:$backprime_marker text_line t:$(!(middle:$backprime_marker &{ return left.length===middle.length;}) (backprime_code / text_line))* (right:$backprime_marker &{ return left.length===right.length; }) newline)

backprime_marker
= "\`\`\`" "\`"*

multiline_rev_card
= @multiline_rev separator_line

multiline_rev
= tag:tag_line? arg1:multiline_rev_before multiline_rev_mark arg2:multiline_rev_after {
return createParsedQuestionInfo(CardType.MultiLineReversed,((tag ?? "") + arg1+"${settings.multilineReversedCardSeparator}"+"\\n"+arg2.trim()),location().start.line-1,location().end.line-2);
}

multiline_rev_before
= $(!multiline_rev_mark nonempty_text_line)+

multiline_rev_after
= $(!separator_line text_line)+

close_card
= $(multiline_before_close? f:close_line e:(multiline_after_close)? e1:(newline annotation)?) {
return createParsedQuestionInfo(CardType.Cloze,text().trim(),location().start.line-1,location().end.line-1);
}

close_line
= ((!close_text [^\\n\\r])* close_text) text_line_nonterminated?

multiline_before_close
= (!close_line nonempty_text_line)+

multiline_after_close
= e:(!(newline separator_line) text_line1)+

close_text
= ${close_rules}

close_equal
= close_mark_equal (!close_mark_equal [^\\n\\r])+ close_mark_equal

close_mark_equal
= "=="

close_star
= close_mark_star (!close_mark_star [^\\n\\r])+ close_mark_star

close_mark_star
= "**"

close_bracket
= close_mark_bracket_open (!close_mark_bracket_close [^\\n\\r])+ close_mark_bracket_close

close_mark_bracket_open
= "{{"

close_mark_bracket_close
= "}}"

inline_mark
= "${settings.singleLineCardSeparator}"

inline_rev_mark
= "${settings.singleLineReversedCardSeparator}"

multiline_mark
= "${settings.multilineCardSeparator}" _* newline

multiline_rev_mark
= "${settings.multilineReversedCardSeparator}" _* newline

end_card_mark
= "${settings.multilineCardEndMarker}"

separator_line
= end_card_mark newline

text_line_nonterminated
= $[^\\n\\r]+

nonempty_text_line
= @$[^\\n\\r]+ newline

text_line
= @$[^\\n\\r]* newline

text_line1
= newline @$[^\\n\\r]*

loose_line
= $(([^\\n\\r]* newline) / [^\\n\\r]+) {
return createParsedQuestionInfo(CardType.Ignore,"",0,0);
}

annotation
= $("<!--SR:" (!"-->" .)+ "-->")

not_newline
= $[^\\n\\r]*

newline
= $[\\n\\r]

empty_line
= $(_* [\\n\\r])

empty_lines
= $empty_line+

nonemptyspace
= [^ \\f\\t\\v\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff]

emptyspace
= _*

_ = ([ \\f\\t\\v\\u0020\\u00a0\\u1680\\u2000-\\u200a\\u2028\\u2029\\u202f\\u205f\\u3000\\ufeff])
`;

if (settings.showDebugMessages) {
console.log(grammar);
}

// const t0 = Date.now();
const parser = generate(grammar);
// const t1 = Date.now();
// console.log("To generate the parser, it took " + (t1 - t0) + " milliseconds.")

return parser;
}
1 change: 1 addition & 0 deletions src/lang/locale/ar.ts
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ Note that this setting is common to both Flashcards and Notes. : أدخل مسا
INLINE_REVERSED_CARDS_SEPARATOR: "فاصل من أجل البطاقات العكسية المضمنة",
MULTILINE_CARDS_SEPARATOR: "فاصل من أجل البطاقات المتعددة",
MULTILINE_REVERSED_CARDS_SEPARATOR: "فاصل من أجل البطاقات العكسية المتعددة",
MULTILINE_CARDS_END_MARKER: "الأحرف التي تدل على نهاية الكلوزات وبطاقات التعلم المتعددة الأسطر",
NOTES: "ملاحظات",
REVIEW_PANE_ON_STARTUP: "تمكين جزء مراجعة الملاحظات عند بدء التشغيل",
TAGS_TO_REVIEW: "وسوم للمراجعة",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/cz.ts
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ Note that this setting is common to both Flashcards and Notes.`,
INLINE_REVERSED_CARDS_SEPARATOR: "Oddělovač pro otočené inline kartičky",
MULTILINE_CARDS_SEPARATOR: "Oddělovač pro víceřádkové kartičky",
MULTILINE_REVERSED_CARDS_SEPARATOR: "Oddělovač pro víceřádkove otočené kartičky",
MULTILINE_CARDS_END_MARKER: "Znaky označující konec clozes a víceřádkových flash karet",
NOTES: "Poznámky",
REVIEW_PANE_ON_STARTUP: "Enable note review pane on startup",
TAGS_TO_REVIEW: "Tag pro revizi",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/de.ts
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,7 @@ Note that this setting is common to both Flashcards and Notes.`,
INLINE_REVERSED_CARDS_SEPARATOR: "Trennzeichen für einzeilige beidseitige Lernkarten",
MULTILINE_CARDS_SEPARATOR: "Trennzeichen für mehrzeilige Lernkarten",
MULTILINE_REVERSED_CARDS_SEPARATOR: "Trennzeichen für mehrzeilige beidseitige Lernkarten",
MULTILINE_CARDS_END_MARKER: "Zeichen, die das Ende von Lückentexten und mehrzeiligen Flashcards kennzeichnen",
NOTES: "Notizen",
REVIEW_PANE_ON_STARTUP: "Öffne Überprüfungswarteschlage beim start",
TAGS_TO_REVIEW: "Zu wiederholende Tags",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ Note that this setting is common to both Flashcards and Notes.`,
INLINE_REVERSED_CARDS_SEPARATOR: "Separator for inline reversed flashcards",
MULTILINE_CARDS_SEPARATOR: "Separator for multiline flashcards",
MULTILINE_REVERSED_CARDS_SEPARATOR: "Separator for multiline reversed flashcards",
MULTILINE_CARDS_END_MARKER: "Characters denoting the end of clozes and multiline flashcards",
NOTES: "Notes",
REVIEW_PANE_ON_STARTUP: "Enable note review pane on startup",
TAGS_TO_REVIEW: "Tags to review",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/es.ts
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ Note that this setting is common to both Flashcards and Notes.`,
MULTILINE_CARDS_SEPARATOR: "Separador para tarjetas de memorización multilínea",
MULTILINE_REVERSED_CARDS_SEPARATOR:
"Separador para tarjetas de memorización multilínea invertidas",
MULTILINE_CARDS_END_MARKER: "Caracteres que denotan el fin de los clozes y tarjetas didácticas de varias líneas",
NOTES: "Notes",
REVIEW_PANE_ON_STARTUP: "Activar panel de revisión de notas al arrancar",
TAGS_TO_REVIEW: "Etiquetas a revisar",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/it.ts
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,7 @@ Note that this setting is common to both Flashcards and Notes.`,
INLINE_REVERSED_CARDS_SEPARATOR: "Separatore per schede all'incontrario sulla stessa riga",
MULTILINE_CARDS_SEPARATOR: "Separatore per schede su più righe",
MULTILINE_REVERSED_CARDS_SEPARATOR: "Separatore per schede all'incontrario su più righe",
MULTILINE_CARDS_END_MARKER: "Caratteri che denotano la fine di carte con spazi da riempiere e carte multilinea",
NOTES: "Note",
REVIEW_PANE_ON_STARTUP: "Abilita il pannello di revisione note all'avvio",
TAGS_TO_REVIEW: "Etichette da rivedere",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/ja.ts
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ Note that this setting is common to both Flashcards and Notes.`,
INLINE_REVERSED_CARDS_SEPARATOR: "インラインの表裏反転フラッシュカードに使用するセパレーター",
MULTILINE_CARDS_SEPARATOR: "複数行のフラッシュカードに使用するセパレーター",
MULTILINE_REVERSED_CARDS_SEPARATOR: "複数行の表裏反転フラッシュカードに使用するセパレーター",
MULTILINE_CARDS_END_MARKER: "クローズと複数行フラッシュカードの終わりを示す文字",
NOTES: "ノート",
REVIEW_PANE_ON_STARTUP: "Enable note review pane on startup",
TAGS_TO_REVIEW: "レビューに使用するタグ",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/ko.ts
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ Note that this setting is common to both Flashcards and Notes.`,
INLINE_REVERSED_CARDS_SEPARATOR: "인라인 반전 플래시카드 구분자",
MULTILINE_CARDS_SEPARATOR: "여러 줄 플래시카드 구분자",
MULTILINE_REVERSED_CARDS_SEPARATOR: "여러 줄 반전 플래시카드 구분자",
MULTILINE_CARDS_END_MARKER: "클로즈와 다중 행 플래시카드의 끝을 나타내는 문자",
NOTES: "노트",
REVIEW_PANE_ON_STARTUP: "Enable note review pane on startup",
TAGS_TO_REVIEW: "리뷰에 사용할 태그",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/pl.ts
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ Note that this setting is common to both Flashcards and Notes.`,
MULTILINE_CARDS_SEPARATOR: "Separator dla kart zamaskowanych wieloliniowych",
MULTILINE_REVERSED_CARDS_SEPARATOR:
"Separator dla kart zamaskowanych odwróconych wieloliniowych",
MULTILINE_CARDS_END_MARKER: "Caracteres que denotam o fim de clozes e flashcards multilineares",
NOTES: "Notatki",
REVIEW_PANE_ON_STARTUP: "Włączyć panel przeglądu notatek przy starcie",
TAGS_TO_REVIEW: "Tagi do przeglądu",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/pt-br.ts
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ Note that this setting is common to both Flashcards and Notes.`,
INLINE_REVERSED_CARDS_SEPARATOR: "Separador para flashcards inline reversos",
MULTILINE_CARDS_SEPARATOR: "Separador para flashcards de múltiplas linhas",
MULTILINE_REVERSED_CARDS_SEPARATOR: "Separador para flashcards de múltiplas linhas reversos",
MULTILINE_CARDS_END_MARKER: "Caracteres que denotam o fim de clozes e flashcards multilinha",
NOTES: "Notas",
REVIEW_PANE_ON_STARTUP: "Habilitar painel de revisão de notas na inicialização",
TAGS_TO_REVIEW: "Etiquetas para revisar",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/ru.ts
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ Note that this setting is common to both Flashcards and Notes.`,
INLINE_REVERSED_CARDS_SEPARATOR: "Разделитель для обратных внутристрочных карточек",
MULTILINE_CARDS_SEPARATOR: "Разделитель для многострочных карточек",
MULTILINE_REVERSED_CARDS_SEPARATOR: "Разделитель для обратных многострочных карточек",
MULTILINE_CARDS_END_MARKER: "Символы, обозначающие конец клозов и многострочных карточек",
NOTES: "Заметки",
REVIEW_PANE_ON_STARTUP: "Включить панель изучения карточек при запуске программы",
TAGS_TO_REVIEW: "Теги для изучения",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/zh-cn.ts
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ Note that this setting is common to both Flashcards and Notes.`,
INLINE_REVERSED_CARDS_SEPARATOR: "单行翻转卡片的分隔符",
MULTILINE_CARDS_SEPARATOR: "多行卡片的分隔符",
MULTILINE_REVERSED_CARDS_SEPARATOR: "多行翻转卡片的分隔符",
MULTILINE_CARDS_END_MARKER: "表示填空和多行闪卡结束的字符",
NOTES: "笔记",
REVIEW_PANE_ON_STARTUP: "启动时开启笔记复习窗格",
TAGS_TO_REVIEW: "复习标签",
Expand Down
1 change: 1 addition & 0 deletions src/lang/locale/zh-tw.ts
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ Note that this setting is common to both Flashcards and Notes.`,
INLINE_REVERSED_CARDS_SEPARATOR: "單行反轉卡片的分隔字元",
MULTILINE_CARDS_SEPARATOR: "多行卡片的分隔字元",
MULTILINE_REVERSED_CARDS_SEPARATOR: "多行翻轉卡片的分隔字元",
MULTILINE_CARDS_END_MARKER: "表示填空和多行闪卡结束的字符",
NOTES: "筆記",
REVIEW_PANE_ON_STARTUP: "啟動時開啟筆記復習窗格",
TAGS_TO_REVIEW: "復習標籤",
Expand Down
Loading