Improve memory usage when no style is applied #659
Labels
Comments
|
I think the best thing to do in this case is to determine how much of a difference in memory occurs between the current approach, where SEG and S are separated from one another, and the previous approach where the two were combined. If it's not much of a difference, this implies a need to optimize something. If there is a difference, I may have introduced a memory leak. The code snippet was not intended to work out-of-box. It served to give you a general direction to pursue in implementing your own version of the idea. |
|
I have started a memory profile session using Netbeans ( is free and has a Java profiler included ). There you can choose to profile memory. Press the delta button for a diff. The first picture is a delta after the project is started, without doing nothing. Many string gets allocated, probably through events.
|
|
Thanks for your work. I plan on looking into this more later on today. |
Try this: SuspendableYes allowRichChange = new SuspendableYes();
EventStream<?> plainTextChange = codeArea.richChanges()
.hook(c -> System.out.println("rich change occurred, firing event"))
.successionEnds(Duration.ofMillis(500))
.hook(c -> System.out.println("user has not typed anything for the given duration: firing event"))
.conditionOn(allowRichChange)
.hook(c -> System.out.println("this is a valid user-intitiated event, not us changing the style; so allow it to pass through"))
.filter(ch -> !ch.isIdentity())
.hook(c -> System.out.println("this is a valid plain text change. firing plaintext change event"));
EventStream<?> dirtyViewport = EventStreams
.merge(
codeArea.estimatedScrollXProperty().values(),
codeArea.estimatedScrollYProperty().values())
.hook(e -> System.out.println("y or x property value changed"))
.successionEnds(Duration.ofMillis(200))
.hook(e -> System.out.println("We've waited long enough, now fire an event"));
EventStreams.merge(plainTextChange, dirtyViewport)
.hook(c -> System.out.println("either the viewport has changed or a plain text change has occurred. fire an event"))
.subscribe(dirtyStyles -> {
// rather than clearing the previously-visible text's styles and setting the current
// visible text's styles....
//
// codeArea.clearStyle(startOfLastVisibleTextStyle, endOfLastVisibleTextStyle);
// codeArea.setStyleSpans(offsetIntoContentBeforeReachingCurrentVisibleStyles, newStyles);
//
// do this entire action in one move
//
// codeArea.setStyleSpans(0, styleSpans)
//
// where `styleSpans` parameters are...
// | unstyled | previously styled | currently visible text | previously styled | unstyled |
// | empty list | computeHighlighting() | empty list |
// compute the styles for the currently visible text
StyleSpans<Collection<String>> visibleTextStyles = computeHighlighting(getVisibleText());
// calculate how far into the content is the first part of the visible text
// before modifying the area's styles
int firstVisibleParIdx = codeArea.visibleParToAllParIndex(0);
int startOfVisibleStyles = codeArea.position(firstVisibleParIdx, 0).toOffset();
int lengthFollowingVisibleStyles = codeArea.getLength() - startOfVisibleStyles - visibleTextStyles.length();
StyleSpans<Collection<String>> styleSpans = visibleTextStyles
// 1 single empty list before visible styles
.prepend(new StyleSpan<>(Collections.emptyList(), startOfVisibleStyles))
// 1 single empty list after visible styles
.append(new StyleSpan<>(Collections.emptyList(), lengthFollowingVisibleStyles));
// no longer allow rich changes as setStyleSpans() will emit a rich change event,
// which will lead to an infinite loop that will terminate with a StackOverflowError
allowRichChange.suspendWhile(() ->
codeArea.setStyleSpans(0, styleSpans)
);
}); |
|
So, using the original rich changes approach used in the demo, it jumps from a styleless ~45 MB on my end to about ~170 MB when the entire area is styled. When I use the approach I created above, it jumps to about ~100 MB. However, due to FXMisc/Flowless#60, the memory continues to increase due to the scroll values emitting events that trigger a re-computation of the syntax highlighting :-/ |
|
Could you please provide the complete class source code for the example above ? I am missing the "getVisibleText()" and this does not compile fine for me. My issue is related only to unstyled document with large files. |
|
My apologies. I posted the same thing in #627, and I included the Here's the full source code: import javafx.application.Application;
import javafx.application.Platform;
import javafx.beans.binding.Bindings;
import javafx.beans.property.SimpleIntegerProperty;
import javafx.beans.property.SimpleObjectProperty;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.IndexRange;
import javafx.scene.control.Label;
import javafx.scene.control.ToolBar;
import javafx.scene.layout.StackPane;
import javafx.scene.layout.VBox;
import javafx.stage.Stage;
import org.fxmisc.flowless.VirtualizedScrollPane;
import org.fxmisc.richtext.CodeArea;
import org.fxmisc.richtext.LineNumberFactory;
import org.fxmisc.richtext.model.Paragraph;
import org.fxmisc.richtext.model.StyleSpan;
import org.fxmisc.richtext.model.StyleSpans;
import org.fxmisc.richtext.model.StyleSpansBuilder;
import org.reactfx.EventStream;
import org.reactfx.EventStreams;
import org.reactfx.SuspendableNo;
import org.reactfx.SuspendableYes;
import org.reactfx.util.FxTimer;
import java.text.NumberFormat;
import java.time.Duration;
import java.util.Collection;
import java.util.Collections;
import java.util.Timer;
import java.util.TimerTask;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class ScalabilityDemo extends Application {
private static final String[] KEYWORDS = new String[] {
"abstract", "assert", "boolean", "break", "byte",
"case", "catch", "char", "class", "const",
"continue", "default", "do", "double", "else",
"enum", "extends", "final", "finally", "float",
"for", "goto", "if", "implements", "import",
"instanceof", "int", "interface", "long", "native",
"new", "package", "private", "protected", "public",
"return", "short", "static", "strictfp", "super",
"switch", "synchronized", "this", "throw", "throws",
"transient", "try", "void", "volatile", "while"
};
private static final String KEYWORD_PATTERN = "\\b(" + String.join("|", KEYWORDS) + ")\\b";
private static final String PAREN_PATTERN = "\\(|\\)";
private static final String BRACE_PATTERN = "\\{|\\}";
private static final String BRACKET_PATTERN = "\\[|\\]";
private static final String SEMICOLON_PATTERN = "\\;";
private static final String STRING_PATTERN = "\"([^\"\\\\]|\\\\.)*\"";
private static final String COMMENT_PATTERN = "//[^\n]*" + "|" + "/\\*(.|\\R)*?\\*/";
private static final Pattern PATTERN = Pattern.compile(
"(?<KEYWORD>" + KEYWORD_PATTERN + ")"
+ "|(?<PAREN>" + PAREN_PATTERN + ")"
+ "|(?<BRACE>" + BRACE_PATTERN + ")"
+ "|(?<BRACKET>" + BRACKET_PATTERN + ")"
+ "|(?<SEMICOLON>" + SEMICOLON_PATTERN + ")"
+ "|(?<STRING>" + STRING_PATTERN + ")"
+ "|(?<COMMENT>" + COMMENT_PATTERN + ")"
);
private final int TEST_ROWS = 400 * 100;
private final Runtime runtime = Runtime.getRuntime();
private final CodeArea codeArea = new CodeArea();
private final Label memoryLabel = new Label();
private Timer updateTimer;
public static void main(String[] args) {
System.setProperty("prism.lcdtext", "false");
System.setProperty("prism.text", "t2k");
launch(args);
}
private String getSampleCode(){
StringBuilder sb = new StringBuilder(String.join("\n", new String[] {
"package com.example;",
"",
"import java.util.*;",
"",
"public class Foo extends Bar implements Baz {",
"",
" /*",
" * multi-line comment",
" */",
" public static void main(String[] args) {",
" // single-line comment",
" for(String arg: args) {",
" if(arg.length() != 0)",
" System.out.println(arg);",
" else",
" System.err.println(\"Warning: empty string as argument\");",
" }",
"\n"
}));
for (int i=0 ; i < TEST_ROWS ; i++) {
sb.append(" System.out.println(\"");
sb.append(Math.random());
sb.append("\");\n");
}
sb.append(" }\n");
sb.append("}\n");
return sb.toString();
}
@Override
public void start(Stage primaryStage) {
codeArea.setParagraphGraphicFactory(LineNumberFactory.get(codeArea));
codeArea.replaceText(0, 0, getSampleCode());
Button gcButton = new Button("Garbage Collect");
gcButton.setOnAction( event -> {
System.gc ();
System.runFinalization ();
});
Button styleButton = new Button("Parse");
styleButton.setOnAction( event -> {
// setupOriginal();
// setupOriginalWithSuccessions();
setupOptimized();
});
styleButton.fire();
ToolBar toolbar = new ToolBar();
toolbar.getItems().addAll( memoryLabel, gcButton, styleButton );
VBox vBox = new VBox();
StackPane stackPane = new StackPane(new VirtualizedScrollPane<>(codeArea));
stackPane.setPrefSize(1200, 880);
vBox.getChildren().addAll( toolbar, stackPane );
Scene scene = new Scene(vBox, 1200, 900);
scene.getStylesheets().add(ScalabilityDemo.class.getResource("java-keywords.css").toExternalForm());
primaryStage.setScene(scene);
primaryStage.setTitle("CodeArea Scalability Test");
primaryStage.show();
FxTimer.runPeriodically(Duration.ofSeconds(1), () -> memoryLabel.setText(memoryStatus()));
}
public void setupOriginal() {
codeArea.richChanges()
.filter(ch -> !ch.getInserted().equals(ch.getRemoved()))
.subscribe(change -> {
codeArea.setStyleSpans(0, computeHighlighting(codeArea.getText()));
});
}
public void setupOriginalWithSuccessions() {
codeArea.richChanges()
.successionEnds(Duration.ofMillis(500))
.filter(ch -> !ch.getInserted().equals(ch.getRemoved()))
.subscribe(change -> {
codeArea.setStyleSpans(0, computeHighlighting(codeArea.getText()));
});
}
public void setupOptimized() {
SuspendableYes allowRichChange = new SuspendableYes();
EventStream<?> plainTextChange = codeArea.richChanges()
.hook(c -> System.out.println("rich change occurred, firing event"))
.successionEnds(Duration.ofMillis(500))
.hook(c -> System.out.println("user has not typed anything for the given duration: firing event"))
.conditionOn(allowRichChange)
.hook(c -> System.out.println("this is a valid user-intitiated event, not us changing the style; so allow it to pass through"))
.filter(ch -> !ch.isIdentity())
.hook(c -> System.out.println("this is a valid plain text change. firing plaintext change event"));
EventStream<?> dirtyViewport = EventStreams
.merge(
codeArea.estimatedScrollXProperty().values(),
codeArea.estimatedScrollYProperty().values())
.hook(e -> System.out.println("y or x property value changed"))
.successionEnds(Duration.ofMillis(200))
.hook(e -> System.out.println("We've waited long enough, now fire an event"));
EventStreams.merge(plainTextChange, dirtyViewport)
.hook(c -> System.out.println("either the viewport has changed or a plain text change has occurred. fire an event"))
.subscribe(dirtyStyles -> {
// rather than clearing the previously-visible text's styles and setting the current
// visible text's styles....
//
// codeArea.clearStyle(startOfLastVisibleTextStyle, endOfLastVisibleTextStyle);
// codeArea.setStyleSpans(offsetIntoContentBeforeReachingCurrentVisibleStyles, newStyles);
//
// do this entire action in one move
//
// codeArea.setStyleSpans(0, styleSpans)
//
// where `styleSpans` parameters are...
// | unstyled | previously styled | currently visible text | previously styled | unstyled |
// | empty list | computeHighlighting() | empty list |
// compute the styles for the currently visible text
StyleSpans<Collection<String>> visibleTextStyles = computeHighlighting(getVisibleText());
// calculate how far into the content is the first part of the visible text
// before modifying the area's styles
int firstVisibleParIdx = codeArea.visibleParToAllParIndex(0);
int startOfVisibleStyles = codeArea.position(firstVisibleParIdx, 0).toOffset();
int lengthFollowingVisibleStyles = codeArea.getLength() - startOfVisibleStyles - visibleTextStyles.length();
StyleSpans<Collection<String>> styleSpans = visibleTextStyles
// 1 single empty list before visible styles
.prepend(new StyleSpan<>(Collections.emptyList(), startOfVisibleStyles))
// 1 single empty list after visible styles
.append(new StyleSpan<>(Collections.emptyList(), lengthFollowingVisibleStyles));
// no longer allow rich changes as setStyleSpans() will emit a rich change event,
// which will lead to an infinite loop that will terminate with a StackOverflowError
allowRichChange.suspendWhile(() ->
codeArea.setStyleSpans(0, styleSpans)
);
});
}
public String getVisibleText() {
return codeArea.getVisibleParagraphs().map(Paragraph::getText).reduce((a, b) -> a + "\n" + b).getOrElse("");
}
public String memoryStatus() {
NumberFormat format = NumberFormat.getInstance();
StringBuilder sb = new StringBuilder();
long maxMemory = runtime.maxMemory();
long allocatedMemory = runtime.totalMemory();
long freeMemory = runtime.freeMemory();
sb.append("Used ");
sb.append(format.format((runtime.totalMemory() - runtime.freeMemory()) / (1024*1024)) );
sb.append(" MB from ");
sb.append(format.format((freeMemory + (maxMemory - allocatedMemory)) / (1024*1024)) );
sb.append(" MB allocated memory. Text length " + codeArea.textProperty().getValue().length());
return sb.toString();
}
private static StyleSpans<Collection<String>> computeHighlighting(String text) {
Matcher matcher = PATTERN.matcher(text);
int lastKwEnd = 0;
StyleSpansBuilder<Collection<String>> spansBuilder
= new StyleSpansBuilder<>();
while(matcher.find()) {
String styleClass =
matcher.group("KEYWORD") != null ? "keyword" :
matcher.group("PAREN") != null ? "paren" :
matcher.group("BRACE") != null ? "brace" :
matcher.group("BRACKET") != null ? "bracket" :
matcher.group("SEMICOLON") != null ? "semicolon" :
matcher.group("STRING") != null ? "string" :
matcher.group("COMMENT") != null ? "comment" :
null; /* never happens */ assert styleClass != null;
spansBuilder.add(Collections.emptyList(), matcher.start() - lastKwEnd);
spansBuilder.add(Collections.singleton(styleClass), matcher.end() - matcher.start());
lastKwEnd = matcher.end();
}
spansBuilder.add(Collections.emptyList(), text.length() - lastKwEnd);
return spansBuilder.create();
}
} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I would like to use the RichTextFx for large SQL files ( can be files from backups ). I tested this and the memory usage is low in the beginning, but increase very fast if I scroll the editor - it later decrease again, meaning there are some temporary large objects created.
I used to test the code from #627
where I disabled the style highlighting.
Could you please check the memory usage for this case ?
My plan is:
In the same issue on the bottom is also a code snippet for highlighting only the current viewport. This does not currently work if I include it in one of the examples. Could you please provide also the correct code for this ?
I think a similar strategy can be used also by the other cases when large files are loaded.
The text was updated successfully, but these errors were encountered: