Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix][ci] Increase thread stack size to 1500k for Maven JVM to prevent StackOverflowError in compilation #18602

Merged
merged 1 commit into from
Nov 24, 2022
Merged

[fix][ci] Increase thread stack size to 1500k for Maven JVM to prevent StackOverflowError in compilation #18602

merged 1 commit into from
Nov 24, 2022

Conversation

lhotari
Copy link
Member

@lhotari lhotari commented Nov 24, 2022 •
edited
Loading

Motivation

  • For some reason compilation can fail with StackOverflowError with the default settings
    • increase thread stack size to 1500k (with -Xss1500k)

Example failure:

Error:  Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project pulsar-broker-common: Fatal error compiling: java.lang.StackOverflowError -> [Help 1]

I've created this PR as a draft so that we can use this if we end up needing to address this issue and it reproduces again.

Modifications

  • Add -Xss1500k to MAVEN_OPTS

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: lhotari#104

@lhotari
[fix][ci] Increase thread stack size to 1500k for Maven JVM to preven…
0be8dc9 
…t StackOverflowError in compilation

- For some reason compilation can fail with StackOverflowError with the default settings
  - increase thread stack size to 1500k (with -Xss1500k)

Example:
Error:  Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project pulsar-broker-common: Fatal error compiling: java.lang.StackOverflowError -> [Help 1]
@lhotari lhotari mentioned this pull request Nov 24, 2022
Closed
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Nov 24, 2022
@lhotari lhotari changed the title [fix][ci] Increase thread stack size to 1500k for Maven JVM to preventt StackOverflowError in compilation [fix][ci] Increase thread stack size to 1500k for Maven JVM to prevent StackOverflowError in compilation Nov 24, 2022
@lhotari
Copy link
Member Author

lhotari commented Nov 24, 2022

It looks like the problem reproduced in https://github.com/apache/pulsar/actions/runs/3539671520/jobs/5942046881#step:10:3748

@lhotari lhotari requested a review from nicoloboschi November 24, 2022 10:43
@lhotari
Copy link
Member Author

lhotari commented Nov 24, 2022

Log from the failed build 

[INFO] Compiling 92 source files to /home/runner/work/pulsar/pulsar/pulsar-broker-common/target/classes
  java.lang.StackOverflowError
  	at lombok.core.AST.fieldsOf(AST.java:234)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:1)
  	at lombok.core.AST.buildWithField0(AST.java:394)
  	at lombok.core.AST.buildWithField(AST.java:292)
  	at lombok.javac.JavacAST.drill(JavacAST.java:506)
  	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:476)
  	at lombok.javac.JavacAST.buildTree(JavacAST.java:285)

@lhotari
Copy link
Member Author

lhotari commented Nov 24, 2022

Attempting to fix the issue first by upgrading Lombok in #18604

@lhotari lhotari mentioned this pull request Nov 24, 2022
Merged
4 tasks
@lhotari
Copy link
Member Author

lhotari commented Nov 24, 2022

the issue in Pulsar might match projectlombok/lombok#1118

@lhotari lhotari marked this pull request as ready for review November 24, 2022 11:51
@lhotari
Copy link
Member Author

lhotari commented Nov 24, 2022

It looks like the only mitigation for the Lombok issue is to increase stack size.

@lhotari
Copy link
Member Author

lhotari commented Nov 24, 2022

/pulsarbot rerun-failure-checks

@codecov-commenter
Copy link

codecov-commenter commented Nov 24, 2022 •
edited
Loading

Codecov Report

Merging #18602 (0be8dc9) into master (598ca5d) will increase coverage by 0.79%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #18602      +/-   ##
============================================
+ Coverage     47.72%   48.51%   +0.79%     
- Complexity     9330     9476     +146     
============================================
  Files           618      618              
  Lines         58568    58609      +41     
  Branches       6093     6097       +4     
============================================
+ Hits          27951    28436     +485     
+ Misses        27589    27134     -455     
- Partials       3028     3039      +11
Flag Coverage Δ
unittests 48.51% <ø> (+0.79%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ker/service/schema/exceptions/SchemaException.java 60.00% <0.00%> (-40.00%) ⬇️
.../pulsar/broker/service/SharedConsumerAssignor.java 68.51% <0.00%> (-9.26%) ⬇️
...lsar/broker/loadbalance/impl/ThresholdShedder.java 38.56% <0.00%> (-4.58%) ⬇️
...ker/resourcegroup/ResourceGroupConfigListener.java 64.38% <0.00%> (-2.74%) ⬇️
...g/apache/pulsar/broker/lookup/TopicLookupBase.java 52.51% <0.00%> (-1.68%) ⬇️
...he/pulsar/broker/admin/v2/NonPersistentTopics.java 60.64% <0.00%> (-1.39%) ⬇️
...sar/broker/loadbalance/impl/LoadManagerShared.java 43.42% <0.00%> (-0.88%) ⬇️
...broker/service/schema/BookkeeperSchemaStorage.java 73.92% <0.00%> (-0.31%) ⬇️
...oker/service/schema/SchemaRegistryServiceImpl.java 62.16% <0.00%> (-0.31%) ⬇️
...sistent/PersistentDispatcherMultipleConsumers.java 58.47% <0.00%> (-0.18%) ⬇️
... and 58 more

@lhotari lhotari closed this Nov 24, 2022
@lhotari lhotari reopened this Nov 24, 2022
@lhotari lhotari merged commit b3d0985 into apache:master Nov 24, 2022
lifepuzzlefun pushed a commit to lifepuzzlefun/pulsar that referenced this pull request Dec 9, 2022
@lhotari @lifepuzzlefun
[fix][ci] Increase thread stack size to 1500k for Maven JVM to preven…
544a7d7 
…t StackOverflowError in compilation (apache#18602)
@lhotari lhotari mentioned this pull request Dec 15, 2022
Merged
lifepuzzlefun pushed a commit to lifepuzzlefun/pulsar that referenced this pull request Jan 10, 2023
@lhotari @lifepuzzlefun
[fix][ci] Increase thread stack size to 1500k for Maven JVM to preven…
3568063 
…t StackOverflowError in compilation (apache#18602)
@lhotari lhotari mentioned this pull request Jan 25, 2023
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nicoloboschi nicoloboschi nicoloboschi approved these changes

Assignees
No one assigned
Labels
doc-not-needed Your PR changes do not impact docs ready-to-test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lhotari @codecov-commenter @nicoloboschi