Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OutOfMemoryError when trying to generate a graph for a big project #65

Open
tisis2 opened this issue Apr 13, 2016 · 4 comments
Open

OutOfMemoryError when trying to generate a graph for a big project #65

tisis2 opened this issue Apr 13, 2016 · 4 comments

Comments

@tisis2
Copy link

tisis2 commented Apr 13, 2016

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at scala.xml.Text$.apply(Text.scala:37)
at scala.xml.UnprefixedAttribute.(UnprefixedAttribute.scala:27)
at de.schauderhaft.degraph.writer.GroupNodeWriter$.apply(Writer.scala:77
)
at de.schauderhaft.degraph.writer.NodeWriter$.apply(Writer.scala:60)
at de.schauderhaft.degraph.writer.GroupNodeWriter$$anonfun$apply$2.apply
(Writer.scala:99)
at de.schauderhaft.degraph.writer.GroupNodeWriter$$anonfun$apply$2.apply
(Writer.scala:99)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike
.scala:245)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike
.scala:245)
at scala.collection.immutable.Set$Set1.foreach(Set.scala:79)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)

    at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.

scala:47)
at scala.collection.SetLike$class.map(SetLike.scala:92)
at scala.collection.AbstractSet.map(Set.scala:47)
at de.schauderhaft.degraph.writer.GroupNodeWriter$.apply(Writer.scala:99
)
at de.schauderhaft.degraph.writer.NodeWriter$.apply(Writer.scala:60)
at de.schauderhaft.degraph.writer.GroupNodeWriter$$anonfun$apply$2.apply
(Writer.scala:99)
at de.schauderhaft.degraph.writer.GroupNodeWriter$$anonfun$apply$2.apply
(Writer.scala:99)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike
.scala:245)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike
.scala:245)
at scala.collection.immutable.HashSet$HashSet1.foreach(HashSet.scala:322
)
at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:
978)
at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:
978)
at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:
978)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:245)

    at scala.collection.AbstractSet.scala$collection$SetLike$$super$map(Set.

scala:47)
at scala.collection.SetLike$class.map(SetLike.scala:92)
at scala.collection.AbstractSet.map(Set.scala:47)
at de.schauderhaft.degraph.writer.GroupNodeWriter$.apply(Writer.scala:99
)
at de.schauderhaft.degraph.writer.NodeWriter$.apply(Writer.scala:60)
at de.schauderhaft.degraph.writer.NodeWriter$.apply(Writer.scala:55)
at de.schauderhaft.degraph.writer.NodeWriter$.apply(Writer.scala:50)
at de.schauderhaft.degraph.writer.Writer$$anonfun$toXml$1.apply(Writer.s
cala:37)

@schauder
Copy link
Collaborator

You have multipe options to work around this issue:

  1. increase the maximum heap size using the -Xmx jvm parameter.
  2. limit the scope of the analysis using include/excludeand the noJars() option. The last is only available for tests.

Also I would be interested in the following information, in order to reduce the memory requirements for the next version:

Approximately how many classes do you analyze?
Does an outputfile get created? If so, how large is it?

@tisis2
Copy link
Author

tisis2 commented Apr 13, 2016

thanks for your fast answer,

i tried to set -Xmx1024m with 32bit version of java (max possible) and got OutOfMemory exception again but later. with 1024m heap a file got created without the option it wasn't created.

i managed to do a whole run with a 64bit jvm with -Xmx4g but it looks like our project is too big to get it on one single diagram...

the project got about 30 000 classes

@schauder
Copy link
Collaborator

Wow, that are a lot of classes.

I'm kind of surprised that it only blows up in the last step, which is writing the result into a graphml file.

It probably should be fairly easy to replace that with something more scalable.
I will try to do that, but I can't promise a date.

But yed will have a hard time handling so many nodes as well, so I'm not at all sure, if this will really help you.

If you are looking for a way just to have a look at your project you might want to to give jqassist a try, which can extract similar information from your project and dumps it into a neo4j database, you can then aggregate your graph until it becomes, suitable for visualization.

@tisis2
Copy link
Author

tisis2 commented Apr 14, 2016

ok thanks i will take a look at it ;)

ya yed had also problems with the graph but after i increased the max heap it could be load and i could take a look at it :) it was a bit laggy but it was intresting to see the result ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants