Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turtle serializer #228

Closed
rubensworks opened this issue Oct 27, 2020 · 2 comments
Closed

Turtle serializer #228

rubensworks opened this issue Oct 27, 2020 · 2 comments

Comments

@rubensworks
Copy link

I seem to be unable to serialize to Turtle when setting the format using serializer.format:Turtle.

java.lang.IllegalStateException: Unexpected activity serializer: Turtle
	at ldbc.snb.datagen.util.LdbcConfiguration.getDynamicActivitySerializer(LdbcConfiguration.java:149)
	at ldbc.snb.datagen.spark.generators.SparkActivitySerializer$$anonfun$apply$1.apply(SparkActivitySerializer.scala:42)
	at ldbc.snb.datagen.spark.generators.SparkActivitySerializer$$anonfun$apply$1.apply(SparkActivitySerializer.scala:33)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:980)
	at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:980)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2101)
	at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2101)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
	at org.apache.spark.scheduler.Task.run(Task.scala:123)
	at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I've skimmed through the issues, and saw this: #161
Does this mean that this generator will not be supporting Turtle any more going forward?
If so, what alternative methods would you suggest to serialize to Turtle?

@szarnyasg
Copy link
Member

szarnyasg commented Oct 27, 2020

Hi @rubensworks,

If your data sets sizes aren't huge, try the stable branch with Hadoop. That still has the Turtle serializer.

We're overhauling the code on the dev branch. Adding and RDF/Turtle serializer is on the list (#227) but has a low priority. So I cannot promise anything.

Gabor

@rubensworks
Copy link
Author

Thanks for the reply!
I'll try to use the stable branch for now then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants