Add option to control number of nodes & partitions

[dszakallas]

The goal of this PR is to facilitate dataset generation on smaller Spark clusters on EMR.

Our experiments show that a machine with 128GB of memory is capable of generating SF3K reliably with 3 blocks per partition given ample disk size to allow for spills (tested with 3.8TB); while less partitions (subsequently, larger block/partition ratio) would introduce OOM errors for this configuration.
This PR:

• sets spark.default.parallelism to the desired partition number (i.e the same value as --num-threads). This makes sure that after wide operations (e.g. the sorting), we don't fall back to a risky, lower value. This is the same behaviour as run.py.
• renames the sf_ratio parameter to sf_per_executor with the same semantics (scale factor for each executor node). The default value is changed to 3000, i.e. 3000 SF for each executor.
• introduces the executors parameter mutually exclusive with the former, to allow for absolutely setting the number of executor nodes.
• analogously to the former two, introduces the sf_per_partition and partitions parameters to set the desired partition number. The default is sf_per_partitions = 100

The following combinations are tested and work. Partition numbers are given as absolute values and they were calculated for each scale factor using the formula ceil(persons_in_sf / block_size / 3), to make sure each partition contains no more than 3 blocks.

./tools/emr/submit_datagen_job.py sf3k_bi 3000 parquet bi --sf-per-executor 3000 --partitions 330 --jar $JAR_NAME --instance-type i3.4xlarge --bucket $BUCKET_NAME  -- --explode-edges --explode-attrs

~11 hours

./tools/emr/submit_datagen_job.py sf10k_bi 10000 parquet bi --sf-per-executor 3000 --partitions 1000 --jar $JAR_NAME --instance-type i3.4xlarge --bucket $BUCKET_NAME  -- --explode-edges --explode-attrs

~ 11 hours

[szarnyasg]

AWS_DEFAULT_PROFILE=ldbc_datagen ./tools/emr/submit_datagen_job.py sf30k-m10-p2567 30000 parquet bi --sf-per-executor 3000 --partitions 2567 --jar ldbc_snb_datagen_2.12_spark3.2-0.4.0+207-6f234d14-jar-with-dependencies.jar --instance-type i3.4xlarge --bucket ldbc-snb-datagen-bi-2021-07 --ec2-key datagen-fedora-box -- --explode-edges --explode-attrs

crashed after 9.5 hours with

        Suppressed: java.io.IOException: Could not get block locations. Source file "/ldbc_snb_datagen/build/graphs/parquet/bi/singular-projected-fk/initial_snapshot/dynamic/Comment
_hasTag_Tag/_temporary/0/_temporary/attempt_202207201819127622827660770100748_0106_m_001240_404839/part-01240-de8ba2d6-41d7-41fd-980c-e51947058914-c000.snappy.parquet" - Aborting...
block==null