Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mysql (and possible other analysers) Errors #93

Open
VincenzoFerme opened this issue Aug 21, 2016 · 0 comments
Open

Mysql (and possible other analysers) Errors #93

VincenzoFerme opened this issue Aug 21, 2016 · 0 comments
Assignees
@VincenzoFerme
Labels
enhancement
Milestone
Next Tasks

Comments

@VincenzoFerme
Copy link
Member

An execution log reporting the error as in the title. Moreover in case of an error, the computation must not be marked as succeeded as it seems to be marked from the following logs.


8/21/2016 6:11:22 PMAnalyser work request queued for script that meets requirements execution_time, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMReceived work request for script execution_time, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMReceived work request for script process_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMDispatching work request for script process_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/trials/processDuration.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:11:22 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:11:22 PMAnalyser work request queued for script that meets requirements process_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMDispatching work request for script execution_time, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMAnalyser work request queued for script that meets requirements number_of_process_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMReceived work request for script number_of_process_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMReceived work request for script construct_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/trials/executionTime.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:11:22 PMDispatching work request for script construct_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:11:22 PMDispatching work request for script number_of_process_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/trials/constructDuration.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:11:22 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:11:22 PMAnalyser work request queued for script that meets requirements construct_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/trials/numberOfProcessInstances.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:11:22 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:11:22 PMAnalyser work request queued for script that meets requirements number_of_construct_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMReceived work request for script number_of_construct_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMReceived work request for script database_size, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:22 PMAnalyser work request queued for script that meets requirements database_size, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:44 PM
[Stage 1:======>                                                (27 + 51) / 240]
[Stage 1:=============================>                        (132 + 50) / 240]
[Stage 1:==========================================>           (191 + 49) / 240]


[Stage 3:===================>                                   (86 + 48) / 240]
[Stage 3:===============================>                      (142 + 51) / 240]
[Stage 3:================================================>     (217 + 23) / 240]
[Stage 4:=====================>                                 (94 + 48) / 240]
[Stage 4:=============================>                        (130 + 54) / 240]
[Stage 4:===============================================>      (209 + 31) / 240]


[Stage 5:======================================>               (171 + 48) / 240]


8/21/2016 6:11:44 PMScript analysers/trials/numberOfProcessInstances.py processed
8/21/2016 6:11:44 PMAnalyser script completed successfully: number_of_process_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:44 PMDispatching work request for script database_size, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:44 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/trials/databaseSize.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:11:44 PMAll trials analysers/trials/numberOfProcessInstances.py for experiment BenchFlow.wfmsTest.1.1 completed/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:11:44 PMAnalyser work request queued for experiment script number_of_process_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:11:44 PMReceived work request for script number_of_process_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:11:45 PM
[Stage 7:=======================================>               (72 + 28) / 100]


[Stage 9:============================>                          (59 + 54) / 115]
[Stage 9:======================================================>(114 + 1) / 115]


[Stage 11:===========================>                         (126 + 49) / 240]
[Stage 11:================================>                    (149 + 73) / 240]
[Stage 12:>                                                       (0 + 0) / 240]
[Stage 12:======================>                              (100 + 53) / 240]
[Stage 12:============================>                        (130 + 50) / 240]
[Stage 12:==========================================>          (193 + 47) / 240]


8/21/2016 6:11:45 PMScript analysers/trials/executionTime.py processed
8/21/2016 6:11:45 PMAnalyser script completed successfully: execution_time, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:45 PMAll trials analysers/trials/executionTime.py for experiment BenchFlow.wfmsTest.1.1 completedDispatching work request for script number_of_construct_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:45 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/trials/numberOfConstructInstances.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:11:45 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:11:45 PMAnalyser work request queued for experiment script execution_time, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:11:45 PMAll requirements met for: number_of_process_instances,execution_time
8/21/2016 6:11:45 PMReceived work request for script execution_time, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:11:45 PMReceived work request for script throughput, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:45 PMAnalyser work request queued for script that meets requirements throughput, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:48 PM
[Stage 7:>                                                       (1 + 48) / 100]
[Stage 7:========>                                              (15 + 48) / 100]
[Stage 7:==========>                                            (19 + 49) / 100]
[Stage 7:================>                                      (30 + 48) / 100]
[Stage 7:====================>                                  (37 + 48) / 100]
[Stage 7:=========================>                             (46 + 48) / 100]
[Stage 7:==============================================>        (85 + 15) / 100]


[Stage 9:======================>                                (48 + 48) / 115]
[Stage 9:============================================>          (93 + 22) / 115]


[Stage 11:=======================================>             (181 + 49) / 240]


[Stage 15:============>                                         (57 + 48) / 240]
[Stage 15:===================================>                 (159 + 49) / 240]


8/21/2016 6:11:48 PMScript analysers/trials/processDuration.py processed
8/21/2016 6:11:48 PMAnalyser script completed successfully: process_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:48 PMDispatching work request for script number_of_process_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:11:48 PMAll trials analysers/trials/processDuration.py for experiment BenchFlow.wfmsTest.1.1 completedAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/experiments/numberOfProcessInstances.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:11:48 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:11:48 PMAnalyser work request queued for experiment script process_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:11:48 PMReceived work request for script process_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:11:48 PM
[Stage 7:>                                                       (0 + 48) / 100]
[Stage 7:===>                                                    (6 + 48) / 100]
[Stage 7:=============>                                         (25 + 48) / 100]
[Stage 7:===============>                                       (29 + 48) / 100]
[Stage 7:=============================>                         (54 + 46) / 100]
[Stage 7:==================================>                    (63 + 37) / 100]
[Stage 7:==============================================>        (85 + 15) / 100]


[Stage 9:====>                                                  (10 + 50) / 115]
[Stage 9:=======================>                               (50 + 48) / 115]
[Stage 9:==================================>                    (72 + 43) / 115]


[Stage 15:==================>                                   (80 + 48) / 240]
[Stage 15:==================================>                  (156 + 49) / 240]


8/21/2016 6:11:48 PMScript analysers/trials/constructDuration.py processed
8/21/2016 6:11:48 PMAnalyser script completed successfully: construct_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:48 PMDispatching work request for script throughput, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:11:48 PMAll trials analysers/trials/constructDuration.py for experiment BenchFlow.wfmsTest.1.1 completedAnalyser work request queued for experiment script construct_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:11:48 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/trials/throughput.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:11:48 PMReceived work request for script construct_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:11:48 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:11:57 PMTraceback (most recent call last):
8/21/2016 6:11:57 PM  File "/app/analysers/trials/throughput.py", line 73, in <module>
8/21/2016 6:11:57 PM    if __name__ == '__main__': main()
8/21/2016 6:11:57 PM  File "/app/analysers/trials/throughput.py", line 69, in main
8/21/2016 6:11:57 PM    query = createQuery(sc, cassandraKeyspace, experimentID, trialID, partitionsPerCore)
8/21/2016 6:11:57 PM  File "/app/analysers/trials/throughput.py", line 36, in createQuery
8/21/2016 6:11:57 PM    tp = npr/(ex*1.0)
8/21/2016 6:11:57 PMTypeError: unsupported operand type(s) for *: 'NoneType' and 'float'
8/21/2016 6:12:00 PM
8/21/2016 6:12:00 PMScript analysers/trials/throughput.py processed
8/21/2016 6:12:00 PMAnalyser script completed successfully: throughput, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:12:00 PMDispatching work request for script execution_time, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:00 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/experiments/executionTime.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:12:00 PMAll trials analysers/trials/throughput.py for experiment BenchFlow.wfmsTest.1.1 completed/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:12:00 PMAnalyser work request queued for experiment script throughput, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:00 PMReceived work request for script throughput, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:03 PM
[Stage 7:=====================================================>  (96 + 4) / 100]


8/21/2016 6:12:03 PMScript analysers/trials/databaseSize.py processed
8/21/2016 6:12:03 PMAnalyser script completed successfully: database_size, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:12:03 PMDispatching work request for script process_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:03 PMAll trials analysers/trials/databaseSize.py for experiment BenchFlow.wfmsTest.1.1 completedAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/experiments/processDuration.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:12:03 PMAnalyser work request queued for experiment script database_size, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:03 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:12:03 PMReceived work request for script database_size, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:05 PM
[Stage 1:============>                                          (53 + 48) / 240]
[Stage 1:============================>                         (126 + 49) / 240]
[Stage 1:==================================================>   (224 + 16) / 240]


8/21/2016 6:12:05 PMScript analysers/trials/numberOfConstructInstances.py processed
8/21/2016 6:12:05 PMAnalyser script completed successfully: number_of_construct_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, trial
8/21/2016 6:12:05 PMDispatching work request for script construct_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:05 PMAll trials analysers/trials/numberOfConstructInstances.py for experiment BenchFlow.wfmsTest.1.1 completedAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/experiments/constructDuration.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:12:05 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:12:05 PMAnalyser work request queued for experiment script number_of_construct_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:05 PMReceived work request for script number_of_construct_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:07 PM
[Stage 6:==============>                                         (12 + 36) / 48]


8/21/2016 6:12:07 PMScript analysers/experiments/numberOfProcessInstances.py processed
8/21/2016 6:12:07 PMAnalyser script completed successfully: number_of_process_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:07 PMDispatching work request for script throughput, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:07 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/experiments/throughput.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:12:07 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:12:13 PM/opt/conda/lib/python2.7/site-packages/scipy/stats/stats.py:249: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
8/21/2016 6:12:13 PM  "values. nan values will be ignored.", RuntimeWarning)
8/21/2016 6:12:13 PMTraceback (most recent call last):
8/21/2016 6:12:13 PM  File "/app/analysers/experiments/processDuration.py", line 69, in <module>
8/21/2016 6:12:13 PM    main()
8/21/2016 6:12:13 PM  File "/app/analysers/experiments/processDuration.py", line 64, in main
8/21/2016 6:12:13 PM    query = createQuery(CassandraRDD, experimentID)
8/21/2016 6:12:13 PM  File "/app/analysers/experiments/processDuration.py", line 23, in createQuery
8/21/2016 6:12:13 PM    metrics = computeExperimentMetrics(dataRDD, "process_duration")
8/21/2016 6:12:13 PM  File "/tmp/spark-5a7e1629-c13b-44cd-b6d8-a89be8204fc6/userFiles-dade03cc-0a73-459d-ab93-a3d33b117fff/commons.py", line 103, in computeExperimentMetrics
8/21/2016 6:12:13 PM    coefficientOfVariation = stats.variation(means).item()*100
8/21/2016 6:12:13 PM  File "/opt/conda/lib/python2.7/site-packages/scipy/stats/stats.py", line 975, in variation
8/21/2016 6:12:13 PM    return a.std(axis) / a.mean(axis)
8/21/2016 6:12:13 PM  File "/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py", line 124, in _std
8/21/2016 6:12:13 PM    keepdims=keepdims)
8/21/2016 6:12:13 PM  File "/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py", line 94, in _var
8/21/2016 6:12:13 PM    arrmean, rcount, out=arrmean, casting='unsafe', subok=False)
8/21/2016 6:12:13 PMTypeError: unsupported operand type(s) for /: 'NoneType' and 'int'
8/21/2016 6:12:15 PM/opt/conda/lib/python2.7/site-packages/scipy/stats/stats.py:249: RuntimeWarning: The input array could not be properly checked for nan values. nan values will be ignored.
8/21/2016 6:12:15 PM  "values. nan values will be ignored.", RuntimeWarning)
8/21/2016 6:12:15 PMTraceback (most recent call last):
8/21/2016 6:12:15 PM  File "/app/analysers/experiments/constructDuration.py", line 77, in <module>
8/21/2016 6:12:15 PM    main()
8/21/2016 6:12:15 PM  File "/app/analysers/experiments/constructDuration.py", line 72, in main
8/21/2016 6:12:15 PM    query = createQuery(CassandraRDD, experimentID)
8/21/2016 6:12:15 PM  File "/app/analysers/experiments/constructDuration.py", line 25, in createQuery
8/21/2016 6:12:15 PM    metrics = computeExperimentMetrics(dataRDD, "construct_duration")
8/21/2016 6:12:15 PM  File "/tmp/spark-b704a4d6-6dd5-4a21-bfd5-08c009ae50b1/userFiles-f7ad064d-30bd-4956-87f5-97e35ddcb116/commons.py", line 103, in computeExperimentMetrics
8/21/2016 6:12:15 PM    coefficientOfVariation = stats.variation(means).item()*100
8/21/2016 6:12:15 PM  File "/opt/conda/lib/python2.7/site-packages/scipy/stats/stats.py", line 975, in variation
8/21/2016 6:12:15 PM    return a.std(axis) / a.mean(axis)
8/21/2016 6:12:15 PM  File "/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py", line 124, in _std
8/21/2016 6:12:15 PM    keepdims=keepdims)
8/21/2016 6:12:15 PM  File "/opt/conda/lib/python2.7/site-packages/numpy/core/_methods.py", line 94, in _var
8/21/2016 6:12:15 PM    arrmean, rcount, out=arrmean, casting='unsafe', subok=False)
8/21/2016 6:12:15 PMTypeError: unsupported operand type(s) for /: 'NoneType' and 'int'
8/21/2016 6:12:16 PM
8/21/2016 6:12:16 PMScript analysers/experiments/processDuration.py processed
8/21/2016 6:12:16 PMAnalyser script completed successfully: process_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:16 PMDispatching work request for script database_size, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:16 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/experiments/databaseSize.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:12:16 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:12:18 PM
8/21/2016 6:12:18 PMScript analysers/experiments/constructDuration.py processed
8/21/2016 6:12:18 PMAnalyser script completed successfully: construct_duration, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:18 PMDispatching work request for script number_of_construct_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:18 PMAnalyser worker submitting script:  [--master local[*] --jars /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --driver-memory 12g --executor-memory 12g --conf spark.executor.heartbeatInterval=300000s --conf spark.storage.blockManagerSlaveTimeoutMs=300000s --conf spark.core.connection.ack.wait.timeout=300000s --conf spark.cassandra.connection.host=cassandra --driver-class-path /usr/spark/pyspark-cassandra-assembly-0.3.5.jar --py-files /app/analysers/commons/commons.py,/usr/spark/pyspark-cassandra-assembly-0.3.5.jar /app/analysers/experiments/numberOfConstructInstances.py {"cassandra_keyspace":"benchflow","trial_id":"BenchFlow.wfmsTest.1.1.1","experiment_id":"BenchFlow.wfmsTest.1.1","config_file":"analysers.configuration.yml","container_id":"mysql","container_name":"mysql","host_id":"host"}]
8/21/2016 6:12:18 PM/apps/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin
8/21/2016 6:12:19 PM
[Stage 2:=====>                                                   (5 + 43) / 48]


8/21/2016 6:12:19 PMScript analysers/experiments/executionTime.py processed
8/21/2016 6:12:19 PMAnalyser script completed successfully: execution_time, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:25 PM
[Stage 2:===============>                                        (13 + 35) / 48]


8/21/2016 6:12:25 PMScript analysers/experiments/throughput.py processed
8/21/2016 6:12:25 PMAnalyser script completed successfully: throughput, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:34 PM
8/21/2016 6:12:34 PMScript analysers/experiments/databaseSize.py processed
8/21/2016 6:12:34 PMAnalyser script completed successfully: database_size, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment
8/21/2016 6:12:37 PM
[Stage 6:====>                                                    (4 + 44) / 48]


8/21/2016 6:12:37 PMScript analysers/experiments/numberOfConstructInstances.py processed
8/21/2016 6:12:37 PMAnalyser script completed successfully: number_of_construct_instances, camunda, 7.5.0, BenchFlow.wfmsTest.1.1.1, mysql, host, experiment

Also refer to benchflow/data-transformers#86. The execution log belong to the same pipeline.

Please test also all the other analysers again in the current state of the system. And add test cases for all the encountered problems.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@VincenzoFerme VincenzoFerme

Labels
enhancement
Milestone
Next Tasks
Development

No branches or pull requests
2 participants
@VincenzoFerme @Cerfoglg