Skip to content

The Inference Python Benchmarks Dataflow job is flaky #1788

The Inference Python Benchmarks Dataflow job is flaky

The Inference Python Benchmarks Dataflow job is flaky #1788

GitHub Actions / Test Results succeeded May 18, 2024 in 0s

All 85 tests pass, 51 skipped in 1h 37m 16s

136 tests   85 ✅  1h 37m 16s ⏱️
  1 suites  51 💤
  1 files     0 ❌

Results for commit 71f2278.

Annotations

Check notice on line 0 in .github

See this annotation in the file changed.

@github-actions github-actions / Test Results

51 skipped tests found

There are 51 skipped tests, see "Raw output" for the full list of skipped tests.
Raw output
apache_beam.examples.inference.tfx_bsl.tfx_bsl_inference_it_test.TFXRunInferenceTests ‑ test_tfx_run_inference_mobilenetv2
apache_beam.examples.ml_transform.ml_transform_it_test
apache_beam.examples.snippets.transforms.elementwise.mltransform_test
apache_beam.examples.snippets.transforms.elementwise.runinference_test
apache_beam.examples.streaming_wordcount_debugging_it_test.StreamingWordcountDebuggingIT ‑ test_streaming_wordcount_debugging_it
apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT ‑ test_bqfl_streaming
apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT ‑ test_bqfl_streaming_with_copy_jobs
apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT ‑ test_bqfl_streaming_with_dynamic_destinations
apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests ‑ test_big_query_write_schema_autodetect
apache_beam.ml.gcp.naturallanguageml_test_it.NaturalLanguageMlTestIT ‑ test_analyzing_syntax
apache_beam.ml.gcp.videointelligenceml_test_it.VideoIntelligenceMlTestIT ‑ test_label_detection_with_video_context
apache_beam.ml.inference.base_test.RunInferenceBaseTest ‑ test_run_inference_with_side_inputin_streaming
apache_beam.ml.inference.huggingface_inference_it_test
apache_beam.ml.inference.huggingface_inference_test
apache_beam.ml.inference.onnx_inference_it_test.OnnxInference ‑ test_onnx_run_inference_roberta_sentiment_classification
apache_beam.ml.inference.onnx_inference_test
apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference ‑ test_torch_run_inference_bert_for_masked_lm
apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference ‑ test_torch_run_inference_bert_for_masked_lm_large_model
apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference ‑ test_torch_run_inference_coco_maskrcnn_resnet50_fpn
apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference ‑ test_torch_run_inference_coco_maskrcnn_resnet50_fpn_v1_and_v2
apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference ‑ test_torch_run_inference_imagenet_mobilenetv2
apache_beam.ml.inference.pytorch_inference_test
apache_beam.ml.inference.tensorflow_inference_it_test.TensorflowInference ‑ test_tf_imagenet_image_segmentation
apache_beam.ml.inference.tensorflow_inference_it_test.TensorflowInference ‑ test_tf_mnist_classification
apache_beam.ml.inference.tensorflow_inference_it_test.TensorflowInference ‑ test_tf_mnist_classification_large_model
apache_beam.ml.inference.tensorflow_inference_it_test.TensorflowInference ‑ test_tf_mnist_with_weights_classification
apache_beam.ml.inference.tensorrt_inference_test
apache_beam.ml.inference.vertex_ai_inference_it_test
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_datatable_multi_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_datatable_single_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_numpy_multi_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_numpy_single_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_numpy_single_batch_large_model
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_pandas_multi_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_pandas_single_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_scipy_multi_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_scipy_single_batch
apache_beam.ml.inference.xgboost_inference_test
apache_beam.ml.transforms.handlers_test
apache_beam.ml.transforms.tft_test
apache_beam.runners.dask.dask_runner_test
apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest ‑ test_metrics_it
apache_beam.testing.analyzers.perf_analysis_test
apache_beam.testing.benchmarks.cloudml.cloudml_benchmark_test
apache_beam.testing.test_stream_it_test.TestStreamIntegrationTests ‑ test_basic_execution
apache_beam.testing.test_stream_it_test.TestStreamIntegrationTests ‑ test_multiple_outputs
apache_beam.testing.test_stream_it_test.TestStreamIntegrationTests ‑ test_multiple_outputs_with_watermark_advancement
apache_beam.transforms.enrichment_handlers.feast_feature_store_it_test
apache_beam.transforms.enrichment_handlers.feast_feature_store_test
apache_beam.transforms.periodicsequence_it_test.PeriodicSequenceIT ‑ test_periodicsequence_outputs_valid_watermarks_it
apache_beam.typehints.pytorch_type_compatibility_test

Check notice on line 0 in .github

See this annotation in the file changed.

@github-actions github-actions / Test Results

136 tests found

There are 136 tests, see "Raw output" for the full list of tests.
Raw output
apache_beam.dataframe.io_it_test.ReadUsingReadGbqTests ‑ test_ReadGbq
apache_beam.dataframe.io_it_test.ReadUsingReadGbqTests ‑ test_ReadGbq_direct_read
apache_beam.dataframe.io_it_test.ReadUsingReadGbqTests ‑ test_ReadGbq_direct_read_with_project
apache_beam.dataframe.io_it_test.ReadUsingReadGbqTests ‑ test_ReadGbq_export_with_project
apache_beam.dataframe.io_it_test.ReadUsingReadGbqTests ‑ test_ReadGbq_with_computation
apache_beam.examples.complete.autocomplete_test.AutocompleteTest ‑ test_autocomplete_it
apache_beam.examples.complete.game.game_stats_it_test.GameStatsIT ‑ test_game_stats_it
apache_beam.examples.complete.game.hourly_team_score_it_test.HourlyTeamScoreIT ‑ test_hourly_team_score_it
apache_beam.examples.complete.game.leader_board_it_test.LeaderBoardIT ‑ test_leader_board_it
apache_beam.examples.complete.game.user_score_it_test.UserScoreIT ‑ test_user_score_it
apache_beam.examples.complete.juliaset.juliaset.juliaset_test_it.JuliaSetTestIT ‑ test_run_example_with_setup_file
apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT ‑ test_bigquery_tornadoes_it
apache_beam.examples.cookbook.bigtableio_it_test.BigtableIOWriteTest ‑ test_bigtable_write
apache_beam.examples.cookbook.datastore_wordcount_it_test.DatastoreWordCountIT ‑ test_datastore_wordcount_it
apache_beam.examples.dataframe.flight_delays_it_test.FlightDelaysTest ‑ test_flight_delays
apache_beam.examples.dataframe.taxiride_it_test.TaxirideIT ‑ test_aggregation
apache_beam.examples.dataframe.taxiride_it_test.TaxirideIT ‑ test_enrich
apache_beam.examples.fastavro_it_test.FastavroIT ‑ test_avro_it
apache_beam.examples.inference.tfx_bsl.tfx_bsl_inference_it_test.TFXRunInferenceTests ‑ test_tfx_run_inference_mobilenetv2
apache_beam.examples.ml_transform.ml_transform_it_test
apache_beam.examples.snippets.transforms.elementwise.mltransform_test
apache_beam.examples.snippets.transforms.elementwise.runinference_test
apache_beam.examples.streaming_wordcount_debugging_it_test.StreamingWordcountDebuggingIT ‑ test_streaming_wordcount_debugging_it
apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT ‑ test_streaming_wordcount_it
apache_beam.examples.wordcount_it_test.WordCountIT ‑ test_wordcount_impersonation_it
apache_beam.examples.wordcount_it_test.WordCountIT ‑ test_wordcount_it
apache_beam.io.fileio_test.MatchIntegrationTest ‑ test_transform_on_gcs
apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT ‑ test_big_query_legacy_sql
apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT ‑ test_big_query_new_types
apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT ‑ test_big_query_new_types_avro
apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT ‑ test_big_query_standard_sql
apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT ‑ test_batch_copy_jobs_with_no_input_schema
apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT ‑ test_bqfl_streaming
apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT ‑ test_bqfl_streaming_with_copy_jobs
apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT ‑ test_bqfl_streaming_with_dynamic_destinations
apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT ‑ test_multiple_destinations_transform
apache_beam.io.gcp.bigquery_file_loads_test.BigQueryFileLoadsIT ‑ test_one_job_fails_all_jobs_fail
apache_beam.io.gcp.bigquery_io_read_it_test.BigqueryIOReadIT ‑ test_bigquery_read_1M_python
apache_beam.io.gcp.bigquery_io_read_it_test.BigqueryIOReadIT ‑ test_bigquery_read_custom_1M_python
apache_beam.io.gcp.bigquery_json_it_test.BigQueryJsonIT ‑ test_direct_read
apache_beam.io.gcp.bigquery_json_it_test.BigQueryJsonIT ‑ test_export_read
apache_beam.io.gcp.bigquery_json_it_test.BigQueryJsonIT ‑ test_file_loads_write
apache_beam.io.gcp.bigquery_json_it_test.BigQueryJsonIT ‑ test_query_read
apache_beam.io.gcp.bigquery_json_it_test.BigQueryJsonIT ‑ test_streaming_inserts
apache_beam.io.gcp.bigquery_read_it_test.ReadAllBQTests ‑ test_read_queries
apache_beam.io.gcp.bigquery_read_it_test.ReadInteractiveRunnerTests ‑ test_read_in_interactive_runner
apache_beam.io.gcp.bigquery_read_it_test.ReadNewTypesTests ‑ test_iobase_source
apache_beam.io.gcp.bigquery_read_it_test.ReadNewTypesTests ‑ test_native_source
apache_beam.io.gcp.bigquery_read_it_test.ReadTests ‑ test_iobase_source
apache_beam.io.gcp.bigquery_read_it_test.ReadTests ‑ test_native_source
apache_beam.io.gcp.bigquery_read_it_test.ReadTests ‑ test_table_schema_retrieve
apache_beam.io.gcp.bigquery_read_it_test.ReadTests ‑ test_table_schema_retrieve_specifying_only_table
apache_beam.io.gcp.bigquery_read_it_test.ReadTests ‑ test_table_schema_retrieve_with_direct_read
apache_beam.io.gcp.bigquery_read_it_test.ReadUsingStorageApiTests ‑ test_iobase_source
apache_beam.io.gcp.bigquery_read_it_test.ReadUsingStorageApiTests ‑ test_iobase_source_with_column_selection
apache_beam.io.gcp.bigquery_read_it_test.ReadUsingStorageApiTests ‑ test_iobase_source_with_column_selection_and_row_restriction
apache_beam.io.gcp.bigquery_read_it_test.ReadUsingStorageApiTests ‑ test_iobase_source_with_column_selection_and_row_restriction_rows
apache_beam.io.gcp.bigquery_read_it_test.ReadUsingStorageApiTests ‑ test_iobase_source_with_native_datetime
apache_beam.io.gcp.bigquery_read_it_test.ReadUsingStorageApiTests ‑ test_iobase_source_with_query
apache_beam.io.gcp.bigquery_read_it_test.ReadUsingStorageApiTests ‑ test_iobase_source_with_query_and_filters
apache_beam.io.gcp.bigquery_read_it_test.ReadUsingStorageApiTests ‑ test_iobase_source_with_row_restriction
apache_beam.io.gcp.bigquery_read_it_test.ReadUsingStorageApiTests ‑ test_iobase_source_with_very_selective_filters
apache_beam.io.gcp.bigquery_test.BigQueryFileLoadsIntegrationTests ‑ test_avro_file_load
apache_beam.io.gcp.bigquery_test.BigQueryStreamingInsertTransformIntegrationTests ‑ test_multiple_destinations_transform
apache_beam.io.gcp.bigquery_test.BigQueryStreamingInsertTransformIntegrationTests ‑ test_value_provider_transform
apache_beam.io.gcp.bigquery_test.PubSubBigQueryIT ‑ test_file_loads
apache_beam.io.gcp.bigquery_test.PubSubBigQueryIT ‑ test_streaming_inserts
apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests ‑ test_big_query_write
apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests ‑ test_big_query_write_insert_errors_reporting
apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests ‑ test_big_query_write_insert_non_transient_api_call_error
apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests ‑ test_big_query_write_new_types
apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests ‑ test_big_query_write_schema_autodetect
apache_beam.io.gcp.bigquery_write_it_test.BigQueryWriteIntegrationTests ‑ test_big_query_write_without_schema
apache_beam.io.gcp.datastore.v1new.datastore_write_it_test.DatastoreWriteIT ‑ test_datastore_write_limit
apache_beam.io.gcp.gcsfilesystem_integration_test.GcsFileSystemIntegrationTest ‑ test_copy
apache_beam.io.gcp.gcsfilesystem_integration_test.GcsFileSystemIntegrationTest ‑ test_rename
apache_beam.io.gcp.gcsfilesystem_integration_test.GcsFileSystemIntegrationTest ‑ test_rename_error
apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest ‑ test_batch_copy_and_delete
apache_beam.io.gcp.gcsio_integration_test.GcsIOIntegrationTest ‑ test_copy
apache_beam.io.gcp.healthcare.dicomio_integration_test.DICOMIoIntegrationTest ‑ test_dicom_search_instances
apache_beam.io.gcp.healthcare.dicomio_integration_test.DICOMIoIntegrationTest ‑ test_dicom_store_instance_from_gcs
apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest ‑ test_streaming_data_only
apache_beam.io.gcp.pubsub_integration_test.PubSubIntegrationTest ‑ test_streaming_with_attributes
apache_beam.io.parquetio_it_test.TestParquetIT ‑ test_parquetio_it
apache_beam.ml.gcp.cloud_dlp_it_test.CloudDLPIT ‑ test_deidentification
apache_beam.ml.gcp.cloud_dlp_it_test.CloudDLPIT ‑ test_inspection
apache_beam.ml.gcp.naturallanguageml_test_it.NaturalLanguageMlTestIT ‑ test_analyzing_syntax
apache_beam.ml.gcp.recommendations_ai_test_it.RecommendationAIIT ‑ test_create_catalog_item
apache_beam.ml.gcp.recommendations_ai_test_it.RecommendationAIIT ‑ test_create_user_event
apache_beam.ml.gcp.recommendations_ai_test_it.RecommendationAIIT ‑ test_predict
apache_beam.ml.gcp.videointelligenceml_test_it.VideoIntelligenceMlTestIT ‑ test_label_detection_with_video_context
apache_beam.ml.gcp.visionml_test_it.VisionMlTestIT ‑ test_text_detection_with_language_hint
apache_beam.ml.inference.base_test.RunInferenceBaseTest ‑ test_run_inference_with_side_inputin_streaming
apache_beam.ml.inference.huggingface_inference_it_test
apache_beam.ml.inference.huggingface_inference_test
apache_beam.ml.inference.onnx_inference_it_test.OnnxInference ‑ test_onnx_run_inference_roberta_sentiment_classification
apache_beam.ml.inference.onnx_inference_test
apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference ‑ test_torch_run_inference_bert_for_masked_lm
apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference ‑ test_torch_run_inference_bert_for_masked_lm_large_model
apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference ‑ test_torch_run_inference_coco_maskrcnn_resnet50_fpn
apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference ‑ test_torch_run_inference_coco_maskrcnn_resnet50_fpn_v1_and_v2
apache_beam.ml.inference.pytorch_inference_it_test.PyTorchInference ‑ test_torch_run_inference_imagenet_mobilenetv2
apache_beam.ml.inference.pytorch_inference_test
apache_beam.ml.inference.sklearn_inference_it_test.SklearnInference ‑ test_sklearn_mnist_classification
apache_beam.ml.inference.sklearn_inference_it_test.SklearnInference ‑ test_sklearn_mnist_classification_large_model
apache_beam.ml.inference.sklearn_inference_it_test.SklearnInference ‑ test_sklearn_regression
apache_beam.ml.inference.tensorflow_inference_it_test.TensorflowInference ‑ test_tf_imagenet_image_segmentation
apache_beam.ml.inference.tensorflow_inference_it_test.TensorflowInference ‑ test_tf_mnist_classification
apache_beam.ml.inference.tensorflow_inference_it_test.TensorflowInference ‑ test_tf_mnist_classification_large_model
apache_beam.ml.inference.tensorflow_inference_it_test.TensorflowInference ‑ test_tf_mnist_with_weights_classification
apache_beam.ml.inference.tensorrt_inference_test
apache_beam.ml.inference.vertex_ai_inference_it_test
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_datatable_multi_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_datatable_single_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_numpy_multi_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_numpy_single_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_numpy_single_batch_large_model
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_pandas_multi_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_pandas_single_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_scipy_multi_batch
apache_beam.ml.inference.xgboost_inference_it_test.XGBoostInference ‑ test_iris_classification_scipy_single_batch
apache_beam.ml.inference.xgboost_inference_test
apache_beam.ml.transforms.handlers_test
apache_beam.ml.transforms.tft_test
apache_beam.runners.dask.dask_runner_test
apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest ‑ test_metrics_it
apache_beam.testing.analyzers.perf_analysis_test
apache_beam.testing.benchmarks.cloudml.cloudml_benchmark_test
apache_beam.testing.test_stream_it_test.TestStreamIntegrationTests ‑ test_basic_execution
apache_beam.testing.test_stream_it_test.TestStreamIntegrationTests ‑ test_multiple_outputs
apache_beam.testing.test_stream_it_test.TestStreamIntegrationTests ‑ test_multiple_outputs_with_watermark_advancement
apache_beam.transforms.enrichment_handlers.feast_feature_store_it_test
apache_beam.transforms.enrichment_handlers.feast_feature_store_test
apache_beam.transforms.external_it_test.ExternalTransformIT ‑ test_job_python_from_python_it
apache_beam.transforms.periodicsequence_it_test.PeriodicSequenceIT ‑ test_periodicsequence_outputs_valid_watermarks_it
apache_beam.typehints.pytorch_type_compatibility_test