sdk: make test_batch_span_processor_scheduled_delay a bit more robust (…

…#3938)

* sdk: make test_batch_span_processor_scheduled_delay a bit more robust

It happened that tests failed because the delay was fired some
microseconds earlier:

>       self.assertGreaterEqual((export_time - start_time) * 1e3, 500)
E       AssertionError: 499.9737739562988 not greater than or equal to 500

Use assertAlmostEqual to accept a similar enough value (delta=25) and
avoid too big values.

Skip tests on windows pypy because of random huge spikes:

E AssertionError: 2253.103017807007 != 500 within 25 delta (1744.1030178070068 difference)

Fix #3911

* opentelemetry-sdk: handle timeout exception in last metric collection

The last metric collection after the thread has been notified to
shutdown is not handling the submission to get a MetricsTimeoutError
exception. Handle that to match what we are doing in the usual loop
collection.

See in TestBatchSpanProcessor.test_batch_span_processor_scheduled_delay
failing with:

opentelemetry-sdk/tests/metrics/test_periodic_exporting_metric_reader.py::TestPeriodicExportingMetricReader::test_metric_timeout_does_not_kill_worker_thread
  \_pytest\threadexception.py:73: PytestUnhandledThreadExceptionWarning: Exception in thread OtelPeriodicExportingMetricReader

  Traceback (most recent call last):
    File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\threading.py", line 1045, in _bootstrap_inner
      self.run()
    File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\threading.py", line 982, in run
      self._target(*self._args, **self._kwargs)
    File "D:\a\opentelemetry-python\opentelemetry-python\opentelemetry-sdk\src\opentelemetry\sdk\metrics\_internal\export\__init__.py", line 522, in _ticker
      self.collect(timeout_millis=self._export_interval_millis)
    File "D:\a\opentelemetry-python\opentelemetry-python\opentelemetry-sdk\tests\metrics\test_periodic_exporting_metric_reader.py", line 87, in collect
      raise self._collect_exception
  opentelemetry.sdk.metrics._internal.exceptions.MetricsTimeoutError: test timeout

---------

Co-authored-by: Diego Hurtado <[email protected]>

opentelemetry-sdk/src/opentelemetry/sdk/metrics/_internal/export/__init__.py

@@ -519,7 +519,13 @@ def _ticker(self) -> None:
                     exc_info=True,
                 )
         # one last collection below before shutting down completely
-        self.collect(timeout_millis=self._export_interval_millis)
+        try:
+            self.collect(timeout_millis=self._export_interval_millis)
+        except MetricsTimeoutError:
+            _logger.warning(
+                "Metric collection timed out.",
+                exc_info=True,
+            )
 
     def _receive_metrics(
         self,

opentelemetry-sdk/tests/trace/export/test_export.py

@@ -461,6 +461,10 @@ def _target():
 
         span_processor.shutdown()
 
+    @mark.skipif(
+        python_implementation() == "PyPy" and system() == "Windows",
+        reason="This test randomly fails with huge delta in Windows with PyPy",
+    )
     def test_batch_span_processor_scheduled_delay(self):
         """Test that spans are exported each schedule_delay_millis"""
         spans_names_list = []
@@ -482,7 +486,7 @@ def test_batch_span_processor_scheduled_delay(self):
         self.assertTrue(export_event.wait(2))
         export_time = time.time()
         self.assertEqual(len(spans_names_list), 1)
-        self.assertGreaterEqual((export_time - start_time) * 1e3, 500)
+        self.assertAlmostEqual((export_time - start_time) * 1e3, 500, delta=25)
 
         span_processor.shutdown()