Better annotate interprocess_exception #3

Dr15Jones · 2022-12-15T20:07:55Z

The IBs with the external generator process are periodically failing with the unhelpful error message

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
	26077_0 process: caught exception 
	boost::interprocess_exception::library_error 15-Dec-2022 03:59:39 CET
	  while serialize lumi
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

That message happens when boost::interprocess_exception constructor is passed an error_info that is set to 0 AND the constructor is not passed a const char*. I went and modified all the places our code might reach a function that calls boost::interprocess_exception and forces it to also pass a const char* saying why the exception is being thrown.

Hopefully this will help track down the problems we see ever so often.

cmsbuild · 2022-12-15T20:08:22Z

A new Pull Request was created by @Dr15Jones (Chris Jones) for branch cms/v1.80.0.

@cmsbuild, @smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

Dr15Jones · 2022-12-15T20:08:54Z

@smuzaffar I compiled all CMSSW code that includes boost/interprocess and had the headers redirected to this changed copy of boost.

fwyzard · 2022-12-15T20:16:32Z

boost/interprocess/detail/managed_open_or_create_impl.hpp

@@ -310,7 +310,7 @@ class managed_open_or_create_impl
               //File existing when trying to create, but non-existing when
               //trying to open, and tried MaxCreateOrOpenTries times. Something fishy
               //is happening here and we can't solve it
-               throw interprocess_exception(error_info(corrupted_error));
+              throw interprocess_exception(error_info(corrupted_error), "file exists when create but not when open");


Suggested change

throw interprocess_exception(error_info(corrupted_error), "file exists when create but not when open");

throw interprocess_exception(error_info(corrupted_error), "do_create_else_open: file exists when trying to create, but not when trying to open");

For consistency with the other cases where you mention the function name explicitly.

fwyzard · 2022-12-15T20:18:01Z

boost/interprocess/managed_shared_memory.hpp

@@ -1,4 +1,4 @@
-//////////////////////////////////////////////////////////////////////////////
+/////////////////////////////////////////////////////////////////////////////


I guess this could be avoided ?

fwyzard · 2022-12-15T20:18:55Z

boost/interprocess/sync/posix/semaphore_wrapper.hpp

@@ -100,13 +100,13 @@ inline bool semaphore_open
      default:
      {
         error_info err(other_error);
-         throw interprocess_exception(err);
+         throw interprocess_exception(err, "semaphore_open unknown type");


Suggested change

throw interprocess_exception(err, "semaphore_open unknown type");

throw interprocess_exception(err, "semaphore_open: unknown type");

fwyzard · 2022-12-15T20:19:16Z

boost/interprocess/sync/posix/semaphore_wrapper.hpp

      }
   }

   //Check for error
   if(handle == BOOST_INTERPROCESS_POSIX_SEM_FAILED){
-      throw interprocess_exception(error_info(errno));
+     throw interprocess_exception(error_info(errno),"semaphore_open sem_open failed");


Suggested change

throw interprocess_exception(error_info(errno),"semaphore_open sem_open failed");

throw interprocess_exception(error_info(errno),"semaphore_open: sem_open failed");

fwyzard · 2022-12-15T20:20:06Z

boost/interprocess/sync/posix/semaphore_wrapper.hpp

@@ -148,7 +148,7 @@ inline void semaphore_init(sem_t *handle, unsigned int initialCount)
   //In the future, a successful call might be required to return 0.
   if(ret == -1){
      error_info err = system_error_code();
-      throw interprocess_exception(err);
+      throw interprocess_exception(err, "semaphore_unlink sem_init failed");


Suggested change

throw interprocess_exception(err, "semaphore_unlink sem_init failed");

throw interprocess_exception(err, "semaphore_init: sem_init failed");

cmsbuild · 2022-12-15T20:56:04Z

Pull request #3 was updated.

Avoid case where exception message is just "boost::interprocess_exception::library_error"

cmsbuild · 2022-12-15T21:02:49Z

Pull request #3 was updated.

Dr15Jones · 2022-12-15T21:06:11Z

One thing of note is from looking at the code, it looks like the message boost::interprocess_exception::library_error was never supposed to be issued and instead the error_info was supposed to give something useful. So this change is just trying to figure out where error_info is not working like it was intended and then to use the extra info to figure out our problem.

smuzaffar · 2022-12-15T23:03:19Z

test parameters:

full_cmssw = true

smuzaffar · 2022-12-15T23:03:29Z

please test

cmsbuild · 2022-12-16T09:57:14Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c3a0f0/29648/summary.html
COMMIT: fc6a197
CMSSW: CMSSW_13_0_X_2022-12-15-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-externals/boost/3/29648/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c3a0f0/29648/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c3a0f0/29648/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 157 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3557521
DQMHistoTests: Total failures: 77
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3557422
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2022-12-16T23:28:21Z

+externals

cmsbuild · 2022-12-16T23:28:38Z

This pull request is fully signed and it will be integrated in one of the next cms/v1.80.0 IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild added externals-pending orp-pending pending-signatures tests-pending labels Dec 15, 2022

fwyzard reviewed Dec 15, 2022

View reviewed changes

Dr15Jones force-pushed the annotate_interprocess_exception branch from 0dcaab1 to 777347c Compare December 15, 2022 20:55

Better annotate interprocess_exception

fc6a197

Avoid case where exception message is just "boost::interprocess_exception::library_error"

Dr15Jones force-pushed the annotate_interprocess_exception branch from 777347c to fc6a197 Compare December 15, 2022 21:02

cmsbuild added tests-started and removed tests-pending labels Dec 15, 2022

cmsbuild added tests-approved and removed tests-started labels Dec 16, 2022

smuzaffar merged commit d4f566f into cms-externals:cms/v1.80.0 Dec 16, 2022

cmsbuild added externals-approved fully-signed and removed pending-signatures externals-pending labels Dec 16, 2022

smuzaffar mentioned this pull request Dec 16, 2022

[boost] Better annotate interprocess_exception cms-sw/cmsdist#8234

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Better annotate interprocess_exception #3

Better annotate interprocess_exception #3

Dr15Jones commented Dec 15, 2022

cmsbuild commented Dec 15, 2022

Dr15Jones commented Dec 15, 2022

fwyzard Dec 15, 2022

fwyzard Dec 15, 2022

Dr15Jones Dec 15, 2022

fwyzard Dec 15, 2022

Dr15Jones Dec 15, 2022

fwyzard Dec 15, 2022

Dr15Jones Dec 15, 2022

fwyzard Dec 15, 2022

Dr15Jones Dec 15, 2022

fwyzard Dec 15, 2022

Dr15Jones Dec 15, 2022

cmsbuild commented Dec 15, 2022

cmsbuild commented Dec 15, 2022

Dr15Jones commented Dec 15, 2022

smuzaffar commented Dec 15, 2022

smuzaffar commented Dec 15, 2022

cmsbuild commented Dec 16, 2022

smuzaffar commented Dec 16, 2022

cmsbuild commented Dec 16, 2022

	throw interprocess_exception(error_info(corrupted_error), "file exists when create but not when open");
	throw interprocess_exception(error_info(corrupted_error), "do_create_else_open: file exists when trying to create, but not when trying to open");

		@@ -1,4 +1,4 @@
		//////////////////////////////////////////////////////////////////////////////
		/////////////////////////////////////////////////////////////////////////////

	throw interprocess_exception(err, "semaphore_open unknown type");
	throw interprocess_exception(err, "semaphore_open: unknown type");

Better annotate interprocess_exception #3

Better annotate interprocess_exception #3

Conversation

Dr15Jones commented Dec 15, 2022

cmsbuild commented Dec 15, 2022

Dr15Jones commented Dec 15, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmsbuild commented Dec 15, 2022

cmsbuild commented Dec 15, 2022

Dr15Jones commented Dec 15, 2022

smuzaffar commented Dec 15, 2022

smuzaffar commented Dec 15, 2022

cmsbuild commented Dec 16, 2022

Comparison Summary

smuzaffar commented Dec 16, 2022

cmsbuild commented Dec 16, 2022