webserver blocked by unidentified reason

[GitHK]

The two errors refer to the same function

  _     ._   __/__   _ _  _  _ _/_   Recorded: 05:21:10  Samples:  1,
 /_//_/// /_\ / //_// / //_'/ //     Duration: 0.703     CPU time: 0.702,
/   _/                      v3.4.2,
,
Program: /home/scu/.venv/bin/simcore-service-webserver --config server-docker-prod.yaml,
,
0.654 instrumented  servicelib/monitor_slow_callbacks.py:23,
└─ 0.654 instrumented  aiodebug/log_slow_callbacks.py:16,
   └─ 0.654 _run  asyncio/events.py:143,
      └─ 0.654 collect_garbage_periodically  simcore_service_webserver/resource_manager/garbage_collector.py:63,
         └─ 0.654 collect_garbage  simcore_service_webserver/resource_manager/garbage_collector.py:89,
            └─ 0.654 remove_orphaned_services  simcore_service_webserver/resource_manager/garbage_collector.py:317,
               └─ 0.654 is_node_id_present_in_any_project_workbench  simcore_service_webserver/projects/projects_api.py:396,
                  └─ 0.654 get_all_node_ids_from_workbenches  simcore_service_webserver/projects/projects_db.py:788,
                     └─ 0.654 __anext__  aiopg/sa/result.py:343,
                        └─ 0.654 fetchone  aiopg/sa/result.py:377,

   _     ._   __/__   _ _  _  _ _/_   Recorded: 05:24:49  Samples:  1,
  /_//_/// /_\ / //_// / //_'/ //     Duration: 0.664     CPU time: 0.665,
 /   _/                      v3.4.2,
 ,
 Program: /home/scu/.venv/bin/simcore-service-webserver --config server-docker-prod.yaml,
 ,
 0.588 instrumented  servicelib/monitor_slow_callbacks.py:23,
 └─ 0.588 instrumented  aiodebug/log_slow_callbacks.py:16,
    └─ 0.588 _run  asyncio/events.py:143,
       └─ 0.588 collect_garbage_periodically  simcore_service_webserver/resource_manager/garbage_collector.py:63,
          └─ 0.588 collect_garbage  simcore_service_webserver/resource_manager/garbage_collector.py:89,
             └─ 0.588 remove_orphaned_services  simcore_service_webserver/resource_manager/garbage_collector.py:317,
                └─ 0.588 is_node_id_present_in_any_project_workbench  simcore_service_webserver/projects/projects_api.py:396,
                   └─ 0.588 get_all_node_ids_from_workbenches  simcore_service_webserver/projects/projects_db.py:788,
                      └─ 0.588 __iter__  _collections_abc.py:760,

Looking at the function's code I was unable to identify what happened.

osparc-simcore/services/web/server/src/simcore_service_webserver/projects/projects_db.py

Lines 788 to 807 in bb62cf8

	async def get_all_node_ids_from_workbenches(
	self, project_uuid: str = None
	) -> Set[str]:
	"""Returns a set containing all the workbench node_ids from all projects
	If a project_uuid is passed, only that project's workbench nodes will be included
	"""
	if project_uuid is None:
	query = "SELECT json_object_keys(projects.workbench) FROM projects"
	else:
	query = f"SELECT json_object_keys(projects.workbench) FROM projects WHERE projects.uuid = '{project_uuid}'"
	async with self.engine.acquire() as conn:
	result = set()
	query_result = await conn.execute(query)
	async for row in query_result:
	result.update(set(row.values()))
	return result

[pcrespov]

This can help and be related with #2757