README and error reporting and controlelr resturct

src/tools/interop/idt/README.md

@@ -212,28 +212,18 @@ RPi users, as needed:
 ```
 idt capture -h
 
-usage: idt capture [-h] [--platform {Android}]
-                   [--ecosystem {PlayServices,PlayServicesUser,ALL}]
-                   [--pcap {t,f}]
-                   [--interface {wlp0s20f3,docker0,lo}]
-                   [--additional {t,f}]
+usage: idt capture [-h] [--platform {Android}] [--ecosystem {PlayServicesUser,PlayServices,ALL}] [--pcap {t,f}] [--interface {wlp0s20f3,lo,docker0,any}]
 
 options:
   -h, --help            show this help message and exit
   --platform {Android}, -p {Android}
-                        Run capture for a particular platform
-                        (default Android)
-  --ecosystem {PlayServices,PlayServicesUser,ALL}, -e {PlayServices,PlayServicesUser,ALL}
-                        Run capture for a particular ecosystem or ALL
-                        ecosystems (default ALL)
+                        Run capture for a particular platform (default Android)
+  --ecosystem {PlayServicesUser,PlayServices,ALL}, -e {PlayServicesUser,PlayServices,ALL}
+                        Run capture for a particular ecosystem or ALL ecosystems (default ALL)
   --pcap {t,f}, -c {t,f}
                         Run packet capture (default t)
-  --interface {wlp0s20f3,docker0,lo}, -i {wlp0s20f3,docker0,lo}
-                        Run packet capture against a specified
-                        interface (default wlp0s20f3)
-  --additional {t,f}, -a {t,f}
-                        Run ble and mdns scanners in the background
-                        while capturing (default t)
+  --interface {wlp0s20f3,lo,docker0,any}, -i {wlp0s20f3,lo,docker0,any}
+                        Specify packet capture interface (default any)
 ```
 
 #### Artifacts
@@ -300,13 +290,53 @@ $ idt capture -h
 usage: idt capture [-h] [--platform {Android}] [--ecosystem {DemoExtEcosystem...
 ```
 
+> **IMPORTANT:** Note the following runtime expectations of ecosystems:  
+> `analyze_capture` will be run as a target of `multiprocessing.Process`, \
+meaning the ecosystem object will be copied into a forked process at this time.
+
 The platform loader functions the same as `capture/ecosystem`.
 
 For each package in `capture/platform`, the platform loader expects a module
 name matching the package name.  
 This module must contain a single class which is a subclass of
 `capture.base.PlatformLogStreamer`.
 
-Note the following runtime expectations of platforms:
+## Project overview
+
+- The entry point is in `idt.py` which contains simple CLI parsing with `argparse`.
+- `log` contains logging utilities used by everything in the project.
+
+For capture:
+
+- `base.py` contains the base clases for ecosystems and platforms.
+- `factory.py` contains the ecosystem and platform producer and controller
+- `loader` is a generic class loader that dynamically imports all classes matching a given subclass.
+- `utils/shell` contains a simple helper class for background and foreground Bash commands.
+- `utils/artifact` contains helper functions for managing artifacts.
+
+For discovery:
+
+- `matter_ble` provides a simple ble scanner that shows matter devices being discovered
+  and lost, as well as their VID/PID, RSSI, etc.
+- `matter_dnssd` provides a smple DNS-SD browser that searches for matter devices
+  and thread border routers.
+
+### Conventions
+
+- `config.py` should be used to hold development configs within the directory where they are needed. 
+  - It may also hold configs for flaky/cumbersome features that might need to be disabled in an emergency. 
+  - `config.py` **should not** be used for everyday operation.
+- When needed, execute builds in a folder called `BUILD` within the source tree. 
+  - `idt_clean_all` deletes all `BUILD` dirs and `BUILD` is in `.gitignore`.
+- Although many things are marked as coroutines, almost all real concurrency in the current implementation comes from multiprocessing. 
+  - A general direction should be decided for the project in the next iteration.
+  - Multiprocessing allows for easier implementation where ecosystems are less likely to block each other
+  - Async allows for better shared states and flexibility
+
+## Troubleshooting
 
--   Connect, Start, and Stop may be called multiple times. i.e. start may be called when already streaming.
+- Change log level from `INFO` to `DEBUG` in `config.py` for additional logging.
+- It is expected that this script will be run on a system with `bash`, `tcpdump`, and `adb` already available.
+- Compiling tcpdump for android may require additional dependencies. 
+  - If the build script fails for you, try `idt_go && source idt/scripts/compilers.sh`.
+- You may disable colors and splash by setting `enable_color` in `config.py` to `False`. 

src/tools/interop/idt/capture/base.py

@@ -41,15 +41,13 @@ async def connect(self) -> None:
     async def start_streaming(self) -> None:
         """
         Begin streaming logs
-        Start should be able to be called repeatedly without restarting streaming
         """
         raise NotImplementedError
 
     @abstractmethod
     async def stop_streaming(self) -> None:
         """
         Stop the capture and pull any artifacts from remote devices
-        Stop should not cause an error even if the stream is not running
         """
         raise NotImplementedError
 
@@ -78,21 +76,23 @@ def __init__(
     async def start_capture(self) -> None:
         """
         Start the capture
+        Platform is already started
         """
         raise NotImplementedError
 
     @abstractmethod
     async def stop_capture(self) -> None:
         """
         Stop the capture and pull any artifacts from remote devices
+        Platform is already stopped
         """
         raise NotImplementedError
 
     @abstractmethod
     def analyze_capture(self) -> None:
         """
         Parse the capture and create + display helpful analysis artifacts that are unique to the ecosystem
-        This function will be run as a separate process to allow real time analysis
+        IMPORTANT: This function will be run as a separate process to allow real time analysis!!!
         """
         raise NotImplementedError
 

src/tools/interop/idt/capture/ecosystem/play_services/config.py

@@ -16,3 +16,5 @@
 #
 
 enable_foyer_probers = True
+test_error_init = False
+test_error_execution = False

src/tools/interop/idt/capture/ecosystem/play_services/play_services.py

@@ -60,6 +60,10 @@ def __init__(self, platform: Android, artifact_dir: str) -> None:
                             '168',  # mDNS
                             ]
 
+        if config.test_error_init:
+            self.logger.critical("Throwing exception in init for test!")
+            raise Exception("Test init exception from Play Services!")
+
     def _write_standard_info_file(self) -> None:
         for k, v in self.standard_info_data.items():
             self.logger.info(f"{k}: {v}")
@@ -94,10 +98,11 @@ async def start_capture(self) -> None:
             verbose_command = f"shell setprop log.tag.gms_svc_id:{service_id} VERBOSE"
             self.platform.run_adb_command(verbose_command)
         self._get_standard_info()
-        await self.platform.start_streaming()
+        if config.test_error_execution:
+            self.logger.critical("Throwing exception in execution for test!")
+            raise Exception("Test exe exception from Play Services!")
 
     async def stop_capture(self) -> None:
-        await self.platform.stop_streaming()
         self.analysis.show_analysis()
 
     def analyze_capture(self):

src/tools/interop/idt/capture/ecosystem/play_services_user/play_services_user.py

@@ -42,10 +42,9 @@ def __init__(self, platform: Android, artifact_dir: str) -> None:
         self.platform = platform
 
     async def start_capture(self) -> None:
-        await self.platform.start_streaming()
+        pass
 
     async def stop_capture(self) -> None:
-        await self.platform.stop_streaming()
         self.show_analysis()
 
     @staticmethod

src/tools/interop/idt/capture/factory.py

@@ -26,13 +26,13 @@
 import capture
 from capture.utils.async_control import get_timeout
 from capture.base import EcosystemCapture, PlatformLogStreamer, UnsupportedCapturePlatformException
-from capture.utils.artifact import safe_mkdir
+from capture.utils.artifact import safe_mkdir, create_standard_log_name
 from log import border_print
 import log
 
 _PLATFORM_MAP: typing.Dict[str, PlatformLogStreamer] = {}
 _ECOSYSTEM_MAP: typing.Dict[str, PlatformLogStreamer] = {}
-_ERROR_REPORT: typing.Dict[str, list[str]] = {}
+_ERROR_REPORT: typing.Dict[str, list[(str, str, str)]] = {}
 _ANALYSIS_MAP: typing.Dict[str, Process] = {}
 
 logger = log.get_logger(__file__)
@@ -93,19 +93,23 @@ async def init_ecosystems(platform, ecosystem, artifact_dir):
                 async with asyncio.timeout_at(get_timeout()):
                     await EcosystemFactory.get_ecosystem_impl(
                         ecosystem, platform, artifact_dir)
-            except UnsupportedCapturePlatformException:
+            except UnsupportedCapturePlatformException as e:
                 logger.error(f"Unsupported platform {ecosystem} {platform}")
-            except TimeoutError:
+                track_error(ecosystem, "UNSUPPORTED_PLATFORM", str(e))
+            except TimeoutError as e:
                 logger.error(f"Timeout starting ecosystem {ecosystem} {platform}")
-            except Exception:
-                logger.error("unknown error instantiating ecosystem")
-                logger.error(traceback.format_exc())
+                track_error(ecosystem, "TIMEOUT", str(e))
+            except Exception as e:
+                logger.error(f"Unknown error instantiating ecosystem {ecosystem} {platform}")
+                track_error(ecosystem, "UNEXPECTED", str(e))
 
 
-def track_error(ecosystem: str, error_type: str) -> None:
+def track_error(ecosystem: str, error_type: str, error_message: str) -> None:
     if ecosystem not in _ERROR_REPORT:
         _ERROR_REPORT[ecosystem] = []
-    _ERROR_REPORT[ecosystem].append(error_type)
+    e = traceback.format_exc()
+    logger.error(e)
+    _ERROR_REPORT[ecosystem].append((error_type, error_message, e))
 
 
 class EcosystemController:
@@ -118,23 +122,28 @@ async def handle_capture(attr):
                 border_print(f"{attr} for {ecosystem}")
                 async with asyncio.timeout_at(get_timeout()):
                     await getattr(_ECOSYSTEM_MAP[ecosystem], attr)()
-            except TimeoutError:
-                logger.error(f"timeout {attr} {ecosystem}")
-                track_error(ecosystem, "TIMEOUT")
-            except Exception:
-                logger.error(f"unexpected error {attr} {ecosystem}")
-                logger.error(traceback.format_exc())
-                track_error(ecosystem, "UNEXPECTED")
+            except TimeoutError as e:
+                logger.error(f"Timeout {attr} {ecosystem}")
+                track_error(ecosystem, "TIMEOUT", str(e))
+            except Exception as e:
+                logger.error(f"Unexpected error {attr} {ecosystem}")
+                track_error(ecosystem, "UNEXPECTED", str(e))
 
     @staticmethod
     async def start():
+        for platform_name, platform, in _PLATFORM_MAP.items():
+            border_print(f"Starting streaming for platform {platform_name}")
+            await platform.start_streaming()
         await EcosystemController.handle_capture("start")
 
     @staticmethod
     async def stop():
         for ecosystem_name, ecosystem in _ANALYSIS_MAP.items():
             border_print(f"Stopping analysis proc for {ecosystem_name}")
             ecosystem.kill()
+        for platform_name, platform, in _PLATFORM_MAP.items():
+            border_print(f"Stopping streaming for platform {platform_name}")
+            await platform.stop_streaming()
         await EcosystemController.handle_capture("stop")
 
     @staticmethod
@@ -150,9 +159,11 @@ async def probe():
         await EcosystemController.handle_capture("probe")
 
     @staticmethod
-    def error_report():
-        for k, v in _ERROR_REPORT.items():
-            print(f"{k}: {v}")
+    def error_report(artifact_dir: str):
+        error_report_file_name = create_standard_log_name("error_report", "txt", parent=artifact_dir)
+        with open(error_report_file_name, 'a+') as error_report_file:
+            for k, v in _ERROR_REPORT.items():
+                log.print_and_write(f"{k}: {v}", error_report_file)
 
     @staticmethod
     def has_errors():

src/tools/interop/idt/capture/loader.py

@@ -44,12 +44,21 @@ def is_package(potential_package: str) -> bool:
         return os.path.exists(init_path)
 
     def verify_coroutines(self, subclass) -> bool:
+        # ABC does not verify coroutines on subclass instantiation, it merely checks the presence of instance methods
         for item in dir(self.search_type):
             item_attr = getattr(self.search_type, item)
             if inspect.iscoroutinefunction(item_attr):
                 if not hasattr(subclass, item):
+                    self.logger.warning(f"Missing coroutine in {subclass}")
                     return False
                 if not inspect.iscoroutinefunction(getattr(subclass, item)):
+                    self.logger.warning(f"Missing coroutine in {subclass}")
+                    return False
+        for item in dir(subclass):
+            item_attr = getattr(subclass, item)
+            if inspect.iscoroutinefunction(item_attr) and hasattr(self.search_type, item):
+                if not inspect.iscoroutinefunction(getattr(self.search_type, item)):
+                    self.logger.warning(f"Unexpected coroutine in {subclass}")
                     return False
         return True
 

src/tools/interop/idt/capture/platform/android/capabilities.py

@@ -1,6 +1,7 @@
 from typing import TYPE_CHECKING
 
 from capture.utils.shell import Bash
+from capture.utils.artifact import create_standard_log_name
 
 if TYPE_CHECKING:
     from capture.platform.android import Android
@@ -20,6 +21,7 @@ def __init__(self, platform: "Android"):
         self.c_has_root = False
         self.c_is_64 = False
         self.c_hci_snoop_enabled = False
+        self.artifact = create_standard_log_name("capabilities", "txt", parent=platform.artifact_dir)
 
     def __repr__(self):
         s = "Detected capabilities:\n"
@@ -56,3 +58,5 @@ def check_capabilities(self):
             if not self.c_hci_snoop_enabled:
                 self.logger.error("Failed to enabled HCI snoop log")
         self.logger.info(self)
+        with open(self.artifact, "w") as artifact:
+            artifact.write(str(self))

src/tools/interop/idt/idt.py

@@ -207,7 +207,7 @@ def command_capture(self, args: argparse.Namespace) -> None:
         # TODO: Write error traces to artifacts
         if EcosystemController.has_errors():
             border_print("Errors seen this run:")
-            EcosystemController.error_report()
+            EcosystemController.error_report(self.artifact_dir)
 
         border_print("Compressing artifacts...")
         self.zip_artifacts()