Merge remote-tracking branch 'origin/main' into exploit/CVE-2021-22941

# Conflicts:
#	agent/definitions.py
#	tests/asteroid_agent_test.py

Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.10-alpine as base
+FROM python:3.11-alpine as base
 FROM base as builder
 RUN apk add build-base
 RUN mkdir /install
@@ -12,4 +12,4 @@ ENV PYTHONPATH=/app
 COPY agent /app/agent
 COPY ostorlab.yaml /app/agent/ostorlab.yaml
 WORKDIR /app
-CMD ["python3", "/app/agent/template_agent.py"]
+CMD ["python3", "/app/agent/asteroid_agent.py"]

README.md

@@ -1,36 +1,70 @@
-# Ostorlab Template Agent
+<h1 align="center">Agent Asteroid</h1>
 
-This repo is a template to build an Ostorlab agent in Python. It ships with good best practices like:
+<p align="center">
+<img src="https://img.shields.io/badge/License-Apache_2.0-brightgreen.svg">
+<img src="https://img.shields.io/github/languages/top/ostorlab/agent_asteroid">
+<img src="https://img.shields.io/github/stars/ostorlab/agent_asteroid">
+<img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg">
+</p>
 
-* Github actions workflow
-* Linting checks
-* Static typing checks with Mypy
-* Running unit test with Pytest
-* Compute test coverage
+_The Asteroid Agent is a powerful scanner specializing in the detection of vulnerabilities that could be exploited remotely._
+
+---
 
 ## Getting Started
+To perform your first scan, simply run the following command:
+```shell
+ostorlab scan run --install --agent agent/ostorlab/asteroid ip 8.8.8.8
+``` 
+
+This command will download and install `agent/ostorlab/asteroid` and target IP `8.8.8.8`.
+For more information, please refer to the [Ostorlab Documentation](https://github.com/Ostorlab/ostorlab/blob/main/README.md)
+
+
+## Usage
+
+Agent Asteroid can be installed directly from the ostorlab agent store or built from this repository.
+
+ ### Install directly from ostorlab agent store
+
+ ```shell
+ ostorlab agent install agent/ostorlab/asteroid
+ ```
+
+You can then run the agent with the following command:
+```shell
+ostorlab scan run --agent agent/ostorlab/asteroid ip 8.8.8.8
+```
+
+
+### Build directly from the repository
+
+ 1. To build the asteroid agent you need to have [ostorlab](https://pypi.org/project/ostorlab/) installed in your machine. If you have already installed ostorlab, you can skip this step.
+
+```shell
+pip3 install ostorlab
+```
+
+ 2. Clone this repository.
+
+```shell
+git clone https://github.com/Ostorlab/agent_asteroid.git && cd agent_asteroid
+```
+
+ 3. Build the agent image using ostorlab cli.
+
+ ```shell
+ ostorlab agent build --file=ostorlab.yaml
+ ```
+
+ You can pass the optional flag `--organization` to specify your organisation. The organization is empty by default.
 
-Here are links to good resources to get started:
-
-* [Write An Agent](https://docs.ostorlab.co/tutorials/write-an-ostorlab-agent.html)
-* [Use Ostorlab](https://docs.ostorlab.co/tutorials/run-your-first-scan.html)
-* [Debugging and Testing Agents](https://docs.ostorlab.co/tutorials/debugging-agents.html)
-* [Ostorlab Internals](https://docs.ostorlab.co/tutorials/life-of-a-scan.html)
-
-## Ideas for Agents to build
-
-Implementation of popular tools like:
-
-* [semgrep](https://github.com/returntocorp/semgrep) for source code scanning.
-* [nbtscan](http://www.unixwiz.net/tools/nbtscan.html): Scans for open NETBIOS nameservers on your target’s network.
-* [onesixtyone](https://github.com/trailofbits/onesixtyone): Fast scanner to find publicly exposed SNMP services.
-* [Retire.js](http://retirejs.github.io/retire.js/): Scanner detecting the use of JavaScript libraries with known
-  vulnerabilities.
-* [snallygaster](https://github.com/hannob/snallygaster): Finds file leaks and other security problems on HTTP servers.
-* [testssl.sh](https://testssl.sh/): Identify various TLS/SSL weaknesses, including Heartbleed, CRIME and ROBOT.
-* [TruffleHog](https://github.com/trufflesecurity/truffleHog): Searches through git repositories for high entropy
-  strings and secrets, digging deep into commit history.
-* [cve-bin-tool](https://github.com/intel/cve-bin-tool): Scan binaries for vulnerable components.
-* [XSStrike](https://github.com/s0md3v/XSStrike): XSS web vulnerability scanner with generative payload.
-* ~~[Subjack](https://github.com/haccer/subjack): Subdomain takeover scanning tool.~~
-* [DnsReaper](https://github.com/punk-security/dnsReaper): Subdomain takeover scanning tool.
+ 4. Run the agent using on of the following commands:
+	 * If you did not specify an organization when building the image:
+    ```shell
+    ostorlab scan run --agent agent//asteroid ip 8.8.8.8
+    ```
+	 * If you specified an organization when building the image:
+    ```shell
+    ostorlab scan run --agent agent/[ORGANIZATION]/asteroid ip 8.8.8.8
+    ```

agent/asteroid_agent.py

@@ -0,0 +1,38 @@
+"""Asteroid Agent is designed to identify known exploitable vulnerabilities in a remote system. The agent expects a
+message of type `v3.asset.ip.[v4,v6]` or `v3.asset.[domain_name,link]`, and emits back messages of type
+`v3.report.vulnerability` with a technical report."""
+import logging
+from rich import logging as rich_logging
+
+from ostorlab.agent import agent
+from ostorlab.agent.mixins import agent_report_vulnerability_mixin
+from ostorlab.agent.message import message as m
+
+logging.basicConfig(
+    format="%(message)s",
+    datefmt="[%X]",
+    level="INFO",
+    force=True,
+    handlers=[rich_logging.RichHandler(rich_tracebacks=True)],
+)
+logger = logging.getLogger(__name__)
+
+
+class AsteroidAgent(agent.Agent, agent_report_vulnerability_mixin.AgentReportVulnMixin):
+    """Asteroid Agent is designed to identify known exploitable vulnerabilities in a remote system."""
+
+    def process(self, message: m.Message) -> None:
+        """Process messages of type `v3.asset.ip.[v4,v6]` or `v3.asset.[domain_name,link]` and performs a network
+        scan. Once the scan is completed, it emits messages of type
+        `v3.report.vulnerability` with the technical report.
+
+        Args:
+            message: message containing the asset to scan.
+        """
+
+        # TODO (benyissa): implement agent logic here.
+
+
+if __name__ == "__main__":
+    logger.info("starting agent ...")
+    AsteroidAgent.main()

agent/definitions.py

@@ -6,6 +6,13 @@
 from ostorlab.agent.mixins import agent_report_vulnerability_mixin as vuln_mixin
 
 
+@dataclasses.dataclass
+class Target:
+    scheme: str
+    host: str
+    port: int
+
+
 @dataclasses.dataclass
 class Vulnerability:
     """Vulnerability entry with technical details, custom risk rating, DNA for unique identification and location."""
@@ -16,28 +23,18 @@ class Vulnerability:
     dna: str | None = None
     vulnerability_location: vuln_mixin.VulnerabilityLocation | None = None
 
-
-@dataclasses.dataclass
-class Target:
-    """Target asset"""
-
-    scheme: str
-    host: str
-    port: int
-
-
 class Exploit(abc.ABC):
     """Base Exploit"""
 
     @abc.abstractmethod
     def accept(self, target: Target) -> bool:
-        """Rule to heuristically detect if specific target is valid.
+        """Rule: heuristically detect if a specific target is valid.
 
         Args:
             target: Target to verify
 
         Returns:
-            List of identified vulnerabilities.
+            True if the target is valid; otherwise False.
         """
         pass
 
@@ -52,3 +49,8 @@ def check(self, target: Target) -> list[Vulnerability]:
             List of identified vulnerabilities.
         """
         pass
+
+    @property
+    def __key__(self) -> str:
+        """Unique key for the class, mainly useful for registering the exploits."""
+        return self.__class__.__name__

agent/exploits_registry.py

@@ -0,0 +1,44 @@
+"""Register for exploits."""
+from collections import defaultdict
+from typing import Type, Any
+
+from agent import definitions
+
+
+class ExploitsRegistry:
+    """Registry class, This class provides a way to store and retrieve callables that generate lists of
+    `definitions.Exploit` objects from a given file name and bytes object.
+    """
+
+    registry: dict[Any, Any] = defaultdict(dict)
+
+    @classmethod
+    def register_ref(
+        cls,
+        obj: definitions.Exploit,
+        key: str = "__key__",
+    ) -> definitions.Exploit:
+        cls.registry[cls.__name__][getattr(obj, key)] = obj
+        return obj
+
+    @classmethod
+    def values(
+        cls,
+    ) -> list[Any]:
+        return list(cls.registry[cls.__name__].values())
+
+
+def register(
+    f: Type[definitions.Exploit],
+) -> Type[definitions.Exploit]:
+    """
+    To be used as a decorator on the exploit class
+
+    Args:
+        f: The class which its object will be registered.
+
+    Returns:
+        The input callable.
+    """
+    ExploitsRegistry.register_ref(obj=f())
+    return f

ostorlab.yaml

@@ -1,9 +1,75 @@
 kind: Agent
-name: template_agent # Agent name, must be unique by organisation to be published on the store.
-version: 0.0.0 # Must respect semantic versioning.
-description: Agent description. # Support for Markdown format.
-in_selectors: # List of input selectors, this is basically the list of messages the agent should receive.
-  - v3.healthcheck.ping
-out_selectors: [] # List of output selectors.
-docker_file_path : Dockerfile # Dockerfile path for automated releases.
-docker_build_root : . # Docker build dir for automated release build.
+name: asteroid
+version: 0.0.1
+description: |   
+  _The Asteroid Agent is a powerful scanner specializing in the detection of vulnerabilities that could be exploited remotely._
+  
+  ---
+  
+  ## Getting Started
+  To perform your first scan, simply run the following command:
+  ```shell
+  ostorlab scan run --install --agent agent/ostorlab/asteroid ip 8.8.8.8
+  ``` 
+  
+  This command will download and install `agent/ostorlab/asteroid` and target the ip `8.8.8.8`.
+  For more information, please refer to the [Ostorlab Documentation](https://github.com/Ostorlab/ostorlab/blob/main/README.md)
+  
+  
+  ## Usage
+  
+  Agent Asteroid can be installed directly from the ostorlab agent store or built from this repository.
+  
+   ### Install directly from ostorlab agent store
+  
+   ```shell
+   ostorlab agent install agent/ostorlab/asteroid
+   ```
+  
+  You can then run the agent with the following command:
+  ```shell
+  ostorlab scan run --agent agent/ostorlab/asteroid ip 8.8.8.8
+  ```
+  
+  
+  ### Build directly from the repository
+  
+   1. To build the asteroid agent you need to have [ostorlab](https://pypi.org/project/ostorlab/) installed in your machine.  if you have already installed ostorlab, you can skip this step.
+  
+  ```shell
+  pip3 install ostorlab
+  ```
+  
+   2. Clone this repository.
+  
+  ```shell
+  git clone https://github.com/Ostorlab/agent_asteroid.git && cd agent_asteroid
+  ```
+  
+   3. Build the agent image using ostorlab cli.
+  
+   ```shell
+   ostorlab agent build --file=ostorlab.yaml
+   ```
+  
+   You can pass the optional flag `--organization` to specify your organisation. The organization is empty by default.
+  
+   4. Run the agent using on of the following commands:
+       * If you did not specify an organization when building the image:
+      ```shell
+      ostorlab scan run --agent agent//asteroid ip 8.8.8.8
+      ```
+       * If you specified an organization when building the image:
+      ```shell
+      ostorlab scan run --agent agent/[ORGANIZATION]/asteroid ip 8.8.8.8
+      ```
+
+in_selectors:
+  - v3.asset.ip.v4
+  - v3.asset.ip.v6
+  - v3.asset.domain_name
+  - v3.asset.link
+out_selectors:
+  - v3.report.vulnerability
+docker_file_path : Dockerfile
+docker_build_root : .

tests/asteroid_agent_test.py

@@ -0,0 +1,6 @@
+"""Unit tests for AsteroidAgent."""
+
+
+def testAgent() -> None:
+    """Fake test."""
+    assert True