IndexError: list index out of range error every time its run with additional flags of --services or --skip-types

[thomas-devoss]

Every time I run awspx with --skip-types or the --services flag, I get the following error:

Traceback (most recent call last):
╭──────────────────────────────────────────────────────────────────────────────────────╮
│ File "/opt/awspx/cli.py", line 385, in main                                          │
│    382         console.start()                                                       │
│    383                                                                               │
│    384     try:                                                                      │
│  ❱ 385         args.func(args)                                                       │
│    386                                                                               │
│    387     except (KeyboardInterrupt, SystemExit):                                   │
│    388         console.stop()                                                        │
│ File "/opt/awspx/cli.py", line 145, in handle_ingest                                 │
│    142     ingestor = IngestionManager(session=session, console=console, services=arg│
│    143                                 db=args.database, quick=args.quick, skip_actio│
│    144                                 only_types=args.only_types, skip_types=args.sk│
│  ❱ 145                                 only_arns=args.only_arns, skip_arns=args.skip_│
│    146                                                                               │
│    147     assert ingestor.zip is not None, "Ingestion failed"                       │
│    148                                                                               │
│ File "/opt/awspx/lib/aws/ingestor.py", line 79, in __init__                          │
│      76         self.load_transitives()                                              │
│      77                                                                              │
│      78         if not skip_actions:                                                 │
│  ❱   79             self.load_actions()                                              │
│      80                                                                              │
│      81         self.zip = self.save(db)                                             │
│      82                                                                              │
│ File "/opt/awspx/lib/aws/ingestor.py", line 250, in load_actions                     │
│     247                                                                              │
│     248                     # This role trusts all IAM entities within this account  │
│     249                     if (action.source().type("AWS::Account")                 │
│  ❱  250                             and action.source().id().split(':')[4] == self.ac│
│     251                                                                              │
│     252                         self.update(Elements(Trusts(properties=action.propert│
│     253                                                     source=action.target(),  │
╰──────────────────────────────────────────────────────────────────────────────────────╯
IndexError: list index out of range

This happens when it gets to 'resolving policy information'

This does not happen when I just run ./awspx ingest however when I run it like this it will not show any connections/attacks etc in the web gui.

Running with the following flags:
./awspx ingest --skip-types "AWS::S3::Object"
./awspx ingest --services EC2, Lambda, IAM

This is running on a MacOS system, inside the docker container pulled down 2 weeks ago (right around the 10th of January)

[beatro0t]

Hi @thomas-devoss,

Thanks for reporting the issue! To be honest with you, I'm a bit stumped - I've encountered the issue myself and believe its been fixed in the develop branch (which I've just merged), so it failing makes sense. What doesn't make sense is that it succeeds without specifying any types or services 🙈

Would you mind pulling the changes, rerunning INSTALL, and letting me know whether or not this has resolved the issue?

[thomas-devoss]

Sure I can do that today and get back with you, thanks!

[rpigu-i]

Hi there,

I am also seeing the same (or similar issues) trying both master and develop branches.

I've attached the traceback below from attempting to run :

awspx ingest --services EC2 LAMBDA IAM

This is on macOS Catalina.

╭──────────────────── Traceback (most recent call last) ─────────────────────╮
│ /opt/awspx/cli.py:385 in main                                              │
│                                                                            │
│   382 │   │   console.start()                                              │
│   383 │                                                                    │
│   384 │   try:                                                             │
│ ❱ 385 │   │   args.func(args)                                              │
│   386 │                                                                    │
│   387 │   except (KeyboardInterrupt, SystemExit):                          │
│   388 │   │   console.stop()                                               │
│                                                                            │
│ /opt/awspx/cli.py:145 in handle_ingest                                     │
│                                                                            │
│   142 │   ingestor = IngestionManager(session=session, console=console, se │
│   143 │   │   │   │   │   │   │   │   db=args.database, quick=args.quick,  │
│   144 │   │   │   │   │   │   │   │   only_types=args.only_types, skip_typ │
│ ❱ 145 │   │   │   │   │   │   │   │   only_arns=args.only_arns, skip_arns= │
│   146 │                                                                    │
│   147 │   assert ingestor.zip is not None, "Ingestion failed"              │
│                                                                            │
│ /opt/awspx/lib/aws/ingestor.py:79 in __init__                              │
│                                                                            │
│     76 │   │   self.load_transitives()                                     │
│     77 │   │                                                               │
│     78 │   │   if not skip_actions:                                        │
│ ❱   79 │   │   │   self.load_actions()                                     │
│     80 │   │                                                               │
│     81 │   │   self.zip = self.save(db)                                    │
│                                                                            │
│ /opt/awspx/lib/aws/ingestor.py:241 in load_actions                         │
│                                                                            │
│    238 │   │   │   │                                                       │
│    239 │   │   │   │   # Skip AWS::Domain principals                       │
│    240 │   │   │   │   self.update(Elements(principal                      │
│ ❱  241 │   │   │   │   │   │   │   │   │    for principal in resource_base │
│    242 │   │   │   │   │   │   │   │   │    if not principal.type("AWS::Do │
│    243 │   │   │   │                                                       │
│    244 │   │   │   │   # Only actions beginning with sts:AssumeRole are va │
│                                                                            │
│ /opt/awspx/lib/aws/ingestor.py:382 in update                               │
│                                                                            │
│    379 │   def update(self, elements):                                     │
│    380 │   │                                                               │
│    381 │   │   for element in elements:                                    │
│ ❱  382 │   │   │   self.add(element)                                       │
│    383 │                                                                   │
│    384 │   def add(self, element):                                         │
│                                                                            │
│ /opt/awspx/lib/aws/ingestor.py:401 in add                                  │
│                                                                            │
│    398 │   │   │   pass                                                    │
│    399 │   │                                                               │
│    400 │   │   else:                                                       │
│ ❱  401 │   │   │   self.console.info(f"Added {element.label()}: ({element} │
│    402                                                                     │
│    403                                                                     │
│    404 class SessionClientWrapper(object):                                 │
│                                                                            │
│ /opt/awspx/lib/graph/base.py:61 in label                                   │
│                                                                            │
│    58 │   │   return [l for l in self.labels()                             │
│    59 │   │   │   │   if l != self.__class__.__name__                      │
│    60 │   │   │   │   ][0]                                                 │
│ ❱  61 │                                                                    │
│    62 │   def labels(self):                                                │
│    63 │   │   return sorted(list(self._labels))                            │
╰────────────────────────────────────────────────────────────────────────────╯
IndexError: list index out of range

Thanks

[beatro0t]

Hey @rpigu-i ,

Thanks for reporting this issue - it looks like an issue with an empty label (probably from parsing an unknown principal in an assume role policy document). Hopefully, this last commit fixes it. Would you mind running docker exec -it awspx bash -c 'git checkout develop; git pull and trying again? Please let me know what the outcome is