README.md

parseltongue

A Python script for parsing credential enumeration tool, dnscmd, and dsquery output, and correlating relevant information to aid analysis.

Dependencies

Version 1.x.x requires Python 2. Version 2.0.0+, requires Python 3 (tested against Python 3.8.2). Neither require any external libraries.

Usage

Parseltongue has extensive built-in help. Run the script with no arguments to see options and examples.

Command-line Options

• -h, --help

  • Prints the default help message generated by ArgumentParser. If this option is specified, all other arguments will be ignored
  • Run with no options for more verbose output

• --debug

  • Adds debugging output to the log file. This will dump various data structures throughout execution to aid in diagnosing processing errors
  • When using the command-line flag or a config file, debugging will only be enabled after the command-line arguments or config file is processed. To enable debugging for the entire execution of the program, you must change the value of in the code directly.

• -s, --show

  • Displays the dsquery commands that parseltongue expects to be run to collect the necessary data
  • If this option is specified, all other arguments will be ignored

• -c CONFIG_FILEPATH, --config CONFIG_FILEPATH

  • Specifies an absolute or relative path to a JSON configuration file. config.json is the default is -c is omitted
  • If config.json does not exist, parseltongue will create it and populate it with the default settings
  • If a non-existent filepath is specified, parseltongue will either: 1) automatically create the file using default settings (if -g is specified), or 2) prompt the user whether or not they want to create the file. If a non-existent config is specified and not created, parseltongue will exit gracefully.

• -g, --generate-config

  • A boolean switch indicating that the specified config file should be created.
  • If no custom config is specified or the specified config file already exists, this option does nothing

• -u, --update-config

  • Saves the current configuration (i.e., options specified as command-line arguments) to the current config file, either the default or the one specified by -c
  • When used with -c and -g, this can be useful for prototyping custom configs for different engagements. Play with the options until you get the output you want, then run with a -u to save the settings.

• -o OUTPUT_DIR, --output OUTPUT_DIR

  • Specifies the output directory where parsed data should be written
  • If provided, this option overrides the value specified in the config file

• -d DELIMITER, --delimiter DELIMITER

  • Specifies the character that should be used to separate multiple objects of the same type (e.g., member attribute of a group). Defaults to "\n" (new line)
  • If provided, this option overrides the value specified in the config file

• -w WORDLIST, --wordlist WORDLIST

  • Specifies input wordlist(s) to be used for rudimentary password cracking.
  • May be either a path to a single file or a directory path containing multiple wordlists (sub-directories will be scanned as well)
  • Each wordlist file should have a single plaintext password per line
  • Especially useful if you have a list of commonly used passwords specific to your customer

• -v {0,1,2}, --verbosity {0,1,2}

  • This controls the verboseness of messages printed to the terminal screen (0 = Minimal, 1 = Normal, 2 = Info); Default: 1
  • Verbosity of the log file is controlled by a config file setting; Default: ['LOGGING']['VERBOSITY'] = 2 (Info)

• [data_filepath [data_filepath ...]]

  • An arbitrary number of files containing system information (i.e., credentials, dnscmd output, dsquery output). Valid file types include:
    • computers
    • credentials (Parseltongue output)
    • cs_export (Cobalt Strike credential export)
    • dcsync
    • dns
    • gpos
    • groups
    • hashdump
    • logonpasswords
    • lsadump
    • ous
    • users
  • Parseltongue auto-detects information from the name of the data file being processed. Therefore, data files must be named according to the following format: <NT domain>_<date>_<file_type>
  • The accepted datestamp format can be configured using the ['INPUT']['DATA']['FILENAME_DATE_FORMAT'] option in the config file. Dates are used to chronologically order data of the same file type so that more recent data overwrites older data, if applicable.
  • Parseltongue can handle both XML and text versions of exported Cobalt Strike credentials; however, XML is highly preferred. The regex for the text version will not properly handle plaintext password entries where the username contains a space; this is a limitation of the text export format, which is space-delimited, not a bug with parseltongue. Since Cobalt Strike credentials may contain data from multiple domains, configuration options are provided that allows users to specify how to handle various situations.
    • Unless overridden by one of the following settings, the NT domain specified in the filename will be ignored in favor of each credential entry's realm field
    • If the realm is a DNS domain rather than an NT domain, the value of ['INPUT']['DATA']['CS_EXPORT']['INVALID_REALM'] will determine how the data is handled.
      • prompt - The user will be prompted for the matching NT domain (default)
      • replace - The DNS domain will be automatically replaced with the NT domain from the filename
      • warn - A warning message will be displayed, but the data will not be modified. This will create separate credential files for any DNS domains encountered, and these credentials will not be matched with Active Directory users which belong to an NT domain.
      • ignore - No warning will be displayed, and the entry will be skipped (i.e., not added to credential output or matched with users)
      • warn_and_ignore - Displays a warning and skips the entry
    • If a valid NT domain is parsed, but it does not match the NT domain specified in the filename, the ['INPUT']['DATA']['CS_EXPORT']['FOREIGN_DOMAIN'] setting is used.
      • include - Includes data from all domains, including local accounts (default)
      • ignore - The entry will be skipped. This option is provided to create cleaner output, if desired; however, it will likely cause issues if trying to process multiple domains at once or if data includes local credentials. Use at your own risk.
    • Invalid realms are processed before foreign domains. Please consider this logic when modifying your settings
    • The ['INPUT']['DATA']['CS_EXPORT']['POPULATE_COMMENT'] setting determines whether or not to include the source and host fields in the comment field
      • append - The source and host will be appended to existing comments; empty comments will be populated
      • empty_only - Only empty comments will be populated; entries with existing comments will not be modified
      • none - The source and host information will not be included

Configuration File Settings

The following are the default configuration settings. If any command-line arguments are specified (e.g., -o, -d), they will override these settings for the current execution.

{
    "INPUT": {
        "DSQUERY_ATTRS": {
            "COMPUTERS": ["dnshostname", "operatingsystem", "operatingsystemversion", "operatingsystemservicepack", "lastlogon", "lastlogontimestamp", "useraccountcontrol", "description", "memberof", "primarygroupid", "location", "objectsid", "adspath"],
            "GPOS": ["displayname", "name", "adspath"],
            "GROUPS": ["samaccountname", "name", "distinguishedname", "objectsid", "primarygroupid", "description", "member", "adspath"],
            "OUS": ["name", "managedby", "description", "gplink", "adspath"],
            "USERS": ["samaccountname", "name", "distinguishedname", "lastlogon", "lastlogontimestamp", "pwdlastset", "useraccountcontrol", "memberof", "description", "objectsid", "primarygroupid", "adspath"]
        },
        "DATA": {
            "FILENAME_DATE_FORMAT": "%Y-%m-%d",
            "CS_EXPORT": {
                "FOREIGN_DOMAIN": "include", # Valid options: "include", "ignore"
                "INVALID_REALM": "prompt", # Valid options: "replace", "prompt", "warn", "ignore", "warn_and_ignore"
                "POPULATE_COMMENT": True
            }
        },
        "WORDLIST": "wordlists\\wordlist.txt"
    },
    "OUTPUT": {
        "DATA": {
            "FILENAME_DATE_FORMAT": "%Y-%m-%d",
            "DIR": "output",
            "MULTI_OBJECT_DELIMITER": "\n",
            "SEPARATE_BY_DOMAIN": false
        },
        "WORDLIST": "wordlists\\wordlist.txt"
    },
    "LOGGING": {
        "OUTPUT_DIR": "logs",
        "VERBOSITY": 2,
        "TIMEFORMAT_FILE": "%Y-%m-%d_%H%M%S",
        "TIMEFORMAT_LOG": "%Y-%m-%d %H:%M:%S",
        "WRITE_FILE": true
    },
    "DEBUG": False,
    "VERBOSITY": 1
}

There are three main sections (Input, Output, and Logging) that group settings related to those features; additional settings at the root level are considered global to the entire script. If a config file omits a required setting, it will fall back to the default value. Technically you could remove everything from the config file except for the outer {}, and it will simply use the default values.

{
    "INPUT": {
        # Change the items in this list to control which attributes Parseltongue attempts to parse
        # IMPORTANT: The last element must be unique, otherwise objects won't be delimited properly
        "DSQUERY_ATTRS": {
            "COMPUTERS": ["dnshostname", "operatingsystem", "operatingsystemversion", "operatingsystemservicepack", "lastlogon", "lastlogontimestamp", "useraccountcontrol", "description", "memberof", "primarygroupid", "location", "objectsid", "adspath"],
            "GPOS": ["displayname", "name", "adspath"],
            "GROUPS": ["samaccountname", "name", "distinguishedname", "objectsid", "primarygroupid", "description", "member", "adspath"],
            "OUS": ["name", "managedby", "description", "gplink", "adspath"],
            "USERS": ["samaccountname", "name", "distinguishedname", "lastlogon", "lastlogontimestamp", "pwdlastset", "useraccountcontrol", "memberof", "description", "objectsid", "primarygroupid", "adspath"]
        },
        "DATA": {
            # This specifies the date format for input data filenames 
            # (e.g., "SGC_2020-03-11_computers.txt")
            # See https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
            "FILENAME_DATE_FORMAT": "%Y-%m-%d",
            
            "CS_EXPORT": {
                # This determines how to handle credentials with a realm different than the NT domain specified in the name of the current file
                "FOREIGN_DOMAIN": "include", # Valid options: "include", "ignore"
                
                # This determines how to handle realms that are DNS domains
                "INVALID_REALM": "prompt", # Valid options: "replace", "prompt", "warn", "ignore", "warn_and_ignore"
                
                # This controls whether 
                "POPULATE_COMMENT": "append" # Valid options: "append", "empty_only", "none"
            }
        },
        
        # This specifies a file or directory containing wordlists (see -w option above)
        # To disable loading a wordlist, set this value to ""
        "WORDLIST": "wordlists\\wordlist.txt"
    },
    "OUTPUT": {
        "DATA": {
            # This specifies the date format for output CSV filenames 
            # (e.g., "SGC_2020-03-11_computers.csv")
            "FILENAME_DATE_FORMAT": "%Y-%m-%d",
            
            # Directory where output files will be written; can be overridden at runtime 
            # using the -o option
            "DIR": "output",
            
            # The delimiter between multiple objects of the same type; can be overridden
            # at runtime using the -d option
            "MULTI_OBJECT_DELIMITER": "\n",
            
            # A boolean indicating whether output files should be further sub-divided by
            # NT domain within the output directory
            "SEPARATE_BY_DOMAIN": false
        },
        
        # This option specifies a custom filepath where all plaintext passwords
        # loaded from wordlist(s) *OR* parsed from data files will be written.
        # If a static filename is specified, the file will be overwritten.
        # Alternatively, a date format can be specified within '<>'; this 
        # will create separate wordlists based on the datetime string
        # For example: "wordlists\\wordlist_<%Y-%m-%d>.txt" would create
        # a unique wordlist per day
        # This could also be used to specify a specific wordlist per engagement
        # For example: "wordlists\\BLACKNIGHT.txt"
        # This can be set to the same value as ['INPUT']['WORDLIST'] to 
        # continually update the same wordlist
        # To disable writing out a wordlist, set this value to ""
        "WORDLIST": "wordlists\\wordlist.txt"
    },
    "LOGGING": {
        # The directory where log files will be written
        "OUTPUT_DIR": "logs",
        
        # The verboseness level of log messages
        "VERBOSITY": 2,
        
        # Timestamp format to use for the filename of each log
        "TIMEFORMAT_FILE": "%Y-%m-%d_%H%M%S",
        
        # Timestamp format to use for each message in the log file
        "TIMEFORMAT_LOG": "%Y-%m-%d %H:%M:%S",
        
        # Boolean controlling whether or not log files should be created
        # Set to false to disable logging
        "WRITE_FILE": true
    },
    
    # Controls whether extra debugging information is written to the log file, see --debug above
    "DEBUG": False,
    
    # The verboseness of terminal output; can be overridden at runtime using the -v option
    "VERBOSITY": 1
}

Examples

• parseltongue.py -s

  • Displays the dsquery commands to run that will generate the output parseltongue expects

• parseltongue.py -c custom_config.json -g

  • Creates a custom_config.json containing the default config settings, if the file does not exist. If the file does exist, the "-g" argument does nothing. If "-g" is omitted and the specified file does not exist, the user will be prompted whether or not to create the file. Either way, the current configuration is printed to the screen. This can be used to double check settings; especially useful when using command-line options to override config file settings.

• parseltongue.py -c custom_config.json -d "|" -u SGC_2020-03-11_users.txt

  • Loads a custom config file (custom_config.json)
  • Uses "|" as a delimiter between multiple values of the same type (e.g., member attribute of a group)
  • Updates custom_config.json to use "|" as the delimiter
  • Parses Active Directory user data

• parseltongue.py -o ../parseltongue_output -w wordlist.txt SGC_2020-03-11_hashdump.txt SGC_2020-03-11_users.txt

  • Specifies a custom output directory; overrides output directory specified in config file for current execution
  • Reads a list of plaintext passwords (one per line) from wordlist.txt; uses these to crack hashes parsed from the hashdump
  • Parses Active Directory user data and enriches each record with the NTLM hash of the user; if a matching plaintext password was found in wordlist.txt, this will also be included

• parseltongue.py SGC_2020-03-11_users.txt SGC_2020-03-11_computers.txt SGC_2020-03-11_groups.txt

  • Parses users, computers, and group data for the SGC domain

Dsquery Commands

The following are the dsquery commands from which parseltongue expects to receive data. Aditional attributes may be added by editing the code (see configurable constants at the top of the file); however, the last element must appear only once per object as it is used as a delimiter. I have only tested with these attributes, so your mileage may vary if you alter them.

Computers

dsquery * -filter "(objectclass=computer)" -attr dnshostname operatingsystem operatingsystemversion operatingsystemservicepack lastlogon lastlogontimestamp useraccountcontrol description memberof primarygroupid location objectsid adspath -limit 0 -l

Users

dsquery * -filter "(&(objectclass=user)(!(objectclass=computer)))" -attr samaccountname name distinguishedname lastlogon lastlogontimestamp pwdlastset useraccountcontrol memberof description objectsid primarygroupid adspath -limit 0 -l

Groups

dsquery * -filter "(objectclass=group)" -attr samaccountname name distinguishedname objectsid primarygroupid description member adspath -limit 0 -l

Organizational Unites (OUs)

dsquery * -filter "(objectclass=organizationalunit)" -attr name managedby description gplink adspath -limit 0 -l

Group Policy Objects (GPOs)

dsquery * -filter "(objectclass=grouppolicycontainer)" -attr displayname name adspath -limit 0 -l