When running dbt-meshify split, get exception "Unable to find a docs block with the name int_hcss_dis_jobs_doc" even though doc block exists

[dmoditto]

Describe the bug

When running dbt-meshify split, get exception "Unable to find a docs block with the name int_hcss_dis_jobs_doc" even though doc block exists

Steps to reproduce

Using Codespaces, ran "dbt-meshify split dm_accounting_pipeline --select +dm_accounting"

Expected results

Expected to split our +dm_accounting pipeline into a new project

Actual results

Got an exception error:

Exception: Unable to find a docs block with the name int_hcss_dis_jobs_doc.

Screenshots and log output

codespace@codespaces-75b401:/workspaces/dbt_cloud$ dbt-meshify split dm_accounting_pipeline --select +dm_accounting
15:12:12 | INFO | Executing dbt parse...
15:12:40 | INFO | Generating catalog with dbt docs generate...
Traceback (most recent call last):
  File "/home/codespace/.local/bin/dbt-meshify", line 8, in <module>
    sys.exit(cli())
  File "/home/codespace/.local/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/codespace/.local/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/codespace/.local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/codespace/.local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/codespace/.local/lib/python3.10/site-packages/dbt_meshify/main.py", line 232, in split
    project = DbtProject.from_directory(
  File "/home/codespace/.local/lib/python3.10/site-packages/dbt_meshify/dbt_projects.py", line 292, in from_directory
    return DbtProject(
  File "/home/codespace/.local/lib/python3.10/site-packages/dbt_meshify/dbt_projects.py", line 315, in __init__
    self.jinja_blocks: Dict[str, JinjaBlock] = self.find_jinja_blocks()
  File "/home/codespace/.local/lib/python3.10/site-packages/dbt_meshify/dbt_projects.py", line 326, in find_jinja_blocks
    blocks[unique_id] = JinjaBlock.from_file(
  File "/home/codespace/.local/lib/python3.10/site-packages/dbt_meshify/storage/jinja_blocks.py", line 63, in from_file
    start, end = cls.find_block_range(file_content, block_type, name)
  File "/home/codespace/.local/lib/python3.10/site-packages/dbt_meshify/storage/jinja_blocks.py", line 36, in find_block_range
    raise Exception(f"Unable to find a {block_type} block with the name {name}.")
Exception: Unable to find a docs block with the name int_hcss_dis_jobs_doc.

System information

Which database are you using dbt with?

• postgres
• redshift
• bigquery
• snowflake
• other (specify: ____________)

The output of dbt debug:

codespace@codespaces-75b401:/workspaces/dbt_cloud$ dbt debug
15:17:19  Running with dbt=1.7.13
15:17:19  dbt version: 1.7.13
15:17:19  python version: 3.10.12
15:17:19  python path: /usr/local/bin/python
15:17:19  os info: Linux-6.2.0-1019-azure-x86_64-with-glibc2.31
15:17:19  Using profiles dir at /workspaces/dbt_cloud/.dbt
15:17:19  Using profiles.yml file at /workspaces/dbt_cloud/.dbt/profiles.yml
15:17:19  Using dbt_project.yml file at /workspaces/dbt_cloud/dbt_project.yml
15:17:19  adapter type: snowflake
15:17:19  adapter version: 1.7.1
15:17:20  Configuration:
15:17:20    profiles.yml file [OK found and valid]
15:17:20    dbt_project.yml file [OK found and valid]
15:17:20  Required dependencies:
15:17:20   - git [OK found]

15:17:20  Connection:
15:17:20    account: fc41459
15:17:20    user: dylanmorrish
15:17:20    database: ANALYTICS_DW
15:17:20    warehouse: COMPOSE_WH
15:17:20    role: SYSADMIN
15:17:20    schema: PROD
15:17:20    authenticator: None
15:17:20    private_key_path: None
15:17:20    token: None
15:17:20    oauth_client_id: None
15:17:20    query_tag: None
15:17:20    client_session_keep_alive: False
15:17:20    host: None
15:17:20    port: None
15:17:20    proxy_host: None
15:17:20    proxy_port: None
15:17:20    protocol: None
15:17:20    connect_retries: 1
15:17:20    connect_timeout: None
15:17:20    retry_on_database_errors: False
15:17:20    retry_all: False
15:17:20    insecure_mode: False
15:17:20    reuse_connections: None
15:17:20  Registered adapter: snowflake=1.7.1
15:17:20    Connection test: [OK connection ok]

15:17:20  All checks passed!

The output of dbt --version:

Core:
  - installed: 1.7.13
  - latest:    1.7.13 - Up to date!

Plugins:
  - sqlserver: 1.7.4 - Up to date!
  - snowflake: 1.7.1 - Update available!
  - fabric:    1.7.4 - Update available!

  At least one plugin is out of date or incompatible with dbt-core.
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Additional context

Are you interested in contributing the fix?

Nope

[Hansk-12]

I have a similar issue on Athena, while creating a group and assiging an owner. I did the same tests as the OP.

[dave-connors-3]

@dmoditto @Hansk-12 thanks so much for the report! we may need a bit more info to debug here. Would each of you mind providing:

1. the filepath (including whether or not this comes from a package!) of where the failing doc block is defined
2. the content of that entire file (you can scrub any sensitive docs, we care mostly about the blocks of jinja in the file)
3. the name of the failing block (which we have in your logs above @dmoditto!)

hopefully that will be enough for us to get to a root cause here! appreciate your patience!

[dmoditto]

@dave-connors-3 sure thing! Below are my answers. In gathering that info, I was able to narrow down further what is going wrong. Seems like whatever is parsing the doc block header doesn't like the two cases I outline below, and found this happens to macro headers as well.

1. /dbt_cloud/models/marts/accounting_gl/integrations/acnt_integrations.md

{% docs int_hcss_dis_jobs_doc%}

The 'int_hcss_dis_jobs' view is used to udpate HCSS HeavyJob via DIS for Construction and PRG Jobs from JDE F0006 Job Master.

{% enddocs %}

Case 1: The Missing Space

In taking a look at the doc block again, I actually found a change that fixes this: simply ensure there is a space at the doc block tag. So {% docs int_hcss_dis_jobs_doc%} changed to {% docs int_hcss_dis_jobs_doc %} fixes the issue.

Unfortunately, it turns out we have numerous doc blocks that lack that final space. So, I'm off to find and replace all those suckers.

Case 2: An Odd Dash

One other case I found is with the following doc block. This is saved in /macros

{% docs snowflake_-_helper_dbt_doc %}

https://github.com/jeremiahhansen/snowflake-helper-dbt

{% enddocs %}

And it is all good when I remove the oddly placed dash

{% docs snowflake_helper_dbt_doc %}

https://github.com/jeremiahhansen/snowflake-helper-dbt

{% enddocs %}

Case 3: The Missing Space Part 2: The Macro Edition

This is a custom macro saved in /macros

{% macro create_schema_yml_with_sf_tags(
        model_name,
        schema_name,
        database_name = target.database,
        generate_columns = True,
        column_names = []
    ) %}

Adding a space between the name of the macro and the first parenthesis fixes this issue; however, I'm unsure if this macro can still be called. I'll explore that

{% macro create_schema_yml_with_sf_tags (
        model_name,
        schema_name,
        database_name = target.database,
        generate_columns = True,
        column_names = []
    ) %}
```


I'll add any other cases I find, but I hope these three help point towards a root cause and resolution!

[dave-connors-3]

@dmoditto Aha! this makes a lot of sense -- we are using some pretty unscientific regex matching to move jinja blocks, and we're likely simply not covering all the valid edge cases of space and special character placement! should be a relatively quick fix! cc: @nicholasyager

[agyemangg-accelins]

I am facing this issue, please when will the above fix be released?