Incorrect module source path generated for local modules

[JonCubed]

I believe that the source path that is generated in cdk.tf.json is incorrect for local modules, it doesn't take into account the output folder structure being stack/<stack name> and it needs to back up one more folder. I believe the issue is on L34

terraform-cdk/packages/cdktf/lib/terraform-module.ts

Lines 33 to 35 in 337acff

	if (options.source.startsWith("./") || options.source.startsWith("../")) {
	this.source = path.join("..", options.source);
	} else {

It should be

this.source = path.join("../..", options.source);

This is how I reproduce it:

1. create project folder and cd into it.
2. run cdktf init --template="python-pip" --local
3. create a tf module at modules/module1
4. add modules/module1/main.tf

resource "aws_s3_bucket" "bucket" {
  bucket = "cdktf-test-bucket"
  acl    = "private"
  force_destroy = true
}

your file structure should look like this

<project folder>
  |-->modules
  |       |-->module1
  |         |- main.tf
  |- cdktf.json
  |- main.py

5. add module reference to cdktf.json

"terraformModules": [
    {
      "name": "module1",
      "source": "./modules/module1"
    }
  ],  

6. run cdktf get
7. update stack to use module

#!/usr/bin/env python

from cdktf import App, TerraformStack
from constructs import Construct

from imports.aws import AwsProvider

from imports.module1 import Module1


class MyStack(TerraformStack):

    def __init__(self, scope: Construct, ns: str):
        super().__init__(scope, ns)

        AwsProvider(self, 'Aws', region="us-east-1")

       Module1(
           self,
           "cdktf_test_bucket",
       )

app = App()

stack = MyStack(app, "cdktf_test_stack")
app.synth()

8. run cdktf diff

you should get errors similar to

2021-09-02T22:50:45.536] [ERROR] default -
│ Error: Unreadable module directory
│
│ Unable to evaluate directory symlink: CreateFile ..\modules: The system
│ cannot find the file specified.


│ Error: Failed to read module directory
│
│ Module directory  does not exist or cannot be read.

[jsteinich]

Seems like a duplicate of #664

[javiortizmol]

Seems like a duplicate of #664

maybe was fixed for typescript, but currently I'm getting the same error using python.

Seems like a duplicate of #664

I'm getting the same error and I have the same configuration of @JonCubed on python, the module binding generation is not working on this case, at least it should copy the hcl module into the .terraform/modules path as works for the modules that come from the terraform registry.

I'm attaching the modules.json generated on .terraform/modules path:

{"Modules":[{"Key":"my-ec2","Source":"./modules/compute","Dir":""},{"Key":"","Source":"","Dir":"."},{"Key":"My-Vpc","Source":"terraform-aws-modules/vpc/aws","Version":"3.7.0","Dir":".terraform/modules/My-Vpc"}]}

Please accept the fix that @JonCubed created.

[jsteinich]

Seems like a duplicate of #664

maybe was fixed for typescript, but currently I'm getting the same error using python.

Apologizes. I wasn't trying to imply that it was fixed; just that there is another issue that is also tracking the same problem.

the module binding generation is not working on this case, at least it should copy the hcl module into the .terraform/modules path as works for the modules that come from the terraform registry.

I don't think I'm completely following what is going wrong. Module binding generation is controlled by adding modules to cdktf.json and running cdktf get. No generation is done for TerraformHclModule. The value is just used as it is in Terraform, though with some complications for relative paths. If you could provide a snippet of your cdktf.json and module references in code, we'd be happy to look into it further.

[javier-caylent]

This is the cdktf.json

{
  "language": "python",
  "app": "pipenv run python main.py",
  "terraformProviders": [
    "aws@~> 3.57.0"
  ],
  "terraformModules": [
    "terraform-aws-modules/vpc/aws",
    {
      "name": "my-compute-module",
      "source": "./modules/compute"
    }
  ],
  "codeMakerOutput": "imports",
  "context": {
    "excludeStackIdFromLogicalIds": "true",
  "allowSepCharsInLogicalIds": "true"
  }
}

The cdktf get command is working, I mean it is generating the python code of my terraform module and I'm able to use it on the main.py.

The issue exists in the source tag (cdk.tf.json), that the cdktf is setting when you run the cdktf synth command, putting there a wrong path to the terraform module.

This the tree of the cdktf.out folder

.
├── manifest.json
└── stacks
    └── tf-cdk
        └── cdk.tf.json

This is the snippet code of the cdk.tf.json generated.

  "module": {
    "my-ec2": {
      "additional_tags": {
        "Name": "python-default"
      },
      "ami": "ami-0ab4d1e9cf9a1215a",
      "instance_type": "t2.micro",
      "key_name": "test_omni",
      "subnet_id": "subnet-0e0326a44203eb17f",
      "vpc_security_group_ids": [
        "sg-522b1e16"
      ],
      "source": "../modules/compute", #This path should be "../../../modules/compute"
      "//": {
        "metadata": {
          "path": "tf-cdk/my-ec2",
          "uniqueId": "my-ec2",
          "stackTrace": [
            "new TerraformElement (/private/var/folders/ng/9zhjb19d11l1d0gwng4n5zjr0000gn/T/jsii-kernel-O7KKLl/node_modules/cdktf/lib/terraform-element.js:21:28)",
            "new TerraformModule (/private/var/folders/ng/9zhjb19d11l1d0gwng4n5zjr0000gn/T/jsii-kernel-O7KKLl/node_modules/cdktf/lib/terraform-module.js:19:9)",
            "new MyComputeModule (/private/var/folders/ng/9zhjb19d11l1d0gwng4n5zjr0000gn/T/jsii-kernel-O7KKLl/node_modules/my-compute-module/modules/my-compute-module.js:11:9)",
            "/private/var/folders/ng/9zhjb19d11l1d0gwng4n5zjr0000gn/T/tmpa5h6sv5p/lib/program.js:8367:58",
            "Kernel._wrapSandboxCode (/private/var/folders/ng/9zhjb19d11l1d0gwng4n5zjr0000gn/T/tmpa5h6sv5p/lib/program.js:8795:24)",
            "Kernel._create (/private/var/folders/ng/9zhjb19d11l1d0gwng4n5zjr0000gn/T/tmpa5h6sv5p/lib/program.js:8367:34)",
            "Kernel.create (/private/var/folders/ng/9zhjb19d11l1d0gwng4n5zjr0000gn/T/tmpa5h6sv5p/lib/program.js:8108:29)",
            "KernelHost.processRequest (/private/var/folders/ng/9zhjb19d11l1d0gwng4n5zjr0000gn/T/tmpa5h6sv5p/lib/program.js:9692:36)",
            "KernelHost.run (/private/var/folders/ng/9zhjb19d11l1d0gwng4n5zjr0000gn/T/tmpa5h6sv5p/lib/program.js:9655:22)",
            "Immediate._onImmediate (/private/var/folders/ng/9zhjb19d11l1d0gwng4n5zjr0000gn/T/tmpa5h6sv5p/lib/program.js:9656:46)",
            "processImmediate (node:internal/timers:464:21)"
          ]
        }
      }
    },

I see that you currently are working in a PR to fix it, but you should have under the radar the solution for the asset approach + TerraformHCLModule.

I created a PR for that https://github.com/hashicorp/terraform-cdk/pull/961/files

[jsteinich]

Thanks @javier-caylent. I'm hopeful that we can get this working better for the next release.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.