atlantis testdrive failed with 422 Validation Failed [{Resource:PullRequest Field:base Code:invalid Message:}] #3092

pomcho555 · 2023-02-01T14:45:19Z

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

I tried atlantis testdriv with following the instruction. But I couldn't let atlantis create a PR, even though I give a personal GitHub token full access.

Reproduction Steps

./atlantis testdrive

Logs

$ ./atlantis testdrive

Welcome to Atlantis testdrive!

This mode sets up Atlantis on a test repo so you can try it out. We will
- fork an example terraform project to your username
- install terraform (if not already in your PATH)
- install ngrok so we can expose Atlantis to GitHub
- start Atlantis

Press Ctrl-c at any time to exit


github.com username: pomcho555

To continue, we need you to create a GitHub personal access token
with "repo" scope so we can fork an example terraform project.

Follow these instructions to create a token (we don't store any tokens):
https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token#creating-a-fine-grained-personal-access-token
- use "atlantis" for the token description
- add "repo" scope
- copy the access token

GitHub access token (will be hidden):
=> forking repo
=> fork completed!
=> terraform found in $PATH at /usr/local/bin/terraform
=> ngrok not found in $PATH.
=> downloading ngrok
=> downloaded ngrok successfully!
=> creating secure tunnel
=> started tunnel!
=> starting atlantis server
⠙=> atlantis server is now securely exposed at https://80fb-113-36-202-140.ngrok.io

=> creating atlantis webhook
=> atlantis webhook created!
=> creating a new pull request
⠹Error: creating new pull request for repo pomcho555/atlantis-example: POST https://api.github.com/repos/pomcho555/atlantis-example/pulls: 422 Validation Failed [{Resource:PullRequest Field:base Code:invalid Message:}]

Usage:
  atlantis testdrive [flags]

Flags:
  -h, --help   help for testdrive

Environment details

Version: atlantis 0.22.3
OS: macOS Ventura 13.0.1 (22A400)

Additional Context

I tried a solution shared in testdrive 422 Validation Failed #2329, but it failed also.

The text was updated successfully, but these errors were encountered:

nitrocode · 2023-02-01T14:57:38Z

I recall having issues with it recently as well.

This could also be related to this issue

@tekumara did you have to do anything special to get testdrive to work in #2329? If so, please tell us and we can document it. Does it work for you using the latest version?

tekumara · 2023-02-13T03:52:58Z

This is what I did to get it working.

nitrocode · 2023-02-13T13:24:26Z

Hmm. I'll have to give this a shot again. I recall also hitting an issue when running testdrive. Thank you for adding that @tekumara

iamgeef · 2023-03-03T02:49:25Z

I've just downloaded the latest release and am getting the same 422 Validation Failed error, even after deleting the existing atlantis fork - it still occurs

Paulius0112 · 2023-03-11T22:24:29Z

Getting into the same problem with the latest tag

Rjerk · 2023-03-15T09:41:28Z

I used an older release and ran testdrive successfully.

This release can work: https://github.com/runatlantis/atlantis/releases/download/v0.19.8/atlantis_linux_amd64.zip

mdmelo · 2023-03-18T20:05:02Z

you also need to downgrade to ngrok 2.x if running ngrok v3.x, i had 3.2.1 installed and ngrok didnt like the config file created by atlantis. you can manually upgrade the ngrok config in /tmp to verify this. downgrade to ngrok v2.3.41 worked around this issue.

this

web_addr: localhost:41414
tunnels:
  atlantis:
    addr: 4141
    bind_tls: true
    proto: http

needs to be this for use with ngrok v3

web_addr: localhost:41414
tunnels:
    atlantis:
        addr: 4141
        schemes:
            - https
        proto: http
version: "2"
region: us

vquie · 2023-03-25T09:25:19Z

I used an older release and ran testdrive successfully.

This release can work: https://github.com/runatlantis/atlantis/releases/download/v0.19.8/atlantis_linux_amd64.zip

Thanks for the hint. On macOS it worked with this one: https://github.com/runatlantis/atlantis/releases/download/v0.19.9/atlantis_darwin_arm64.zip
Did not try the 0.20.x though but 0.21.x was not working.

nitrocode · 2023-03-25T14:08:39Z

So 0.19.9 is from October 2022

Here are all the changes in the testdrive module

https://github.com/runatlantis/atlantis/commits/3212fe9cbdb831b0bf1ffd4e2851fbfed4215e73/testdrive

Most of the changes seem to be package upgrades

The pr issue seems to be related to here

atlantis/testdrive/testdrive.go

Lines 255 to 260 in 3212fe9

    
           pullRequestURL, err := githubClient.CreatePullRequest(githubUsername, terraformExampleRepo, "example", "main") 
        
           if err != nil { 
        
           	return errors.Wrapf(err, "creating new pull request for repo %s/%s", githubUsername, terraformExampleRepo) 
        
           } 
        
           s.Stop() 
        
           colorstring.Println("[green]=> pull request created![reset]")

atlantis/testdrive/github.go

Lines 73 to 98 in 3212fe9

    
           func (g *Client) CreatePullRequest(ownerName string, repoName string, head string, base string) (string, error) { 
        
           	// First check if the pull request already exists. 
        
           	pulls, _, err := g.client.PullRequests.List(g.ctx, ownerName, repoName, nil) 
        
           	if err != nil { 
        
           		return "", err 
        
           	} 
        
           	for _, pull := range pulls { 
        
           		if pull.Head.GetRef() == head && pull.Base.GetRef() == base { 
        
           			return pull.GetHTMLURL(), nil 
        
           		} 
        
           	} 
        
           	// If not, create it. 
        
           	newPullRequest := &github.NewPullRequest{ 
        
           		Title: github.String("Welcome to Atlantis!"), 
        
           		Head:  github.String(head), 
        
           		Body:  github.String(pullRequestBody), 
        
           		Base:  github.String(base), 
        
           	} 
        
           	pull, _, err := g.client.PullRequests.Create(g.ctx, ownerName, repoName, newPullRequest) 
        
           	if err != nil { 
        
           		return "", err 
        
           	} 
        
           	return pull.GetHTMLURL(), nil

Upstream example

https://github.com/google/go-github/blob/31350112a17a67bdc672f2a73865a2985b15fd60/example/commitpr/main.go#L164-L175

Maybe the base should be master instead of main? We changed to main in Nov 2022 and this pr changed the base to main in testdrive #2662 (v0.21.0).

Perhaps the fork doesnt have a main branch?

atlantis/testdrive/testdrive.go

Lines 39 to 40 in 3212fe9

    
           var terraformExampleRepoOwner = "runatlantis" 
        
           var terraformExampleRepo = "atlantis-example"

atlantis/testdrive/testdrive.go

Lines 117 to 121 in 3212fe9

    
           colorstring.Println("\n=> forking repo ") 
        
           s.Start() 
        
           if err := githubClient.CreateFork(terraformExampleRepoOwner, terraformExampleRepo); err != nil { 
        
           	return errors.Wrapf(err, "forking repo %s/%s", terraformExampleRepoOwner, terraformExampleRepo) 
        
           }

Yep looks like the example is still set to master

https://github.com/runatlantis/atlantis-example

@lkysow could you add a main branch for now to the above? I do not have permission.

We could also update testdrive to always retrieve the base branch of the fork prior to creating the pull request.

The ngrok version is interesting

atlantis/testdrive/utils.go

Line 39 in 3212fe9

const ngrokDownloadURL = "https://bin.equinox.io/c/4VmDzA7iaHb"

atlantis/testdrive/testdrive.go

Lines 154 to 157 in 3212fe9

    
           colorstring.Println("=> downloading ngrok") 
        
           s.Start() 
        
           ngrokURL := fmt.Sprintf("%s/ngrok-stable-%s-%s.zip", ngrokDownloadURL, runtime.GOOS, runtime.GOARCH) 
        
           if err = downloadAndUnzip(ngrokURL, "/tmp/ngrok.zip", "/tmp"); err != nil {

Latest stable is https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-v3-stable-linux-amd64.tgz from ngrok website. The new url contains a v3 meaning the testdrive must be using the previous stable which I assume is v1/v2. Maybe we should see what changes need to be made to make v3 compatible.

Here is the current ngrok config that's v1/v2 compatible and breaking for v3

atlantis/testdrive/testdrive.go

Lines 175 to 183 in 3212fe9

    
           	ngrokConfig := fmt.Sprintf(` 
        
           version: 1 
        
           web_addr: %s 
        
           tunnels: 
        
             atlantis: 
        
               addr: %d 
        
               bind_tls: true 
        
               proto: http 
        
           `, ngrokAPIURL, atlantisPort)

Seems like the config could be run through an upgrade command

ngrok config check
ngrok config upgrade

Or use the config posted above by @mdmelo

-version: 1
+version: "2"
+region: us
 web_addr: %s 
 tunnels: 
   atlantis: 
     addr: %d
-    bind_tls: true
+    schemes:
+    - https
     proto: http

nitrocode · 2023-03-27T22:42:50Z

I asked @lkysow to update the atlantis-example and get did (thanks Luke) repo with a main branch until the above changes can be fixed. This won't fix ngrok needing to be v1 or v2 but it will fix opening the pr against the branch.

https://github.com/runatlantis/atlantis-example/tree/main

nitrocode · 2023-05-04T02:07:28Z

Is this still an issue or can it be closed?

pomcho555 · 2023-05-07T06:14:26Z

It works well with v0.23.5. So I think you can close this issue.

pomcho555 · 2023-05-07T06:19:02Z

BTW, if you re-try atlantis testdrive with already forked atlantis-example repo, it may spit out the following error.

=> creating a new pull request
⠙Error: creating new pull request for repo pomcho555/atlantis-example: POST https://api.github.com/repos/pomcho555/atlantis-example/pulls: 422 Validation Failed [{Resource:PullRequest Field:base Code:invalid Message:}]

To solve this, you need to remove the existing repo.

nitrocode · 2023-05-07T12:08:47Z

You can delete your fork or just create the main branch there and it should work, no?

nitrocode · 2023-05-07T12:19:54Z

I suppose to avoid this error we should also avoid hard coding the target branch name here.

I don't see the CreatePullRequest in the current golang github sdk. I do see this similar Create function which doesn't specify the branch name which we could use instead.

https://github.com/google/go-github/blob/1b6f3a38bfe4ec346f64f5fc66cba5e4f254e94a/github/pulls.go#L257

We should also bump the sdk for testdrive to match the v52 in atlantis server

pomcho555 added the bug Something isn't working label Feb 1, 2023

nitrocode added the Stale label May 4, 2023

nitrocode closed this as completed May 7, 2023

nitrocode removed the Stale label May 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

atlantis testdrive failed with 422 Validation Failed [{Resource:PullRequest Field:base Code:invalid Message:}] #3092

atlantis testdrive failed with 422 Validation Failed [{Resource:PullRequest Field:base Code:invalid Message:}] #3092

pomcho555 commented Feb 1, 2023 •

edited by nitrocode

Loading

nitrocode commented Feb 1, 2023 •

edited

Loading

tekumara commented Feb 13, 2023

nitrocode commented Feb 13, 2023

iamgeef commented Mar 3, 2023

Paulius0112 commented Mar 11, 2023

Rjerk commented Mar 15, 2023

mdmelo commented Mar 18, 2023 •

edited

Loading

vquie commented Mar 25, 2023

nitrocode commented Mar 25, 2023 •

edited

Loading

nitrocode commented Mar 27, 2023 •

edited

Loading

nitrocode commented May 4, 2023

pomcho555 commented May 7, 2023

pomcho555 commented May 7, 2023

nitrocode commented May 7, 2023

nitrocode commented May 7, 2023

atlantis testdrive failed with 422 Validation Failed [{Resource:PullRequest Field:base Code:invalid Message:}] #3092

atlantis testdrive failed with 422 Validation Failed [{Resource:PullRequest Field:base Code:invalid Message:}] #3092

Comments

pomcho555 commented Feb 1, 2023 • edited by nitrocode Loading

Community Note

Overview of the Issue

Reproduction Steps

Logs

Environment details

Additional Context

nitrocode commented Feb 1, 2023 • edited Loading

tekumara commented Feb 13, 2023

nitrocode commented Feb 13, 2023

iamgeef commented Mar 3, 2023

Paulius0112 commented Mar 11, 2023

Rjerk commented Mar 15, 2023

mdmelo commented Mar 18, 2023 • edited Loading

vquie commented Mar 25, 2023

nitrocode commented Mar 25, 2023 • edited Loading

nitrocode commented Mar 27, 2023 • edited Loading

nitrocode commented May 4, 2023

pomcho555 commented May 7, 2023

pomcho555 commented May 7, 2023

nitrocode commented May 7, 2023

nitrocode commented May 7, 2023

pomcho555 commented Feb 1, 2023 •

edited by nitrocode

Loading

nitrocode commented Feb 1, 2023 •

edited

Loading

mdmelo commented Mar 18, 2023 •

edited

Loading

nitrocode commented Mar 25, 2023 •

edited

Loading

nitrocode commented Mar 27, 2023 •

edited

Loading