Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix adapter JSON tests to have the right test structure #1589

Merged
merged 19 commits into from
Dec 2, 2020

Conversation

mansinahar
Copy link
Contributor

We recently discovered that the RunJSONBidderTest function, responsible for running the adapter specific JSON tests does not complain if the JSON test doesn't follow the required structure and we also found that a lot of the adapters weren't actually following the required struct, which meant those bid responses weren't actually getting asserted while running tests.

This PR fixes the JSON tests for all adapters except Beachfront (Beachfront is addressed in #1578)

I updated the RunJSONBidderTest function minimally on my local machine to actually complain when a JSON test doesn't adhere with the required structure and fixed them accordingly. However, I am not pushing the RunJSONBidderTest changes as part of this PR as it will likely be harder to review and so will push those in a follow-up PR.

@SyntaxNode SyntaxNode requested review from bsardo and hhhjort November 17, 2020 15:22
@mansinahar mansinahar mentioned this pull request Nov 20, 2020
@SyntaxNode SyntaxNode requested review from guscarreon and removed request for hhhjort November 23, 2020 15:18
@SyntaxNode SyntaxNode assigned guscarreon and unassigned hhhjort Nov 23, 2020
Copy link
Collaborator

@bsardo bsardo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good, just some nitpicks.

"bid": {
"adm": "<div id=\"123456789_ad\"><script>!function(){console.log\"Hello, world.\";}();<\/script><\/div>",
"id": "uuid",
"impid": "uuid",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be "impid": "some_test_ad_id",?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question but looks their adapter modifies the impid in the bid to id
Ref: https://github.com/prebid/prebid-server/blob/master/adapters/beintoo/beintoo.go#L206

Comment on lines 13 to 17
"protocols": [
1,
2,
5
],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious, are you putting every element of a json array on its own line by choice or is that something that is happening automatically via a json formatter?
IMO small lists of integers are easier to read if it's all on one line. I'm not saying we need to revert these type of changes; I'd just like to understand our stance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I agree with you. Small lists like this can be on their own line. This for me happened automatically via a JSON formatter. I tried to play around with it's settings to see if I can configure it to not do that but didn't have much luck. Changed it manually for this file but there could be others that still have such lists auto-formatted.

If you know of a VSCode JSON formatter where I can control this, please lemme know. I will be happy to have that in my setup and will also update all the other files.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which formatter are you using?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use Beautify.

Copy link
Contributor

@guscarreon guscarreon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me just one suggestion: In order to avoid future adapters from misspelling one of the json fields ("mockBidRequest", "httpCalls", "expectedBidResponses", "expectedMakeRequestsErrors", or "expectedMakeBidsErrors") do you think it'd be convenient to add a check? I believe a validation would be helpful to make sure the tests are actually asserting the fields. Something inside adapters/adapterstest/test_json.go that could look like:

func verifyJsonTestFields(spec testSpec, filename string) error {
    emptyBidRequest := openrtb.BidRequest{}

    // Verify bid request is not empty
    if cmp.Equal(spec.BidRequest, emptyBidRequest) {
        return fmt.Errorf("Json field mockBidRequest should not be empty. Test %s\n", filename)
    }

    // Verify HttpCalls is not empty
    if len(spec.HttpCalls) == 0 && len(spec.MakeRequestErrors) == 0 {
        return fmt.Errorf("We expect either httpCalls or expectedMakeRequestsErrors to be empty but not both. Test %s\n", filename)
    }

    return nil
}
adapters/adapterstest/test_json.go

Thoughts?

@mansinahar
Copy link
Contributor Author

@guscarreon Yes, that's something that I am gonna put as a follow-up PR. Didn't want to overload this further so kept it limited to just modifying the JSON tests themselves but yeah that one is coming soon :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants