-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
client: Add option to enable hairpinMode on Nomad bridge #15961
Conversation
shows whether nomad's bridge network has hairpin mode enabled
require.NotPanics(t, func() { out = buildNomadBridgeNetConfig(*tc.b) }) | ||
if tc.name == "bad_name" { | ||
fmt.Println(string(out)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm curious why print the output for "bad_name" ? and not any others?
and I find myself wanting to assert more than just "doesn't panic" but on the fence about what specifically to suggest short of full-text string matching...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The bad_name test was to see if JSON stuffing was a concern. While it appears to be, it will cause the generateAdminChainRule
function to break. This PR doesn't actually introduce the JSON stuffing issue and ultimately, bridge names would benefit from some additional validation since they aren't built until they are needed the first time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed the test since it doesn't actually test whether or not the go_template rendering breaks, as opposed to making broken JSON. Even in the original code, we used %s
inside of quotes rather than leveraging %q
for JSON safety. That could be a nice "make it better than it was" item we could add though.
Co-authored-by: Daniel Bennett <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I might like to see the value-add of extra unit test coverage to confirm we produce valid json, but this is still quite a good, so LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some minor suggestions, but LGTM!
@@ -184,7 +184,7 @@ func newNetworkConfigurator(log hclog.Logger, alloc *structs.Allocation, config | |||
|
|||
switch { | |||
case netMode == "bridge": | |||
c, err := newBridgeNetworkConfigurator(log, config.BridgeNetworkName, config.BridgeNetworkAllocSubnet, config.CNIPath, ignorePortMappingHostIP) | |||
c, err := newBridgeNetworkConfigurator(log, config.BridgeNetworkName, config.BridgeNetworkAllocSubnet, config.BridgeNetworkHairpinMode, config.CNIPath, ignorePortMappingHostIP) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The list of arguments is getting uncomfortably long, I wouldn't oppose a new config struct the be passed here. Although bridgeNetworkConfiguratorConfig
is kind of weird 😬
Another option would be to pass config
, but that has the downside of not making it clear which config values are relevant.
@@ -54,7 +58,7 @@ func newBridgeNetworkConfigurator(log hclog.Logger, bridgeName, ipRange, cniPath | |||
b.allocSubnet = defaultNomadAllocSubnet | |||
} | |||
|
|||
c, err := newCNINetworkConfiguratorWithConf(log, cniPath, bridgeNetworkAllocIfPrefix, ignorePortMappingHostIP, buildNomadBridgeNetConfig(b.bridgeName, b.allocSubnet)) | |||
c, err := newCNINetworkConfiguratorWithConf(log, cniPath, bridgeNetworkAllocIfPrefix, ignorePortMappingHostIP, buildNomadBridgeNetConfig(*b)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
c, err := newCNINetworkConfiguratorWithConf(log, cniPath, bridgeNetworkAllocIfPrefix, ignorePortMappingHostIP, buildNomadBridgeNetConfig(*b)) | |
c, err := newCNINetworkConfiguratorWithConf(log, cniPath, bridgeNetworkAllocIfPrefix, ignorePortMappingHostIP, buildNomadBridgeNetConfig(b)) |
And then you receive a pointer in buildNomadBridgeNetConfig
.
tmpl, err := template.New("cniConf").Parse(nomadCNIConfigTemplate) | ||
if err != nil { | ||
// Panic on error for catching issues in testing | ||
panic(err) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you can move this to the top of file so the template is parse just once at initialization, and the use Must
to panic, like:
var nomadBridgeTmpl = template.Must(template.New("cniConf").Parse(nomadCNIConfigTemplate))
// TODO: Consider exporting these directly from bridgeNetworkConfigurator | ||
// so they aren't repeated in the input struct | ||
type templInput struct { | ||
AllocSubnet string | ||
BridgeName string | ||
HairpinMode bool | ||
CNIAdminChainName string | ||
} | ||
|
||
tIn := templInput{ | ||
AllocSubnet: b.allocSubnet, | ||
BridgeName: b.bridgeName, | ||
HairpinMode: b.hairpinMode, | ||
CNIAdminChainName: cniAdminChainName, | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah cool, so yeah, I think having a config struct passed to bridgeNetworkConfigurator
could help here as well.
err = tmpl.Execute(&out, tIn) | ||
if err != nil { | ||
// Panic on error for catching issues in testing | ||
panic(err) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The docs are not very clear in all possible error modes here, so I think it may be better to return err
and let the allocrunner handle that? It may be a transient issue that a retry would fix.
- `bridge_network_hairpin_mode` `(bool: false)` - Specifies if hairpin mode | ||
is enabled on the network bridge created by Nomad for allocations running | ||
with bridge networking mode on this client. You may use the corresponding | ||
node attribute `nomad.bridge.hairpin_mode` in constraints. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- `bridge_network_hairpin_mode` `(bool: false)` - Specifies if hairpin mode | |
is enabled on the network bridge created by Nomad for allocations running | |
with bridge networking mode on this client. You may use the corresponding | |
node attribute `nomad.bridge.hairpin_mode` in constraints. | |
- `bridge_network_hairpin_mode` `(bool: false)` - Specifies if hairpin mode | |
is enabled on the network bridge created by Nomad for allocations running | |
with bridge networking mode on this client. You may use the corresponding | |
node attribute `nomad.bridge.hairpin_mode` in constraints. When hairpin mode | |
is enabled allocations are able to reach their own IP and port. Changing this | |
value requires a reboot of the client host to take effect. |
I think? 😅
Feel free to modify any of this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For now, I think I'd like to omit the additional text so we can get the merge done. Then test the behavior around this on a running system, and then if needed come back and add it in a followup PR. I've added #16023 to track this
Switch back to sprintf implementation to prevent bonus complexity around error handling. Updated string to use %q in places where the user can supply a string value that could break the JSON. Added a valid JSON check to the test.
Awesome work! |
Hi @A-Helberg this shipped in Nomad 1.5.0-beta.1 and will ship in the GA release. |
bridge_network_hairpin_mode
client config setting