Add a "fully utilised" toggle to IP ranges

[nogaff]

NetBox version

3.0.10

Feature type

Change to existing functionality

Proposed functionality

1. The calculation for prefix utilisation should not assume that child IP ranges are fully utilised. The default assumption should be that an IP range is not utilised other than by defined IP addresses.
2. IP ranges should have a checkbox to toggle on a "fully utilised" state, (similar to the "is pool" state on prefixes). This "fully utilised" state should cause the parent prefix to evaluate the IP range as having 100% utilisation.

Use case

Currently the calculation for prefix utilisation assumes that any and all child IP ranges have 100% utilisation. Thus, if a prefix is fully populated with consecutive IP ranges, it will always show 100% utilisation, regardless of the number of defined IP addresses.

That assumption may make sense if an IP range is used for DHCP since NetBox cannot know how much of the DHCP pool is consumed, but if an IP range is reserved for devices with static IP addresses and has spare capacity, that 100% utilisation assumption is no longer valid and gives an incorrect overview of the prefix.

The proposed change will allow a static IP range to be properly documented without falsely representing the utilisation of the parent prefix, while still allowing a DHCP pool to be defined as a fully utilised IP range. Take the following example:

Prefix 10.0.0.1/24 contains:

• IP Range 10.0.0.1-100: static IP addresses, 30 connected devices
• IP Range 10.0.0.101-254: DHCP pool

Prefix utilisation in current NetBox version: 100%
Prefix utilisation with proposed changes: 73%

Furthermore, this change would not take away the possibility of recreating the current behaviour, because if a user wishes to mark a static IP range as fully utilised, ignoring any defined IP addresses, then the proposed checkbox will allow them to do so.

Database changes

I'm not familiar with the database schema but I would assume the addition of a "fully utilised" checkbox field would require a new boolean column in the ipam_iprange table.

External dependencies

No response

[jeremystretch]

The calculation for prefix utilisation should not assume that child IP ranges are fully utilised.

I'm not sure I agree Suppose I have a prefix 192.0.2.0/24, within which I've designated a range 192.0.2.1 through 192.0.2.20. When considering the utilization of the prefix, shouldn't we treat those IPs as utilized?

[DanSheps]

My view is the same as @jeremystretch's.

I understand some people may have a different use case, however this is going to be one of those things where 50% of the people feel one way, and 50% of the people feel the other. Both are valid viewpoints and equally right, we just have to choose the best one.

I think from our perspective, we have the best method chosen, as you can get easily get the utilization infromation by recursing to the point you need but determining prefix utilization if we take away the existing functionality requires queries against the different API's (IP Address, IP Range, IP Prefix) and calculations to determine the utilization. What we have now is the "safer" method.

[nogaff]

Trying to be objective for a moment and set aside what I'd personally find useful, I'm afraid I have to disagree with the above replies from a logical viewpoint as well.

This really comes down to how you define the word "utilised", but in this context, I would say "utilised" is synonymous with "allocated to or consumed by a device", (as born out by the behaviour of prefixes prior to the addition of IP ranges). Would anyone contest that?

If the very definition of the word is disputed depending on where it's being applied, then we may as well close this now, because the only way a decision could ever be made would be by community vote, or by decree of the project maintainers. 😄

But if we do accept that definition, then an IP range which does not have all of its individual IP addresses assigned to specific devices cannot possibly be considered "fully utilised". It is either partially utilised, or unutilised, and I believe the prefix should therefore treat it as such.

However, @jeremystretch seems to be taking the view that an IP range should be considered fully utilised purely as a result of acknowledging its conceptual existence, which seems logically questionable to say the least. The fact that you've decided that a range of 192.0.2.1-20 is to be used for *some purpose* does not implicitly mean those 20 IP addresses have now been consumed. Each of them is still available until such time as a device claims them or they are explicitly reserved in some way.

Yes, we know that a DHCP pool presents a unique case due to the unknown consumption of IP addresses within the pool, and we know there are scenarios where you would want to reserve an entire IP range for use by a known set of devices in the near future, but those are exactly the situations that the proposed checkbox would cater for.

determining prefix utilization if we take away the existing functionality requires queries against the different API's (IP Address, IP Range, IP Prefix) and calculations to determine the utilization. What we have now is the "safer" method.

Admittedly I'm unfamiliar with the code of this project, but I don't see any negatives there. Surely that just means some conditions and a bit of extra math are required. Where does "safe" come into it? (Genuine question; not meant rhetorically).

I think from our perspective, we have the best method chosen, as you can get easily get the utilization infromation by recursing to the point you need

Yes, that's easy... until someone goes and defines a /16 prefix containing 200 ranges, and suddenly has to add up 200 individual utilisations to figure out what capacity should be left in the prefix (even more difficult if the ranges are not of equal size). I'm not suggesting that's likely, but people do weird things, (and that's their prerogative).

I'm of the belief that the code shouldn't care what the user is doing; it should accurately present the data the user has provided, however bizarre it might be.

[jeremystretch]

However, @jeremystretch seems to be taking the view that an IP range should be considered fully utilised purely as a result of acknowledging its conceptual existence, which seems logically questionable to say the least. The fact that you've decided that a range of 192.0.2.1-20 is to be used for some purpose does not implicitly mean those 20 IP addresses have now been consumed.

Let's say I want to find a /24 prefix that has room for a range of 100 IPs. I scroll down the list of prefixes and find one that shows only 10% utilization. Great, that should have room! But wait: it already has a range of 200 IPs defined inside it, so that won't work. Let me try another one. Nope, that one also already has some ranges carved out and not enough room left. I keep moving down the list, checking prefixes one at a time until I find one that actually has enough room. Boy, this sure would have been easier if NetBox just counted the designated ranges as utilized space.

That's why NetBox reports IP ranges as utilized space within the parent prefix.

[nogaff]

Boy, this sure would have been easier if NetBox just counted the designated ranges as utilized space.

Then set the "fully utilised" checkbox on those ranges if that's your use case. Problem solved!

I see where you're coming from, but your argument is based on what information the user is looking to extrapolate from the data held in the database, which is anyone's guess.

You want to quickly find a prefix with spare capacity for an extra range. With the checkbox enabled, you can.

I want to quickly see if any of my prefixes are running out of capacity and will need to be expanded soon. With the checkbox disabled, I can.

With no checkbox at all, only your use case is possible.

If the option is there, then the user can decide what they want to get out of it for themselves.

[jeremystretch]

Then set the "fully utilised" checkbox on those ranges if that's your use case. Problem solved!

But each range itself is not fully utilized, so this would be incorrect.

There's really no need to discuss this further. The scope of this FR is to allow marking an IP range as fully utilized, which has been accepted. We will not be altering the manner in which NetBox calculates prefix utilization.

[nogaff]

But each range itself is not fully utilized, so this would be incorrect.

That's certainly true, but at that point it would be down to the user to decide how they wish to use the tools provided, rather than having no choice in the matter.

There's really no need to discuss this further. The scope of this FR is to allow marking an IP range as fully utilized, which has been accepted. We will not be altering the manner in which NetBox calculates prefix utilization.

Fair enough. Personally, I see little value in being able to mark an IP range as fully utilised if it changes nothing about how the data is presented at the prefix level. That was the entire point of this FR, (which was created in response to another user reporting essentially the same issues I've raised). Full utilisation at the IP range level is likely to be obvious from any roles/tags/descriptions that have been entered.

Still, I accept your decision.

Perhaps there is another way to address this issue in a separate FR, by making the prefix display itself configurable (i.e. an option to choose how the prefix utilisation is calculated), if you'd be open to that idea?

Otherwise, I'll just drop it here and say thank you for your time!

[DanSheps]

Marking this as needs milestone.

Scope of this FR:

1. Add boolean field to the model to mark a IP Range as fully utilized
2. Add related forms/fields to form classes and views
3. Add/adjust tests where applicable

[teixemf]

Marking this as needs milestone.

Scope of this FR:

1. Add boolean field to the model to mark a IP Range as fully utilized
2. Add related forms/fields to form classes and views
3. Add/adjust tests where applicable

Since the current behavior is to increment the Prefix Utilization bar when an IP Range is created, I suggest make the boolean fully utilized=True by default.

[DanSheps]

As mentioned, this is not going to change the manner in which we calculate prefix utilization.

[bluikko]

I'm not sure I agree Suppose I have a prefix 192.0.2.0/24, within which I've designated a range 192.0.2.1 through 192.0.2.20. When considering the utilization of the prefix, shouldn't we treat those IPs as utilized?

Doesn't that depend on how the range is used?

For me ranges are usually not used as "pools". They are used to document parts of a subnet - and only for informative purposes both for humans and for automation for assigning IP addresses to machines.
For example:

• I have a prefix 192.0.2.0/26.
• Out of 192.0.2.0/26, IP addresses .0 to .19 are to be used by front-end servers.
• Out of 192.0.2.0/26, IP addresses .20 to .29 are to be used by middleware servers.
• Out of 192.0.2.0/26, IP addresses .30 to .34 are to be used by loadbalancers.
• ... and so on.
• IP addresses inside of the range, say .0 to .19, may have some IP addresses assigned to the front-end servers, and it may have several IP addresses still free reserved for say automatic elasticity for spinning up more VMs on demand.
• Out of the IP addresses inside the subnet, some are used and some are free but it is partitioned for different uses for a stack of servers.
• Each server in the subnet has IP address configured with the actual /26 prefix length since they need to talk to each other and that is the real subnet.

[jeremystretch]

I'll admit that after re-reading this thread a year later, I'm more convinced by @nogaff's arguments. I do still have concerns about the change's impact on allocating new ranges, but setting mark_utilized to True would effectively retain the existing behavior, so it's probably not that important.

[jeremystretch]

For now, I'm going to keep this as-is, to mirror the behavior of the mark_utilized toggle on the Prefix model. I'm open to possibly introducing a second toggle to influence how a range counts toward the utilization of its parent prefix(es), but that's a longer conversation and should entail discussion of the prefix model as well.

[nogaff]

I'm open to possibly introducing a second toggle to influence how a range counts toward the utilization of its parent prefix(es), but that's a longer conversation and should entail discussion of the prefix model as well.

@jeremystretch Sounds rather like the other FR I opened (#8263) which got automatically closed due to inactivity, although I'm not sure whether you're thinking of a toggle at the prefix level, or at the IP range level. Perhaps it's worth reopening that one for discussion?