DNS record management

[ryanmerolle]

Basic support for DNS resolution for mapping individual IPs to provide additional metadata/logic to assist with searches and investigation of individual IP usage.

[shaneshort]

At this point I'd be happy to just have a hostname field for the specific IP address, I think that would handle my needs.

[x-zeroflux-x]

I think that adding the host name field will be needed as part of the solution anyway. As you will need to be able to place the dns record somewhere within the platform. Probably would not hurt to add that now and then tie the hostname field into the dns integration later when this gets looked at.

[ventris]

Adding a hostname field would improve the use for the IPAM by itself and would simplify adding new records.

[jeremystretch]

I'm happy to add a hostname field to the IPAddress model if it won't conflict with any future plans for DNS support. I haven't given much thought to how that would be implemented though.

[rdujardin]

What we could do for instance :

• Having a char field 'hostname' in IPAddress, for reverse DNS
• Having an object DNSZone storing SOA data, with fields : name (charfield), ttl (integer), soaname (char), soacontact (char), serial (char), refresh (integer), retry (integer), expire (integer), minimum (integer) ; all fields required
• Having an object DNSRecord with fields : name (charfield), type (charfield : MX, A, AAAA, CNAME, etc), priority (integer), a foreign key on DNSZone, a foreign key on IPAddress, and if IP is not needed for instance in a CNAME record, a value charfield ; only type, zone and either ip or char value are required

A question is : in DNS zone files, records can have empty name, which means the record designate the origin of the zone, how can we take this into account ? we can allow name to be empty but then how records could be selected in list ? or we can enter '@' in name to mean origin.

What do you think about it ?

[x-zeroflux-x]

The hostname field at the very least will not impact future dns implementations as long as you want to connect the DNS to the device name and not have it entirely as its own module under a new submenu, This really plays out on how you would like it to work flow visually.

I have an idea in response to @rdujardin which i think is really beneficial to us as service providers.
Bare with me as i explain as i dump my thoughts onto this post.

We should do something that no one else does well or like us have to use custom scripts to cron and check records to auto create zones for customers that are not provisioned. That is to auto generate PTR records and insert them into DNS.

How to handle default PTR record creation?
Using Subnet 10.0.0.0/24, An example of this would be as follows;

• Add Subnet to DCIM
• Have option to auto provision PTR records for subnet and insert them into DNS platform
• Generation based on a set standard of subnet in reverse and a domain name that is user configured.
  An example of how this may look for the host .1 would be 1.0.0.10-in.addr.usersetdomain.com

When adding a subnet into DCIM have an option to automatically setup default PTR records
This then would automatically login to DNS, Check for existing record and if none exist, then create a default set of records for the subnet, if exist - perhaps ignore or show popup modal with some options on how to handle that.

How do we handle linking to devices and assigning / un-assigning resolution?
From there it gets a bit more involved as you will need to have certain checks and references back to existing records and default records within DNS. If we said that the best way to manage work flow is by using the hostname field on the device as being the main insertion point for PTR records, which i am not sure how this would work yet as this would need to be able to support multiple PTR records for multiple IPs assigned to the device, but say this is what was used as a reference.

Adding a PTR record
Edit the device that is located in the rack and add custom PTR record into hostname field. This maybe smtp.mailserver.com. This process would need to check if any record exists if none exist then create as per hostname field entry.

Removing a PTR record - Removing Device
If you are to remove the device from the rack this would need to remove the PTR record as part of this removal process. I would suggest handling this either way;

If subnet has the auto generate default records selected, then removing this device would remove this PTR entry and repopulate this with the default record used for that IP.

if subnet does not have the auto generate records selected, removing this device should remove the PTR record from DNS and leave this record empty in the DNS server.

Changing a PTR record
Changing a record should be as simple as value is X please change to Y. If value does not exist in DNS server then populate with Y anyway.

That is all i have thought about thus far - and i am by no means an expert on how this could look and feel and work throughout DCIM, perhaps others may want to add some value to this topic as there maybe a different way to look at this?

[rdujardin]

I don't know if I have well understood your message, but a DNS should be associated to an IPAddress, not to a device. The device has a primary IP and so it's already indirectly associated to a hostname. In the "reality", i.e. in bind zone files, we only associate hostnames and IPs, there are no devices.

We can make a new DNS menu, it's a menu more but DNS is essential for potentially a big part of users. IPAM menu is already quite full.

About the PTR auto populating, it's a good idea. About the automatically connecting to DNS server, it's maybe quite overkill ? Netbox provides a readonly API, which can be used in external scripts to export configs, it may be sufficient ?

What do you think ?

[x-zeroflux-x]

My comments are based on my opinion and do not necessarily represent the industry as a whole, so with that in mind - see below :)

Having Netbox connect to the PowerDNS API - is a major bonus when you are working with thousands of ip addresses. As it just removes a single step of administration which is unnecessary if you use it the way we do. My suggestion while this would benefit us greatly was thought about and taken into consideration that others would benefit from this too if they were using a similar Netbox / PowerDNS setup.

With exporting of files, you maybe also assuming that one would be using bind and/or the non database driven version that uses config files in for example /var/named. I know some providers still may use file based configuration but we personally don't.

If you look at this from a work flow chart from top to bottom, Ideally I would like to see Netbox as the top product in that chart and it can make connections to other platforms below such as DNS / ROUTERS / FIREWALLS, etc not just to have a place to store all your data/assets on your network but to really integrate and manage them. This is obviously a different conversation for a different comment but if you have Netbox making the connections and not the other way around this software becomes an integral part of day to day management and deployment in your business.

Within regards to how it flows within Netbox - I am not sure either which is why i left that open for discussion, other products are attaching ip to device or a way of linking them to the device in the rack which allows you to find most configuration in one view. Netbox may not follow this path and other people may have a better idea on how this may work?

Thoughts?

[shaneshort]

I think that this is probably best left to some kind of plugin based solution. I'd rather something that's a little bit more flexible than hunkering netbox down with stuff that's probably specific to people's own workflow.

I'd suggest not putting too much specific dns zone related information into netbox itself, but rather enough information that the rest of my infrastructure can nut it out based on information inside netbox.

[ryanmerolle]

Yea I would agree with @shaneshort . @rackcentral, you feature seems like we would be getting ahead of ourselves with provisioning functionality before creating the DCIM/IPAM source of truth Stretch set forth to first create. Maybe this is a separate feature later down the road, if not a plugin as @shaneshort mentioned.

[Armadill0]

An integration of the Powerdns API would be a great feature! Core or plugin doesn't matter for me, as long as it works.

And I think the better solution is to attach an IP to an interface. In this way we can have multiple interfaces with different IPs and their associated DNS records. Those are linked to the devices, which allows us to identify a single system via multiple connections.

[x-zeroflux-x]

While i respect what you both have to say, @shaneshort @ryanmerolle.

I think that you can not call Netbox a DCIM, (DataCentre Infrastructure Management) tool, if you do not consider that management is important enough to include in the design of your overall software.

DNS is essential to anyone running a network using a DCIM software solution. You have built the heart of this solution and possibly what could be the main central management tool for service providers moving forward.

This could be an amazing software if you consider the possibility of allowing it to be more involved in the management role of provider networks.

Thanks for your time.

[shaneshort]

@rackcentral I'm not suggesting that what you're chasing isn't useful nor wanted, I'm simply stating that building-in use-case specific things like powerdns isn't the right course of action. Making plugins for those use cases, would be

[rdujardin]

@rackcentral it's true that netbox should be a true management tool and not just a manual database, but for the moment the project seems not to be following this path but to focus on core features. a writable API is planned for version 2.0, so we can imagine for instance making a simple core simply handling the data model, then making more powerful/specific I/O features in plugins using the API.

[x-zeroflux-x]

@rdujardin @shaneshort - No problems,

Thanks for your time - I look forward to seeing the development path to see where it takes Netbox for future releases.

[rdujardin]

Ok. So what about my proposal (of having a field hostname in ip for PTR, new objects DNSZone and DNSRecord and a new DNS menu) ?

[rdujardin]

Hello, I'm currently working on DNS and I'm thinking to emit a pull request soon, so I would like to talk a bit about it in order to make some changes if wanted.

As I have proposed it, I created a new app for DNS, with its new menu and its two new objects : Zone and Record. I've also added a host name field to IPAddress to handle the reverse DNS. Zone and Record objects are as follows :

• Object Zone, storing SOA data, with fields : name (charfield), ttl (integer), soa_name (char), soa_contact (char), soa_serial (char), soa_refresh (integer), soa_retry (integer), soa_expire (integer), soa_minimum (integer), description (charfield) ; all fields required except of description
• Object Record with fields : name (charfield), type (charfield : MX, A, AAAA, CNAME, etc), priority (integer), a foreign key on a Zone, a foreign key on an IPAddress, and if IP is not needed for instance in a CNAME record, a value charfield ; and a description charfield ; only type, zone and either ip or char value are required

If you want to create a A/AAAA record with an IP you don't want in your database, if you're not in charge of its reverse or if it's an IP you don't own for instance, you can do it by setting the IP in the value charfield instead of setting an IPAddress foreign key.

So I have created these objects and all what goes with : views (list, detail, add, import), filters, API serializers and views, bulk operations, CSV import/export, new documentation page, etc.

The zone detail views lists all related records, zones and records are searchable/filterable with almost all their fields. On some char fields, filters are a multi select field on values present in database, it's for instance the case of record type : if you have A, NS, MX and CNAME records in your database, these will be the four possibilities in the filter.

I also have modified a bit some IPAM objects to handle the reverse DNS : prefixes are considered as reverse zones, so they have SOA fields too (optional), and on IP address detail view I added a list of related DNS records.

Serials are not editable because they are automatically updated as soon as something is modified in zones : the zone itself, or one of its records (edition/adding/deletion), or one of the IP addresses related to one of its records. The serial is formatted like this : YYYYMMDDN with N a positive integer.

I have also handled exports in BIND format, which is compatible with all DNS servers directly or through an import : each zone can be exported, and reverse DNS is handled too. A prefix, which is a reverse zone, can be exported. DNS reverse zones are /16 or /24 in IPv4 and /n*16 in IPv6, in IPv4 if the prefix is smaller than /24 the whole /24 corresponding zone is exported (possibly with IPs from the other prefixes in the same zone), if it's bigger the several corresponding zones are exported. BIND exports can also be accessed through the API, so it's possible to automate your DNS conf from Netbox.

I have also created a special view in DNS menu which allow to export all the reverse zones in a single shot.

I'm currently working on some additional helpful features : auto-create the corresponding A/AAAA record if applicable when you set or modify the host name field of an IP address, and auto-create the IPs and prefixes which don't exist yet when you import records.

I think I've said most of what I've done and what I'm still working on, tell me your thoughts about it, if you agree or not with a feature, if you have suggestions, ... (you can look at my fork repository if you want to see by yourself what I've already done).

[Armadill0]

Nice work!

My two cents: The hostname field should be renamed to something more generic, because this could also be a Cluster- or an HA-IP. Maybe something like "fqdn" would be a better choice.

[rdujardin]

I agree. In fact I already thought of renaming it to something like "reverse dns" or "ptr" because it's basically what the field is. What about it ?

[Armadill0]

I think PTR would be fine and precise.

Btw: auto-creation of corresponding A/AAAA record would be awesome! 😃

[Dave-Snigier]

Taking that approach with the serial may cause problems as it only allows for 10 changes each day (assuming you start the count at 0). Typically I've seen YYYYMMDDNN as a serial format which allows for 100 changes in a day which is sufficient when manually editing as you can batch a bunch of changes together.

I fear incrementing the serial for every individual change automatically would rapidly exhaust the serials for a day.

You could move to a simple incrementing serial which would wrap around when it reaches the max. Another option might be to only increment the serial when that zone file is exported (through the GUI or API).

Thanks for your work on this feature!

[rdujardin]

Sorry, I have not correctly described it. For the moment, the serial is in fact formatted YYYYMMDD + a number of unlimited size, there's no constraint on the number of digits, except of the size of the field itself which is 100 characters. Is it okay like this ?

[joachimtingvold]

Keep in mind that the NN part has to be a fixed number of digits (zero padded) from the start, since 20160727101 > 2016072801.

[Dave-Snigier]

@jallakim is correct. The serial needs to be fixed and unfortunately bind9 seems to be restricted to a max length of 4294967295 which only leaves us two digits to increment with.

[joachimtingvold]

Incrementing the serial whenever exporting or accessing through API would be a nice approach, yes. Only if there are changes since last export/poll, though.

[rickmur]

I'm also highly interested in seeing the merge being integrated in the NetBox release. DNS management is the only killer feature missing at the moment.

[jsenecal]

I'm wondering if there would be some interrest in having DNS support based around lexicon and/or based around existing work done in #398 - This might be something that could be combined with the way the reports feature works for updating/fetching records periodically or manualy. (As I believe implementing a celery task runner would be somewhat too complex for the scope of this project.)

If @jeremystretch agrees on this, I could start working on it fairly soon as I need this feature implemented and would rather have it included in netbox than have an external set of scripts to implement this with custom fields.

[ghost]

@jsenecal @jeremystretch It will be really great, we could have this dns features added

[dBitech]

I would agree, DNS admin out of netbox would be immensely useful.

…

On Wed, Sep 19, 2018 at 3:33 AM learn3r ***@***.***> wrote: @jsenecal <https://github.com/jsenecal> @jeremystretch <https://github.com/jeremystretch> It will be really great, we could have this dns features added — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#166 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJ1FTuynZhfZLLw5J6UQQ8fIF5wXOGVWks5uch0agaJpZM4JDKzT> .

[candlerb]

Not just DNS management, but a bit wider:

• A domain is an asset, from a given registrar, with an expiry date. (Would be very useful to be warned before domains expire!)
• Similarly would like to keep tracks of certificates, and be warned of expiry dates
• DNS record management sits on top, but you need to be careful what you're asking for.

Is Netbox the authoritative source of all your DNS records - you dump zone files from it to your DNS servers periodically? That only works with BIND-type backends.

Or does Netbox make API calls to add / change / delete DNS records in some other system? Maybe Netbox has its own DNS tables, with database triggers to log changes. Those triggers can in turn be picked up by an external DNS update tool which talks to your DNS infrastructure.

Or should Netbox treat the DNS itself as a database, doing live DNS queries and updates??

[eimann]

I'd rather see DNS in netbox as an addition so some kind of existing DNS infrastructure with a few built-in APIs.

I don't want to domain and certificate stuff in a DCIM, that's the wrong place.

[luqasz]

I don't want to domain and certificate stuff in a DCIM, that's the wrong place.

Neither do I.

[spotlesscoder]

I think that you can not call Netbox a DCIM, (DataCentre Infrastructure Management) [...]

thanks for writing out the long form for people like me who are not experienced in this field.. makes life so much easier

[eperegudov]

NetBox is not only DCIM and also IPAM tool, and whereas IPAM and DNS management are inextricably linked together, it's seem like that having of basic DNS for network equipment inside NetBox will be a good option.

If this feature is not important for someone, then just do not use it.

[luqasz]

I just don't want any certificates, PKI etc. in netbox. DNS is fine.

[ghost]

NetBox is not only DCIM and also IPAM tool, and whereas IPAM and DNS management are inextricably linked together, it's seem like that having of basic DNS for network equipment inside NetBox will be a good option.

If this feature is not important for someone, then just do not use it.

I fully agree with @joniknsk as netbox is IPAM at least it's good to keep track of DNS records of organisation, but we can use other tools to manage those records with API

[DanSheps]

I think some people are conflating IPAM and other infrastructure management solutions.

IPAM is IP Address Management, I would encourage everyone to familiarize themselves with the terminology. While some IPAM's do indeed provide DNS management, not all do.

I don't disagree with hostname and perhaps even record management, as that would be part of device management, but running your whole DNS infrastructure in netbox is not something I would care to see and I hope the maintainers do not look at implementing this. Same with integrating PKI infrastructure, your PKI should be managed elsewhere. I am sure when DNS is implemented, there will be a webhook system implemented as well with it that you can use to hook into your DNS infrastructure.

When all else fails, Read the Docs.

NetBox intends to represent the desired state of a network versus its operational state. As such, automated import of live network state is strongly discouraged. All data created in NetBox should first be vetted by a human to ensure its integrity. NetBox can then be used to populate monitoring and provisioning systems with a high degree of confidence.

Changing DNS, by it's vary nature, is a more operational state, versus a desired state.

[shaneshort]

Did this ever get anywhere? I'm worried it got bogged down in over-reaching feature requests when half of us were simply asking for a field in the table

[DouglasHeriot]

Should I just create a custom field "FQDN" on the "IP Address" object on my own installation of Netbox? Or would it be helpful to add FQDN as an actual field in the table properly? With #568 import/export custom fields being worked on, that's one less downside to a simple custom field as a solution for those that want this.

[jeremystretch]

I've been thinking about this for a while, and I feel we're best off simply adding a hostname or fqdn field to the IPAddress model to store an arbitrary name. NetBox has reached a size and level of adoption where any further major additions to its scope could seriously jeopardize its maintainability. There are already numerous tools available for DNS management, all of which likely do a better job than NetBox would. I'd rather us focus on making it easier to integrate NetBox with others tools, so that a DNS management application can manipulate IP addresses within NetBox, for example.

[netops2devops]

I see that this issue has been closed but just wanted to chime in and quickly mention about another open source IPAM solution called "NetDot" which was developed by university of Oregon which offers both DHCP and DNS integration along with IPAM. It's developed in Perl but comes with a Python Restful API client to push/pull data into Netdot. We use it at my home institution and it works really well. Hope this helps. https://github.com/cvicente/Netdot

[Dzordzu]

In my opinion it should have NOT been closed. Single DNS is (obviously) great option, but not the best one. The best option IMHO will be a separate page (and table in db). Duplicating IP addresses is not the option, 'casue it is not the place for it
I opt to reopen this issue