Rework config builders

[drkitty]

Resolves issue #1002.

This branch ended up being much larger than I originally intended it to be, which is part of why it took so long.

Here's a one-liner to sort the zone files and normalize their whitespace to make comparison easier:

find $zone_file_dir -type f -exec bash -c 'grep -v "; Serial" "{}" | sed -r "s/\s+/ /g; s/\s+$//; /^$/ d" | sort -o "{}"' \;

Major changes:

• I rewrote most of the DNS build system because it felt badly organized, inflexible, difficult to understand, and larger than it needed to be. No guarantees on the flexibility of the new builder, but it's more straightforward and I tried to avoid unnecessary classes and state.
• The DNS and DHCP builders now manually sync the staging and production config directories, without using version control.
• The DNS and DHCP builders now use file sizes to do the sanity check, rather than line counts, mostly because I had to reimplement the sanity check now that we're not using Git, and using file sizes was slightly easier.
• The DNS builder removes zone files and directories from the production directory if they don't exist in Cyder.
• During a dry run, no email is sent on failure and the stop file is neither created nor checked.
• All DNS objects now use their own TTL rather than their SOA's. If this is wrong, please tell me.
• I added one data migration and one schema migration. They're numbers 9998 and 9999 at the moment. The numbers should be changed to the lowest available ones once this is merged.

Minor changes:

• I renamed the bindbuild command to dns_build, for better consistency with the dhcp_build command. Let me know if that was a stupid idea.
• The DNS and DHCP builders now share significantly more code, hopefully making it easier to maintain them.
• I changed the DNS and DHCP builders from classes into functions, without sacrificing configurability (as far as I can tell).
• I changed a couple of the options to the DNS and DHCP builders to better reflect their behavior with the new stage/prod syncing system.
• I renamed a few DNS and DHCP build settings for various reasons. Most significantly, the BINDBUILD dict is now called DNSBUILD to match the other name changes.
• I moved some files around for various reasons. Everything should still work.
• I changed the wording slightly in build failure emails.
• Tests still pass.

Option migration guide:

• -f → -a (build All zones).
• -p was removed; syncing to the prod directory is now the default unless -n (dry ruN) is passed.
• All other options remain the same.

[murrown]

So you're removing the ZONES_WITH_NO_CONFIG setting?

[drkitty]

Oops.

[drkitty]

Seems to work properly now.

[murrown]

Why not sort the build files themselves? I'm assuming that, unlike DHCP builds, somewhere down the line someone is going to want to look at the DNS build files eventually. That's why we've added in buttons in Cyder to view the builds, and why we arrange each record into neatly aligned columns.

[murrown]

ZONES_WITH_NO_CONFIG doesn't work, gives this message:

Exception: `named-checkzone akademiaprima.pl /tmp/dns_stage/pl/akademiaprima/akademiaprima.pl.public` failed in /home/ch/cyder

=== stdout ===
zone akademiaprima.pl/IN: has no NS records
zone akademiaprima.pl/IN: not loaded due to errors.

Shouldn't need to do named-checkzone on zone files that aren't in the config.

[drkitty]

If we don't need to check the zone, there's certainly no reason to build it then.

[murrown]

For some records, the TTL is written to builds as None instead of a number. This might be caused by a botched data migration or something.

[drkitty]

You migrated beforehand?