Per-domain DNS on Linux using a local caching server

More adventures in sysadmin time!

I had a bit of a sticky conundrum to deal with this evening. After setting up local DNS yesterday, I realized I'd have a problem for machines I have that connect to the Red Hat VPN. Here's the conundrum: my local DNS server can resolve, say, 'nas.happyassassin.net' - a box that has no public DNS record because I don't want to allow external access to it - but the Red Hat DNS server obviously can't. But my local DNS server can't resolve, say, 'topsecret.redhat.com', a box on the Red Hat VPN but again with no public DNS record.

How do I set things up so a box connected to the RH VPN can resolve both hosts on the local network and hosts on the RH VPN?

edit: Peter Robinson has pointed out that if you use NetworkManager, it should just handle this. I still run static network.service on my servers - I think because it just wasn't plausible to use NM when I initially configured them. I could probably switch to NM, now, I'll have to look into it. But the below may still be of use to others.

You'd think just listing both nameservers in /etc/resolv.conf would work (perhaps with a delay while it hits up the one that won't work, for the case where the one that won't work is listed first), but it doesn't seem to. This is how things wind up if you just use everything 'out of the box', and in that configuration, I can resolve stuff on the VPN (whose servers wind up listed first) but not the local network (whose servers wind up listed second). I don't know why; if someone does, do let me know.

Unless I missed something somewhere, it sure ain't simple. One option would be to have your router connect to the VPN, of course, but that has other drawbacks, and anyway, the firmware I have on my router at present isn't capable of acting as a VPN client. I poked through name solving documentation until it become pretty clear that this just isn't something you can really do 'simply'. There's no nsswitch option to say 'send lookups for .redhat.com to THIS name server but lookups for .happyassassin.net to THIS name server'.

But! You can do it with a bit of finesse and a local copy of dnsmasq. Basically the approach is to set up dnsmasq as a simple local caching name server using the router as its 'upstream' server, and then have the openvpn bring-up process write a little bit to dnsmasq's configuration which tells it 'use these DNS servers I just found for all redhat.com lookups'. As long as the VPN isn't up, dnsmasq is basically just forwarding all requests to the router; when the VPN comes up it keeps doing that for almost all requests, but requests for redhat.com addresses get sent to the RH server instead. Here's how I did it:

Install dnsmasq. Edit /etc/dnsmasq.conf - very few changes needed here, you just want to set listen-address=127.0.0.1 (to make sure nothing outside of the local box sees it, as we're just using it for this trick - we get caching as a bonus), and resolv-file=/etc/dnsmasq-resolv.conf (or any other file that isn't resolv.conf).

EDIT added later, forgot it at first: Edit /etc/resolv.conf so it simply reads: search=yourdomain yourvpndomain nameserver 127.0.0.1

i.e., send everything to dnsmasq. I suppose you could add the Google or OpenDNS addresses as a fallback in case everything goes south. Also edit /etc/sysconfig/ifcfg-(ifname) and add PEERDNS="no", which should stop the network service overwriting resolv.conf every time you bring up the connection (this step is inexplicably missed out of every 'use dnsmasq as a caching server' guide I've ever read, so I had to figure it out for myself).

If you actually need to have your new 'not-resolv.conf' file populated via DHCP every time you bring up a connection you have a bit of a conundrum to solve here; fortunately I don't, because I'm just always going to want it to use 192.168.1.1, so I simply hand-write it with 'search happyassassin.net redhat.com' and 'nameserver 192.168.1.1'.

Now the tricky bit. This is the bit that'll vary as well depending on exactly why you want to use per-domain DNS. But for me, this is how it goes. The recommended openvpn config scripts for our VPN include this wonderful bit of elegant shell scripting. On VPN up:

if [ -n "${dns[*]}" ]; then for i in "${dns[@]}"; do sed -i -e "1,1 i nameserver ${i}" /etc/resolv.conf || die done fi

On VPN down:

if [ -n "${dns[*]}" ]; then for i in "${dns[@]}"; do sed -i -e "/nameserver ${i}/D" /etc/resolv.conf || die done fi

So, I just munged that up a bit, and made it do this instead. On VPN up:

if [ -n "${dns[*]}" ]; then for i in "${dns[@]}"; do sed -i -e "1,1 i server=/redhat.com/${i}" /etc/dnsmasq.d/redhat.conf || die done systemctl try-restart dnsmasq.service fi

On VPN down:

if [ -n "${dns[*]}" ]; then for i in "${dns[@]}"; do sed -i -e "/server=\/redhat.com\/${i}/D" /etc/dnsmasq.d/redhat.conf || die done systemctl try-restart dnsmasq.service fi

Beautiful, innit? Just beautiful. Instead of haphazardly munging up resolv.conf we're now haphazardly munging up a dnsmasq config snippet. MUCH better. You need to make sure redhat.conf exists and contains a single line before that mess will work. What that does, with a following wind, is write lines like this to /etc/dnsmasq.d/redhat.conf when the VPN comes up, and delete them when it goes down:

server=/redhat.com/XX.YY.ZZ.FOO

which in dnsmasq syntax means 'send requests for this domain to this server'.

I've tested this like twice for thirty seconds, so I'm pretty sure it's bulletproof! Please leave comments indicating exactly how I have sinned against nature this week, I'm always willing to learn...

Comments

Matěj Cepl wrote on 2013-09-23 21:54:
In the similar situation (luther.ceplovi.cz is known on the public Internet as 90.177.109.184, but in the home LAN as 192.168.0.13). In the end the best solution I've found was to create this script as /etc/NetworkManager/dispatcher.d/99-setHosts: #!/bin/bash # The script is invoked by NetworkManagerDispatcher, like this: # 99-dnsmasq # Well, acutally it is more complicated than -- this way it works with regular # eth0, but with VPN it is more confusing: # INTERFACE=eth0 UPDOWN=down # INTERFACE=eth0 UPDOWN=up # INTERFACE=eth0 UPDOWN=vpn-down # INTERFACE=tun0 UPDOWN=vpn-up INTERFACE=$1 UPDOWN=$2 MYSSID="MYSI|KOCKY" MYNETID="xx:xx:xx:xx:xx:xx" # MAC addr of known machine PATH=$PATH:/sbin logger -p user.info -t NMdispatch-dnsmasq "INTERFACE=$INTERFACE, UPDOWN=$UPDOWN" if [[ "$UPDOWN" =~ up$ ]] ; then NETID=$(nm-tool |awk -F, "/\*$MYSSID/ { print \$2 ; }" | tr -d '[:space:]') sed -i -e '/luther\.ceplovi\.cz/d' /etc/hosts if nm-tool |grep -s -E 'Device: wlp3s0.*(KOCKY|MYSI)' >/dev/null 2>&1 ; then echo -e '192.168.0.13\tluther.ceplovi.cz luther social.ceplovi.cz social' \ >>/etc/hosts logger -p user.info -t NMdispatch-dnsmasq \ "Adding 192.168.0.13 to /etc/hosts" else logger -p user.info -t NMdispatch-dnsmasq \ "Not addding 192.168.0.13 to /etc/hosts" fi pkill -HUP -f dnsmasq || /bin/true fi The result is that I have automatically set up luther.ceplovi.cz in /etc/hosts as 192.168.0.13 when I am on the home LAN. However, when I am on the public Internet (via wifi in a pub or something) /etc/hosts stays the same.
matej.ceplovi.cz wrote on 2013-09-23 21:57:
Better formatting of the script is at http://paste.fedoraproject.org/41688/
adamw wrote on 2013-09-23 23:39:
congratulations, that's actually more horrible than mine!
Samuel Sieb wrote on 2013-09-28 00:41:
The reason putting both DNS servers in your resolv.conf file doesn't work is because it tries the first server and gets a not found (NXDOMAIN) result. The normally correct assumption is that the domain really doesn't exist so there's no reason to try another server. It would be nice if there was some way to tell it to keep trying the other servers, but I don't think there is any such option. However, as you found, there does seem to be a good workaround using dnsmasq that might actually be easy to use soon. I've been contemplating this same issue myself recently and might try out that workaround as well.