Writing a DNS Client
This post is part of a series, Byte-Sized Go, in which I tackle mini-projects related to network programming in Go.
Introduction
As part of another project, I was wanting to implement a simple DNS server from scratch. I decided to write a simple client first as a way to work out the basic shapes I would need for the server (parsing flag bits in the header, reading/writing bytes over the wire and encoding/decoding them into Go structs). Working through this project ended up being even more enlightening than I expected. Many of the pieces of DNS I had taken for granted or completely overlooked stood out in a new light.
I started by reading RFC 1035, which lays out the entire specification for a standard domain name system. Adherence to this document, now almost 40 years old, is what allows every machine on the internet to share meaningful domain name information across all sorts of operating systems and applications.
Interesting Pieces
Some of the skills I practiced through this project include:
- Bitwise operations: working with DNS header flags where multiple flags are stored within a byte, or even crossing byte boundaries
- Processing raw bytes: encoding and decoding between useful Go structs and byte streams which can be sent over a network connection
- Implementing an RFC spec: reading the official protocol specifications and translating them to working code
- Binary protocol design: understanding why fixed-width fields, length-prefixed data, and network byte order (big-endian) exist and how they interact
- Go interfaces: io.Reader, io.Writer, and how bytes.Buffer, bytes.Reader, and net.Conn all compose naturally because they satisfy the same interfaces
- UDP networking: sending and receiving raw datagrams, and what constraints that puts on message size
- Package design: separating protocol logic from CLI concerns, deciding what to export, and designing a clean public API surface
- Unit testing: writing idiomatic (table-based) unit tests to confirm accurate function in happy paths and edge cases
I want to call out a couple areas in more depth that were especially interesting from a technical perspective.
Compression pointers
DNS messages encode domain names, such as “google.com”, by breaking each section apart at the “.”; this unit is called a label. In the byte stream, labels are prepended by their length as an indication of how many of the following bytes should be read to retrieve a label. Within a label, each character is a single ASCII-encoded byte. So, for example, “google.com” can be represented as the Go byte slice:
[]byte{0x06, 'g', 'o', 'o', 'g', 'l', 'e', 0x03, 'c', 'o', 'm', 0x00}
Go conveniently converts runes to integers for us, which makes it easier to parse this example, but you could also write this as:
[]byte{0x06, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x03, 0x63, 0x6f, 0x6d, 0x00}
The first byte, 0x06, tells the client to read the 6 following bytes (‘g’, ‘o’, ‘o’, ‘g’, ’l’, ’e’) as a label,
then expect another length byte.
The eighth byte, 0x03, marks the length of “com”.
The terminal byte, 0x00, signals the end of the record.
In a simple implementation with relatively few answers to return, such as a single A record, this is likely fine as-is. In practice, though, we might quickly run into issues with length. The default behavior of DNS is to operate over UDP, which limits message length to a max of 512 bytes. The specification also allows a response to indicate truncation when a complete response is too long for a UDP datagram, which should prompt the client to retry over TCP, but there is another solution we can try first: compression.
By reusing the labels from earlier records, we can reduce length by omitting redundant records. This is accomplished using compression pointers. Labels are intentionally restricted to lengths of 63 octets or less, so a label length byte would never exceed 6 bits. If the first two bits of a length byte are set to 1, it indicates that the byte and its following byte should be treated as a 2-byte pointer instead, where the remaining 14 bits are the offset of the target byte.
In practice, let’s imagine a response message for “google.com” which has multiple answer records,
perhaps an A record and an MX record.
Since the A record will have already contained the labels for “google.com”,
the MX record might only need to add a label for “mail.google.com”.
It could specify the “mail” label as normal, a length byte of 0x04 followed by 4 ASCII bytes,
then use a pointer to point to the previous occurances of “google” and “com”.
The pointer would point to the first byte of the “google” label based on how many bytes into the message it occurs.
The client would continue to read labels from that point until it reached the terminal 0x00 byte after “com”.
This might be better illustrated as:
Message buffer (byte offsets shown on left):
Offset 0: [ header - 12 bytes ]
Offset 12: \x06 g o o g l e <- "google" label (length 6)
Offset 19: \x03 c o m <- "com" label (length 3)
Offset 23: \x00 <- root label (end of name)
Offset 24: \x00\x01 <- QTYPE (A)
Offset 26: \x00\x01 <- QCLASS (IN)
Offset 28: \x04 m a i l <- "mail" label (length 4)
Offset 33: \xC0\x0C <- pointer to offset 12 ("google.com")
Offset 35: \x00\x0F <- QTYPE (MX)
Offset 37: \x00\x01 <- QCLASS (IN)
Bitwise operations
Most of the pieces of a DNS message can be broken into whole bytes, which are generally easier to work with. The primary exception to this is in the header section. The RFC illustrates this section well:
1 1 1 1 1 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| ID |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
|QR| Opcode |AA|TC|RD|RA| Z | RCODE |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| QDCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| ANCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| NSCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| ARCOUNT |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
The 3rd and 4th byte contain flags which don’t require an entire byte to store. By packing them all into these 2 bytes, we can make better use of our limited byte budget.
Parsing these flags in Go is a pretty simple exercise in bitwise operations, but bitwise operations, at least in my experience, aren’t generally very common in most areas of modern software, so it’s fun to have a simple, practical application.
A clean way to parse the flags is to define a set of constants which store the bit locations:
const (
flagQR = 1 << 15
flagAA = 1 << 10
flagTC = 1 << 9
flagRD = 1 << 8
flagRA = 1 << 7
)
I didn’t end up needing to do much with the flags in my simple initial implementation of the client, but a simple example shows up on the server side. When a server builds a response message, many of the fields are the same as the query, so the easiest thing is to reuse the query message and just modify it with response information. One of the first steps is to flip the QR flag; 0 indicates a query and 1 indicates a response.
m.Header.Flags |= flagQR
The “inclusive or” operator is used here to ensure QR in the flags of a message m is set to 1.
This could also be used defensively on the client side to verify that the QR bit is 1 for any received response as expected.
if r.Header.Flags & flagQR == 0 {
// QR flag is 0, so the response message is malformed
}
A server could do that opposite, ensuring that the client sent a proper query:
if r.Header.Flags & flagQR != 0 {
// QR flag is 1, so the query is malformed
}
And to combine any number of the flags, such as when building a new message, we can use another inclusive or:
message := Message{
Header: Header{
ID: 42,
Flags: flagQR | flagAA, // sets the QR bit to indicate a response and the AA bit to indicate authority for the domain in question
QDCount: 1,
ANCount: 1,
NSCount: 0,
ARCount: 0,
},
...
Using the Utility
I implemented the client as the first subcommand of a larger project I’m calling “Netgo”. Netgo will be a collection of common terminal network utilities.
The DNS client is called with ng diglet [flags] [arg].
It currently accepts exactly 1 name as an argument.
Optional flags include:
-t: Record type (default A)-s: Server address (default 9.9.9.9)-p: Server port (default 53)
Here’s a lookup of “google.com” as an example of the output:
❯ ng diglet google.com
----------------------------
---------- diglet ----------
----------------------------
--- Questions ---
Server: 9.9.9.9:53
Name: google.com
Type: A
----------------------------
--- Answers ---
Type TTL Data
A 54 142.250.72.206
Potential Improvements
I’ll probably continue to add to this client over time as I want to work out additional DNS shapes, such as:
- Upgrading to TCP after truncation
- Reverse lookups