NUM, MODL and Unpacker

NUM

Namespace Utility Modules (NUM) is a way to store and retrieve data using DNS TXT records, use cases for the technology are standardised using modules.

NUM records are identified by URIs like num://numexample.com:1/, where 1 is the module number – in this case, module 1 is a way to standardise the storage of contact data.

Existing data serialisation formats like JSON/YAML/TOML and others are badly suited to storing serialised objects in DNS TXT (line breaks and quotes are bad in DNS TXT), and Base64/other encoding makes records far less efficient (even when compressed).

To ensure NUM uses the DNS as efficiently as possible, we needed to create our own data serialisation format.

MODL

We created a compact, DNS-friendly data serialisation format called Minimal Object Description Language (MODL), which easily translates to JSON.

A NUM Lookup for the URI num://numexample.com:1/ results in the following DNS query and resulting NUM record, written in MODL:

$ dig 1._num.numexample.com TXT +short

@n=1;o(n=NUM Example Co;s=Example Strapline;c[(t(d=Customer Service;v=+441270123456));(fb(v=examplefacebook));(in(v=exampleinstagram));(tw(v=exampletwitter))])

The MODL library converts the NUM record into a JSON format that all developers are familiar with:

{
    "@n": 1,
    "o": {
        "n": "NUM Example Co",
        "s": "Example Strapline",
        "c": [
            {
                "t": {
                    "d": "Customer Service",
                    "v": "+441270123456"
                }
            },
            {
                "fb": {
                    "v": "examplefacebook"
                }
            },
            {
                "in": {
                    "v": "exampleinstagram"
                }
            },
            {
                "tw": {
                    "v": "exampletwitter"
                }
            }
        ]
    }
}

It's cool that we've stored this object so efficiently in DNS, but it's not very easy for developers to work with. We needed a way to unpack this compact, character-efficient object into something that developers can work with intuitively.

Unpacker

Unpacker takes a compact JSON object and unpacks it into something more useful. For example expanding keys like o to organisation and n to name. It can also rewrite values and restructure our object to make it easier to work with.

Once the compact JSON object has been Unpacked it looks like this see how the process works:

{
  "@n": 1,
  "organisation": {
    "object_display_name": "Organisation",
    "name": "NUM Example Co",
    "slogan": "Example Strapline",
    "contacts": [
      {
        "telephone": {
          "object_display_name": "Telephone",
          "description_default": "Call",
          "description": "Customer Service",
          "prefix": "tel:",
          "method_type": "core",
          "value": "+441270123456",
          "hours": null
        }
      },
      {
        "facebook": {
          "object_display_name": "Facebook",
          "description_default": "View Facebook profile",
          "description": null,
          "prefix": "https://www.facebook.com/",
          "method_type": "third_party",
          "controller": "facebook.com",
          "value": "examplefacebook"
        }
      },
      {
        "instagram": {
          "object_display_name": "Instagram",
          "description_default": "View Instagram profile",
          "description": null,
          "prefix": "https://www.instagram.com/",
          "method_type": "third_party",
          "controller": "instagram.com",
          "value": "exampleinstagram"
        }
      },
      {
        "twitter": {
          "object_display_name": "Twitter",
          "description_default": "View Twitter profile",
          "description": null,
          "prefix": "https://www.twitter.com/",
          "method_type": "third_party",
          "controller": "twitter.com",
          "value": "exampletwitter",
          "value_prefix": "@"
        }
      }
    ]
  }
}

Result

The result is efficient storage and transfer over DNS, with beautiful objects returned to developers. Here's an example client implementation:

https://contacts.apps.num.uk/?s=numexample.com

Chicken and Egg?

Of course, in this case the example only works because the protocol has been adopted for our test domain name numexample.com. Crucially, to help overcome the chicken-and-egg problem, the protocol has two NUM query locations.

Independent NUM Zone

The first query is to what we call the Independent NUM Zone, using the example domain example.com, the independent NUM Zone is at _num.example.com

Hosted NUM Zone

A query is made to the hosted NUM Zone if the query to the independent zone fails. The hosted NUM Zone for example.com is at example.com.9.h.1.num.net (it uses a Zone Distribution technique to spread the load on num.net but that's a topic for another day)

Populating the NUM Server

We're in the process of populating the NUM Server with millions of pieces of data for UK companies using public datasets like Common Crawl, Companies House and public-facing company websites. For the largest companies, we're manually creating records – making this useful company data available to developers for free, with unlimited access and unrestricted use.

To take a look at a few examples click:

• Aviva
• Barclays
• Bupa