Just in Time

Amazon Web Services (AWS) began recommending chrony some time ago. The documentation [3] explains that AWS runs the Amazon Time Sync Service, to which it recommends connecting from Elastic Compute Cloud (EC2) instances. Sounding a little like a James Bond movie, the AWS docs go on to explain that the service "uses a fleet of satellite-connected and atomic reference clocks in each region to deliver accurate current time readings." In English, I suspect that means each continent's varying clusters of AWS data centers have access to atomic clocks for extra accuracy.

Not surprisingly, AWS appreciates the value to their customers of getting time syncing working correctly. As mentioned, it's a critical part of any production system's health. AWS docs confirm the /etc/chrony.conf path for that config file, so I'll hazard a guess that's the location of the file on RHEL derivatives, as well. AWS suggests adding the line

server 169.254.169.123 prefer iburst

to the config file. If you're using their in-house Amazon Linux 2 OS for your server instance, you can ignore that instruction because it already defaults to using the AWS time service.

The funny-looking IP address above is in the LINKLOCAL-RFC3927-IANA-RESERVED address range (169.254.0.0/16), which isn't routed out onto the Internet, so it is a fast, local way of syncing with the AWS internal systems. Note that it ends with 123, which is the usual NTP port, making it easier for reference.

Only a Question of Time

The Amazon Time Sync Service topic segues perfectly into considerations about the time server sources you can choose to use. I'll start by saying that if you're using AWS, you almost never have a reason not to trust their time service, but I'll then add that because timekeeping is such a critical service, you might consider adding some extra redundancy in the form of other NTP sources. Should you not be using AWS, you should definitely think about which sources you have in place for redundancy.

Before continuing further, I'm going to refer you back to the chrony FAQ page [2] and the section How can I improve the accuracy of the system clock with NTP sources?. There, you'll find information on tweaking the time servers to which you ultimately choose to sync.

In my experience with upstream time servers that are geographically close but not all in the same country, using the public servers listed on the NTP site [4] is a very effective approach. Note the warning on the site about denial-of-service attacks:

NTP users are strongly urged to take immediate action to ensure that their NTP daemons are not susceptible to being used in distributed denial-of-service (DDoS) attacks. Please also take this opportunity to defeat denial-of-service attacks by implementing Ingress and Egress filtering through BCP38.

A link is offered for BCP38 [5] that details the clever approaches for filtering out attacks (at the network Access Control List level) that plagued NTP for a while.

Time Works Wonders

The NTP site has a very useful page dedicated to helping you choose time servers near you [6] and how round-robin is used to iterate through a list of servers presented to an NTP server by the Domain Name System (DNS).

Figure 2 shows that a typical NTP-style hostname actually points to multiple time servers, which allows lists of time servers to be gathered easily into "pools." I'm in Europe, so I can add the pool servers to my chrony config file that are geographically close [7],

server 0.europe.pool.ntp.org
server 1.europe.pool.ntp.org
server 2.europe.pool.ntp.org
server 3.europe.pool.ntp.org

Figure 2: Four NTP servers under one DNS entry.

by prepending server to each line. This setup offers 16 IPv4 time servers; additionally, 2.europe.pool.ntp.org offers four IPv6 clocks with which to connect.

If you're worried that the number of community-volunteered clocks will reduce over time, in Europe alone, you can see that you should have no issues in that respect (Figure 3).

Figure 3: Europe's pool looks pretty healthy, boasting lots of clocks.

However, take note of the comment on the page for European servers: "In most cases, it's best to use pool.ntp.org to find an NTP server (or 0.pool.ntp.org, 1.pool.ntp.org, etc. if you need multiple server names). The system will try finding the closest available servers for you."

The clever (NTP) protocol that assists with timekeeping is innately designed to measure response times from servers that are geographically disparate and then compensate against the inherent network latency.

If you want to get your hands dirtier with NTP and chronyd (or ntpd, of course), then a word of advice would be to remember that DNS entries for the clock servers (e.g., 0.pool.ntp.org through 3.pool.ntp.org) point to a randomized set of servers that are updated each hour, which helps to distribute load. Bear in mind, however, that if you're working in a hospital and life depends on equipment running to the correct time, you're probably going to need a different pool of servers than someone who's watching YouTube at home. Business and domestic needs might be quite different.