SNTP, Time, TZ, libc

Time

With the Arduino ESP8266 Core version number 2.4.0, the newlib C Library (libc) was added. And thus support for a familiar collection of time functions has been added. A manual for the newlib C Library can be found here

TZ Variable and tzset()

For libc to handle timezone and daylight savings time for a locale, the TZ environment variable must be set. A description of the format can be found here and a little bit more information on the POSIX format can be found here. However, I found this site, pub.opengroup.org(scroll down to TZ topic), to have a more complete and concise expression of the POSIX format. Assuming you have a configured Linux system running, you might find the POSIX formated TZ string you need by entering the following at a command prompt.

tail -1 /etc/localtime

While looking through the possible extracted TZ strings, I noticed that some of the results have less than/greater than symbols for quoted values. From my read of the source code for .../newlib/libc/time/tzset_r.c, the use of less than/greater than symbols for quoting values for std and dst are not supported by libc.

I have noticed, that on today's Linux systems, the TZ environment variable is not set. However, the value is embedded in the files /etc/local/time or /usr/share/zoneinfo/localtime. These are not text files. You can use tail -1 filename to skip the binary stuff and displaying the text-based POSIX formated TZ value.

After setting TZ, a call to tzset() will parse the TZ environment variable and set the shared timezone information for libc/time functions. After the call to tzset(), we no longer need to tie up memory with the TZ environment variable; however, unsetenv() does not work as you would expect. unsetenv() will remove the variable from the environment table; however, its memory allocation is not free-ed.

TMI: The shared timezone information consists of three external variables: _timezone, _daylight, and tzname. Also, there are structures __tzinfo_type and __tzrule_type, that hold the specifics for handling time change.

Due to the legacy nature of libc, the way the environment table works is less than optimal for an embedded environment. When environment variables are unset with unsetenv() or updated using setenv() the old memory allocations are never released. This was done this way because there was no way to notify that a change had been made and the old value was invalid. Thus small leaks were allowed. This was tolerable with a transient application, where the leaks would be free-ed at termination. This behavior is not so nice in a memory tight embedded environment, where there is no application to terminate. It would be best that you only use setenv() in "run once" init() code.

This method no longer works with current core: To work around this I use the following function to handle processing a tzset() call. I set up a temporary environment table for tzset() to reference, then restore the old table pointer afterward. While tzset() does save a copy of the pointer to the environment variable, it parsed. It is only used to compare against a future TZ environment variable for change detection. In the libc code the only places I see that call tzset() are: 1) setenv() will check to see if you are setting TZ and call tzset() and 2) When strftime.c is compiled for regression testing, a call to tzset() is made; however, compiled as a library function, no calls are made.

CAVEAT: To avoid confusion, if you use this function setTimeTZ(), then only use setTimeTZ() to update timezone information. Never call tzset() directly or call setenv() to set TZ. Also, a call to tzset() with out an environment variable TZ present, will reset timezone information back to GMT.

//*** This method does not work with current Arduino ESP8266 core release.

#include <time.h>

extern char **environ;
void setTimeTZ(const char *tz) {
  char **old_environ = environ;
  char * tzEnv[2] = { (char *)tz, NULL };
  environ = &tzEnv[0];
  tzset();
  environ = old_environ;
}

void setup() {
  ...
  setTimeTZ("TZ=PST8PDT,M3.2.0,M11.1.0");
  configTime(0, 0, "pool.ntp.org");
  // or   configTime(0, 0, "0.pool.ntp.org", "1.pool.ntp.org", "2.pool.ntp.org");
  ...
}
...

Also in the call to configTime(), use 0 for timezone and daylight options. This sets up the lwIP sntp code to return GMT zone time. You want GMT zone time to pass to the libc time function calls.

DHCP and NTP Server - Observations

• The character arrays used to identify the NTP Servers in the call to configTime() must be global, static or permanently allocated memory. These pointers may be referenced long after the calling function has exited.
• All configured NTP Server Names are discarded, when the DHCP server provides one or more NTP Servers.
• When the DHCP server has not been configured to give out NTP Server Names, then the NTP Server Names provided by configTime() are used.
• The above summary is my interpretation of the code at tools\sdk\lwip2\builder\lwip2-src\src\apps\sntp\sntp.c.
  • A call to dhcp_set_ntp_servers() will overwrite any NTP Server Names set by a previous call to configTime() or sntp_setservername().
  • When dhcp_set_ntp_servers() is called with a list of NTP Server Addresses, any remaining possitions, not set from the list, are cleared.
  • dhcp_set_ntp_servers() is called as part of the DHCP process. When lwIP is built with the defines: LWIP_DHCP and SNTP_GET_SERVERS_FROM_DHCP set.
  • Calls to sntp_setserver(), when built with SNTP_SERVER_DNS defined, will set to NULL the NTP Server Name set by a previous call to sntp_setservername(). Note, this is how dhcp_set_ntp_servers() is clearing previously set NTP Server Names.

Bash Script to Extact a List of TZ Values

This simple script will write a csv file of TZ values, based on the information located at /usr/share/zoneinfo/posix on a Linux System.

#!/bin/bash
#
#   Copyright 2018 M Hightower
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

zoneinfodir="/usr/share/zoneinfo/posix"

# print tz string from files in directory $1
printtz(){
    local afile adir

    # Check %1 for files
    while read -u3 afile; do
      echo "\"${afile}\",\"`tail -1 ${afile}`\""
    done 3< <(
         stat -L -c"%F %n" ${1}/* |
         grep -E ^regular\ file |
         cut -d\  -f3)

    # Check %1 for directories
    while read -u3 adir; do
      $0 "" "" "" "${adir}"
    done 3< <(
         stat -L -c"%F %n" ${1}/* |
         grep -E ^directory |
         cut -d\  -f2)
}

if [[ -n "${4}" ]]; then
  printtz "$4"
else
  startat=`echo "${zoneinfodir}//" | wc -m`
  printtz "${zoneinfodir}" | cut -c1,${startat}- | sort
fi