TECH DEVIANCY

last updated, April 22, 2011

Advanced Time Considerations

December 2006

Scope of Discussion

The time keeping features of modern computers is something often taken for granted until one actually has to create some software that utilizes them. The good news is that this area has been around for a while and therefor some outstanding solutions exist for most situations. The bad news is that computer time is still widely misunderstood and often the cause of major, yet avoidable problems. For the sake of giving me a good resource to point people to who could use a good run down on this area, I have written this article. So if you ever need to work on a program that in any way uses clocks, calendars, and computers read on and you might catch something you never thought about before.

The Basic Problem

The basic problem is that our calendar system is actually very complicated. Days of the week, months, leap years, time zones, and other details can quickly add up to a huge mess. Let us say we need to know the date and time 5 minutes from now. Simple right? Start making a program to do that, for that matter do it properly, and your head will explode. Now throw in a situation were multiple users need to utilize the program from different places in the world all simultaneously connected through the internet in different time locals. Starting to see the challenge? Remember Y2K?

This is of course not even to mention even more demanding tasks such as translating times to something other than the Gregorian calendar (the calendar system most commonly recognized in the west as "normal"). Perhaps you have clients in Israel, China, Iran, India, or even Russia, that would like to do something with a different calendar. Though this may not seem like something you would ever need to worry about there are much more common tasks that are deceptively simple.

Think of a real time data logger that needs to save a value every single hour. How should it behave during the switch in and out of daylight saving time? How should a PVR behave in the same situation?

By and large, one does not really need to do that much to solve these kinds of problems as long as he or she has some understand of what is going on. Too often programmers' and administrators' sentiment is that of "let the OS worry about it", or "just use the library functions without thinking". This is really the wrong attitude seeing as how so many things can go wrong.

The Basic Solution

The root of the solution to simplifying computational time keeping is very simple. Separate the task of tracking the passage of time, from that of calendar operations. To put it another way, the science of atomic clocks is another field of study from astronomy. The majority of hardware and software around today at some level tracks time in terms of the number of seconds past 1970 UTC. This way programs can simply think of the time and date as a simple integer. Every second has a unique and unambiguous number. Saving the time something happened requires just one number. Moving forward or backward in time requires simple arithmetic. Time before midnight December 31 1970 UTC is expressed as a negative number. When a date and time needs to be presented to a human, this integer gets rendered into what ever calender system, time zone, and format is needed. The reverse takes place when the time is input from a human. This is often referred to as Unix time, or even ANSI C time.

So to answer the above question, the data logger and PVR never even need to think about daylight saving time except for displaying, and getting times from users. Typically this is all done with relatively standard C library functions. Higher level languages of course wrap around these (if you are lucky enough to even get some exposed means to use them). The basic clock function is time(), and the basic calendar render / parser routines are localtime(), gmtime(), strftime(), and mktime() defined in time.h. See my portable time in C cheat sheet below. Check with your man pages or other documentation for further details, and various other functions. Windows of course does not disappoint those expecting its usual silliness, by providing a dozen or so supplemental API functions to preform the exact same tasks with much more difficulty. Consult the Windows platform SDK for details. It is also worth noting that any serious time aware application should not be run on any Win 9x/ME version.

Solutions have Problems

32 bit Platforms

On most 32 bit platforms, a signed 32 bit integer is used for the time value. This gives an effective range of December 12, 1901 8:45:52 PM GMT through January 1, 2038 3:14:07 AM GMT. The Y2K "crisis" has come and gone, but the Y2038 issue is quite real. Many systems already have solutions in place. 64 bit Unix systems use a 64 bit integer, and are unaffected. More likely to be an issue are file formats that use a 32 bit number. If you are designing a system to be used for a while, or that holds historical records prior to 1902, you will need more than 32 bits. Some formats that intended to only hold dates in the present use unsigned 32 bit. This trades dates prior to 1970 for longer future use. A full 64 bits of file storage can be hard to justify in a binary file format. Often a 40 or 48 bits may be used, or even a 48 bits for seconds, and 16 bits for fractions of a second. Even still, it certainly beats 64 or more bytes for storing time as text.

Note that in addition to a host being 64 bit, the C runtime libraries must also support time beyond a 4 byte integer. A quick test on 64 bit Fedora Core 4 installation running on x86_64, yields no problems for years like 1712 and 2135. For years before 1900 I used a negative number in the tm structure. Be careful, because that same test on a 32 bit box generated a segmentation fault.

Daylight Saving Time

Daylight saving time is unequivocally one of the most profoundly idiotic ideas in the history of the human race. In only stands to reason that it of course causes problems with software. DST can and is taken into account in calendar and locale functions on most platforms. The primary problem is that at any time a government can, and will change the rules for moving in and out of DST. What this means is that if a system really wants to use a DST scheme, it needs to have a way to be updated. For example, in 2005 the US decided to change to a new rule starting in 2007. What sort of chaos this creates with older systems with out of date locale definitions remains to be seen. See also: Daylight Saving Time History.

Leap Seconds

UTC, or universal coordinated time, throws in a serious complexity. UTC adds some leap seconds into the mix at various points throughout its calendar. These leap seconds manifest themselves as the 61st second of the minute they occur. Further more UTC leap seconds, the rarer still double leap second, and the yet unused day with one less second, only happen when "they" say so. So in the sense of the current time, a system must be connected to an outside time source to stay aware of those events. In the sense of historic time, a lookup table is needed. This is really just another detail of our calendar system, and since the integer time programming style is to internally be calendar agnostic, this is not that big of an issue. More serious is that with UTC the very definition of a day, minute, and hour is different! Since it is very common for applications to use minutes, hours, and days, the most popular strategy is basically to ignore leap seconds. Here are some of the effects ignoring leap seconds in your applications, and caused by work arounds built into operating systems.

  • Be aware that when rendering a time into the "UTC calendar" the number 61 can be used for the number of seconds past the minute.
  • The modern unix linear integer time representation has some numbers that represent two different seconds in the UTC calendar. It is also possible for a number to not represent any second in the UTC calendar. Check platform documentation for details of how to find out which numbers this applies to. The negative effect of using a number line that does not 100% reflect the number of elapsed seconds since its inception, is arguably more than offset by the benefits of not having consider minutes that are not 60 seconds.
  • The effects of these leap second events are rare enough to be comparable to the artifacts of clock synchronization
  • Time series data sets utilizing a single integer representation of time intervals should use the first number in the interval for the representation. For example, 2:00 for the interval 2:00 - 3:00.
  • Presumably, the occasionally available difftime() takes into account leap seconds.
  • For more gory details check out this wikipedia page.
  • Another way to look at the situation is to say, even though integer time style programming uses the integer to track the amount of time passed, it really does so only for the purpose of simplifying calendar operations. Therefor, if the overall goal is to make calendar operations easy to perform for a computer program, it is acceptable for the unix time to differ a few seconds from the real number of seconds since 1970 if it makes the calendar correct. A computer would make a terrible choice for a machine to count the number of real elapsed seconds over a long period of time anyway.
  • When asked "What is time?", Einstein is reported to have said, "time is what the clock on the wall says."

Common Implementation Challenges

As great as integer time style programming is, this snazzy approach still hasn't made its way everywhere yet. Other times it can still be unclear how to proceed. There are also plenty of puzzles given to us by our friends in Redmond to solve. Let's review a few I have encountered in recent years.

.Net

The .Net runtime internally holds time as the number of milliseconds since 1900 UTC. One would think converting between Unix and .Net time would be a matter of moving between seconds and milliseconds and then applying a 70 year offset. Not so fast, because at least with C#, dotneters don't use this integer for what I'm calling ".Net time". Instead they use objects in the form of the DateTime, and TimeSpan data types. What is needed is a means to convert from a Unix time_t integer to a .Net DateTime object and back. Here is my example of how to get the job done.

Over the river and through the woods, from .Net to Unix time we go...

 /* convert a unix time_t into a DateTime object in local time */
 public static DateTime time_convert_local(int unix_time)
 {
  DateTime unix_utc, epoch_offset = new System.DateTime(1970, 1, 1, 0, 0, 0);

  unix_utc = epoch_offset.AddSeconds( (double) unix_time);
  return unix_utc.ToLocalTime();
 }

 /* convert a unix time_t into a DateTime object in univeral time */
 public static DateTime time_convert_utc(int unix_time)
 {
  DateTime unix_utc, epoch_offset = new System.DateTime(1970, 1, 1, 0, 0, 0);

  unix_utc = epoch_offset.AddSeconds( (double) unix_time);
  return unix_utc;
 }

 /* convert a DateTime object in local time to a unix time_t */
 public static int DateTime_convert_local(DateTime dotnet_time)
 {
  DateTime epoch_offset = new System.DateTime(1970, 1, 1, 0, 0, 0);
  TimeSpan ts = dotnet_time.ToUniversalTime().Subtract(epoch_offset);
  return (int) ts.TotalSeconds;
 }

MS SQL server

If you ever encounter a MS SQL server, note that it does have a native data type for time called DateTime. Unfortunately it is in no way time zone aware. Thus, if you wanted to store time correctly in a MS SQL server, there are two options. The first is of course to store it as an integer in Unix time. The second would be to use the DateTime data type, but store all the times in UTC. The challenge arises when an existing database uses DateTime in a local time zone that uses daylight saving time. This situation actually arises in many software applications that do not use integer time style programming.

For starters there will be one hour a year that can not be used at all - the hour before the switch from daylight to standard time. There will also be one hour each year that is not used at all - the hour skipped over during the switch to DST. There is nothing can be done about this. Going from Unix time to the local time is a routine operation. Going from the DateTime fields in the database's locale time to Unix time will also work, provided mktime() is using a locale definition that can figure out witch times during the year are in DST, and ST. The trick is to set the tm_isdst member of the tm structure to -1. See Converting an arbitrary date in the local time locale to Unix time below.

Web Applications

It would certainly be useful if there was a good way to find out the local time zone that a web client is connecting from. This way a page could render its time zones correctly for each visitor. There is a way in javascript to return the time zone offset in minutes. To some extent, this could be resolved to a timezone in a lookup table. I recommend two very important design points for web applications: always specify the time zone when displaying a time, and give the user a way to change the output to a different time zone. For example, if a corporate intranet application is to be used by personal in three time zones, a drop down box could allow for the selection of any one of those three plus UTC. A conference call at 3:00 PM could be anything, but one at 3:00 PM CST is at least a real time even you are in MST. A drop down box allowing you to render the time as 2:00 PM MST is even better. Also don't forget to use the daylight saving time version of the time zone name. Because many places do not observe DST, people may mentally convert a time wrong even when they are aware of the differences in zones.

Pulling a Browser's Time Zone
<script language="JavaScript">
 var visitortime = new Date();
 document.write('<input type="hidden" name="x-VisitorTimeZoneOffset" ');
 if(visitortime)
 {
  document.write('value="' + visitortime.getTimezoneOffset() + '">');
 } else {
  document.write('value="JavaScript not Date() enabled">');
 }
</script>

<noscript>
 <input type="hidden" name="x-VisitorTimeZoneOffset" 
 value="Browser not JavaScript enabled">
</noscript>

Calander Functions in C cheat sheet

Getting the current Unix Time

#include <stdio.h>
#include <time.h>

int main(void)
{
 time_t now;
 now = time(NULL); /* time(&now) works to */
 printf("current unix time is %d\n", now);
 return 0;
}

Getting the current local time as text

#include <stdio.h>
#include <time.h>

int main(void)
{
 char time_string[1024];
 strftime(time_string, 1024, "%m/%d/%Y %H:%M:%S %Z", localtime(time(NULL))); 
 printf("%s\n", time_string);
 return 0;
}

Getting the current local time as broken down values

#include <stdio.h>
#include <time.h>

int main(void)
{
 struct tm *time_details = localtime(time(NULL)); 
 printf("for example the day of the month is %d\n", time_details->tm_mday);
 return 0;
}

Getting the current universal time as broken down values

#include <stdio.h>
#include <time.h>

int main(void)
{
 struct tm *time_details = gmtime(time(NULL)); 
 printf("for example the day of the month in UTC is %d\n", time_details->tm_mday);
 return 0;
}

Getting the current universal time as text

#include <stdio.h>
#include <time.h>

int main(void)
{
 char time_string[1024];
 strftime(time_string, 1024, "%m/%d/%Y %H:%M:%S Universal Coordinated Time", gmtime(time(NULL))); 
 printf("%s\n", time_string);
 return 0;
}

Converting an arbitrary date in the local time locale to Unix time

#include <stdio.h>
#include <time.h>

int main(void)
{
 time_t moment;
 struct tm unparsed;

 unparsed.tm_year  = 2006 - 1900; /* the year here is the years past 1900 */
 unparsed.tm_mon   = 12 - 1;      /* month starts a 0 */
 unparsed.tm_mday  = 21;
 unparsed.tm_hour  = 16;
 unparsed.tm_min   = 50;
 unparsed.tm_sec   = 1;
 unparsed.tm_isdst = -1;          /* set this to -1 */

 moment = mktime(&unparsed);
 return 0;
}

Getting the time zone offset value for the current locale - needs fixing

#include <stdio.h>
#include <time.h>

int main(void)
{
 tzset();
 printf("the current locale time is universal time adjusted by %d seconds\n", timezone);
 return 0;
}

Fining out if the current locale is currently in daylight saving time

#include <stdio.h>
#include <time.h>

int main(void)
{
 struct tm *time_details = localtime(time(NULL)); 
 
 /* note that localtime() and gmtime() internally call tzset() */

 if(time_details->tm_isdst)
  printf("right now in the current locale it is daylight saving time\n");
 else
  printf("right now in the current locale it is not daylight saving time\n");

 return 0;
}

Getting the text for the current locale's current time zone string

#include <stdio.h>
#include <time.h>

int main(void)
{
 struct tm *time_details;

 tzset();

 if(daylight)
 {
  time_details = localtime(time(NULL)); 
 
  printf("right now in the current locale the timezone string is '%s'\n",
         tzname[time_details->tm_isdst]);

 } else {

  printf("in the current locale timezone string is allways '%s'\n", tzname[0]);

 }

 return 0;
}

Converting an arbitrary time in universal time to Unix time

#include <stdio.h>
#include <time.h>

int main(void)
{
 int difference;
 time_t a, b, moment;
 struct tm unparsed, *temp;

 unparsed.tm_year  = 2006 - 1900; /* the year here is the years past 1900 */
 unparsed.tm_mon   = 12 - 1;      /* month starts a 0 */
 unparsed.tm_mday  = 21;
 unparsed.tm_hour  = 16;
 unparsed.tm_min   = 50;
 unparsed.tm_sec   = 1;
 unparsed.tm_isdst = -1;          /* set this to -1 */

 a = mktime(&unparsed);
 temp = gmtime(&a);
 b = mktime(temp);

 difference = a - b;
 moment = a + difference;  /* moment now stores the unix time of unparsed in UTC */

 return 0;
}

Converting an arbitrary time in an arbitrary time zone to Unix time (not portable)

#include <stdio.h>
#include <time.h>
#include <stdlib.h>

int main(void)
{
 time_t moment;
 struct tm unparsed;
 char old[256];

 if(getenv("TZ") == NULL)
  old[0] = 0;
 else
  snprintf(old, 256, getenv("TZ"));

 setenv("TZ", "UTC", 1); /* set to a value in the tree /usr/share/zoneinfo/ */

 unparsed.tm_year  = 2006 - 1900; /* the year here is the years past 1900 */
 unparsed.tm_mon   = 12 - 1;      /* month starts a 0 */
 unparsed.tm_mday  = 21;
 unparsed.tm_hour  = 16;
 unparsed.tm_min   = 50;
 unparsed.tm_sec   = 1;
 unparsed.tm_isdst = -1;          /* set this to -1 */

 moment = mktime(&unparsed); 

 if(old[0] == 0)
   unsetenv("TZ");
 else
   setenv("TZ", old, 1);
 

 return 0;
}

Rendering a Unix time in an arbitrary time zone (not portable)

#include <stdio.h>
#include <time.h>
#include <stdlib.h>

int main(void)
{
 time_t moment = 1148248201;
 struct tm *time_details;
 char old[256];

 if(getenv("TZ") == NULL)
  old[0] = 0;
 else
  snprintf(old, 256, getenv("TZ"));

 setenv("TZ", "UTC", 1); /* set to a value in the tree /usr/share/zoneinfo/ */ 

 time_details = localtime(moment); 

 if(old[0] == 0)
   unsetenv("TZ");
 else
   setenv("TZ", old, 1);
 

 return 0;
}

Contributing Credits

April 2011. Pointer syntax error fixed by Jan Theodore Galkowski


© 2006, 2011 C. Thomas Stover

cts at techdeviancy.com

back