The Lesser Known 'Epoch'-ness Monster

Have you ever heard of or dealt with UNIX Epoch (also known as POSIX) time? Ok, so maybe I’m not talking about a monster, but an information technology system for describing time. I’m not planning to retype what UNIX epoch represents, so I’ll send you to this wiki page Unix Epoch. (Here you can read a good explanation and history behind it.) However, I will speak to how you could encounter this time encoding/format in the incident response and digital forensics fields as well as providing a simple solution to a small nuisance that I recently came across.

Throughout an infosec professionals’ career, there are many situations in which one may come across an epoch timestamp. Here’s a couple examples:

• Viewing packet capture files using tools such as tcpdump and tshark. By default, tcpdump and tshark will display the UNIX epoch time when capturing packets (on non-Windows machines)
• MySql database analysis. MySql supports UNIX epoch time.
• Squid proxy logs can be displayed in UNIX epoch time.
• A lot of software running on a *NIX operating system will internally log time in UNIX epoch time, although some may display the time in a converted form by default. Essentially, analyzing data sourced from a linux, UNIX, or Mac OS X platform may be subject to displaying this time format to the analyst.

Recently, I was provided a csv log file to analyze. Below is a file I generated to provide a quick example.

Notice the first column… It contains a 32 bit integer that represents UNIX epoch time. This time representation is the amount of seconds since 01/01/1970 UTC/GMT. I don’t know about you, but there’s no way that I’m going to convert this time into a format that makes sense to me in my head. Back to the log file… There were thousands of lines in this log file so using an online utility such as http://www.onlineconversion.com/unix_time.htm to convert each timestamp manually would not be practical. Three initial solutions came to mind during the brainstorming process.

Possible Solution #1

The first possible solution deals with doing the calculations to determine how many seconds/hrs/days/years the 32 bit integer represents. I most likely would need to determine how many days/hours/seconds existed in each year, while realizing caveats such as leap seconds not being counted, etc. This could be done, but it would require a lot of time to write and then thoroughly test. After the calculations portion is vetted, I would need to append this timestamp to the original log file. This option will be a last resort if I can’t leverage and existing way of performing this action.

Possible Solution #2

The second possible solution includes writing a bash script that would grab the value of the first column of every line, post it to an online website that has already invested the time into performing the calculations, read and parse the results from the web page, and then copy that to a temp file to be appended to the original file. I realized that if I were to pursue this route, I could potentially DoS the website, especially if the log file had thousands of lines in it (I would need to create a single request for each line in repetition). This is not ideal and I would most likely not choose this option for such large conversions.

Possible Solution #3

The third solution that came to mind includes writing a script to convert each epoch date per line and then writing this to a new file. This solution would use a cool feature of the “coreutils” suite of tools. Specifically, the little known use of the “@” switch within the coreutils “date” command. Note that the version of coreutils that this was tested on was ‘GNU coreutils v8.13’. Another note: To determine the version of coreutils that your *NIX system is running, just execute any of the coreutils applications with the switch “--version”.

$ date --version

date (GNU coreutils) 8.13.

…

Needless to say, this is the solution that I chose to pursue.

Below is the command that performs the conversion from UNIX epoch time to an output that is easily understood by analysts.

$ date -d @1382406394

Mon Oct 21 21:46:34 EDT 2013

Here’s the nifty little bash script that takes the first column from each line in the file, converts the UNIX epoch time to your systems current locale (utilizing the date command) and writes the new line to standard out, allowing you to save the output to a new file if desired. A little awk foo was leveraged as well...

This short article was written to provide an infosec analyst with a simple understanding of UNIX epoch time and it was also written to show a quick solution to a problem that I encountered. You never know what type of problems you will run into, so having an experienced set of problem solving skills is essential to an infosec professional.