Reading data from GPS devices

Training Support

Article from Issue 153/2013
Author(s):

With a small GPS receiver on his wrist, Mike has been jogging through San Francisco neighborhoods. While catching his breath, safe at home, he visualizes the data he acquired while running with Perl.

A few years ago, portable GPS devices looked more like the clunky cellphones of the early 1990s. Today, athletes no longer need to drag along that much extra weight, as devices like the Garmin Forerunner 10 [1] have shrunk to the size of digital LED watches from the 1970s (Figure 1). These ultimate sports accessories log geographic coordinates during runs.

Figure 1: The wristwatch-sized GPS receiver logs the coordinates of points on the route traveled with timestamps.

Thus, runners can see how fast they are currently traveling and whether they need to speed up or slow down to achieve their own time goals. After completing all of this muscular activity, runners can then enjoy the experience of logging new speed records, viewing the running route on a map, reviewing the miles traveled, or marveling at an altitude profile of the route.

After plugging the Garmin 10 device into a USB port on my Ubuntu machine (Figure 1), the Linux kernel immediately detects the device as a storage unit and mounts the files stored on the device under /media/GARMIN. However, to do this, I did need a special adapter cable that clings to the GPS watch like a creature from the movie Alien and that came with the Garmin package. In the GARMIN/ACTIVITY directory, Linux users will find the FIT files in which the manufacturer stores motion activities in a proprietary binary format.

Athletes can upload these files to a newly created account on the Garmin website [2] and then view a presentation of the run data (Figure 2). The example shows an approximately five-mile route around Lake Merced – a lake near the Pacific Ocean just south of San Francisco  – which I ran for this Perl column in about 40 minutes.

Figure 2: On the Gamin website, athletes can upload their FIT data for a graphical display.

The graph shows total time, average time per mile (8:45 minutes), and elevation change (300 feet). If you prefer metric units, you can change to kilometers and meters on the website.

The Six-Second Cycle

The GPS device determines the runner's position approximately every six seconds with the help of earth-orbiting navigation satellites; it then stores the data points as geographical latitudes and longitudes and measures the current speed and distance traveled.

Unlike other GPS units, the Garmin 10 does not determine the current altitude above sea level during the run; apparently, this would require more expensive hardware to get right. But the Garmin website effortlessly fills this topographic gap in the movement profile based on the coordinates, using a server-side static elevation profile that probably knows the altitude of any inhabited part of the earth. Unless you are running up the stairs in a high rise, you will receive accurate altitude information for your run in this way.

Do It Yourself

Even the neatest website can be improved; some people want to remodel their data for seemingly esoteric purposes. Instead of painstakingly decrypting the proprietary format, you can pick up the Garmin SDK online [3]. Although the SDK does not define a Perl API, it does document all the data structures used in the log data.

On the basis of this information, developer Kiyokazu Suto built a Perl module [4] under a public domain-like license, but so far it has not been uploaded to CPAN. From the developer's website, you can download Garmin::FIT, which reads FIT files; Figure 3 shows an example of how the fitdump utility included with the module outputs the data.

Figure 3: The raw data read from the FIT format shows that the device determines its latitude and longitude, the current speed, and the distance traveled approximately every six seconds.

Despite the format having been disclosed, extracting the data from the binary blob proves to be a Sisyphean task. The Garmin::FIT module offers the print_all_fields() method, which I leveraged to hash up the dumper in Listing  1 [5]; it produces the output in Figure 3. To process the data, developers need to dig deeper and write their own functions.

Listing 1

fittest

01 #!/usr/bin/perl
02 use warnings;
03 use strict;
04 use Garmin::FIT;
05
06 my $fit = Garmin::FIT->new();
07 $fit->file("354I2029.FIT");
08 $fit->
09 data_message_callback_by_name
10   ('', \&dump_it);
11
12 $fit->open();
13 $fit->fetch_header();
14 1 while $fit->fetch();
15
16 #############################
17 sub dump_it {
18 #############################
19  my ($self, $desc, $v) = @_;
20
21  if ($desc->{message_name}) {
22   print
23     "$desc->{message_name} ";
24  } else {
25   print "Unknown ";
26  }
27
28  print "(",
29    $desc->{message_number},
30    "):\n";
31
32  $self->print_all_fields(
33   $desc, $v, indent => '  ');
34 }

Extracting from FIT

Listing 2 thus takes on the task of reshaping the FIT data and producing an easily readable YAML file. To do this, line 14 loads a FIT file passed in at the command line into the Garmin::FIT module. The goal of the subsequent procedure is to create a Perl array with the data from the record entries in the FIT file and then call the DumpFile() function from the YAML module to dump them as YAML data into a .yaml file of the same name.

Listing 2

fit2yaml

01 #!/usr/bin/perl
02 use warnings;
03 use strict;
04 use Garmin::FIT;
05 use YAML qw( DumpFile );
06
07 my ($file) = @ARGV;
08 die "usage: $0 fit-file"
09   if !defined $file;
10 (my $yaml_file = $file) =~
11   s/.fit$/.yaml/i;
12
13 my $fit = Garmin::FIT->new();
14 $fit->file($file);
15
16 my $messages = [];
17
18 $fit->
19 data_message_callback_by_name
20   (
21  '',
22  sub {
23   my $msg = message(@_);
24   push @$messages, $msg
25     if $msg;
26   return 1;
27  }
28   );
29
30 $fit->open();
31 $fit->fetch_header();
32 1 while $fit->fetch();
33
34 print DumpFile($yaml_file,
35  $messages);
36
37 #############################
38 sub message {
39 #############################
40  my ($fit, $desc, $v) = @_;
41
42  if (!$desc->{message_name}
43   or
44   $desc->{message_name} ne
45   "record")
46  {
47   return undef;
48  }
49
50  my $m = {};
51
52  foreach
53    my $i_name (keys %$desc)
54  {
55
56   next if $i_name !~ /^i_/;
57   (my $name = $i_name) =~
58     s/^i_//g;
59
60   my $pname = $name;
61   my $attr =
62     $desc->{ 'a_' . $name };
63   my $i = $desc->{$i_name};
64   my $invalid =
65     $desc->{ 'I_' . $name };
66
67   $m->{$pname} =
68     $fit->value_cooked(
69    "", $attr,
70    "", $v->[$i]
71     );
72  }
73
74  return $m;
75 }

To search the FIT data, line 19 calls the data_message_callback_by_name() method and sets up a callback that the FIT parser invokes for each entry found. The callback function message defined beginning in line 38 extracts the important values from the given parameters and puts them together to create a new data structure.

As Figure 4 shows, the current total run distance does not reside directly in the distance entry of the variable $v presented to the callback. Instead, the data structure $desc contains a number of values that the user must combine in mysterious ways to arrive at the desired result. To get the total distance, for example, the i_distance key in $desc contains an array index number (the number 4 in this case), which can be used to extract the desired total distance from the $v array as $v[4]. Then, fit2yaml combines this with the unit m for meters and determines the result with the Garmin::FIT value_cooked() method.

Figure 4: The Garmin format proves to be idiosyncratic. The distance, for example, is coded in $v[4].

The method still needs the value for a_distance (scaling and units) and the value for I_distance (validity scope) so that it can finally arrive at the desired value for the distance. Because the format can store all kinds of data in a tiny space, this method was most likely chosen to save space.

For simplicity's sake, Listing 2 does not attempt to handle all the supported data formats but focuses on the record entries with the run data sampled every six seconds. Other logged events, such as where and when the runner pressed the Start button, are ignored to limit the scope of the script.

Figure 5 shows the finished YAML data: a list of records that each contain a distance field (distance traveled from the start in kilometers), position_lat (latitude), position_long (longitude), speed (in meters per second), and timestamp (current time).

Figure 5: These entries show the Garmin GPS data in YAML format, as determined by the script in Listing 2.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Perl: Plotting GPS Data

    Perl hackers take to the hills with a navigation system that provides a graphical rendering of a hiking tour.

  • Perl: Automating Color Correction

    If you have grown tired of manually correcting color-casted images (as described in last month's Perl column), you might appreciate a script that automates this procedure.

  • Perl: Jawbone UP Data

    The Jawbone UP electronic bracelet measures the wearer's daily activity and nocturnal sleep patterns. If you are bored by the official smartphone app, you can create your own Perl scripts and access your data via the unofficial web API.

  • GPS Tools

    Almost all manufacturers of GPS devices use proprietary formats to save routes, tracks, and waypoints. Vendors unfortunately rarely offer Linux software for uploading and downloading or processing the data. Four GPS editors keep Linux users on the right track.

  • Orca

    Monitor and troubleshoot Linux system performance with the free and powerful Orca.

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95

News