Extract and analyze GPS data with Go
Simply Complicated
This is by no means as simple as in scripting languages such as Python, where hashes are also unsorted, but a sort
command on the hash keys quickly returns them in order, for a loop to process the entries. In Go, the for
loop starting in line 22 of Listing 3 first needs to collect all the keys in the hash map and push them into a newly created array slice. The sort.Slice()
function then sorts the slice by time in ascending order, starting in line 25.
With an array slice of strings, you could do the whole thing quickly with sort.Strings()
, but because the hash keys are time.Time
type data, line 26 still has to define a callback function that tells the sort algorithm which of two array values at indexes i
versus j
is now larger. Fortunately, the time.Time
type has a Before()
function that returns exactly this result, so the callback function simply returns the result of that, because that's exactly what the comparison function is supposed to determine.
With the keys sorted by time in ascending order, the for
loop starting in line 31 can now pass the hash entries, also sorted in a separate array slice, to the go-chart
plotting library that line 5 in Listing 3 pulls from GitHub. It draws visually appealing charts as bar diagrams, pie charts, or function graphs. In the case at hand, the data exists as a series of values at specific time points, so line 36 defines the chart as chart.TimeSeries
and sets the fill and stroke color to blue. You can compile the program with
go build activity.go gpxread.go
because it loads in the gpxFiles()
and gpxPoints()
functions from Listing 1 to support its plot instructions. The output it generates is an image file named activity.png
(Figure 4).
Real World
But given the GPX coordinates in geographical longitude and latitude, how do you discover the country in which they are located, or even the state, city, or street? This function is the domain of geomapping tools. This is actually a reverse case – it's not a matter of inferring the geolocation from a given address but discovering the city from the given coordinates. To do this, you ultimately need a huge database that assigns places on the Earth's map to their GPS coordinates in as much detail as possible.
To perform this calculation, there are various services online vying for the favor of their paying customers, who are then allowed to access their treasure trove of data via API. Google Maps originally offered this kind of mapping free of charge, but the boffins at Google withdrew this offer some time ago and now require a credit card to register. The company then bills you if you exceed a certain number of requests.
But if you look around online, you will find a number of freemium offers from some providers such as OpenCage [4], where you can register by email and receive an API token for a limited number of requests for a trial. As an example of using the API, the GeoRev()
function, starting in line 25 of Listing 4, converts a combination of the latitude and longitude in float64 format to the state at that location. This means that 37.7, -122.4
becomes California
and 49.4, 8.7
becomes Bavaria
.
Listing 4
georev.go
01 package main 02 03 import ( 04 "encoding/json" 05 "fmt" 06 "github.com/peterbourgon/diskv" 07 "io/ioutil" 08 "net/http" 09 "net/url" 10 ) 11 12 type GeoState struct { 13 ApiKey string 14 Cache *diskv.Diskv 15 } 16 17 func NewGeoState() *GeoState { 18 state := GeoState{ 19 ApiKey: "<API-Key>", 20 Cache: diskv.New(diskv.Options{BasePath: "cache"}), 21 } 22 return &state 23 } 24 25 func (state *GeoState) GeoRev(lat, lng float64) string { 26 key := roundedLatLng(lat, lng) 27 28 res, err := state.Cache.Read(key) 29 if err != nil { 30 res = []byte(state.GeoLookup(lat, lng)) 31 state.Cache.Write(key, res) 32 } 33 34 return string(res) 35 } 36 37 func roundedLatLng(lat, lng float64) string { 38 return fmt.Sprintf("%.1f,%.1f", lat, lng) 39 } 40 41 func (state *GeoState) GeoLookup(lat, lng float64) string { 42 u := url.URL{ 43 Scheme: "https", 44 Host: "api.opencagedata.com", 45 Path: "geocode/v1/json", 46 } 47 q := u.Query() 48 q.Set("key", state.ApiKey) 49 q.Set("q", roundedLatLng(lat, lng)) 50 u.RawQuery = q.Encode() 51 52 resp, err := http.Get(u.String()) 53 if err != nil { 54 panic(err) 55 } 56 57 body, err := ioutil.ReadAll(resp.Body) 58 if err != nil { 59 panic(err) 60 } 61 return stateFromJson(body) 62 } 63 64 func stateFromJson(txt []byte) string { 65 var data map[string]interface{} 66 json.Unmarshal(txt, &data) 67 68 results := data["results"].([]interface{})[0].(map[string]interface{}) 69 return results["components"].(map[string]interface{})["state"].(string) 70 }
Figure 5 shows the detailed JSON response provided by the OpenCage server for a coordinate in the San Francisco metropolitan area. Everything from the zip code to the street name, country, state, and so on is shown there. Of course, this only works if Listing 4 (in line 19) contains a valid API key, which is available on the OpenCage site if you register your email (no credit card required).
Using Go to extract data from a server response with JSON data is always difficult if there is no corresponding Go type on the client side that can reproduce the data structure down to the last detail. The alternative is a hack that relies on type assertion to force the data into hash maps with interface{}
types. The stateFromJson()
function works its way through the results
, components
, and state
entries in the JSON data before it finds the nugget it's looking for, while Go uses type assertions to constantly assure the function that the next part really is of the expected type.
Smart Cache
Of course, it would be pretty stupid to bomb the server with thousands of requests for almost identical track points – on the one hand, because this requires a round trip across the Internet each time, and on the other, because the limited quota of the free trial account would be quickly consumed.
This is why Listing 4 defines a cache/
directory, where the simple discv
cache implementation from GitHub permanently caches previously retrieved API results for later use. The roundedLatLng()
function, starting in line 37, rounds the longitude and latitude to one decimal place for incoming requests and first looks to see if there is already a result in the cache. If not, it fetches the corresponding geodata with a web request to the server's API and stores what it needs from it in the cache. While the program is running, the cache fills up visibly with the incoming requests, as a quick look at the cache
directory in Figure 6 reveals.
Armed with this package, Listing 5 can now draw a pie chart indicating which states the collection of GPX files cover in total and where the most hiking steps were recorded. To do this, in the perState
hash map, it increments the entry under the state string key by one. The more track points there are in the respective region, the higher the count value, and the bigger the pie slice in the graph later on.
Listing 5
states.go
01 package main 02 03 import ( 04 "github.com/wcharczuk/go-chart/v2" 05 "os" 06 ) 07 08 func main() { 09 geo := NewGeoState() 10 perState := map[string]int{} 11 12 for _, path := range gpxFiles() { 13 for _, pt := range gpxPoints(path) { 14 state := geo.GeoRev(pt.Latitude, pt.Longitude) 15 perState[state]++ 16 } 17 } 18 19 vals := []chart.Value{} 20 for state, count := range perState { 21 vals = append(vals, chart.Value{Value: float64(count), Label: state}) 22 } 23 24 pie := chart.PieChart{ 25 Width: 512, Height: 512, 26 Values: vals, 27 } 28 29 f, _ := os.Create("states.png") 30 defer f.Close() 31 pie.Render(chart.PNG, f) 32 }
Listing 5 generates an image file named states.png
, and Figure 7 shows the results: My GPX files are mostly from California but also feature the German states of Bavaria, Baden-Württemberg, and Lower Saxony.
« Previous 1 2 3 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Red Hat Adds New Deployment Option for Enterprise Linux Platforms
Red Hat has re-imagined enterprise Linux for an AI future with Image Mode.
-
OSJH and LPI Release 2024 Open Source Pros Job Survey Results
See what open source professionals look for in a new role.
-
Proton 9.0-1 Released to Improve Gaming with Steam
The latest release of Proton 9 adds several improvements and fixes an issue that has been problematic for Linux users.
-
So Long Neofetch and Thanks for the Info
Today is a day that every Linux user who enjoys bragging about their system(s) will mourn, as Neofetch has come to an end.
-
Ubuntu 24.04 Comes with a “Flaw"
If you're thinking you might want to upgrade from your current Ubuntu release to the latest, there's something you might want to consider before doing so.
-
Canonical Releases Ubuntu 24.04
After a brief pause because of the XZ vulnerability, Ubuntu 24.04 is now available for install.
-
Linux Servers Targeted by Akira Ransomware
A group of bad actors who have already extorted $42 million have their sights set on the Linux platform.
-
TUXEDO Computers Unveils Linux Laptop Featuring AMD Ryzen CPU
This latest release is the first laptop to include the new CPU from Ryzen and Linux preinstalled.
-
XZ Gets the All-Clear
The back door xz vulnerability has been officially reverted for Fedora 40 and versions 38 and 39 were never affected.
-
Canonical Collaborates with Qualcomm on New Venture
This new joint effort is geared toward bringing Ubuntu and Ubuntu Core to Qualcomm-powered devices.