Extract and analyze GPS data with Go
Simply Complicated
This is by no means as simple as in scripting languages such as Python, where hashes are also unsorted, but a sort
command on the hash keys quickly returns them in order, for a loop to process the entries. In Go, the for
loop starting in line 22 of Listing 3 first needs to collect all the keys in the hash map and push them into a newly created array slice. The sort.Slice()
function then sorts the slice by time in ascending order, starting in line 25.
With an array slice of strings, you could do the whole thing quickly with sort.Strings()
, but because the hash keys are time.Time
type data, line 26 still has to define a callback function that tells the sort algorithm which of two array values at indexes i
versus j
is now larger. Fortunately, the time.Time
type has a Before()
function that returns exactly this result, so the callback function simply returns the result of that, because that's exactly what the comparison function is supposed to determine.
With the keys sorted by time in ascending order, the for
loop starting in line 31 can now pass the hash entries, also sorted in a separate array slice, to the go-chart
plotting library that line 5 in Listing 3 pulls from GitHub. It draws visually appealing charts as bar diagrams, pie charts, or function graphs. In the case at hand, the data exists as a series of values at specific time points, so line 36 defines the chart as chart.TimeSeries
and sets the fill and stroke color to blue. You can compile the program with
go build activity.go gpxread.go
because it loads in the gpxFiles()
and gpxPoints()
functions from Listing 1 to support its plot instructions. The output it generates is an image file named activity.png
(Figure 4).
Real World
But given the GPX coordinates in geographical longitude and latitude, how do you discover the country in which they are located, or even the state, city, or street? This function is the domain of geomapping tools. This is actually a reverse case – it's not a matter of inferring the geolocation from a given address but discovering the city from the given coordinates. To do this, you ultimately need a huge database that assigns places on the Earth's map to their GPS coordinates in as much detail as possible.
To perform this calculation, there are various services online vying for the favor of their paying customers, who are then allowed to access their treasure trove of data via API. Google Maps originally offered this kind of mapping free of charge, but the boffins at Google withdrew this offer some time ago and now require a credit card to register. The company then bills you if you exceed a certain number of requests.
But if you look around online, you will find a number of freemium offers from some providers such as OpenCage [4], where you can register by email and receive an API token for a limited number of requests for a trial. As an example of using the API, the GeoRev()
function, starting in line 25 of Listing 4, converts a combination of the latitude and longitude in float64 format to the state at that location. This means that 37.7, -122.4
becomes California
and 49.4, 8.7
becomes Bavaria
.
Listing 4
georev.go
01 package main 02 03 import ( 04 "encoding/json" 05 "fmt" 06 "github.com/peterbourgon/diskv" 07 "io/ioutil" 08 "net/http" 09 "net/url" 10 ) 11 12 type GeoState struct { 13 ApiKey string 14 Cache *diskv.Diskv 15 } 16 17 func NewGeoState() *GeoState { 18 state := GeoState{ 19 ApiKey: "<API-Key>", 20 Cache: diskv.New(diskv.Options{BasePath: "cache"}), 21 } 22 return &state 23 } 24 25 func (state *GeoState) GeoRev(lat, lng float64) string { 26 key := roundedLatLng(lat, lng) 27 28 res, err := state.Cache.Read(key) 29 if err != nil { 30 res = []byte(state.GeoLookup(lat, lng)) 31 state.Cache.Write(key, res) 32 } 33 34 return string(res) 35 } 36 37 func roundedLatLng(lat, lng float64) string { 38 return fmt.Sprintf("%.1f,%.1f", lat, lng) 39 } 40 41 func (state *GeoState) GeoLookup(lat, lng float64) string { 42 u := url.URL{ 43 Scheme: "https", 44 Host: "api.opencagedata.com", 45 Path: "geocode/v1/json", 46 } 47 q := u.Query() 48 q.Set("key", state.ApiKey) 49 q.Set("q", roundedLatLng(lat, lng)) 50 u.RawQuery = q.Encode() 51 52 resp, err := http.Get(u.String()) 53 if err != nil { 54 panic(err) 55 } 56 57 body, err := ioutil.ReadAll(resp.Body) 58 if err != nil { 59 panic(err) 60 } 61 return stateFromJson(body) 62 } 63 64 func stateFromJson(txt []byte) string { 65 var data map[string]interface{} 66 json.Unmarshal(txt, &data) 67 68 results := data["results"].([]interface{})[0].(map[string]interface{}) 69 return results["components"].(map[string]interface{})["state"].(string) 70 }
Figure 5 shows the detailed JSON response provided by the OpenCage server for a coordinate in the San Francisco metropolitan area. Everything from the zip code to the street name, country, state, and so on is shown there. Of course, this only works if Listing 4 (in line 19) contains a valid API key, which is available on the OpenCage site if you register your email (no credit card required).
Using Go to extract data from a server response with JSON data is always difficult if there is no corresponding Go type on the client side that can reproduce the data structure down to the last detail. The alternative is a hack that relies on type assertion to force the data into hash maps with interface{}
types. The stateFromJson()
function works its way through the results
, components
, and state
entries in the JSON data before it finds the nugget it's looking for, while Go uses type assertions to constantly assure the function that the next part really is of the expected type.
Smart Cache
Of course, it would be pretty stupid to bomb the server with thousands of requests for almost identical track points – on the one hand, because this requires a round trip across the Internet each time, and on the other, because the limited quota of the free trial account would be quickly consumed.
This is why Listing 4 defines a cache/
directory, where the simple discv
cache implementation from GitHub permanently caches previously retrieved API results for later use. The roundedLatLng()
function, starting in line 37, rounds the longitude and latitude to one decimal place for incoming requests and first looks to see if there is already a result in the cache. If not, it fetches the corresponding geodata with a web request to the server's API and stores what it needs from it in the cache. While the program is running, the cache fills up visibly with the incoming requests, as a quick look at the cache
directory in Figure 6 reveals.
Armed with this package, Listing 5 can now draw a pie chart indicating which states the collection of GPX files cover in total and where the most hiking steps were recorded. To do this, in the perState
hash map, it increments the entry under the state string key by one. The more track points there are in the respective region, the higher the count value, and the bigger the pie slice in the graph later on.
Listing 5
states.go
01 package main 02 03 import ( 04 "github.com/wcharczuk/go-chart/v2" 05 "os" 06 ) 07 08 func main() { 09 geo := NewGeoState() 10 perState := map[string]int{} 11 12 for _, path := range gpxFiles() { 13 for _, pt := range gpxPoints(path) { 14 state := geo.GeoRev(pt.Latitude, pt.Longitude) 15 perState[state]++ 16 } 17 } 18 19 vals := []chart.Value{} 20 for state, count := range perState { 21 vals = append(vals, chart.Value{Value: float64(count), Label: state}) 22 } 23 24 pie := chart.PieChart{ 25 Width: 512, Height: 512, 26 Values: vals, 27 } 28 29 f, _ := os.Create("states.png") 30 defer f.Close() 31 pie.Render(chart.PNG, f) 32 }
Listing 5 generates an image file named states.png
, and Figure 7 shows the results: My GPX files are mostly from California but also feature the German states of Bavaria, Baden-Württemberg, and Lower Saxony.
« Previous 1 2 3 Next »
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Fedora 40 Beta Released Soon
With the official release of Fedora 40 coming in April, it's almost time to download the beta and see what's new.
-
New Pentesting Distribution to Compete with Kali Linux
SnoopGod is now available for your testing needs
-
Juno Computers Launches Another Linux Laptop
If you're looking for a powerhouse laptop that runs Ubuntu, the Juno Computers Neptune 17 v6 should be on your radar.
-
ZorinOS 17.1 Released, Includes Improved Windows App Support
If you need or desire to run Windows applications on Linux, there's one distribution intent on making that easier for you and its new release further improves that feature.
-
Linux Market Share Surpasses 4% for the First Time
Look out Windows and macOS, Linux is on the rise and has even topped ChromeOS to become the fourth most widely used OS around the globe.
-
KDE’s Plasma 6 Officially Available
KDE’s Plasma 6.0 "Megarelease" has happened, and it's brimming with new features, polish, and performance.
-
Latest Version of Tails Unleashed
Tails 6.0 is based on Debian 12 and includes GNOME 43.
-
KDE Announces New Slimbook V with Plenty of Power and KDE’s Plasma 6
If you're a fan of KDE Plasma, you'll be thrilled to hear they've announced a new Slimbook with an AMD CPU and the latest version of KDE Plasma desktop.
-
Monthly Sponsorship Includes Early Access to elementary OS 8
If you want to get a glimpse of what's in the pipeline for elementary OS 8, just set up a monthly sponsorship to help fund its continued existence.
-
DebConf24 to be Held in South Korea
Busan will be the location of the latest DebConf running July 28 through August 4