HTML5 Offline
Offline-capable applications with HTML5
ByAn offline cache in your browser and a bit of HTML5 acrobatics combine for interactive web applications that keep working even when the Internet connection breaks down.
Web browsers were originally simply viewers for static HTML documents. Since HTML2, browsers have had the capability to capture simple user input with forms and send it to a database on the Internet. For this to work, of course, the system must be connected to the Internet. At every disturbance of the network connection, exasperated users lose their work.
HTML5 includes some powerful features that let web applications operate offline. In this article, I introduce these offline technologies in detail. I’ll also show you how to build an example application that implements an offline-capable tool for saving geo-coordinates (geolocations). This app runs on any devices that supports HTML’s offline features, including Firefox (currently v14) , Opera (v12), Chrome (v21), Safari, and Internet Explorer (v9).
Understanding Offline HTML
To operate without an Internet connection, an offline HTML5 app must keep an offline copy of the application and any application data. HTML5 defines the following components:
- application cache – a cache used for storing components of the application itself, such as HTML and JavaScript files.
- Web Storage – used for storing key-value pairs
- Web SQL – used for storing data in an SQL database
- IndexedDB – stores data in a NoSQL database.
To start, I’ll describe these important offline storage components before putting them to work in a real HTML5 offline application.
Application Cache
The application cache is controlled by a cache manifest file that is deposited on the web server. The manifest in Listing 1 (article code is available online) causes the browser to store all files in the application cache that are listed after the keyword CACHE. The cache manifest supports both relative and absolute URL paths.
Listing 1: Cache Manifest
01 CACHE MANIFEST 02 03 CACHE: 04 index.html 05 js/jquery-1.7.1.min.js 06 js/script.js 07 js/jquery.mobile-1.1.0.min.css 08 js/jquery.mobile-1.1.0.min.js 09 js/OpenLayers.js 10 11 NETWORK: 12 /couchdb/ 13 *
Users must give permission before the browser is allowed to save the files. If the browser does not ask, users can change the settings – for example, via the Firefox context menu under View Page Info | Permissions. In Figure 1, the user has allowed data to be stored in the application cache (see the last setting at the bottom).
The URLs in the NETWORK section of the cache manifest (Listing 1) do not land in the application cache; the browser instead requests them each time anew from the web server. The * wildcard allows the system to access every URL in the web. Developers associate an HTML file with the cache manifest by specifying a path in the manifest attribute (Listing 2).
Listing 2: Integrating the Manifest
01 <!DOCTYPE HTML> 02 <html manifest="geotrack.appcache"> 03 [...] 04 </html>
This path is relative to the HTML file. The first time it is called up, the browser reads the cache manifest asynchronously and stores the files listed in the CACHE section in the application cache of the browser. After that, each time the HTML file is called up, the web browser verifies the validity of the manifest. If the browser is in offline mode or no network connection is available, the browser loads the files from the application cache.
If the manifest has changed even one bit, the web browser reads it again and loads the files from the CACHE section into the application cache once again. Chrome lets you have a look at how the cache manifest is processed (Figure 2).
For the web browser to recognize the manifest, the corresponding MIME type must be set on the Apache web server by adding the
AddType cache-manifest .appcache
line to your config file.
Web Storage
Cookies make it easy to keep track of a session under the HTTP protocol, but they are ill-suited for permanent storage of application data in a web browser. For this reason, the W3C created Web Storage in 2009. In contrast to the alternatives Web SQL and IndexedDB, this kind of storage is included in the current versions of all web browsers.
Web Storage offers two data storage types: Session storage saves data per tab, and local storage saves per URL. Both session storage and local storage deposit key-value pairs, but only the data in local storage remains permanently. The window.LocalStorage object implements local storage with JavaScript (Listing 3).
Listing 3: Local Storage
01 window.LocalStorage = { 02 storage: window.localStorage, 03 04 save : function(obj) { 05 var pos = this._unpack('pos'); 06 pos[obj.key] = obj; 07 this.storage.setItem('pos', JSON.stringify(pos)); 08 }, 09 10 read : function(key, cbk) { 11 cbk(this._unpack('pos')[key]); 12 }, 13 14 _unpack : function(key) { 15 var data = this.storage.getItem(key) 16 return (data)?jQuery.parseJSON(data):{}; 17 } 18 };
The storage attribute in line 2 refers to the local storage. The save method in line 4 stores objects in a field. Line 7 then writes this with the setItem() method to the local storage. To obtain a value, stringify() translates the field into a character string in JSON format.
An object is read with the read method (line 10). Like save, this method also first unpacks the field of the object with unpack(). Unpack reads the serialized field from local storage with getItem() (line 15). Finally, parseJSON() reconstructs the object. The read method uses the results of the read operation to call up the user-defined callback function from the parameter list (see line 10).
Web SQL
Although local storage is perfectly adequate for permanently storing key-value pairs, it is not suitable for retaining structured data. The processing duration of the stringify() method grows quickly with the size of the object to be serialized (Figure 3).
W3C also created Web SQL, which can access an SQL database from web browsers with JavaScript. That solution is controversial, however, and further development of the standard was discontinued at the close of 2010. Web SQL is included in the current versions of the Opera, Chrome, and Safari web browsers.
One example for Web SQL is shown in Listing 4.
Listing 4: Wrapper Object for Web SQL
01 window.WebSQL = { 02 _open : function() { 03 return window.openDatabase('geos', '', 'myDB', 50000000); 04 }, 05 06 reinstall : function(vers) { 07 var db = this._open(); 08 db.changeVersion(db.version, vers, function(t) { 09 t.executeSql('CREATE TABLE pos (id, lat float, lon float)'); 10 }); 11 }, 12 13 read : function (key, cbk) { 14 this._open().readTransaction(function(t) { 15 t.executeSql('SELECT * FROM pos WHERE id = ?', [key], 16 function(t, d) {d.rows.length?cbk(d.rows.item(0)):''}, 17 function(t, e) {console.log(e.message)}); 18 }); 19 }, 20 21 save : function(obj) { 22 this._open().transaction(function(t,r) { 23 t.executeSql('INSERT INTO pos (id, lon, lat) VALUES (?, ?, ?)', 24 [obj.key, obj.lon, obj.lat]) 25 }); 26 } 27 };
Before the SQL instructions, line 3 must first open the database. The openDatabase() method requires the name of the database, the version, a display name, and the presumed size of the database in bytes. An empty version number will always open the newest version.
In line 14, readTransaction() reads the data; transaction() (line 22) is for writing. Both methods run asynchronously (see the box “Asynchronous Method Invocation”). The executeSql() method in lines 9, 15, and 23 executes SQL instructions. The question mark in the SQL statements is a wildcard that is replaced by the elements in the fields that follow.
The success callback from executeSql() in line 16 calls up the user-defined callback function cbk for the first line of the results – d.rows.item(0). The error callback in line 17 writes an error message to the console. The database schema can be changed with the changeVersion() method (line 8). This function requires the old and new version numbers. The instructions for manipulating the schema are transferred by the success callback.
IndexedDB
Mozilla refuses to implement Web SQL in Firefox, and depicting objects in the columns of a database table (object-relational mapping) is complicated. Many consider it easier to use an object store, such as the IndexedDB component. IndexedDB stores objects according to their keys and, with transactions and indexes, includes important aspects of a database system capable of supporting multiple users. IndexedDB has been partially implemented in the current versions of Firefox and Chrome, as well as in Microsoft’s upcoming Internet Explorer 10.
Listing 5 shows IndexedDB at work: Line 2 references the database object with vendor prefixes of several browsers. In line 6, the JavaScript code opens the database, and in line 12, it starts a transaction. The first parameter of the transaction method delivers a list of object stores that are to be processed. The second marks read transactions with 0 and writes with 1; from version 13 on, Firefox uses readonly and readwrite, as prescribed by the standard. Only then does the following objectStore() method allow access to the data in the pos object store.
Listing 5: Wrapper Object for IndexedDB
01 window.IndexedDB = { 02 indexedDB : window.indexedDB || window.webkitIndexedDB || window.mozIndexedDB, 03 storeParams: {keyPath:'key', autoIncrement:false}, 04 05 open : function(cbk, mod) { 06 var orq = this.indexedDB.open('geos'); 07 orq.onsuccess = function(e) { 08 if (window.mozIndexedDB) { 09 try { 10 cbk(e.target.result.transaction(['pos'], mod).objectStore('pos')); 11 } catch(l) { 12 cbk(e.target.result.transaction(['pos'], ((mode === 0)?'readonly':'readwrite')).objectStore('pos')); 13 } 14 } else { 15 cbk(e.target.result.transaction(['pos'], mod).objectStore('pos')); 16 } 17 }}, 18 19 reinstall : function() { 20 this.indexedDB.deleteDatabase('geos'); 21 var orq = this.indexedDB.open('geos'); 22 orq.onupgradeneeded = function(e) { 23 e.target.result.createObjectStore('pos', IndexedDB.storeParams); 24 }; 25 orq.onsuccess = function(e) { 26 if (e.target.result.setVersion) { //chromium 27 var crq = e.target.result.setVersion(2); 28 crq.onsuccess = function(e) { 29 e.target.source.createObjectStore('pos', IndexedDB.storeParams); 30 }}}; 31 }, 32 33 read : function(key, cbk) { 34 this.open(function(obst) { 35 obst.get(key).onsuccess = function(e) {cbk(e.target.result);} 36 }, 0); 37 }, 38 39 save : function(obj) { 40 this.open(function(obst) {obst.put(obj);}, 1); 41 } 42 };
In line 35, read uses the get() method to read the object asynchronously with the key key. The success callback calls up the user-defined callback function cbk() for the result of the read operation. The save in line 39 uses the put() method to store an object. IndexedDB extracts the key from keyPath: 'key' in line 3. This agreement gives IndexedDB the path to find the key in an object.
The code in lines 23 and 29 delivers the parameter while the object store pos is being created by the createObjectStore() method. After the pos database is deleted in line 20, the code lets the onupgradeneeded event create it anew. This also occurs when a database is first opened. In the Chrome browser, the setVersion() method comes into action in the success callback of the opening request. Line 27 specifies a higher version number.
Uniform API
The three wrapper objects in Listings 3 to 5 offer a uniform API for storing objects. The autoConnect method from the DataConnector object in Listing 6 selects the best interface. If none of the storage methods is appropriate, throw causes an error.
Listing 6: Initializing the Offline Cache
01 window.DataConnector = { 02 autoConnect : function() { 03 if (IndexedDB.indexedDB) { 04 return IndexedDB; 05 } else if (window.openDatabase) { 06 return WebSQL; 07 } else if (window.localStorage) { 08 return LocalStorage; 09 } 10 throw new Error({'message': 'no storage class available!'}); 11 } 12 };
An Example Application
The application cache and the storage methods describe previously allow the developer to build a web application capable of surviving offline. In the rest of this article, I describe the programming of the example application shown in Figure 4.
The demo installation, Geotracker (in German), lets the user save and display geo-coordinates on a mobile device. With a click on New Entry, the user can mark a new position in the map. The current position is already set. After saving the location, the app first deposits the position in one of the three local data storage areas and then places a red marker on the map. The position also lands in a list below the map. Although geolocation is now a standard feature on smartphones and other mobile devices, this simple example will show you how to build an application with HTML5’s important new offline functionality.
Geotracker employs a collection of different web technologies. Local Storage, Web SQL, and IndexedDB ensure that the application data is available offline. The Geolocation API determines the location of the mobile device, and the OpenLayers library shows the position on a map. jQuery Mobile draws up a graphic interface that works on both desktops and smartphones. Finally, CouchDB stores the locations on the server side, so they are available for multiple users.
As a restart of the browser demonstrates, the saved data is retained, and the application can even start without the need for a network connection. Those who are interested in what goes on under the surface should use the current versions of the Firefox, Opera, or Chrome web browser for testing. Chrome supports all technologies described in this article. Furthermore, Google’s web browser provides an uncluttered console for inspecting the web application. You can summon the console with the Ctrl+Shift+I keyboard shortcut.
The web browser can determine the current position of a mobile device with the Geolocation API. Firefox and Chrome determine the position with a query to Google, which analyses the information of neighboring WiFi networks and the IP address. The Geolocation API packages position data in an object. This data includes the latitude and longitude; the accuracy in degrees; the altitude above sea level, as well as the altitude accuracy (altitudeAccuracy) in meters; the current speed on the ground (velocity) in meters per second; and the direction of movement (heading) in degrees – relative to 0° north and increasing clockwise.
Position Assessment
The Geolocation API is implemented in all current versions of the Firefox, Opera, Chrome, Safari, and Internet Explorer web browsers, but unfortunately, it doesn’t work in Firefox under Linux. The Geolocation Shim script is an alternative way to determine your geolocation with Google. In any case, the WiFi interface should be activated for the operation.
Listing 7 shows how the programmer utilizes the Geolocation API. The getCurrentPosition() function in lines 3 to 7 determines the position asynchronously. The success callback is found in the first parameter of the function, and the error callback is in the second.
Listing 7: Wrapper Object
01 window.Geolocation = { 02 current : function() { 03 navigator.geolocation.getCurrentPosition( 04 function(pos) {Geolocation.success(pos)}, 05 function(err) {console.log(err.code);}, 06 {highAccuracy:true, maximumAge:0, timeout:1000} 07 ); 08 }, 09 10 watch : function() { 11 navigator.geolocation.watchPosition( 12 function(pos) {Geolocation.success(pos)} 13 ); 14 }, 15 16 success : function(pos) { 17 Geolocation.latitude = pos.coords.latitude; 18 Geolocation.longitude = pos.coords.longitude; 19 } 20 };
The object in the third parameter configures the call: highAccuracy determines the position as accurately as possible, which could use quite a bit of power; maximumAge allows the use of saved geolocations that are not older than specified, and timeout determines the time after which the positioning will abort. Both entries use milliseconds as a unit. The maximumAge:0 setting will not resort to previously saved geolocations.
The watchPosition() function observes the current position permanently. Should the position change noticeably, the code executes the success callback, which saves the current latitude and longitude.
OpenLayers
Geolocations can be shown on a map with the help of the OpenLayers free JavaScript library. Currently, OpenLayers is available in version 2.11. Listing 8 shows how the script element in line 5 integrates the library in an HTML document. The div container in line 9 collects the map.
Listing 8: OpenLayers
01 <!DOCTYPE html> 02 <html> 03 <head> 04 <meta charset="UTF-8"/> 05 <script src="js/OpenLayers.js"></script> 06 <script src="listing-12.js"></script> 07 </head> 08 <body> 09 <div id="osmap" style="width:512px;height:256px"></div> 10 </body> 11 </html>
Listing 9 demonstrates how positions can be displayed using JavaScript. Line 6 initializes the map area, line 7 adds a layer with OpenStreetMap, and line 9 adds the layer for the markers. The addMarker() method in line 15 marks a geolocation by creating a clone of the image object in line 12 and then inserts transformed coordinates with coords (line 20). Finally, the script centers the map on the new markers.
Listing 9: Wrapper Object for OpenLayers
01 window.Map = { 02 map: undefined, 03 ipath : 'img/marker.png', 04 05 init : function() { 06 this.map = new OpenLayers.Map({div:"osmap", zoom:12}); 07 this.map.addLayer(new OpenLayers.Layer.OSM()); 08 this.marker = new OpenLayers.Layer.Markers(); 09 this.map.addLayer(this.marker); 10 var size = new OpenLayers.Size(21,25); 11 var offset = new OpenLayers.Pixel(-(size.w/2), -size.h); 12 this.icon = new OpenLayers.Icon(this.ipath, size, offset); 13 }, 14 15 addMarker : function(lon, lat) { 16 this.marker.addMarker(new OpenLayers.Marker(this.coords(lon, lat), this.icon.clone())); 17 this.map.setCenter(this.coords(lon, lat)); 18 }, 19 20 coords : function (lon, lat) { 21 return new OpenLayers.LonLat(lon, lat).transform( 22 new OpenLayers.Projection("EPSG:4326"), 23 new OpenLayers.Projection("EPSG:900913") 24 ); 25 } 26 };
jQuery Mobile
jQuery Mobile, currently in version 1.1.0, is an open source framework used for creating graphical user interfaces for web browsers in smartphones and tablets. The jQuery Mobile framework includes many pre-configured buttons, lists, dialogs, and search filters that developers would otherwise have to take the trouble to create and test themselves.
jQuery Mobile, which is based on HTML5 and the jQuery JavaScript library, is compatible with a growing number of environments, such as Apple iOS 3.2 to 5.0, Android 4.0, Windows Phone 7 to 7.5, Blackberry 7, Firefox Desktop 4 to 9, Firefox Mobile (10 Beta), Opera Desktop 4 to 5, Chrome Desktop 11 to 17, Safari Desktop, and Internet Explorer 7 to 9.
The preconfigured elements from jQuery Mobile can be used with custom data attributes (Listing 10). After lines 6 to 8 have loaded the files for jQuery Mobile, the data-theme data attribute in line 11 selects corporate design "c" for the page.
Listing 10: Data Attributes for jQuery Mobile
01 <!DOCTYPE HTML> 02 <html manifest="geotrack.appcache"> 03 <head> 04 <meta charset="UTF-8"/> 05 <title>Geo Tracker</title> 06 <link rel="stylesheet" href="css/jquery.mobile-1.1.0.min.css"/> 07 <script src="js/jquery-1.7.1.min.js"></script> 08 <script src="js/jquery.mobile-1.1.0.min.js"></script> 09 <script src="listing-9.js"></script> 10 </head> 11 <body data-theme="c"> 12 <div data-role="page" id="overview"> 13 <div data-role="header"> 14 <h1>Geotracker</h1> 15 </div> 16 <div data-role="content"> 17 <a data-role="button" data-rel="dialog" href="#addDetail">New Entry</a> 18 <h2>Entries</h2> 19 <div id="listcontainer"> 20 <ul id="list" data-role="listview"></ul> 21 </div> 22 </div> 23 </div> 24 </body> 25 </html>
Programmers can create their own designs with the ThemeRoller tool. The data-role attribute identifies the container in line 12 as a single page, and the attribute in line 13 is the header for that page. Line 16 specifies the content area, and line 17 defines the a element as the button for adding new locations.
The New Entry button in line 17 of Listing 10 integrates a dialog by referencing the element with the addDetail ID in the data-rel="dialog" attribute. Listing 11 shows the source code of the element – a form for entering the location.
Listing 11: Dialog with jQuery Mobile
01 <div data-role="page" id="addDetail"> 02 <div data-role="content"> 03 <form id="detail"> 04 <div data-role="fieldcontain"> 05 <label for="lat">Key</label> 06 <input type="text" name="key" id="key"> 07 <label for="lat">Latitude</label> 08 <input type="text" name="lat" id="latitude"> 09 <label for="long">Longitude</label> 10 <input type="text" name="long" id="longitude"> 11 <fieldset class="ui-grid-a"> 12 <div class="ui-block-a"> 13 <input id="addCancel" data-rel="back" type="button" value="Cancel"/> 14 </div> 15 <div class="ui-block-b"> 16 <input data-rel="back" type="submit" value="Save"/> 17 </div> 18 </fieldset> 19 </div> 20 </form> 21 </div>
Listing 12 describes the behavior of the application in the case of different events.
Listing 12: Events
01 $('#addDetail').live('pageinit', function() { 02 $('#detail').submit(function() { return add();}); 03 $('#addCancel').click(close); 04 }); 05 06 $('#addDetail').live('pagebeforeshow', function() { 07 $('#latitude').val(Geolocation.latitude); 08 $('#longitude').val(Geolocation.longitude); 09 });
The first time the element with the addDetail ID is loaded, it triggers the pageinit event in line 1.
The submission of the form – the submit event (line 2) – calls up the add() function. A click on the cancel button – the click event – closes the dialog (line 3). Before the dialog triggered by the pagebeforeshow event (line 6) appears, JavaScript enters the current values for the geographic latitude and longitude in the text fields. The example application transfers the collected data to a CouchDB database.
Online Data
CouchDB is a NoSQL database that stores documents in JSON format according to a key. The CouchDB database includes a REST API and can be managed easily with jQuery. After installation, the CouchDB server runs under port 5984. A reverse proxy integrates the CouchDB in the domain of the web application. Listing 13 shows the configuration of an Apache web server for this purpose.
Listing 13: Proxy for CouchDB
01 <VirtualHost *:80> 02 DocumentRoot /home/pa/www 03 04 ProxyPass /couchdb http://localhost:5984 05 ProxyPassReverse /couchdb http://localhost:5984 06 </VirtualHost>
The definition of the proxy is in line 4. This file should be saved under /etc/apache2/sites-available/offline and linked to /etc/apache2/sites-enabled. Before restarting the Apache server with apache2ctl restart, the default configuration should be changed from Port :80 (in line 1 of Listing 13) to :8080, for example.
The following HTTP query creates an empty database named geos:
curl -X PUT http://localhost/couchdb/geos
Listing 14 shows the synchronization of the offline cache with the CouchDB database. The toSync() method in line 4 serializes the object with stringify to a JSON document and sends it per HTTP PUT to the geos database.
Listing 14: Database Synchronization
01 window.Sync = { 02 syncurl : '/couchdb/geos', 03 04 toSync : function(obj) { 05 $.ajax({ 06 url : Sync.syncurl+'/'+obj.key, 07 contentType : 'appcliation/json', 08 type : 'PUT', 09 data : JSON.stringify(obj) 10 }); 11 } 12 };
The document key is appended at the end of the URL in accordance with the REST API. The content type of the HTTP query is application/json. With the application cache and the data storage for application data (Local Storage, Web SQL, and IndexedDB), HTML5 includes everything needed to make offline-capable web applications so that the apps are always available and no longer influenced by network connection disturbances.
Conclusion
jQuery and HTML5 help you avoid cost-intensive development of different applications for the different platforms. This brief look at how to build an offline-capable web application should be enough to get you started with the new offline features of HTML5.
The Author
Andreas Möller has been developing Internet-based software for a decade, including database and web applications, as well as systems for single-source publishing. Currently, he works as a freelance author and consultant.