Making your mobile app work offline means that you try to provide, as much as possible, the same user experience whatever the network condition is: up, down or even slow or inconsistent in speed.

Note: if you are looking for an off-the-shelf solution for offline, I wrote another post about Data sync and offline support with Dropbox Datastore.

Why you should care about offline mode

Here are several reasons:

  • the most used smartophone apps are trying more and more to provide content while offline. And these are the apps where users are spending most of the time on their smartphones. So they get used to being able to read content in apps even when the network is off. Also, apps like Evernote even let users create and edit notes while offline.
  • your office sure has a kick ass wifi, but in the wild the user of your app has a totally different experience. Sometimes there is no mobile network at all, e.g. in the subway. In the train the network can go up and down. Sometimes it’s slow, sometimes it’s fast. The biggest cities in the world have 4G now, but even in these big cities there are still a lot of areas where the moblie network is slow.
  • moblie networks are different: latency is high, speed is very inconsistent, which is the worst in terms of user experience. There is a great talk by Ilya Grigorik from Google: “Faster Websites: Crash Course on Frontend Performance (Part 1/2)”, Devoxx 2012, that explains why mobile networks are so different. Relevant part starts at 1:14:30 and ends at 1:24:50.

To put yourself in the shoes of your app user, you should definitely use tools that let you test your application in various network conditions. For example, on OS X and iOS, you can use the “Network Link Conditioner”, which let’s you set different network conditions like Edge, 3G, 100% packet loss.

My BusyBox time tracking app

## Let’s implement an Evernote-like app

So let’s implement an Evernote-like app, that let’s you read, create and edit text notes whatever the network conditions are. I really like Evernote. I am not such a big fan of their design, and I think the user interface is a bit cluttered now, because they put a great lot of features in this app (looks like the Evernote team will work on improving this: http://blog.evernote.com/blog/2014/01/04/on-software-quality/). But the fact that I can read and edit notes wherever I am without having to think about the network is a bliss. In my opinion Evernote offline mode is what makes the difference in terms of user experience.

Here are the git repositories that contain the client side and server side code that you can use as an example: https://github.com/creynaud/notes-server and https://github.com/creynaud/notes-iphone-app. These are not production ready, but still they give a good idea of what you will have to do to add an offline mode to your app. The client side code is an iOS native app. The server side code is a Django web app that uses Django REST framework to provide a REST API. It can be deployed on Heroku. Note that though the example client code is for iOS, the general discussion is still relevant even if you are doing Android, Windows Phone or web development.

It all comes down to versioning your objects

Before diving into the details, I want to stress that implementing an offline mode in a native app is not that difficult. At the end it all comes down to tracking the version of your objects. So for the note app example, each note will have a revision, i.e a unique revision that is incremented by the server each time the object is modified. Once your objects have a revision, it is easy to reconcile client and server data, detect which object update needs to be fetched from the server, or whether a conflict should be raised on a note edition. We will see how later in this post. Also the most important thing when implementing the offline mode is to get the REST API right, because the REST API is the key to handling client/server data reconciliation.

Let’s implement the offline mode in 3 steps

Reading notes while offline

Storing the data locally

As your application is offline, you cannot reach the server (“Thank you Miss Obvious!” - “You’re welcome :)”), so you will have to read the notes from a local storage. In the example iOS native code I use Core Data/SQLite as the local storage, but you really can use whatever you want. Note also that you can store the raw JSON data (again, either in SQLite or whatever suits your need), it’s really your choice. On the one hand maybe you need the power of SQL to make queries, on the other hand going schema less can be convenient if your data format evolves a lot. It really depends on your needs. In my example, I used Core Data because I am a big fan of NSFetchedResultsController.

When you implement an offline mode, the offline mode becomes the default mode. There is no way to predict how long a request will take, so you will 1st have to show the data that is already there in the local storage. You may reach to the server in a 2nd time to display more data if the network is available.

Leveraging http caching mechanisms

While you will store the data in a local storage, you should still try to leverage http caching mechanism as much as possible. Remember that I mention that the central mechanism in implementing an offline mode is to version your objects? Good news is the revision will also be useful to leverage http caching.

Here are the relevant http headers for http caching:

  • Cache-Control
  • Etag and If-None-Match
  • or Last-Modified and If-Modified-Since

But let’s see how these work with an example using the Etag header.

The following GET request is a GET on one note, the one which has “45472b6c-0b06-4040-96c8-f8c369c22ec0” as its uuid. The interesting thing is that the server returns an Etag header in the response. The Etag header value is “1-a7ef01db-e55e-477b-bc06-1eda2502181b”, which is the revision of the note.

claire@poulette:~$ curl http://127.0.0.1:8000/api/notes/45472b6c-0b06-4040-96c8-f8c369c22ec0/ -v -H 'Authorization: Basic Y2xhaXJlOnRlc3Q=' | python -mjson.tool
> GET /api/notes/45472b6c-0b06-4040-96c8-f8c369c22ec0/ HTTP/1.1
> User-Agent: curl/7.32.0
> Host: 127.0.0.1:8000
> Accept: */*
> Authorization: Basic Y2xhaXJlOnRlc3Q=

< HTTP/1.0 200 OK
< Date: Tue, 25 Feb 2014 17:36:37 GMT
< Server: WSGIServer/0.1 Python/2.7.4
< Vary: Accept, Cookie
< X-Frame-Options: SAMEORIGIN
< ETag: "1-a7ef01db-e55e-477b-bc06-1eda2502181b"
< Content-Type: application/json
< Allow: GET, PUT, DELETE, HEAD, OPTIONS, PATCH
{
    "date": "2014-02-25T17:30:21Z",
    "revision": "1-a7ef01db-e55e-477b-bc06-1eda2502181b",
    "text": "This is some text content.",
    "title": "Title",
    "uuid": "45472b6c-0b0
}

If I repeat this GET request, the server will give me the same response, so it will resend the note again. This is not what I want. I would like the server to send me the note JSON only if the note changed since the last version. So I am going to tell the server what is the last revision of the note that I am aware of, by using an If-None-Match header:

claire@poulette:~$ curl http://127.0.0.1:8000/api/notes/45472b6c-0b06-4040-96c8-f8c369c22ec0/ -v -H 'Authorization: Basic Y2xhaXJlOnRlc3Q=' -H 'If-None-Match: "1-a7ef01db-e55e-477b-bc06-1eda2502181b"'
> GET /api/notes/45472b6c-0b06-4040-96c8-f8c369c22ec0/ HTTP/1.1
> User-Agent: curl/7.32.0
> Host: 127.0.0.1:8000
> Accept: */*
> Authorization: Basic Y2xhaXJlOnRlc3Q=
> If-None-Match: "1-a7ef01db-e55e-477b-bc06-1eda2502181b"

< HTTP/1.0 304 NOT MODIFIED
< Date: Tue, 25 Feb 2014 17:44:54 GMT
< Server: WSGIServer/0.1 Python/2.7.4
< X-Frame-Options: SAMEORIGIN
< ETag: "1-a7ef01db-e55e-477b-bc06-1eda2502181b"
< Vary: Accept, Cookie
< Content-Length: 0
< Allow: GET, PUT, DELETE, HEAD, OPTIONS, PATCH

This time the server answers with 304 not modified, and does not answer any content, because the note did not change.

Now let’s repeat this request after having modified the note via the server web app:

claire@poulette:~$ curl http://127.0.0.1:8000/api/notes/45472b6c-0b06-4040-96c8-f8c369c22ec0/ -v -H 'Authorization: Basic Y2xhaXJlOnRlc3Q=' -H 'If-None-Match: "1-a7ef01db-e55e-477b-bc06-1eda2502181b"'
> GET /api/notes/45472b6c-0b06-4040-96c8-f8c369c22ec0/ HTTP/1.1
> User-Agent: curl/7.32.0
> Host: 127.0.0.1:8000
> Accept: */*
> Authorization: Basic Y2xhaXJlOnRlc3Q=
> If-None-Match: "1-a7ef01db-e55e-477b-bc06-1eda2502181b"

< HTTP/1.0 200 OK
< Date: Tue, 25 Feb 2014 17:47:43 GMT
< Server: WSGIServer/0.1 Python/2.7.4
< Vary: Accept, Cookie
< X-Frame-Options: SAMEORIGIN
< ETag: "2-2702d3b4-28f6-43f2-ac7f-519f5bf25afb"
< Content-Type: application/json
< Allow: GET, PUT, DELETE, HEAD, OPTIONS, PATCH
{
    "uuid": "45472b6c-0b06-4040-96c8-f8c369c22ec0",
    "revision": "2-2702d3b4-28f6-43f2-ac7f-519f5bf25afb",
    "date": "2014-02-25T17:30:21Z", "title": "Title Updated",
    "text": "This is some text content."
}

The server answers with 200 again and provides the new version of the note, and the Etag value now provides the new revision of the note.

If you are developing on iOS, you can try to make these calls inside your app code and spy the http traffic with the tcpflow command line tool or Charles. See https://github.com/creynaud/notes-iphone-app/blob/master/KeepANote/KNAppDelegate.m. The http caching comes out of the box on iOS thanks to NSURLCache. If you are interested in this subject, you can also read this NSHipster post: http://nshipster.com/nsurlcache/.

Creating notes while offline

As your application is offline, again you cannot reach the server. So you will have to store the new note locally until the network connection is restored. Here are the steps that you can follow:

  • Store (e.g in SQLite) the JSON document that needs to be posted, with a flag telling that it is not posted to the server yet
  • Try to post the JSON document to the server in the background
  • Mark the JSON document has successfully posted only if POST succeeds
  • In case of failure, retry to post the JSON document during next sync with the server

Evernote shows a little icon on your note if it is not synchronized with the server. This icon goes away once the synchronization is done. This is a good way to let the user know what happens without being too intrusive.

Editing and deleting notes while offline

Here come the conflicts

Editing and deleting notes is similar to notes creation. The big difference though is that when you let your user update notes while offline, conflicts will show up (even if there is no multi-user edition).

You should build the conflict detection inside your REST API. I am talking about detection, not resolution. How you solve a conflict really depends on your application. Let’s come back to conflict detection first with an example.

How to detect conflicts via your REST API

Let’s imagine I updated the note with uuid “45472b6c-0b06-4040-96c8-f8c369c22ec0” via the web app while my smartphone was offline, then I continued editing the same note on my smartphone while still not turning the network on. Here is what the PUT request to update the note on the server would look like when the network goes back on:

claire@poulette:~$ curl -X PUT -d '{"uuid": "45472b6c-0b06-4040-96c8-f8c369c22ec0", "revision": "2-2702d3b4-28f6-43f2-ac7f-519f5bf25afb", "date": "2014-02-25T17:30:21Z", "title": "Title Updated", "text": "This is some text content."}' -H "Authorization: Basic Y2xhaXJlOnRlc3Q=" -H "Content-Type: application/json" -H "Accept: application/json" http://127.0.0.1:8000/api/notes/45472b6c-0b06-4040-96c8-f8c369c22ec0/
{"error": "Conflict, you are trying to update an old revision of this object"}

This returns an error, because a conflict has been detected. The last revision that is known for the note on my device is “2-2702d3b4-28f6-43f2-ac7f-519f5bf25afb”, but this is not the current revision of the note on the server.

Let’s do another request to get the last version of the note on the server:

claire@poulette:~$ curl -v -H "Authorization: Basic Y2xhaXJlOnRlc3Q=" -H "Accept: application/json" http://127.0.0.1:8000/api/notes/45472b6c-0b06-4040-96c8-f8c369c22ec0/ | python -mjson.tool
> GET /api/notes/45472b6c-0b06-4040-96c8-f8c369c22ec0/ HTTP/1.1
> User-Agent: curl/7.32.0
> Host: 127.0.0.1:8000
> Authorization: Basic Y2xhaXJlOnRlc3Q=
> Accept: application/json

< HTTP/1.0 200 OK
< Date: Wed, 26 Feb 2014 11:47:27 GMT
< Server: WSGIServer/0.1 Python/2.7.4
< Vary: Accept, Cookie
< X-Frame-Options: SAMEORIGIN
< ETag: "3-e2f79e44-4440-41c2-8aef-86aa67fb8602"
< Content-Type: application/json
< Allow: GET, PUT, DELETE, HEAD, OPTIONS, PATCH
{
    "date": "2014-02-25T17:30:21Z",
    "revision": "3-e2f79e44-4440-41c2-8aef-86aa67fb8602",
    "text": "This is some text content.",
    "title": "Title Updated Through the Web interface",
    "uuid": "45472b6c-0b06-4040-96c8-f8c369c22ec0"
}

Indeed the revision of the note is now “3-e2f79e44-4440-41c2-8aef-86aa67fb8602”.

Detecting conflicts via the REST API prevents you from overwritting a note modification without being aware of the change. Now let’s move on to conflicts resolution.

How to solve conflicts

In the case of the note taking app, a good way to solve the conflict is to show to the user both texts (the current version on the device, and the current version on the server), and let him keep what he wants. In some other applications, you may choose to proceed in another way. Sometimes letting the latest version in time win can do just fine.

Back to the example of note “45472b6c-0b06-4040-96c8-f8c369c22ec0”, we need to solve the conflict. Here is the new PUT request, with a new content that is a merge of the 2 versions of the note and that takes into account the last revision:

claire@poulette:~$ curl -X PUT -d '{"uuid": "45472b6c-0b06-4040-96c8-f8c369c22ec0", "revision": "3-e2f79e44-4440-41c2-8aef-86aa67fb8602", "date": "2014-02-25T17:30:21Z", "title": "Title Updated in the mobile app and the web interface", "text": "This is the new content after conflict resolution."}' -H "Authorization: Basic Y2xhaXJlOnRlc3Q=" -H "Content-Type: application/json" -H "Accept: application/json" http://127.0.0.1:8000/api/notes/45472b6c-0b06-4040-96c8-f8c369c22ec0/
{
    "uuid": "45472b6c-0b06-4040-96c8-f8c369c22ec0",
    "revision": "4-09cb45ba-92b4-49b6-a367-48077b3ce2e2",
    "date": "2014-02-25T17:30:21Z",
    "title": "Title Updated in the mobile app and the web interface",
    "text": "This is the new content after conflict resolution."
}

This time the PUT request succeeded and the server set the note revision to “4-09cb45ba-92b4-49b6-a367-48077b3ce2e2”: conflict solved!

## Two words about the server side

I used the Django REST framework to implement the REST API on the server side. The REST API comes out of the box except for the following points (but this was very easy to add):

  • You need to add a UUID and a Revision in the Note model
  • PUT and DELETE requests must be rejected with a 400 (Bad request) if the note revision is not specified
  • PUT and DELETE requests must be rejected with a 409 (Conflict) if the specified revision is not the current one on the server side
  • The ETAG has to be added to the headers for GET requests

Again, the server side Django code is available on github: https://github.com/creynaud/notes-server, so you can have a look at the implementation.

Let’s take this offline

I hope you are now ready to add an offline mode in your mobile app! Let me know what you think about it. Would you go for it? Why? Why not? I would also love to hear from you if you implemented an offline mode in your app, or have interesting links or thoughts to share on the subject.

Thanks for reading!

Claire