Late july, we get contacted by iAgentur, a Swiss mobile application development agency. They had a request from the online newspaper 20min.ch. They wanted to build a special map, with geolocalized photo called: Moment Swiss.
The idea is simple: Ask all 20min users to post a picture, using the 20min application to build the biggest picture album of the country.
Of course they were going to have thousands of pictures, this is why they needed our maptimize clustering engine to display them on google map.
They had an important request: to be able to handle an heavy load. When the project will start, they will send a push notification to all users (iOS and Android). They have about two million users. It means many of them will post at the same time a picture. The expectation was to have about 1000 requests per second.
That’s a lot. Especially for a high computation algorithm like maptimize. But our latest version is an extension of ElasticSearch which is very easy to scale in the cloud.
We decided to use Digital Ocean for cloud hosting. Fair price, very nice API and the best support we’ve ever seen for a cloud hosting company so far.
After a couple of days, we had a scalable configuration. We were able to add/remove clustering nodes with one command line.
3 companies involved in that project:
- iAgentur did the iPhone and the Android application
- 20min did the data layer (image storage and resize, votes, …) and the web page with map and pictures
- maptimize for the clustering engine and REST API to sync data from the data layer and to display data on google map.
The D Day
We were informed when the push notification had been sent to the two million users. Of course, we were looking at traffic and how our service will be respond. It’s amazing to see how so many people react instantly to a notification!
We went from nothing to about 900 requests per second in less than one minute. The estimate was correct, I was impressed. After the peak, we stayed at about 500 requests per second for few hours.
Everything was perfect. Of course, We had few connections timeout. Not very long, not long enough to find what’s going. It was only for few seconds, and just at the start.
Now, after log analysis, we understood what happened. We made benchmarks on data read access and write access. But at the beginning we had a huge amount of read AND write access at the same time, much more than expected.
We already gave some numbers. Here is the graph of most important requests per minute for 3 days.
About device access during those three days we had:
- more than 5.5 million map interactions (pan/zoom) on mobile device.
- about 850 000 map interactions from the web app.
- more than 1.5 million picture views on mobile.
- about 320 000 picture views from the web app.
That was a really great project to work on with two great partners. Today, we even know more about the scalability of our maptimize service. If you are interested by using maptimize for your service of for an event, you can contact use at firstname.lastname@example.org.
Next article will be about another maptimize project done for the BBC and the German election.