Optimization!

We just rolled out an update to twitterbump. The first thing you will notice is our new color scheme. We think that these colors are nicer on the eyes and give it a calmer feel, but let us know what you think!

The biggest part of this update was optimization and scalability. Over the weekend I watch this presentation form Google: http://sites.google.com/site/io/building-scalable-web-applications-with-google-app-engine.

It really helped me understand some of the issues that come with scaling websites and it gave me some ideas on how to better twitterbump. One of the first things that it mentions is that the data store that the Google app engine uses is more like a hash table then a relational database. And it works a lot better if you use it that way. So in this release we replace queries with keys where ever possible. This should speed things up dramatically. Using keys will put our calls in O(1) instead of the O(n) we get with the queries. However, since keys can't be changed once they are created this will cause some weirdness with some of our current data until is pushes itself out. The weirdness will mostly be some possible duplicate entries and bump counts displaying as 0 on old data.

The second thing that we changed is that we are now using the built-in data cacher, memcache, very heavily. We are caching any queries that we preform so we avoid that O(n) time waster again, but we are also caching some of the data that we receive from outside sources, like twitter. When you bumped a tweet in the past we would save the entire tweet xml in the datastore so we could rebuild the tweet object later. Now we just store the tweet object in memcache and only rebuild it if the data is stale. We are also caching your friends timeline, but just for a minute. If you think that is too long let us know. We might reduce it to 30 seconds.

So try out the new version and let us know what you think! Also let me know if you think this was too technical or want any further information on anything.

Thanks!