MemcacheD + PHP + MySQL = Dream Team
When using development frameworks like CakePHP, Zend or even Smarty, I always took their ability to cache data for granted. After all, I hadn't built all of my applications using frameworks and those that were lacking didn't seem to be hurting too badly. It just seemed like a bonus for using their environments and while I was very aware of the fact that caching reduces stress on the database, I hadn't really built anything that was in dire need of caching as a means of improving overall functionality.
Once I started learning about memcached, I actually started to think about my non-frameworked applications and whether there were any noticeable lags. It wasn't long before I had thought of a few glaringly obvious examples where (mem)caching could be used to significantly improve performance. But before I go into those examples, here is a brief explanation of what memcached does for you (taken directly from memcached.org):
What is Memcached?
Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.
Memcached is simple yet powerful. Its simple design promotes quick deployment, ease of development, and solves many problems facing large data caches. Its API is available for most popular languages.
So first on my list was Blogflare.com, with its MySQL-based statistical cruncher running on regular intervals and its PHP-based image tickers, the load on the database was strained on a very regular basis. I noticed if I just adjusted the simple database calls to push updates to memcached and then check memcached for data before I even touch the database, performance significantly increased on the front-end. Additionally, this same improvement was helpful in terms of serving out ticker images that are normally database-driven. With these two simple changes, load on the database dropped enough to make scaling much less of a pain. This is relevant, too, as the site has grown steadily over the past few months and shows no signs of stopping.
The next item on the list is one I can't actually link you to but an application that I've built for the University's Art School to handle finance planning and reporting for graduate students. After a few moments of thinking about its structure I realized the database is very read-heavy, particularly in the reporting area. Reports are generated from stored procedures and perform some pretty heavy calculations on each student's data in order to provide administrators with a very accurate picture of spending and planned spending. Updating the database calls for insert/update/delete methods to perform calls to memcached first proved to be ENORMOUSLY effective at speeding up use of the database. While this helps to reduce load on the server, the most important factor in this case was how much better the overall user experience was. Lagginess became intermittent rather than constant.
When I was researching memcached and wondering about how such a thing would scale, I found a clipping from this article entitled How to Dramatically Speed Up Your Web Application: An Introduction to memcached very helpful-
No doubt if you took Computer Science in school you were cautioned of the temptation to abuse caches because there is a law of diminishing returns in regards to the size of your cache: the larger your cache gets the more costly it is retrieve and store information within it. Memcache however is not heavily constrained in this way, because the cache at large is made up of lots of little caches. This allows memcached to be much more responsive even when the cache itself begins to reach sizes that might be really inefficient in other circumstances.
So this little bit managed to assuage my fears and give me the confidence to jump right in and start testing it with my lesser-applications. The results, even this early, have been astounding and I can now see how they are a very vital part of modern web application development.