When using development frameworks like CakePHP, Zend or even Smarty, I always took their ability to cache data for granted. After all, I hadn't built all of my applications using frameworks and those that were lacking didn't seem to be hurting too badly. It just seemed like a bonus for using their environments and while I was very aware of the fact that caching reduces stress on the database, I hadn't really built anything that was in dire need of caching as a means of improving overall functionality.

Once I started learning about memcached, I actually started to think about my non-frameworked applications and whether there were any noticeable lags. It wasn't long before I had thought of a few glaringly obvious examples where (mem)caching could be used to significantly improve performance. But before I go into those examples, here is a brief explanation of what memcached does for you (taken directly from memcached.org):

What is Memcached?

Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

Memcached is simple yet powerful. Its simple design promotes quick deployment, ease of development, and solves many problems facing large data caches. Its API is available for most popular languages.

So first on my list was Blogflare.com, with its MySQL-based statistical cruncher running on regular intervals and its PHP-based image tickers, the load on the database was strained on a very regular basis. I noticed if I just adjusted the simple database calls to push updates to memcached and then check memcached for data before I even touch the database, performance significantly increased on the front-end. Additionally, this same improvement was helpful in terms of serving out ticker images that are normally database-driven. With these two simple changes, load on the database dropped enough to make scaling much less of a pain. This is relevant, too, as the site has grown steadily over the past few months and shows no signs of stopping.

The next item on the list is one I can't actually link you to but an application that I've built for the University's Art School to handle finance planning and reporting for graduate students. After a few moments of thinking about its structure I realized the database is very read-heavy, particularly in the reporting area. Reports are generated from stored procedures and perform some pretty heavy calculations on each student's data in order to provide administrators with a very accurate picture of spending and planned spending. Updating the database calls for insert/update/delete methods to perform calls to memcached first proved to be ENORMOUSLY effective at speeding up use of the database. While this helps to reduce load on the server, the most important factor in this case was how much better the overall user experience was. Lagginess became intermittent rather than constant.

When I was researching memcached and wondering about how such a thing would scale, I found a clipping from this article entitled How to Dramatically Speed Up Your Web Application: An Introduction to memcached very helpful-

No doubt if you took Computer Science in school you were cautioned of the temptation to abuse caches because there is a law of diminishing returns in regards to the size of your cache: the larger your cache gets the more costly it is retrieve and store information within it. Memcache however is not heavily constrained in this way, because the cache at large is made up of lots of little caches. This allows memcached to be much more responsive even when the cache itself begins to reach sizes that might be really inefficient in other circumstances.

So this little bit managed to assuage my fears and give me the confidence to jump right in and start testing it with my lesser-applications. The results, even this early, have been astounding and I can now see how they are a very vital part of modern web application development.

Sweet Date

Sweet Date is a very straight-forward dating service web site. People sign up, browse and find people that interest them, and they can either send them an instant message or they can send them a message using the site's built-in mailing system.

As far as the look goes, I chose blue because I think it looks great and because I thought I'd try adhering to some standards for emotional targetting this time (e.g. Color Wheel Pro / Blue). Additionally, I'm taking my time with the feel of the site as I don't want it to have a "thrown-together" appearance. Instead, I want people to know intuitively how to use the web site and where to go if they need help.

The reason I'm able to pay attention to these elements is because I'm using Smarty Templating Engine again, which makes development go a lot faster. Now I can focus on jQuery and Photoshop.

While I'm very aware that dating web sites are a dime-a-dozen these days, I've always wanted to build one for several reasons:

1. Ads pay reasonably well
2. The structure is reasonably simple
3. I need more large-scale web sites on my resume

Although at the moment, the web site is currently being built, the basic frame is in place with the look & feel largely intact. Once registration is open, feel free to open an account but the site will be largely in beta until I feel it is relatively complete.

Profiles Free

Profiles Free, I'll admit, was something made purely out of curiosity. I wanted to see whether I could build a site that Google would like enough to keep users coming back to while at the same time generating a small stream of income, and it does just that.

The site incorporates Google AdSense and affiliate marketing ads in a very "in your face" kind of style.

One of the best things about this site is that it's almost maintenance-free. I check on it about once a month just to make sure it's still there and functioning.

All of the back-end code is done in PHP and the database is, of course, MySQL. Instead of using any kind of JavaScript library on the site, I decided it was a bit overkill for a site like this and just went with the light-weight GreyBox framework to display images in modal forms.

Sitdiary

Sitdiary started back in 2001 as a very simple idea: I wanted other people to be able to post blog entries about anything they wanted. What started as a small idea soon grew into a user-base of over 250,000, where at its peak had over 100 users online at any given time.

Time after time, the bandwidth wasn't enough for this "small" site and I was forced to continuously upgrade on a monthly basis. At one point, the cost became more than I could handle and that's when the popularity of the site started to take a downward turn.

As the site would be out of commission for days at a time, the user base soon dwindled down to around 50,000 and all of a sudden, bandwidth wasn't too much of an issue.

Of course, if I knew then what I know now, I would have done quite a few things differently. For one, I would've chosen a catchier name. Secondly, I would've added some serious caching to this site to reduce overall demand on files and database. Thirdly, I wouldn't have built the database in such a fashion. When I started building Sitdiary, I figured the best way to build the database would be to give each user their own tables for: entries (posts), comments and friends. While this might work well for a few thousand users, the method quickly degrades over time as it is an unnecessary waste of space. Fourthly, I would have added a payment plan for the really dedicated users to help pay for the bandwidth costs.

Since then, I've upgraded the database to a more standardized format and after testing the speed before and after, I can tell you that the standardized format is significantly faster. I've also added page caching using Smarty.

Overall, I'm glad Sitdiary is still running despite its bumpy and uncertain past.

Bill Manager

Bill Manager was initially built as a way to remind me to pay my own bills, but as I talked to other people I found that they were interested in such a service -- especially for free.

On top of reminders, the site also features a few informational articles, a series of tips on financial responsibility and calculators.

Currently, the site is still going an upgrade to facilitate multiple users but should be up shortly.

It should be noted that since this site was created, several others have popped up on the 'bill manager' search that do pretty much the same thing. This is no surprise, but I'm in the process of coming up with a killer feature to set it apart (if only temporarily).

The Weight Loss Tracker

The Weight Loss Tracker is exactly as its name implies: a weight loss tracker.

Users create an account and enter their weight loss goal and starting weight. Immediately, they can begin logging their weight on a daily basis. This way, they can see their progress over time.

Additionally, the site provides code that will allow the user to display a graphical ticker displaying their progress on their way to their weight loss goal.

This site was the result of a personal challenge I gave to myself: to create a simple, fully functioning website (with code documentation!) within ten hours. And actually I completed the site in about seven (7) hours and spent the rest of the time making the code and documentation prettier.

Blogflare.com

Blogflare.com, started in 2007, was made to allow blogs of all varieties to be ranked according to traffic and several other factors. In addition, they receive free traffic and better search engine rankings as a result.

The biggest challenge for a site like this is the ongoing calculation of traffic statistics. Version 1.0 of the website calculated statistics every hour on the hour, which seemed efficient at the time but drained resources, especially on a shared hosting environment.

Version 2.0 of the website changed things up a bit. After testing, I found it was best to calculate ranks every 5-10 minutes to keep the data load small and the impact on performance minimal during the update. The reason for this is that data isn't allowed to grow too large before being reset in that five minute window.

Of course, I've toyed with the idea of spacing the updates 30 seconds to one minute apart but at that point, the updates become noticeable.