Take care. This article is likely outdated.

About writable storage in modern hosting

To understand the issue of writable (or not writable) storage, you must understand the history and development of web hosting technology. Let me begin with …

A short history of web hosting

Web hosting is as old as the (commercial) Internet itself. Once there was the infrastructure allowing you to offer your most important product / ideas / photos of your cat to the whole world, people discovered their overwhelming urge to do just that. However, not many people had the things needed to make their stuff available in the world wide web; so companies were founded to provide the metal and cables for them. Here we were: web hosting providers. On the one hand, gratis hosting services, such as good ol' tripod.com, Angelfire (both are now owned by former concurrent Lycos) and GeoCities stepped forward to give away ad sponsored websites for everyone for free. Most of the time with easy-to-use website builders but quite limited in their capabilities. On the other hand, people needed more capable (in terms of available technology and dedicated resources) and above all ad-free services. So paid, shared web hosting providers came to pass and after a very short time, this became a highly competitive market where competitors praised their products in ever more strident voices. Your typical hosting offer of those days read like this:

  • CGI-bin & PHP incl.
  • 1,000 GB Traffic
  • 1,000 MB Storage
  • unlimited MySQL Databases
  • 18 ½ mailboxes + unlimited mail forwards
  • Norton Anti Virus 30 day trial free shipped

Of course, those are still around (and most of the websites in the interwebs are probably relying on them). After those (or around the same time) another kind of hosting was born: Do not buy a ready made environment, but the raw resources, setup your environment yourself - the root server. And the derivatives of this, such as vServers, housing your own metal in a provided rack and so on. Each of them (ad-sponsored hosting, shared website hosting, root servers, …) has their own purpose and target audience. However, since those kind of hosting offers were first invented, the Internet constantly grew, new technologies were introduced and previously edge-case-requirements became mainstream. One of them: scalability. The plain website hosting might give you, the developer, a good starting point, as everything is setup for you and you just have to upload your code and are good to go. However, it is most of the time, if not always, quite limited in it's resources. Maybe you upgrade from your M package to L, then XL – but that's about it. Also when upgrading from M to L to get, let's say, an additional MySQL database, your traffic also grows from 5,000 GB to 20,000 GB  (or so). Might sound good in the first place, but if you don't need this (and most of the time: you don't), you still have to pay for it. In any case: there is a limited size to which your application can grow and you have to figure out yourself where to go from there. So that is (one possibility) when dedicated root (or managed) servers come into play. You could use the whole resources of the system for your own purposes. But now you have to face other problems: your old environment included probably all kinds of hidden failover measurements (eg your storage might have been in a big redundant storage array of your shared hosting provider). With your own root server, you have basically figure out everything yourself - hello sys-admin. That's where most developers either change careers or outsource those things to people who do this for a living (or just create a free botnet c&c server for everybody to use). In any case, growing above the limits of a single (however large) machine is for most developers (with limited or non sys admin skills) impossible. Sure, there are a lot of companies which can provide the infrastructure and skills - if can afford them. And this  is where the new kind of hosting providers stepped in. Since Google App Engine and innovative hosting providers like Heroku - everything is rethought and renegotiated. One of the major benefits resulting: you, the developer, can scale resources in real-time to an extend which was impossible beforehand. This new idea of what website hosting is was feasible not only because of new technologies, such as IaaS cloud providers, but also because the website developers became web application developers - real programmers ;). Let me qualify this, before you smash your monitor out of anger. When I began programming websites, back in 1999/2000, I wrote CGI scripts in Perl and/or PHP. The developers point of view was kind of page-based. Single sites, which consisted of layout stuff (HTML, CSS, Javascript) and a bit of logic. Nearly everybody had to be web designer and web developer in one. Nowadays, you have "pure" web developers, aware of far more sophisticated concepts such as MVC and MOVE. The website has become a web application with dedicated logic for separated layers.

With or without writable storage

And this is why companies like Heroku can afford to provide a totally different hosting environment which does not even include former corner stones such as writable storage. Now, there is a large number of web programmers which are up for it and do look at this not as an constraint but a possibility - or so it is praised. However, not everything new is good and what works for Ruby or Python developers not necessary work as well for PHP coders. In PHP, you have a huge legacy of established and widely used applications. Of course, if you, the PHP developer, start from scratch, writing a completely new app, you could do it probably just as well as the next Ruby guy.

Why without writable storage anyway?

If you are not a hosting provider, you might think: Why not just keep the writable storage? Why drop it in the first place? Well, here is the problem: Those new hosting providers with their fancy scalability need an infrastructure which is scalable to the same degree as their end-user-product! Those infrastructures, eg provided by Amazon AWS and Rackspace Cloud, come with their own quirks and problems. For example: you cannot predict the IP addresses of your next virtual machine. This one thing right here renders most, if not any, classic failover solutions impossible - or at least so inefficient or fragile that you cannot use them in any production environment. But without them, no hosting provider in their right mind would offer any kind of storage! So we and any other creator of such a modern hosting environment, have to come up with new solutions, which are neither widely documented in the Internet nor tested in millions of hours (hence new). If you cannot or want not expend the effort: no writable storage.

Contra writable storage

Having no writable storage available in the first place forces you to use other storage providers, eg Amazon S3. A direct result is, that any request to the "static" contents in the storage are not directed to the same machines your app runs on. This not only frees resources on the app server, but is also gives you a good starting situation for putting a CDN on top later on.

Share & discuss this: