isn't quite ashamed enough to present

jr conlin's ink stained banana

:: Little Fluffy Abstractions

Clouds are the in thing among us computer people. We talk about how they do amazing things, and can provide incredible value to everything. We hear about how important it is to move to The Cloud, build our businesses in The Cloud and otherwise use Clouds for absolutely anything we can possibly think of.

So, what the hell is The Cloud anyway?

Well, let's start with one that everyone can agree on.

cloud1
cloud/kloud/ a visible mass of condensed water vapor floating in the atmosphere, typically high above the ground.



That was easy! So basically, the way to improve your project and data flow is to reduce it to water vapor! So AWS is just a network of tea kettles you rent.

Ok, so no, not really.

Let's talk about what a cloud is in terms of computers and building systems.

cloud2
The Cloud is an abstraction. It's shorthand for "stuff goes here". The Cloud has all the messy, horribly detailed crap. It's what's between "Collect Underpants" and "Profit".

It was created by folks a long time ago to contain stuff that wasn't really relevant to the things you're talking about. It's a swirling mass of machines, data, protocols and chaos that does something that may or may not be useful, but you need it and dwelling on it too much might make your head explode.

As JWZ has noted, "The Cloud, aka The Internet".

As fun as it is to look at clouds from a distance, or not try to think about what's in them too much, eventually, you're going to have to. Particularly if you actually want to do anything that involves The Cloud.

So, let's look a bit deeper into The Cloud. First off, this might help. The stuff behind the panel? That's The Cloud. Things tend to be a little messy in there, and they probably don't always work the way you'd expect or want them to.

Fortunately, there are a few reasonably simple rules:

  1. The Cloud is not free.
    Sadly, we're still a few years from having the Enterprise available to make us a Martini, so things still cost money. The amount of money tends to vary, and some folks are even kind enough to donate things, but servers, disks, power, cooling, network cables, bandwidth, monitoring personnel, colocation facilities, rent, healthcare, and occasional snacks for the break room still have bills due at the end of the month. What can complicate matters is that not all of those things cost the same, and each has a different way to charge you. Some may be "up front" charges, others might be monthly, yet others might only be for how much you're using. Figuring out the costs for this can be tricky at best, and usually something you can't determine until you've got some idea about what you're really going to build.
  2. The Cloud is not reliable
    All sorts of things can cause services to disappear. Things like a business closing or a backhoe can cause unexpected outages that might last for quite some time. For that matter, a machine you've been talking to might be glitchy, get pulled and you're routed somewhere else without your knowledge, and the data you've just pushed is now gone. There are ways that you can make The Cloud reliable, but then you're butting against the other rules.
  3. The Cloud is not fast
    Sure, there are some things that a cloud can do pretty fast, like find a phrase in Shakespeare or find a viable method to fold a certain protein, but for getting stuff on and off The Cloud, you can be better off unplugging a 1TB drive getting in your car and driving an hour. My lovely home on the outskirts of Silicon Valley has a "high speed" network connection with a massive 725Mb upload speed. (in reality i get about 600Mbps) Sounds awesome, but then you remember we're talking bits not Bytes, to upload a 1 MB file, it's: 8,388,608 bits / 600 bits per second for about 13,981 seconds or long enough to explain why i don't use an online backup site. There are faster nets i could use, but i don't for a plethora of reasons.
  4. The Cloud hates you
    Because things cost money in The Cloud, the folks that run things REALLY want to keep from living under a bridge. This means that they do things that seem counter intuitive or obviously hostile. For instance, a single computer has a very limited number of both inbound and outbound connections it can host. You can do things to push the upper limits of this, but those come with trade-offs about what else you can do with the machine and (again) what it costs. Things like routers, hubs and other bits of networking magic are essentially specialized computers that have the same sorts of restrictions. This means that "long lived" connections are generally not a good idea, and stuff on the network tries it's very, very best to not do those. Routers may decide at any point, "You've been here too long, go away now", and break the connection. If they're polite, they'll tell you, but that's not always the case. In addition, they can do things to make it look like you're connected, when you're really not. Things like reply to certain types of pings or other keep-alive messages as if the remote client was still connected, because they'll try re-establishing the connection later if there's "real" data that needs to be sent.

Again, it's possible to create a system that has flawless reliability and connectivity, has high speed connections everywhere, and doesn't hate you, but that will definitely NOT be free.

Working with Clouds is about finding the compromises that work. It's about understanding that it's shorthand. Honestly, it bothers me no end when folks talk about systems developed by Facebook, Netflix or Google and talk about re-implementing them for their project, without understanding that Facebook, Netflix and Google have hundreds of millions of dollars a month going into these things. There's a reason that Amazon is happy to offer AWS and encourages folks to use it.

You just need to be a bit more aware of what goes on inside. You need to be willing to spend a lot of time testing, checking, auditing, and logging things, as well as making sure that there are things like escalation paths in the case of emergencies, sunsetting plans in case things need to be shut down, and a plan for how you're going to provide future versions of your interface while supporting folks that can't or won't go to the latest and greatest.

When you build for the cloud, shipping is just the first step.

Blogs of note
personal Christopher Conlin USMC memoirs of hydrogen guy rhapsodic.org Henriette's Herbal Blog
geek ultramookie

Powered by WordPress
Hosted on Dreamhost.