isn't quite ashamed enough to present

jr conlin's ink stained banana

:: Why The Commenters on Hacker News Are Terrible Engineers

CNN published an article a bit ago talking about Silicon Valley’s Poor. It’s a look at the stark reality of where i live and how hard it is for folks who don’t happen to be fortunate enough to be working in the current, hot industry. The article is really well done, and while i don’t agree that rent subsidies are the right approach to the problem, the bulk of the article makes a stirring case.

So, it was no huge surprise that it showed up on Hacker News, and the comments are truly depressing. Not just because they’re full of the usual self centered, overly superior viewpoints that make reading comments there sad making (really, if you don’t want to get into the same sort of spittle fueled rage i’m in, don’t read the comments), but because they fail for one very simple spectacular reason.

They show how bad these guys are at even understanding complex systems.

First off, let’s run a few numbers, shall we? i mean, i’m an engineer, so before i start a conversation about a given problem, i want to make sure i understand the scope and scale. That’s what being an engineer is about.

First off, let’s look at the population. Silicon Valley comprises of the southern portions of the Bay Area. The counties are Santa Clara (from Palo Alto south to encompass Gilroy and east to cover Milpitas) and San Mateo (up the peninsula to San Francisco. i could easily argue that it includes portions of Santa Cruz (to the south toward the Ocean) & Almeda (which covers the eastern Bay Area), but those are partial at best. i could also argue that San Francisco is part of this as well, since a vast majority of folks live there and commute, but those numbers are immaterial to the conversation at this point.

The population of those two counties works out to be 2,699,483. While the total population is not employed, it’s fair to say that because of the high number of bachelors, it’s fair to factor out 30% of the population as being non-job holders (retired, children, infirm, etc.) It’s also fair to say that the Census is not fully accurate either, so there’s some fuzz here, for sanity, we’ll say the working population is two million.

Working off the number of employees for the top companies in Santa Clara (plus google and facebook because, really? they’re missing?) gives about 756,000. Mind you, that includes big hire numbers like HP, Intel and Lockheed Martin which do NOT have their entire workforce located here, but should cover the countless “startups” and smaller companies like SmugMug, Imgur & NetApp. So, figure 40% of the employed here work for high-tech. Figure another 200,000 folks work in equally high pay positions like patent attorneys, doctors, financials and you’ve got about 50% of the population working in “High Pay” fields.

Monthly rent in the two counties is about $2,650 a month, meaning $31,800 a year, just for housing expenses. If we say that weekly living expenses are $100 (for things like food, clothing, internet access, fuel, utilities, commuting, etc.) are a conservative $200 a week, that means that total cost of living is about $42,000. Now add tax to that (~30%) and you’ve got to earn a minimum of $55,000. That means that the minimum wage for the county has to be around $26.50 an hour. (Living Wage notes the number at $12.01 for Santa Clara, so i’m being VERY generous here.)

“No big deal”, you say. “i earn more than that.”

That’s because you’re special, but remember that you’re less than half the work force. There are more folks cleaning yards, checking your groceries, picking up your trash, teaching your kids, cooking and serving your meals at restaurants, driving trucks to supply businesses and all the myriad of things you need for a society to function. They’re not making $26.50 an hour. They’re making a lot less. i’ll remind you that minimum wage in Santa Clara is $8 an hour.

“So, they should move out of the county!”

Ok, sport. Let’s say they do. Well, San Benito County to the south has Hollister, where rent for a 2BR apartment is $1,200 a month, plus a 1+ hour commute. Tracy, out in the valley is also about $1,200 with a 1+ hour commute. (These commutes go up SIGNIFICANTLY during rush periods, so we’ll double the commute time to 2 hours one way.) That means that folks have to spend 4 hours a day in their car just getting to or from work. Mind you, a lot of folks are doing just that, so there’s not a lot of jobs out in those parts, so yeah, commuting isn’t really avoidable. So, your fuel costs just went up, as well as car maintenance costs. After all, car breaks down and can’t go to work? Well, you’re not getting paid that day, bucko.

Oh, yeah, and God forbid you have kids. It’s not like you can lock them in their cage for the day and call the neighbors to go feed them if you’re running late. (Granted, we’re talking about valley engineers here, so breeding is probably not going to be a major concern.)

So much for saving money, huh?

Granted, i’m not a social engineer either, so i have no clear answer to this. i will note that there are dozens of empty office parks in Sunnyvale with new ones going up all the damn time. Getting some of that zoned as Commercial/Residential would help increase available housing driving the rents down. Likewise, putting some of the acquired wealth in the Valley toward improved infrastructure and better mass transit would make the place a helluva lot better than putting another damn McLaren or Telsa on the roads around here. i also think it’s high time we up the minimum wage around here to $12 and put in rent caps. i’ve seen that apartment that’s “a deal” at $2,000 a month. You painted it last century. You’ve made enough friggin’ money you greedy bastard, and yeah, $12 an hour means your latte’s getting pricier. As you said before, you can afford it. If not, maybe you should move.

But, again, i’m not a civic engineer. Still, if this is the way that these apes approach a complex problem, no wonder they all fell in love with Mongo.

:: Bias Confirmation

Over the past few weeks, i’ve been working on creating “demo” apps for SimplePush integration into third party message transport systems. If i’ve lost you in the previous sentence’s word salad, feel free to bail on this post, because it’s going to get a wee bit cranky, and a whole lot geeky. Actually, strike that, reverse it.

Background Info

Wherein i talk about why i walked into the tarpit

Right, so let’s start from the beginning. Mozilla is rolling out a messaging system for programmers. There’s a lot of these sort of things out there, but mostly they’re like email was back in the bad ol’ days of Prodigy, AOL and Compuserve. Folks who used those systems couldn’t send email to folks who were on other systems, or at least, couldn’t do so easily. Then came some discussion, standards meetings, and the Simple Mail Transport Protocol (SMTP) was introduced. Now, various folks could send mail to other various folks, and after copious arm twisting, customer demands, and folks leaving to use The Internet (because that didn’t have weird restrictions like the older platforms did), the companies caved and converted to SMTP. Much joy was to be had. Well, for a little while at least.

Fast forward a few decades and we’re back in the same spot again, only this time, it’s with how servers can send messages to your devices. Turns out, that’s kinda tricky. Your laptop, phone, tablet, watch, belt buckle, shoelaces, and fridge can pretty easily connect up to sites like Facebook, Google, and RedTube (your belt, sadly, is REALLY into bondage for some reason). That’s because those sites pay money to be at well known locations on the internet. You, however, don’t. You move around. One minute you’re at home, then 30 minutes later you’re at work, or headed down the freeway. Turns out that you moving around so much makes it hard to get something to you. (Imagine that your cellphone number changed between locations, and a few times while you’re between locations. That’d make it a bit hard for folks to get in touch with you.) Turns out exactly that happens with your various gadgets. That means that if you wanted to get stuff like chat messages, or surf reports, or a lot of other things, that could lead to all sorts of problems.

Turns out there are a bunch of ways that folks have tried to solve this problem, and have been trying to solve it for a while. Apple, & Google are the big players in this area (much like Prodigy, AOL and CompuServe were with email in the very early days of the internet). Sadly, none of these play nice together. You can’t use the same system to talk to Google that you do for Apple. Heck, you can’t even send the same sort of data to Google that you can to Apple.

Thus, very smart folk have sat down and created a standard for all of this. This standard is still in the works, but there’s a prototype that i’ve been working with for a while. That’s SimplePush, and much like email, it tries to make things easier for sites you want to hear from to send you things you’re actually interested in getting. Now, because those big systems are big systems, and because those big systems are VERY TIGHTLY INTEGRATED into devices, it makes sense that we try to use those notification systems to carry some of the data we want to send.

For some systems, this can be interesting.

And then there’s Apple.


In which i note that i’m not a fan…

i’m not a fan of Apple. i’ve a long, and somewhat checkered history with the company, and while i appreciate that they have an excellent eye for product design that has lead to a good deal of improvement in tech, i’ll note that Disney has had an equally large impression on amusement park theme rides, but i’m rather happy with my car not following a rail and limited to 15mph.

Fortunately, i’ve had little need to deal with Apple products. i get my music from Amazon, watch movies on Netflix, generally work on Unix systems (i tend to run Windows as the UI client, but most of the apps i run there are Open Source.) i have no beef with other folks using Apple products and, frankly, the office here is pretty much filled with Apple laptop chargers because of the enormous number of Macs in the building.

Suffice to say that i’m a bit biased against the legacy of Steve, but still reasonably polite to the odd Apple recruiter that called.

The Procession of Ducks

In which we learn it is better to be eaten whole by alligators, than nibbled to death by ducks

Right, so first i needed to build a version of SimplePush that used Google Cloud Messaging as it’s carrier. i grabbed a copy of Android Studio, fired it up, and in about 6 days, used Java to write a simple demo app that let me connect to the server, and fire messages that were then echoed back to the app over GCM. i’m not going to claim that it’s a masterpiece of elegant code or anything, but it works well enough to test out if things on the server are working.

About the only really annoying thing is that the key bindings in Android Studio can’t be mapped to Vim, so i would occasionally write :w! in the code. i knew Java from a previous job, and it’s bloody java so it’s not like it’s hard to find things. If you like, you can play with the demo using the built in emulator or “side load” it to any device you’d like.

That done, i turned my attention to Apple Push Notification System (APNS). And there in lies my tale of woe.

Apparently, to build iOS apps of any sort, i need to get xCode (an compiler environment for Macs), an iOS device, and a $99 Apple iOS developmental license. i was able to borrow a mac from work, i had my wife’s old iPad2 as the device, but had to pony up the $99+tax fee out of pocket. If i were to go the cheapest possible route, i’d have to pay $500 for a mac mini + $250 for an ipad mini + the $100 iOS development license. Why do i need a real device? Because the APNS requires a real device. It doesn’t work in an emulator. So, close to a grand out of pocket vs what i would have had to spend for GCM, which was $0.

Receipts filed for next years tax season, i decided to dig in. And that’s when i encountered ObjectiveC.
[[intent alloc] withSarcasmAndSlightMalice: @Seriously];?
Granted, i could have used Swift, but in either case, this is a language specific to writing apps for a specific platform. That’s a bit like having to learn Esperanto to order a lunch. i won’t get into the oddities of ObjectiveC (and there are a lot of them), but i’ll note that you really do need to use xCode to write the app. This is because there are whole portions of the app that can only really be built by dragging connectors around like you’re running a 1920’s era switchboard.

Then there’s the certificates. God, the certificates. i get TLS. It’s wonderful for security because it’s using really large bits of math in order to prove you are you and i am i, and you get it for free with browsers. Granted, for this, i need a certificate saying who i am. i then need another certificate that is signed against the first certificate to indicate the app is my app (ok, that seems fair). i then need to export the certificate parts, convert them to a different format, so i can turn them into another format and store them on my server for the connection back to the APNS server which (obviously) needs to have the certificate it knows about me.

This is why OAuth is popular.

Oh, and in addition, i’ll need to get the long, complicated device id off of the iOS device using a completely program that tries to sell you Kanye West and Pitbull songs so that it can be one of the 1000 devices you’ve specifically blessed to run this app outside of the closely guarded and completely arbitrarily governed sanctum they’ve established. Because, obviously, why would anyone ever want to build testing or demo apps?

Anyway, back to the demo app. Once you’ve got the crux of things (hopefully) working, you then have to try sending the actual notification. This is done using a binary format. Well, actually, one of two. There’s the oldest format which consists of a “0” + the length of a token + the token + the length of the message + the message in JSON format. The message JSON needs to be constructed according to a reasonably simple structure, and after you feed the beast, you get a response back indicating if the message was accepted. Most examples will refer to this format.

Then there’s the new format, which has a 2 + the length of all the items combined + the items [ 1 + length of the token + token, 2 + length of the message + the message, 3 + length of an id + the id, 4 + length of the expiration time + expiration time, 5 + length of the priority flag + priority flag]. This information is bundled up and fed to the beast and you get a response back indicating if the message was rejected. You get nothing back if it was accepted. Mind you, it could take a second or two to process the data, so you ought to hang around a bit waiting, just in case. Oh, and after the third message or so, the server will drop your connection, so you’ll have to reconnect. Golly, and because i nearly forgot, you’ll also want to hang onto messages you’ve sent recently because there’s no ACK and you’ll need to resend them later if they’re requested. Because i’m a nice guy, here’s a PHP script built off the most common example that uses either v0 or v2 APNS.

Oh, and also? Even if the server accepts the message and says there’s no problem. There may be a problem and the service will just silently eat the message. So, if you do something silly and screw up the length of a field, or don’t specify the expiration with just enough buffer, or maybe have a typo in the JSON data, the message is helpfully shot into the void never to be heard from again. This makes debugging a bit of a challenge, but hey! It’s apple! It just works! Unless it doesn’t, in which case it’s your fault and you should fix it, you moron.

(Honestly, it would be lovely to have a service set up that would actually verify the data you’re sending in to it and see if it’s correct. i should write one.)

The Sorrow

In which the hero walks to the sunset behind La Brea

Much swearing and twiddling later, it is possible to use APNS to send data to apps. i’m still missing a few additional things, and will have to create them before this goes fully live, but it’s good enough for early prototyping.

So, in theory, i’m over the worst of it. i should be happy, right?

Well, no. You see, having built this, i recognize that this is more than a one or two week process. i’m going to need to have a stable mac environment to work in for a while. That means i’m going to have to swap the loaner macbook for a mini. i’m also going to have to get a mini for my home because there will be days i’ll want to work from home, and i’ve discovered that remote desktop services on macs (or at least on the loaner) is… problematic. Fortunately, i’ll be able to toss the little crapbox into the garage where it can spin it’s fans up and not keep the rest of the house awake. It also means that i’ll be hanging onto the wife’s old iPad for a while, mostly for the same reasons (hopefully, i won’t have to get a second one of those).

Apple, the craft glitter of computing.

Once that i am actually DONE with this crap, i might see if i can put Linux on the mini and use it as a NAS or something. Having hot plugged a USB 3.0 NTFS drive into the mac and not seeing it show up does not bode well, but that could well be a FUSE solvable problem.

All i can say is that this experience has not endeared me to apple at all. (Seriously? Damn near every platform out there has used Ctrl as the principal modifier key, but apple decides to use it’s own, and has the key sit right next to the spacebar? No wonder they want you to use the mouse for everything.) Frankly, sometimes it seems that Apple exists just to ebb joy out of my life.

But, people love them, and are happily effervescent about the Emperor’s wardrobe choices. Even if he’s standing on his thrown, gyrating his hips, and making helicopter noises.

:: Cutting Out the Brain Candy

i’ve come to realize that i’ve been a horrible slacker.

Not just here (this blog exists pretty much for my own evil purposes as well as my private well to scream down), but in a lot of things. i’ve become a consumer. i’ve come to realize that when i’m bored, i simply consume. i dine on a regular digital diet of fatty brain food like reddit, twitter, and crappy youtube videos.

i’ve come to realize that i’ve started consuming more than i create, by quite a lot, really. i’m not sure when (although i have an inkling) and i’m not sure why (again, inkling), but i’ve let myself stop creating things. To me, at least, this is horrible.

i used to drive myself constantly to make something. Heck, i used to post inanity to this blog daily. i gave myself a strict schedule to write a new chapter of a serial thing every two weeks (with a minimum of five pages). i hacked on damn near anything i could get my hands on and have bins filled with bits an pieces to dozens of unfinished projects. i would never give myself the opportunity to be bored.

Yeah, that’s no longer a problem.

Which is a problem.

As an engineer, i hate problems, so i’m going to solve that particular problem.

i’m making it a point to make at least one something every month. That includes conceptualizing, building and documenting whatever in however long the month is. Big things may get divided into littler things, but each thing needs to be able to stand alone as a thing. If i don’t finish one thing in a month, i have to finish two the next month. Oh yeah, and blog posts don’t count (well, not as just a blog post, gotta be more to it than that. Just writing a post is too easy.)

i’ve tried being bored. It sucks and makes my brain fat. i’m not going to do it any more.

:: The Streetcar Problem

Streetcar-CrashProgramming nerds love theoretical technical problems like The Prisoner’s Dilemma, or The Dining Philosophers. Here’s a fun mental exercise for your afternoon: What would happen to your favorite library/service/product if the principal engineer was suddenly killed by a streetcar? For an astonishing number of services, the answer will probably not be “continue to work seamlessly”.

(i’ll note that i decided to break this out of a different post about Being Nice for Selfish Reasons: Work Edition. i decided that it’s a question that needed it’s own post.)

Most projects, like most movements or causes, tend to have a singular leader. There’s a lot of reasons for that, including the facts that running a successful project kinda requires someone to make hard decisions, say “No, we shouldn’t do that” a fair bit, and “We’re going to go in this direction, in spite of your personal interests”. There are ways to present all of those statements, sometimes in friendly, happy ways, other times like a overbearing jerk, but there are ways. The problem is also the Achilles Heel of Open Software Projects: Getting someone else to give a damn about what you’re building. You have to be nice enough or provide enough value that people want to help you and deal with your pushy ideas on how to do things.

There’s a weird psychological edge to successful software development that many folks don’t think about. If you even think about the tools and services you use, you probably associate them with one or a very small group of lead folks. These are the captains of the project and in some respects are cult leaders for their ideas, even if they aren’t in direct control. Python has Guido; jQuery has jresig; PhP has Rasmus; etc. Larger projects like Webkit require a large, organizational structure to deal with the huge, complicated code base. However, even these projects have a small group or even an individual who is the acknowledged leader. This is pretty much how humans set things up.

So, what happens when such an individual leaves a project? Usually, it’s “not good things”. In the sudden vacuum, there can be a power struggle as folks fight to become the replacement, or things drift as the next person in line has a slightly different view of how to do things. In rare instances, it’s possible for the project to continue or improve, but that’s usually because one of the goals of the individual at the helm was to ensure that he or she is replaceable. Yeah, not a lot of folks think that way. It’s very uncomfortable for people to confront their lack of permanence.

All of this does impact you, however. You invest in a given resource. Whether you like it or not, you become dependent on that resource more and more. When the streetcar comes, you’re also subject to the repercussions. If you’re lucky, skilled and have the time, you can fork the project and keep it alive long enough to find a suitable replacement. If not, well, you’re dorked.

It can also be exceptionally difficult to keep your product “Streetcar proof”. Sure, there’s principals of abstraction, but the big problem is that there are reasons you’ve picked a technology or service over it’s competitors, and those reasons will be reflected in the core of your product’s design. Replacing or degrading those will not be easy, or potentially possible for whoever is maintaining your code.

Yeah, that’s right. You’re also subject to Streetcars.

So, when your personal streetcar arrives, how will folks deal with it?

:: Building A Worse Toaster

Sadly, i’m rolling misses on my Google Fu this morning, but i believe that C.J. Cherryh wrote a short story about an elderly woman who wanted a new toaster. It being the future, the device delivered itself and promptly started doing the laundry, fixing the car, reprogramming the TV and all sorts of other wondrous tasks. The Matron, tutted, jabbed a fork deep into the machine. It sputtered, fell over, sparked and finally a warmed, lightly browned piece of toast popped out of a slot.

While the story is funny, it’s also really useful to remember when building a tool.

Ultimately, you want your tool to be as simple as possible. This is the general theory of Unix. Little things that work well together. The problem is that building simple things tends to not be very popular.

As a software engineer, i am both subject to, and constant advocate of, feature creep. i want things that just do a bit more. i mean, sure i could have something that’s just a front end to a database that allows me to enter a blob of text and then later display it,… but it could also have an inline editor, ooh, and a search function, and it’d be neat to have plugins… And, well, you wind up with WordPress.

Same with things like jquery, which started off as fairly simple and small, then became, well, jquery1. Even protocols go through that sort of progression (mind you, i already ranted a bit about “human vs. machine readable“). Darn near everything tends to start simple, but get increasingly more complicated.

As is usual, a number of things tweaked me about this point. One was an old rant about how even though people spend most of their lives in front of a computer now, most don’t know how to use one. The other post talks about developer inequality and a longing for Hypercard. Both pretty much highlight the same thing, and i could argue against both equally.

For the first, i’ll note that computers are media delivery devices and people tend to prefer to consume more than create, and that’s ok. A fair number of people have no idea how to maintain their auto or understand the mechanics of municipal transit but use it to come and go. Granted, if they’re taking an intro course on how to do car repair, probably a good idea to show them how to change the oil and rotate the tires, much like an intro class to computers ought to discuss programming.

Thing is, you can teach programming to kids. You can teach auto repair to kids. Hell, you can teach quantum mechanics to kids. You just have to make them want to learn about it. Javascript isn’t hard. Packing it full of complex crap, frameworks and other stuff to make it “easy” makes it hard, but by-and-large most kids can grok the fundamentals pretty darn fast. No, they’re not going to be able to understand using Promises to trigger component events in the Shadow DOM by the end of the week, but i wouldn’t expect that.

Give ‘em two weeks.

Likewise straight HTML is delightful because it’s actually nearly readable. Pretty print it and it’s pattern matching you can easily wrap your head around. Heck, explain to someone that The Web® really is basically a bunch of fancy telnet calls, and it’s like watching a light bulb go off.

Sadly, that may go away with HTTP22 or SPDY, s/telnet/curl/ and you’re basically back in the game again.

The final thing to note about all of this is that folks will surprise the hell out of you. They’re not going to follow your arbitrary rules and will pretty much use whatever you hand them as a hammer. Done right, that’s not a problem.

In fact, it could be a huge benefit.

Good Sir Tim Berners-Lee never expected people to use his protocol to do face to face video chats. Heck, he wasn’t all that hot on documents having things other than links, but he was willing to let folks be creative with his toy. Heck, i’ve built stuff, given folks instructions on how to properly use it, and they’ve blissfully ignored me again and again.

This is typical (regardless of my continually trying to either ignore or change it), and should simply be part of any service you build. If anything, that sort of abuse should be lauded.

So, when you’re building your next toaster (or library, app, interface, device, whatever), ask yourself:

  • Is this doing the least it can to be useful?
  • Can it talk to other things easily?
  • Does this tool make zero presumptions about the user or the way it will be used?

Answering those questions will pretty much tell you how successful your thing will be.
Otherwise, you’re just making a worse toaster.

1i’m not knocking jQuery as bad, i’m just saying that people have decided that jQuery is how you write javascript. Javscript has evolved and adopted a lot of the things that jQuery started. The problem is that folks immediately turn to jQuery $(“.foo”) rather than use local commands document.getItemsByClassName(“foo”). There are good reasons to use jQuery. There are good reasons not to. Ultimately, though, you should balance the decision on what you need to do rather than knee jerk use a tool.

2It’s funny how long HTTP2

Blogs of note
personal Christopher Conlin USMC memoirs of hydrogen guy Henriette's Herbal Blog
geek ultramookie

Powered by WordPress
Hosted on Dreamhost.