isn't quite ashamed enough to present

jr conlin's ink stained banana

:: Bias Confirmation

Over the past few weeks, i’ve been working on creating “demo” apps for SimplePush integration into third party message transport systems. If i’ve lost you in the previous sentence’s word salad, feel free to bail on this post, because it’s going to get a wee bit cranky, and a whole lot geeky. Actually, strike that, reverse it.

Background Info

Wherein i talk about why i walked into the tarpit

Right, so let’s start from the beginning. Mozilla is rolling out a messaging system for programmers. There’s a lot of these sort of things out there, but mostly they’re like email was back in the bad ol’ days of Prodigy, AOL and Compuserve. Folks who used those systems couldn’t send email to folks who were on other systems, or at least, couldn’t do so easily. Then came some discussion, standards meetings, and the Simple Mail Transport Protocol (SMTP) was introduced. Now, various folks could send mail to other various folks, and after copious arm twisting, customer demands, and folks leaving to use The Internet (because that didn’t have weird restrictions like the older platforms did), the companies caved and converted to SMTP. Much joy was to be had. Well, for a little while at least.

Fast forward a few decades and we’re back in the same spot again, only this time, it’s with how servers can send messages to your devices. Turns out, that’s kinda tricky. Your laptop, phone, tablet, watch, belt buckle, shoelaces, and fridge can pretty easily connect up to sites like Facebook, Google, and RedTube (your belt, sadly, is REALLY into bondage for some reason). That’s because those sites pay money to be at well known locations on the internet. You, however, don’t. You move around. One minute you’re at home, then 30 minutes later you’re at work, or headed down the freeway. Turns out that you moving around so much makes it hard to get something to you. (Imagine that your cellphone number changed between locations, and a few times while you’re between locations. That’d make it a bit hard for folks to get in touch with you.) Turns out exactly that happens with your various gadgets. That means that if you wanted to get stuff like chat messages, or surf reports, or a lot of other things, that could lead to all sorts of problems.

Turns out there are a bunch of ways that folks have tried to solve this problem, and have been trying to solve it for a while. Apple, & Google are the big players in this area (much like Prodigy, AOL and CompuServe were with email in the very early days of the internet). Sadly, none of these play nice together. You can’t use the same system to talk to Google that you do for Apple. Heck, you can’t even send the same sort of data to Google that you can to Apple.

Thus, very smart folk have sat down and created a standard for all of this. This standard is still in the works, but there’s a prototype that i’ve been working with for a while. That’s SimplePush, and much like email, it tries to make things easier for sites you want to hear from to send you things you’re actually interested in getting. Now, because those big systems are big systems, and because those big systems are VERY TIGHTLY INTEGRATED into devices, it makes sense that we try to use those notification systems to carry some of the data we want to send.

For some systems, this can be interesting.

And then there’s Apple.

Disclosure

In which i note that i’m not a fan…

i’m not a fan of Apple. i’ve a long, and somewhat checkered history with the company, and while i appreciate that they have an excellent eye for product design that has lead to a good deal of improvement in tech, i’ll note that Disney has had an equally large impression on amusement park theme rides, but i’m rather happy with my car not following a rail and limited to 15mph.

Fortunately, i’ve had little need to deal with Apple products. i get my music from Amazon, watch movies on Netflix, generally work on Unix systems (i tend to run Windows as the UI client, but most of the apps i run there are Open Source.) i have no beef with other folks using Apple products and, frankly, the office here is pretty much filled with Apple laptop chargers because of the enormous number of Macs in the building.

Suffice to say that i’m a bit biased against the legacy of Steve, but still reasonably polite to the odd Apple recruiter that called.

The Procession of Ducks

In which we learn it is better to be eaten whole by alligators, than nibbled to death by ducks

Right, so first i needed to build a version of SimplePush that used Google Cloud Messaging as it’s carrier. i grabbed a copy of Android Studio, fired it up, and in about 6 days, used Java to write a simple demo app that let me connect to the server, and fire messages that were then echoed back to the app over GCM. i’m not going to claim that it’s a masterpiece of elegant code or anything, but it works well enough to test out if things on the server are working.

About the only really annoying thing is that the key bindings in Android Studio can’t be mapped to Vim, so i would occasionally write :w! in the code. i knew Java from a previous job, and it’s bloody java so it’s not like it’s hard to find things. If you like, you can play with the demo using the built in emulator or “side load” it to any device you’d like.

That done, i turned my attention to Apple Push Notification System (APNS). And there in lies my tale of woe.

Apparently, to build iOS apps of any sort, i need to get xCode (an compiler environment for Macs), an iOS device, and a $99 Apple iOS developmental license. i was able to borrow a mac from work, i had my wife’s old iPad2 as the device, but had to pony up the $99+tax fee out of pocket. If i were to go the cheapest possible route, i’d have to pay $500 for a mac mini + $250 for an ipad mini + the $100 iOS development license. Why do i need a real device? Because the APNS requires a real device. It doesn’t work in an emulator. So, close to a grand out of pocket vs what i would have had to spend for GCM, which was $0.

Receipts filed for next years tax season, i decided to dig in. And that’s when i encountered ObjectiveC.
[[intent alloc] withSarcasmAndSlightMalice: @Seriously];?
Granted, i could have used Swift, but in either case, this is a language specific to writing apps for a specific platform. That’s a bit like having to learn Esperanto to order a lunch. i won’t get into the oddities of ObjectiveC (and there are a lot of them), but i’ll note that you really do need to use xCode to write the app. This is because there are whole portions of the app that can only really be built by dragging connectors around like you’re running a 1920’s era switchboard.

Then there’s the certificates. God, the certificates. i get TLS. It’s wonderful for security because it’s using really large bits of math in order to prove you are you and i am i, and you get it for free with browsers. Granted, for this, i need a certificate saying who i am. i then need another certificate that is signed against the first certificate to indicate the app is my app (ok, that seems fair). i then need to export the certificate parts, convert them to a different format, so i can turn them into another format and store them on my server for the connection back to the APNS server which (obviously) needs to have the certificate it knows about me.

This is why OAuth is popular.

Oh, and in addition, i’ll need to get the long, complicated device id off of the iOS device using a completely program that tries to sell you Kanye West and Pitbull songs so that it can be one of the 1000 devices you’ve specifically blessed to run this app outside of the closely guarded and completely arbitrarily governed sanctum they’ve established. Because, obviously, why would anyone ever want to build testing or demo apps?

Anyway, back to the demo app. Once you’ve got the crux of things (hopefully) working, you then have to try sending the actual notification. This is done using a binary format. Well, actually, one of two. There’s the oldest format which consists of a “0” + the length of a token + the token + the length of the message + the message in JSON format. The message JSON needs to be constructed according to a reasonably simple structure, and after you feed the beast, you get a response back indicating if the message was accepted. Most examples will refer to this format.

Then there’s the new format, which has a 2 + the length of all the items combined + the items [ 1 + length of the token + token, 2 + length of the message + the message, 3 + length of an id + the id, 4 + length of the expiration time + expiration time, 5 + length of the priority flag + priority flag]. This information is bundled up and fed to the beast and you get a response back indicating if the message was rejected. You get nothing back if it was accepted. Mind you, it could take a second or two to process the data, so you ought to hang around a bit waiting, just in case. Oh, and after the third message or so, the server will drop your connection, so you’ll have to reconnect. Golly, and because i nearly forgot, you’ll also want to hang onto messages you’ve sent recently because there’s no ACK and you’ll need to resend them later if they’re requested. Because i’m a nice guy, here’s a PHP script built off the most common example that uses either v0 or v2 APNS.

Oh, and also? Even if the server accepts the message and says there’s no problem. There may be a problem and the service will just silently eat the message. So, if you do something silly and screw up the length of a field, or don’t specify the expiration with just enough buffer, or maybe have a typo in the JSON data, the message is helpfully shot into the void never to be heard from again. This makes debugging a bit of a challenge, but hey! It’s apple! It just works! Unless it doesn’t, in which case it’s your fault and you should fix it, you moron.

(Honestly, it would be lovely to have a service set up that would actually verify the data you’re sending in to it and see if it’s correct. i should write one.)

The Sorrow

In which the hero walks to the sunset behind La Brea

Much swearing and twiddling later, it is possible to use APNS to send data to apps. i’m still missing a few additional things, and will have to create them before this goes fully live, but it’s good enough for early prototyping.

So, in theory, i’m over the worst of it. i should be happy, right?

Well, no. You see, having built this, i recognize that this is more than a one or two week process. i’m going to need to have a stable mac environment to work in for a while. That means i’m going to have to swap the loaner macbook for a mini. i’m also going to have to get a mini for my home because there will be days i’ll want to work from home, and i’ve discovered that remote desktop services on macs (or at least on the loaner) is… problematic. Fortunately, i’ll be able to toss the little crapbox into the garage where it can spin it’s fans up and not keep the rest of the house awake. It also means that i’ll be hanging onto the wife’s old iPad for a while, mostly for the same reasons (hopefully, i won’t have to get a second one of those).

Apple, the craft glitter of computing.

Once that i am actually DONE with this crap, i might see if i can put Linux on the mini and use it as a NAS or something. Having hot plugged a USB 3.0 NTFS drive into the mac and not seeing it show up does not bode well, but that could well be a FUSE solvable problem.

All i can say is that this experience has not endeared me to apple at all. (Seriously? Damn near every platform out there has used Ctrl as the principal modifier key, but apple decides to use it’s own, and has the key sit right next to the spacebar? No wonder they want you to use the mouse for everything.) Frankly, sometimes it seems that Apple exists just to ebb joy out of my life.

But, people love them, and are happily effervescent about the Emperor’s wardrobe choices. Even if he’s standing on his thrown, gyrating his hips, and making helicopter noises.

    What do you think, sirs?

    Advertisers! Be sure to read our
    Advertisement Policy!

    :: Cutting Out the Brain Candy

    i’ve come to realize that i’ve been a horrible slacker.

    Not just here (this blog exists pretty much for my own evil purposes as well as my private well to scream down), but in a lot of things. i’ve become a consumer. i’ve come to realize that when i’m bored, i simply consume. i dine on a regular digital diet of fatty brain food like reddit, twitter, and crappy youtube videos.

    i’ve come to realize that i’ve started consuming more than i create, by quite a lot, really. i’m not sure when (although i have an inkling) and i’m not sure why (again, inkling), but i’ve let myself stop creating things. To me, at least, this is horrible.

    i used to drive myself constantly to make something. Heck, i used to post inanity to this blog daily. i gave myself a strict schedule to write a new chapter of a serial thing every two weeks (with a minimum of five pages). i hacked on damn near anything i could get my hands on and have bins filled with bits an pieces to dozens of unfinished projects. i would never give myself the opportunity to be bored.

    Yeah, that’s no longer a problem.

    Which is a problem.

    As an engineer, i hate problems, so i’m going to solve that particular problem.

    i’m making it a point to make at least one something every month. That includes conceptualizing, building and documenting whatever in however long the month is. Big things may get divided into littler things, but each thing needs to be able to stand alone as a thing. If i don’t finish one thing in a month, i have to finish two the next month. Oh yeah, and blog posts don’t count (well, not as just a blog post, gotta be more to it than that. Just writing a post is too easy.)

    i’ve tried being bored. It sucks and makes my brain fat. i’m not going to do it any more.

    :: F*ck You, Brain.

    i feel fortunate that i am reasonably mentally healthy. There’s a lot of people who actually have depression, bi-polar disorder, ADD and other conditions and those people have my respect.

    That said, i can understand a tiny fraction of what they go through. i tend to be very hard on myself. i tend to favor criticism more than complements. This is normal, of course, and chances are exceptionally good you do the same thing. i know this because a few years ago when i felt i was at a very low point i wanted to understand what the hell was wrong with me. It was a pattern i found myself falling into and since i didn’t enjoy it, i wanted to know what i could do to solve the problem.

    Turns out, little, since we as a species are pretty firmly hardwired for it. i found that at least for me, understanding what was going on did help. Later on, finding books like You Are Not So Smart really helped me understand even more of this sort of self inflicted crap.

    Part of this is also my industry of choice. Long ago, i consigned myself to be a tradesman in the midst of artists. There are programmers that craft things of magic and joy. These are wonderful creations that improve life in this veil of tears immeasurably. Granted, those things often need to be maintained and supported. At times, bits need to be built out of things stronger than pixie dust and gossamer, and i take a bit of joy trying to make sure that folks don’t know that some of the “magic” is now rebar.

    i’ll also admit that i’m not the best at what i do. Even after working a lifetime in the industry. There are others that i work with that make me look like an amateur. These are people i can greedily learn from, and i’m damn thankful of the opportunity, but it also means that in a culture that spotlights “Rock Stars” and “Ninjas”, i’ll be the guy near the back of the auditorium clapping.

    All of this can be a bit… i don’t want to say depressing, but it’s certainly draining. i’ve had numerous, very dark thoughts, but generally can dismiss them. There are days where i feel reasonably good about myself and completely understand that in very short order, my mood will be completely reversed by something. i’ve learned that this swing happens in both directions, and frankly it’s best to simply not let the pendulum swing that hard in any direction.

    Take, for instance, right now. At this time, i’m feeling very low. i was responsible for a bit of infrastructure. i drafted the design, implemented code, and got it working, and felt reasonably good about myself. Projects started relying on it. Outside groups started playing with it. There was a significant challenge, i needed a bit of help, and we met the challenge with some tweaking.

    The code has since been rewritten by someone far younger than me and is, in many respects, far better than what i originally built. It’s lighter weight, more responsive and probably more maintainable than what i had built. i already see ways that it could be improved and expanded.

    It’s also probably going to be a failure point because of things outside of my control, which may cause the projects that were relying on it to also fail. None of the breaking points are my fault, and i’ve noted what the problems are, how to address them, and what actions are available to all. When i dismiss fault, i am being very clear. The failure is due to a behavior in a system i did not code for nor did we clearly understand at the onset of the project. If this system is removed, the fault is also removed, however this system is required for it’s own reasons. It’s a bit like saying “Well, the crop harvest failed due to the dam break.”

    Still, it’s damn hard for me to shake the “This is your fault, and you suck as a human being” mindset. The program was written in a language and construct i wasn’t fully familiar with. The younger engineer is, so that’s why the code is better after his attention. The failing system is one that i am also unfamiliar with, and the subsystem that is an issue came as a surprise to a good many folk. i view all of these as being “excuses”. My psyche demands that excuses don’t matter, only results. (Yes, i was brought up in a strict, military household, why do you ask?)

    People talk about failure as a benefit. It’s how one learns. It’s inevitable, and constant, and what makes success so remarkable is that it’s uncommon enough to be remarkable. This does not make failure any less pleasant. Culturally, and personally, it’s a stigma. Feeling that way is irrational, but very common.

    At this point, i’ll also think about the various other projects that i feel the need to accomplish, and the sorry state that they’re in, the 60+ articles moldering in my unread queue and a thousand other reasons i have to beat myself up. i should be accomplishing more. i’m not. i should be more creative. i’m not. “Nobody else knows how big a screw up you are” as Mr. Savage points out, but i am acutely aware of it.

    Like i said at the top, there are folks out there with serious medical issues who are constantly struggling with far bigger demons than me. i admire every day that they succeed in beating those demons and they have every right to scoff at me and my personal pity party. If you’re one of those, i welcome your well earned derision.

    Possibly, you may feel the same way i do. Here’s to letting you know you’re not alone.

    And thanks for reading my bit of personal therapy.

    :: Disposable Medium

    i’ll admit that i generate content that is disposable. i tweet, occasionally post crap to facebook and sometimes toss images up on imugr. Sometimes, i comment on reddit. All of that is disposable.

    It’s disposable because i don’t own or manage the forum or pay for the servers. It’s not mine to archive, index or monetize. My account may be blocked or dropped for reasons i have no control over. Whatever i generate may go away tomorrow and i’ll have zero say in the matter. If i’m lucky, i’ll be able to pull a portion of the content off of these sites, but who knows if i’ll be able to do anything with it.

    Sites, even very popular ones, shut down all the time. It’s a well documented fact of life. All the stuff you put on that site disappears when they decide it’s no longer profitable/viable/fun anymore. Heck, there’s even a team that tries to rescue your crap before it disappears forever. All of that kinda rose up when i tweeted:

    i was half kidding, but honestly a bit serious. For reasons i’m still not quite sure about, Medium has become the latest blog substitute. The replies i got on that tweet were equally unsatisfactory. Yes, posts do look pretty-ish, provided you can figure out to get images to do what you want, but you’re pretty damn limited, and let’s face it, every post is basically “Large Hipster style banner pic with Headline, followed by paragraphs of 22px Georgia font. Ok, so this blog clearly illustrates that i have zero modern styling sense, but it’s not terribly difficult to duplicate that look.

    You’re also fairly limited in the sort of things you can do in your text. In my blog, i’ve got carte blanc. There, i’ve got the limited styling features that they demand i use. i suppose the next item would probably be the network effects, i post there because all the cool kids post there, therefore i’m a cool kid too! This is like standing in front of stuffed toys and claiming to have given a Ted Talk. (Granted, there are some that probably should have stuck with the stuffed toys, like this one:



    $7500 spent to hear that talk. Money well spent, huh? And granted, that one was probably for laughs, so there’s this one instead). Granted, since folks don’t seem to like RSS anymore, it’s a bit harder to find interesting articles outside of twitter, facebook, reddit, buzzfeed, digg, slashdot, your aunt’s forwarding email, bathroom walls, …

    Honestly, it’s the message, not the platform that makes one cool or interesting.

    i get that some folks don’t want to run their own site. That’s fine, it can be a hassle to make sure that software is up-to-date. What i don’t really understand is the value folks are seeing. Is it like the early days of the internet where things that weren’t written in comic sans on a gray background were considered to be believable, published articles? Is it that these folks don’t really want to be all that closely associated with what they’re creating? Is it that blogger/wordpress.com/facebook/google+ isn’t the latest new thing? (Ok, Google+ is probably never really going to be a thing.) Is it that they don’t care if what they’re spending time making lasts longer than a reddit karma cycle? Do these folks not realize that the most effective way to build a known presence on the web is to have a known presence on the web?

    i have absolutely no idea.

    For throw away stuff, i don’t mind using disposable services, but i think i’ll keep using stuff i have some control over for things i want to stick around.

    :: The Streetcar Problem

    Streetcar-CrashProgramming nerds love theoretical technical problems like The Prisoner’s Dilemma, or The Dining Philosophers. Here’s a fun mental exercise for your afternoon: What would happen to your favorite library/service/product if the principal engineer was suddenly killed by a streetcar? For an astonishing number of services, the answer will probably not be “continue to work seamlessly”.

    (i’ll note that i decided to break this out of a different post about Being Nice for Selfish Reasons: Work Edition. i decided that it’s a question that needed it’s own post.)

    Most projects, like most movements or causes, tend to have a singular leader. There’s a lot of reasons for that, including the facts that running a successful project kinda requires someone to make hard decisions, say “No, we shouldn’t do that” a fair bit, and “We’re going to go in this direction, in spite of your personal interests”. There are ways to present all of those statements, sometimes in friendly, happy ways, other times like a overbearing jerk, but there are ways. The problem is also the Achilles Heel of Open Software Projects: Getting someone else to give a damn about what you’re building. You have to be nice enough or provide enough value that people want to help you and deal with your pushy ideas on how to do things.

    There’s a weird psychological edge to successful software development that many folks don’t think about. If you even think about the tools and services you use, you probably associate them with one or a very small group of lead folks. These are the captains of the project and in some respects are cult leaders for their ideas, even if they aren’t in direct control. Python has Guido; jQuery has jresig; PhP has Rasmus; etc. Larger projects like Webkit require a large, organizational structure to deal with the huge, complicated code base. However, even these projects have a small group or even an individual who is the acknowledged leader. This is pretty much how humans set things up.

    So, what happens when such an individual leaves a project? Usually, it’s “not good things”. In the sudden vacuum, there can be a power struggle as folks fight to become the replacement, or things drift as the next person in line has a slightly different view of how to do things. In rare instances, it’s possible for the project to continue or improve, but that’s usually because one of the goals of the individual at the helm was to ensure that he or she is replaceable. Yeah, not a lot of folks think that way. It’s very uncomfortable for people to confront their lack of permanence.

    All of this does impact you, however. You invest in a given resource. Whether you like it or not, you become dependent on that resource more and more. When the streetcar comes, you’re also subject to the repercussions. If you’re lucky, skilled and have the time, you can fork the project and keep it alive long enough to find a suitable replacement. If not, well, you’re dorked.

    It can also be exceptionally difficult to keep your product “Streetcar proof”. Sure, there’s principals of abstraction, but the big problem is that there are reasons you’ve picked a technology or service over it’s competitors, and those reasons will be reflected in the core of your product’s design. Replacing or degrading those will not be easy, or potentially possible for whoever is maintaining your code.

    Yeah, that’s right. You’re also subject to Streetcars.

    So, when your personal streetcar arrives, how will folks deal with it?

    Blogs of note
    personal Christopher Conlin USMC memoirs of hydrogen guy rhapsodic.org Henriette's Herbal Blog
    geek ultramookie

    Powered by WordPress
    Hosted on Dreamhost.