Oops! Something went sideways.

Looks like the styling got goofed up. Sorry about that, unless it's what you wanted. If this isn't what you were looking for, try force refreshing your page. You can do that by pressing Shift + F5, or holding Shift and clicking on the "reload" icon. (It's the weird circle arrow thing "⟳" just above this page, usually next to where it says https://blog.unitedheroes.net...)

isn't quite ashamed enough to present

jr conlin's ink stained banana

:: Ec-COVID-Nomics

SARS/Corona Virus 2019 (COVID19) is a terrible disease, on a lot of fronts. The thing i really can’t get over are folks that say stuff like this:

A NextDoor post where someone proudly claims their going to a 40-50 person gathering because the disease has a \

i mean, sure? If you’re young-ish and fortunate, you do have a fairly good chance of getting through it alive. Hooray?

Of course, that’s not really the problem, at least, that’s not the most significant problem you face in the US.

Let’s consider what you face if you get the disease.

First off, there’s dealing with the disease itself. For some folk, it’s nothing. As in they have no symptoms what-so-ever. Other folks require hospitalization. How your body reacts to COVID is pretty much anywhere in-between, and there’s no knowing what it will be. There are also potential long term considerations as well, since it’s still quite a new disease and nobody is quite sure how it will impact everyone.

You may only be “sick” for a few days get “better” and since you got the ‘rona, feel you don’t need to worry about wearing a mask. You’re now a spreader since you’re still contagious since the virus is still very much present in your system. (You could also be asymptomatic, which means you have the disease, but aren’t showing or feeling any symptoms. Feel free to read up about “Typhoid Mary” if you want a nice, historical record of how this could happen.)

But let’s say you’re unfortunate enough to actually require hospitalization. Because we’re America, once you’re released, you’re looking at a bill of anywhere from $32,000 to $73,000 (depending on how good your coverage is). It can also be a whole lot more than that, depending on where you get your care.

i don’t know if you’re able to buy a car right now out of pocket, but that’s kind of the numbers you’re looking at. If you’re not, you’re going to have to figure out where to get that money. Again, since we’re America, you’ll probably turn to the age old practice of finding someone to sue. If not you, don’t worry, your insurance company will probably do it for you. They don’t want to spend that sort of money either, so if they can find someone who exhibited clear, reckless behavior, you bet they’ll be right on top of that.

Of course, if you’re in the clear and someone you’ve contacted afterwards develops COVID, well, let’s just say that announcing your open defiance of strongly suggested health guidelines may not be quite as bold as you had thought.

(i honestly believe that this is the major reason that the US has not implemented Contact Tracing like Canada has. i’m pretty sure someone figured out that having a clear path between litigant and plaintiff may not be fantastic.)

What’s more, again, since we’re America, and our health providers don’t like pre-existing conditions, this is something that could actually come back to haunt you years from now.

So, yeah, that’s why i have zero intention of going to large gatherings so long as COVID is still very much a thing.

:: Why Are You Doing That?

i’ve been doing a fair bit of mentoring lately. i guess because i’m obviously old and folks think i’ve got some wisdom about anything. To be fair, i am old.

Anyway, recently i got into a discussion with someone who’s been thinking a fair bit about her career. She started off doing data work, then did a bit of UI/Front end stuff, and just didn’t find it super fun or compelling. Honestly, very understandable.

i’ve always hated the semi-utopian thing about “Find a job doing what you love”. i’ll just come out and say that’s incredibly rare. There’s a reason that they can do shows about folks who manage to do that, it’s because they’re unique enough to be interesting. The rest of us? Yeah, we’re not as fortunate.

Don’t get me wrong. i have training for programming and some level of skill at it, but what makes me happy is not making servers go ping, but fixing problems and clearing tasks off of lists. i could do that anywhere and feel just as much sense of accomplishment. What makes me excited is not what i do, but why i do it.

Call me an idealist, but i actually really do want to make things better for people. To that end, i view personal privacy and security currently woefully lacking. It’s out there, but it’s not the path of least resistance and so folks tend to skip over it. Working for mozilla gives me the daily opportunity to fix things to be easier, more secure, and more private. That’s the reason i’m still working there and not, i dunno, CTO of some ad network that resells organs harvested from orphans or something.

What i do is not glamorous. My peers and i keep a bunch of back-end services running. We’re not going to be top of HackerNews. Heck if we get double digit count of stars on github, we’ll wonder what the hell happened. Still, we have around half a billion active connections and deliver messages in less than 100ms, juggle nearly a petabyte of encrypted user data, and write in the latest version of Rust because it’s the most performant and cheapest option for doing all of that.

Still, i’m working for a company that’s philosophically aligned with my interests, so yeah, i’ll deal with the frustrations and stress just fine, thank you.

Of course, there are down sides. i will never be invited on to the stage to talk about what i do. That means that promotions and bonuses are rare events. You’ll never know if i do my job right, but you’ll sure as hell know when i do it wrong.

:: The Internet Hates Long Lived Things

First off, this is not about ageism. i’m talking about long lived connections. There are a few folk out there that believe that you can hold a connection between two devices open forever. This is not the case. There are a lot of reasons that a great many things will actively fight your long lived connection. So, here are a few insights from someone who has dealt with Very Long Connections in Webpush and was once naive like you.

Why does the internet hate long lived connections?

Short answer: Money.

Longer answer:
The internet is not free.

Everything about the internet costs money, because everything requires either power or devices. Devices are way more costly because you not only need to buy and power them, you need to shelter, maintain, inspect, and eventually replace them. This includes everything from colocation farms to servers to cables to the conduits that carry the cables and the folks who’s jobs it is to do all that sheltering, maintaining and inspection. The costs may be near infinitesimal for a 10 byte ping, but they’re there, and they add up surprisingly fast.

i’ll also add in that connections between devices also have a software cost. Turns out, there are a limited number of connections that a given computer can accept. There are also constraints depending on the language you use, how much memory you have installed, how fast your CPU is, and how many files you need to have open. There are fun ways to tweak that number and get really high counts, but if you’re doing any actual work with them, you’re going to hit that upper limit. If you’re doing real, serious work (like running TLS so things are secure) boy golly are you going to hit that number and it’s not going to be anywhere near that 10 million connection number someone built for Erlang.

So, in that sort of world where having connections that are basically doing nothing but tying up resources, connections are not going to stick around. You may not want to pay for them, and neither do any of the dozens of intermediary companies what want to maximize profits. They’ll spot a connection as being underused and will simply drop it, since there is probably some other company that wants to use it and send lots of capital producing data over it.

There are tons of reasons a connection could be killed at any time and a whole lot of incentive to ignore any requests you might make to keep a low bandwidth connection up. This includes various “Keep Alive” packets helpfully provided by protocol authors. Those tend to be very light weight dedicated Ping/Ack packets that are sent on a regular cycle. They’re useful if you’ve got a lull for a few minutes, but anything longer than that and the connection is toast. You’re better off crafting a NoOp type message that you fire off regularly. Granted, i fully expect that those will be dropped in the future too once providers use stuff like packet inspection machine learning to further reduce costs and free up “idle” connections.

Well, what about using stateless UDP instead of stateful TCP?

It’s not a bad idea, really. It’s the reason that QUIC is the base for HTTP 3.0, and it’s very clever about making sure that packets get handled correctly. Packets are assigned Server Ids, and cryptography is isolated so data corruption doesn’t cause blockages. Even though, if there’s a connection severance, it’s still dependent on the Client getting back to the Server. The server needs to be at a known, fixed address. That’s neato for things like HTTP, but less so for things like WebPush where the client could be waiting hours or days for a response, and unless the client is actively monitoring the connection (remember, built in KeepAlive packets ain’t enough), it’s basically doing long polling, so you’re kind of back at square one.

(There’s definitely something to be said about that for things like WebPush. WebPush’s “Immediate receipt” requirement, like relativistic travel, depends a great deal on the perspective of the parties involved. That’s a topic for another post.)

So, be mindful young protocol developer/designer. The internet is out to get your long lived connection dream and will dance on it’s grave at every opportunity.

:: Web Pushless


i’m one of the nice people that brought you WebPush. That lovely tech that is probably one of the most user hated things to roll out. i work on the back-end bits. i still hold that in spite of idiot web marketing folk who don’t want us to have nice things, web push is still really useful, but that’s not important right now.

What i want to talk about today is something i’ve been asked a good deal lately:

“How do i provide push notifications on mobile devices if i can’t use device native Push?”

Ok, that probably sounds like a really weird question, but let me explain a few things.

How Push works:

i’m not going to go into super detail here, but suffice to say that Web Push provides a way for servers you’re not connected to currently to send you messages that you’ve agreed to having delivered. It’s super easy for you to send messages to servers since they don’t move around and change their IP address every 15 minutes. Your phone may well do that. So what we do internally is have your phone connect to one of a bunch of servers we run, then it sits around waiting for you to send a message. For things like laptops or desktops which have big batteries or are always plugged in, that’s a great solution. For phones, however things are a bit different.

How Push works on Mobile Devices:

i do believe that Donald does not approve of your battery usage.

Your phone doesn’t want to be on. It wants to power down as much as possible so that your battery doesn’t die after an hour or so. It has LOTS of VERY AGGRESSIVE power management things it does in order to facilitate that. It will also flag any app that consumes “too much” battery and point at it like Donald Sutherland in Invasion of the Body Snatchers. Naturally, since having a reliable connection from your devices maker’s servers to your device is actually really useful, they’re very forgiving about any that they might set up. In fairness, they have a lot of neat tricks they can pull at very low levels to keep your CPU asleep and the battery usage minimal that they’re absolutely not going to let your J. Random Application take advantage of.

So instead they offer a way for you to piggyback on top of their protocols. That’s what Firefox on Android (and to some extent Firefox on iOS and Amazon FireTV) does. The data we send over these bridges is still encrypted because the decryption key is in the User Agent (the actual “Firefox” application).

The problem is, running machines that your device connect to for long periods is kind of expensive. As in people in the Accounts Payable department screaming “WHY DOES THIS BILL HAVE SO MANY ZEROES!?” sort of expensive. There are things you can do to control costs, but frankly, when you’re talking about hundreds of millions of phones calling in, you’re talking about having at least a few good boxes just to handle the loads. It’s kinda depressing how much doing nothing costs, particularly in setting up a secure link that does nothing, but that’s our problem more than yours. (Although, next time you want to buy something, if you were to search for it in the AwesomeBar, click on one of the ads and buy it from there, that’d be swell!)

That’s one of the reasons that Google put their messaging system under Google Play. It’s an app that only gets installed on authorized Google Android devices, and (surprise) it costs money to do that. You can absolutely grab a copy of the Android Open Source, customize it to fit your phone’s hardware platform and get rolling. You might even be able to sideload some versions of Google Play onto those devices, but Android is the most used phone platforms on the planet and even Google has pipers to pay.

So, how do you do Push if Push isn’t there?

Thus we come to the (potentially) million dollar question.

So, if you’ve got an off-brand Android phone, it’s probably using the Open Source release of Android, which does not have Google Play. Honestly, it probably doesn’t have a lot of services. So what options are there?

  1. Polling: This is probably the easiest. When your app is active (or if you set up a timer) you could have it poll a well known server address and check to see if there are any messages. You want to be careful with this, to avoid “thundering herds” where all the devices suddenly check at once and swamp your servers. You can randomize things a bit, but i’ve also seen some devices that “helpfully” round sleep timers to a nearest interval (e.g. you thought you said sleep for 5 minutes? Oh, well, we slept for 15 since that means less CPU.) Some experimentation and monitoring your servers may be required.

    Pro: here is that it’s fairly straightforward and simple to do.
    Con: it’s not exactly “timely”. Good for “Remember John’s Birthday tomorrow” less for “your tea kettle is boiling”.

  2. Active Reception: This one is a bit trickier. Basically, when your app is active, it connects to your servers using WebSocket, HTTP/2 or whatever protocol and actively pulls and listens for messages. This can provide much faster message deliveries while the user is present and attentive.

    Pro: Quick message delivery with feedback.
    Con: Could be complex and doesn’t work when the device is sleeping.

  3. Combo: This one combines the two above steps. You have a small stub program that checks a URL to see if there are any messages pending, and if so, spins up your app to do a full connection. The connection processes everything, then lets the device go back to sleep.

    Pro: Almost exactly like Push, sort of.
    Con: Complex, and probably buggy. Dances the line between “efficient” and “here come the howler monkeys”

Sadly, i believe that any of these would probably constitute a “savvy business opportunity” for some startup, and while i’ve not looks, i would not be surprised in the least if there was a company out there that was offering a service like one of these. i don’t think it would be free though, mostly because of the costs associated with it.

:: Chaos and Kindness

There are two completely different events that have happened in the last week that i need to think about. i tend to find that i think most when i am on a keyboard, so yay you ineffable void and ad bot now reading this, you get more words!

1) Mozilla’s Layoffs.
social media love
The company i work for announced layoffs for about 70 out of 1000 employees. The folks were chosen by project and role, that part is normal. What’s not normal was something i don’t think i’ve ever seen another company do. The employees were not treated like modern lepers and tossed out the door.

Instead, they were told that they were going to be laid off, but still had access to most of the things they needed. This included company mail, internal Slack channels, resources, etc. Folks inside the company rallied to support them. Spreadsheets were created that had employee info and prospective or recommended hires from folks networks. Social Media networks hosted “#MozillaLifeboat” to help get folks on their feet fast, and many very positive words were said in praise of those who were let go.

The folks who we let go were treated like humans. There was an all hands meeting held a couple of days after the layoffs occurred. The folks laid off were encouraged to attend, ask really hard questions, and were given good answers.

Ask yourself, “Would your company have done that? Could they have done that?”

Granted, mozilla works pretty hard on not hiring sociopaths and jerks, so it’s just not really the culture to be terrible to each other. Still, i’ve been through five rounds of layoffs, and had never seen that level of trust.

As remarkably smooth that incredibly disruptive and painful experience was, it did absolutely drive home a point i’ve been thinking for years: You need to be most loyal to the friends and colleagues you meet in your career than to anywhere you happen to work. Any employer that demands faithfulness solely to them is a huge risk to your professional and personal life. Your friends are who will help you, your employer is not. If you work for somewhere you can’t get that, it’s a HUGE red flag. The money might be good, but the risk is tremendous. i can say with first hand knowledge that getting paid well at a place that doesn’t respect you as a person eats at you in subtle ways.

A side product is that you remember that you’re dealing with people, and as such, folks are making it up as they go. Folks want you to believe that there’s a plan and direction, but quite often, there’s not. More often than not, there’s just a rough guess and a general feeling dressed up in powerpoint slides and bold rhetoric. Again, unless you’ve got sociopaths at the helm, layoffs hurt the folks making the decisions about who stays and who goes. Even if they are sociopaths, the company is giving up the money invested in the person and whatever income that person could have brought in.

(Oh, and if you’re ever working somewhere and see absolutely no sign of remorse or regret when an executive talks about layoffs, leave. i’m talking about actual regret, not “Sorry to see those folks go :sad face emoji: it’s terrible. Anyway, who else here is excited to see the Project Foo we’re launching!? [loud, upbeat techno music]”. Yeah, after that, spend the rest of the day polishing up the resume and sending notes to your network about potential leads.)

2) Actix drama

i’ll preface to say that i don’t know all the details about the drama around actix-web. As i understand, there were some concerns around coding practices, a single maintainer, and some folks who may have been jerks. Coding practice discussions are part of any open source projects, single maintainers are concerning for anything other than a small package that’s just starting, and half of the world are jerks.

What happened was that the project maintainer pulled the library code off of github and announced he was done with open source. Honestly, that’s good, because i believe he didn’t know what open source really was.

Let me diverge a bit here.

Open source is about trust.

When you decide to use a package, you are extending trust that:

  • The program/library/package works.
  • It will continue to do so.

Bug fixes, improvements, documentation, etc are also part of that, but kinda fit into the list above. Open source can sometimes be called “Free as in puppy” in that you might be getting into a lot more than you expected.

It’s very rare that the trust is broken. There are ways for a package maintainer to step away from a given package. They could ask a larger group to take over. They could pass it on to someone else. They could “archive” the package and let someone else fork it into a new version. Almost never does anyone just yank their code down in the same way that you almost never see an argument end with someone throwing a temper tantrum. It’s sad because while the author may have been a talented engineer, i can no longer trust anything that they produce.

Would things have been different if folks were not jerks? Probably. Likewise, i think folks were presuming a level of emotional maturity that may not have been present. i don’t fault the author for his actions, even though i’m deeply impacted by them. i’ll survive, reassess and move on. i’m saddened by them, but i look forward to the growth that i hope he gets to experience.

So, how do these things both relate?

In essence, it’s about people. It’s about remembering that at the end of the day, we’re all real, breathing, mentally weird beings and not just clever bags of thinking meat. Sure, there are some openly hostile folk out there, and there are trolls, dirtbags, grifters, and fools, but those tend to be the painful exceptions, rather than the rules.

As Michelle McNamara often said, “It’s chaos, be kind“.

Blogs of note
personal Christopher Conlin USMC Henriette's Herbal Blog Where have all the good blogs gone?
geek ultramookie

Powered by WordPress
Hosted on Dreamhost.