isn't quite ashamed enough to present

jr conlin's ink stained banana

:: Could Someone Check My Math?

Dear Lazyweb,

i am not a math whiz. i get math, but not to the level that it's effortless music. So i'd appreciate ye far smarter folk double checking my thinking here.

The Problem
For the Notifications stuff i'm working on, i need a token that is Really Hard to Guess. That means Lots of Entropy stuffed into a token.

Because this is going through SMTP, i have a hard limit of 64 characters. While also not part of the RFC (or anything else, experience with various MTAs have shown that you're limited to a through z and 0 through 9. Using a handy Entropy Bit Calculator, that shows that:

log2(36)*64 ~= 331 Bits of Entropy.

That means you have a 1 in 2331 (or 4,374,501,449,566,023,848,745,004,454,235,242,730,706,338,861,700,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000) chance of guessing it. (In case you're curious, that's 4 duotrigintillion)

That's great, and all, but does lead one to wind up getting a token that is 64 characters of random crap, like:
wgmrbj3d9pcv1op6opprw7hvjgzimzonakq3m02egtjh8zs2bzyuxnowfx5e9fxu

So i was asked to research how good/bad an idea it might be to make the token a little less random.

The Solution

Specifically, i was asked if i could create a token based on words. That means you'd get a token like:
faucet.variorum.hambling.baiters.harmed.stoa.haysel.glandular.61
i arbitrarily decided to separate the words with periods and back-fill with numbers.

That leads to some interesting math.

i pulled the English Open Word List and collected all the words into a single file (dropping the words that have non-ascii characters). That produced a catalog of 128,766 candidate words. A crappy token generator script later, i found that the mean token consisted of 8 words and was about 61 characters long. By my guess, each token would consist of
[a-z]{2} + // shortest word length was 2 characters
[a-z\.]{59} + // the bulk of the token would be character or a period
[a-z0-9\.]{3} // the remainder would be character, number or period

This would lead me to think that the base entropy would be something like:
(log2(26)*2) + (log2(27)*59) + (log2(37)*3)
or about 306 bits of entropy. (1 in 130 novemvigintrillion odds)

The real problem

But i'm not sure that's true.

That's supposing that the core of the characters being used are taken randomly from that pool of 27 available characters. They're not. These are using words in the English language which means that there are certain character distributions which diminish the pool of potential entropy. Granted, that's masked somewhat by the varying word lengths, but that still means that there aren't a hell of a lot of words that have J, Q, X or Z in them, and that the default letter set that Wheel of Fortune picks for you sucks.

If one takes only the letters that have a greater than 10% chance in 1000 of appearing, then you wind up with about 15 meaning you've got:

log2(26)*2 + log2(27)*59 + log2(37)*3 = 30 bits of entropy, or 1 in 1,073,741,824

That number is no where near good enough. It's even pronounceable.

So, really, the question comes down to: "How much entropy am i losing by using real words?"

Thanks, Lazyweb!

i'll just keep refreshing the page until you answer.

As pointed out in the comments, the short answer is "It's a bad idea". You wind up picking 8 items out of a ~ 128K pool and that's not enough to prevent folks from reasonably guessing.

'tis good to have friends with big brains.

:: Family History, Part 2

More Conlin Family History that i did not make up just now (maybe, ok, definitely). What follows, i have pieced together from local legends, family lore, and the recently discovered wall etchings at the Ros Comáin Home for the Unsettled)

Elias Bertrand Conlin made it his life's mission to do something about the greatest act of evil ever perpetrated by one man against humanity. The senseless loss of millions of lives and the complete destruction of the island of Crete were well known and documented by the historians of the mid 1800's to the point where no greater insult could be made in any language than to call another person a Minion of Brännakattungar.

When Elias was just ten, his father had taken him one summer on a solemn pilgrimage. For days, the train chugged past the fields of graves that nearly filled what remained of France, across the deserts of Northern Italy and finally south to Greece. Father paid the 80 drachma fare and they sailed to see the mighty clouds of steam rise above the Mediterranean where the ocean still lapped against the still cooling dome of raw mantel. At night, even 50 years later, Elias wrote how the sky glowed a faint orange and the smell of molten metal lingered in the breeze. It was a deeply disturbing experience for one so young, but much like the new granite it set Elias' mind about what needed to be done.

He spent much of his adult life deep in studies of math and the new sciences that were cropping up around him. He journeyed to talk to the greatest minds, spent many a sleepless week pouring over thesis and tome, and during his travels, he spent time tinkering with gadgets and mechanisms of his own design. In his 50's he spent what little money he had remaining on a one room apartment in Diakofti. For several days, he hauled crates, lengths of metal pipe, and chemicals into the apartment. Thick black cables snaked from the windows to a set of dynamos he secured in the courtyard. These droned with a regular thrum that made some of the older women of the village nervously cross themselves.

Not much is known of the night when lightning from the apartment struck the heavens. Men speak of how the dynamos shrieked like demons as the black diesel smoke billowed into the starry night. Children speak of the dancing curtains of green light that hung above the village. After the echos of the mighty shock has faded to the point where the frightened cries of the sheep could be heard, the bravest men of the village were met by Elias at his door. His service revolver, held limply by his side, still smoking.

"Is it there?" the blood splattered Elias cried to the shocked villagers. The men had no idea what Elias was talking about and stood, dumb, staring at the crazed foreigner. A cautious voice from the back of the crowd asked, "What?"

Elias angrily pushed his way past the ignorant farmers and ran outside. He started to smile when he no longer saw the hellish orange cloud on the horizon, then laughed joyfully when he spotted the distant lights of Crete on the horizon.

He bruskly grabbed one of the confused men who followed him to the shore. "Tell me what you know of Stephan Brännakattungar!" he demanded.

The villager, having overcome his shock, fired back at Elias. "Look, you, i don't care who the hell that is, but your devilish contraption scared three of my sheep to death." The other men joined his angry tirade with their own concerns and demands for reparations.

Elias stared at the men with his jaw slackened in surprise. Do these men not realize what he had just done? Do they actually believe that their livestock are more important that the rescue of millions? Elias felt his own blood boil and without a word stormed back to his apartment, slammed the door, and a second bolt of lightning cracked through the sky.

He re-emerged to the familiar sent of molten rock and faint orange sky and walked sternly up to the now significantly more Asian villagers. "There, are you happy?" Elias sneered.

"What? No, of course not." The villagers cried. One of the refugees spoke up in broken Greek, "My home in Beijing, full of metal demons. i come here, with rest of family to start new life. Now you do same thing that destroy home. You Stop!"

It was then, as if on summoning, that one of the ironically named Protectors descended from the sky. Beams of death burned several shoreline homes as it's voice boomed "ILLEGAL KNOWLEDGE USED. TERMINATION INITIATING."

Elias knew better than to dawdle as the rest of the village ran screaming into the hills. His door had barely closed when the third bolt split the island's night.

Elias exited his apartment to a fresh sea dawn and a cry of "Now do you see what i've saved you from!?"

From here, i have no record of what happened to the Elias' equipment after the villagers had destroyed it. Only that Elias was labelled as dangerously insane was exiled from the island once they released him. Apparently, he spent a portion of his later years writing highly detailed historical fiction, which he swore was true. He was later "hospitalized" after a rather distressing incident at a Cretan barbeque restaurant.

:: Welcome to the Internet

My oldest niece is twelve. That means that in less than a year, she'll be able to sign up for stuff without parental permission.

Being the geeky uncle i am, i figured i should let her know what to expect.

Pretty soon, you're going to be 13. It's an important year in your life, and as i'm sure you're aware, it's the year you can have an account on sites like Twitter, Gmail and Facebook. It's a point where we think that you're old enough and wise enough to do two things: act like an adult, and take a bit of advice from your geeky uncle.

TL;DR (Too Long; Didn't Read)

  1. Listen first
  2. Know your friends.
  3. Nothing is private.
  4. Stay on guard.
  5. Don't feed the trolls.
  6. Create more than you consume.

Right, so what the heck do i mean by all that?

Listen First

Back when the internet was young and dinosaurs used it to discredit the threat of asteroids, folks used to encourage newcomers to “lurk before you leap”. It was kind of like walking into a room full of conversations. It's polite to walk up to a group and listen to what they're talking about for a while before talking. Think of it this way. You and some of your classmates are in the middle of a conversation about martial arts techniques. You're debating the effectiveness of various holds and stances when a little kid comes up and starts talking about how some dogs are brown. Yeah, it might be cute, but if that kid keeps doing it, things get annoying really fast.

Don't be that little kid.

Read older posts from folks. Listen to what they're talking about (for a few days at least). Listen to how they present themselves. Find out what they find funny or interesting. If it's something that you also find interesting, ask good questions and be respectful. More importantly, if they tend to talk about stuff that makes you angry or scared, say nothing and just walk away.

Lurking keeps you from saying something stupid or from drawing unwanted attention. That's the value of lurking.

Know your friends

On a similar note, all these new, spiffy social sites want to be your friend. Heck, they'll even point out lots of other people who want to be your friend. Folks like the girl you met last summer. Or maybe some boy in your class. Or that creepy guy down by the bus station. Or the woman with 15 cats who screams at her television about communists, Or even someone you've never heard of. It will be tempting to add them all. Everyone likes friends, right? What harm can that do?

Actually, a lot. See the problem is that those people get access to your information. Not just your address and phone numbers, but who you like to hang out with, where you like to go, what your friends are talking about and everything else. i'm sure folks have already talked about the scary stuff, but it can also be a lot more subtle than that.

Imagine that there's a company that sells squid ink soda (now with extra ink). There are a few ways that they can get you to buy a can of carbonated squid ink. They could run ads with celebrities, or cute cartoons of killer whale families, or big signs in Times Square. Those all work, but what's even more effective is to get your friends to tell you how much they love coming home and cracking open a can of fizzy squid juice.

Now, let's say that they create a video featuring their mascot, Squiddo, playing with a kitten. It makes you laugh, and you see a Like button. Sure, you think, i'll click that button and share that info to my friends. Actually, you're doing more than that. When you click that button or become a fan of Squid Ink Soda, they get to see your info, as well as who you're friends with. Suddenly, you may see ads featuring friends and schoolmates next to cans of Squid Ink. You start thinking, “Huh, everybody must like Squid Ink soda. i should try a can.”

What's more, they may start seeing your picture as well and start thinking the same thing. Maybe they go out and buy cans of Squid Ink Soda because they like and trust you. Then they try a can, realize that it's absolutely revolting and think you're an idiot for getting them to drink that awful crap.

Your personal information and friends are really, really valuable. That's why all these sites and companies want it. That's why it's really, really hard to get that info back out of most of these services. Fortunately, you can turn a lot of this stuff off, provided you know how or be a little drastic, or are willing to dig through all the settings and check frequently for new ones. Treat that info like you would a pile of gold coins or gems you have hidden in a cave. Don't just tell everyone about it or you won't have any.

Let's also say that in 5 years or so, you want to go get a part-time job. You put on your best professional clothes and head over for the screening interview. They greet you, sit you down and run a background check on you. That's when they found that you've been friends for 5 years with some kids that recently stole money from work or have a drug habit. Suddenly, you're a potential trouble maker and maybe you should consider looking for work somewhere else.

Colleges are also looking at stuff like social connections as well. Some folks are even demanding that before they hire you, you hand over your login and password so that they can check any private messages you've gotten. (i'll note that this is legal, but questionable and there are a few court cases pending about this, but for now, employers and schools are within their rights to ask for it.)

Nothing is Private

That touches on one of the key things. Nothing online is ever private. If you put something on the Internet, it's public and forever. Facebook is particularly bad at this because they have a habit of resetting your privacy to be more public. That aside, even personal messages can be made public by bad hackers or by bad security on a site.

credit Gene HanThe rule of thumb i use is: Never say anything online unless you're comfortable telling every human being that. There's a lot of things i don't say online (and i'm a fairly public person on the Internet). If you feel like you're going to burst unless you tell someone something, i understand. i frequently write messages to no one and then send them straight to the trashcan.

Even if someone absolutely promises to never, ever tell anyone or share something you gave them because you will kill them if it ever gets out, if you send it over the internet, it's gotten out. Some people lie and post it. Others aren't as careful and it's stolen. Sadly, a lot of folks don't think that way, thus why there are lots of embarrassing photos taken from camera phones that are all over the internet.

If you have to talk to someone to tell them something important, get up, go to that person and tell them to their face. If you can't do that, call them. If you can only talk to them via the internet, let me know and i'll show you how to do it safely, but understand that once that message is in their hands, it's fair game to go on the internet.

Stay on guard

i'd love not to have to tell you this, but you're a girl. Hang on, it's not what you think. i'd love to tell you that the Internet is full of smart folks who don't care if you're a boy or girl or a dog with very good typing skills, but it's not true. There's a group of people that are awful. They're nasty, creepy, and really don't care about you at all, or even worse, want to take advantage of you.

Just like how you can't always predict who the nice person will be, crappy people come from all walks. If you saw them on the street you'd think they were pillars of society, or punks, or sweet little old grandma's. It's always a good idea to be a little suspicious. Be courteous, but always be ready to say “No” or call the authorities.

If it sounds like too good to be true, it's a trap. Always ask why a person or company wants something from you. Know that when you use something like Facebook, you're not the customer, you're what's being sold.

So, why is being a girl important? Partly because, as a girl, you're really sensitive toward emotion and social stuff. (That's normal, by the way, and the product of a few million years of evolution.) It means that you like to talk to people and try to think of how others might react to something you say. Sadly, a lot of folks aren't as in tune with that and are jerks. This will make you angry. It will make you cry. It will make you want to curl up in the corner of your closet until the planet is consumed by the sun.

At least it will if you've not been careful. i won't lie and say that it will never happen, but when it does, you'll be able to deal with it a lot better. Basically, don't let the Internet get to you.

Don't feed the trolls.

There are exceptionally crappy people who enjoy making life miserable for other people. They look for how to push your buttons and get you screaming. They're very good at it. They might say things like how pugs are obviously brain damaged and that anyone who liked them obviously didn't care about dogs. They might say things like how the only good Marine is a dead Marine. Yeah, those are just light hearted teases compared to the kind of crap they normally say. These people are trolls.

They're looking to get you angry. They want to see you cry and fly out of control because that's uproariously funny to them. Because you're a girl, they'll pick and tease you relentlessly and make creepy comments, because they know it will make you cry. They feed off your fury and will keep pushing your buttons until you're absolutely insane with rage. When you reply to them, that feeds them and makes them want to continue abusing you. Don't feed the trolls.

When you see someone saying something offensive, ignore it. (Depending on how offensive it is, you might be able to flag it for someone else to look at and have that account shut down, but trolls will always come back.)

This is not to say that people who have different opinions than you are trolls. Just that if you feel yourself getting angry and provoked, you're probably being trolled. If you feel that's the case, stop talking. Let the troll say she won and ignore them. Let them do their little happy dance under whatever bridge they live, because it's the only highlight to their miserable existence. Eventually, if enough folks spot the troll and refuse to play along, the troll will leave.

(Again, power of lurking will often show trolls and show you who to avoid talking to.)

Create more than you Consume

By and large, though, the internet is not an awful place. It's kind of like the world. There are parts that are horrible, and other parts that are awesome. The parts that are really wonderful is usually stuff made by folks like yourself.

People write. They draw, they make films, they sing songs and you can see just how amazing people can be. Be original, create stuff that folks haven't seen or heard. Be inspired, read stories, find short movies on Youtube, giggle at pictures on CuteOverload and IcanHasCheezburger. Be aware that someone else may have created what you enjoy, and respect how they want to distribute it. Some creators may be open with their creations, like “Sita sings the Blues” and want it shared with everyone. Others (like most Hollywood movies and Recording Artists) want tight control over who gets to hear what they want you to hear, and that's fine too. If they don't want to share, it's their choice. Respect it like you'd respect your own stuff. If you're not sure, presume that they don't want to share. It's generally safer.

If you do share, realize that someone may copy it and claim it as their own. It happens. Ask them to not do that or call them out on it. Always be polite and professional, even if they're not. More importantly, if you keep making stuff, folks will know who the real creative person is.

That's about it for now. Like i said, we're trusting you on the Internet because we hope you're mature enough and smart enough to deal with it. Remember, the internet is full of people, both good and bad. Use your head, pick the choice that involves thinking about it harder, and you'll do fine.

:: Notifications and Bipostal

So, i should probably talk about what the heck i've been doing at Mozilla, shouldn't i?

Well, with things finally starting to surface, i'm a bit more comfortable talking about them. The first part of what i'm working on is Notifications. What the heck is "Notifications" you ask? Well, it's kinda tricky.

The elevator pitch i like to give is "Somewhere between Instant Messaging and Email is 'Notifications'". It's a way for sites to semi-anonymously send messages to a user. Communication is one way right now, mostly for simplicity sake between the site and the user, but there's precious little to prevent the communication from going either way.

Ah, this is our floor, shall we get out of the elevator and actually talk about this? Cool.

The history lesson

A little over a year ago, a couple of damn bright interns spent their summer building a prototype notification system that used AMQP and a few other things to pass messages back and forth. The cool thing is that it allowed browsers to talk to browsers, or sites to talk to browsers or really anything to talk to anything. You could get twitter announcements in your chrome, or send a tab to your mobile device or all sorts of things. It was spiffy, but unfortunately, had issues. A fairly large one was relying on AMQP, meaning a persistent socket connection. That's expensive on a whole slew of levels, not including trying to convince your grandma to punch a hole into her firewall.

So, as is the case with a lot of good ideas, we headed back to the whiteboard to figure out what elements we can use. Some things, like sending a tab to a device, turned out to work better if we used something like sync. That still left a few other features that we wanted.

Enter the BrowserID

BrowserID is cool. The ability to log into a site by selecting what email you want to provide to them is amazingly simple! Granted, if you're logging into a site like GnomeBondage.com, you probably don't want to give them an email that will let them fill your work email box with things you may not want your employer to see.

That's why you want something that is a bit harder for them to associate back to you. And that's what i've been working on.

(Originally, Bipostal (BrowseriD Postal Services, no, really. Stop giggling like that.) was meant to be a later addition to the Push Notifications stuff. Because BrowserID pushed forward, though, the need was higher for that part.)

So, Bipostal generates a token that is specific for you and the third party site (say example.org). The token is ~64 base36 characters resulting in 64*(log2(36) ~= 5.17) = 330 bits of entropy or 2187250724783011924372502227117621365353169430893212436425770606409952999199375923223513177023053824 possible combinations. That's pretty large. Plus, we're doing a number of things to prevent spammers and other ne'er do wells from sending in just random garbage.

When a site wants to send you a note, the send it to an address like "[email protected]browserid.org". We make sure it's legit, strip out the fancy HTML cruft, and sent it to you. You can also quiet messages to that address (if some site turns out to be overly chatty) or delete that ID. In the future, sites can include bits of JSON in their email that can get pulled out and sent to you as notifications. All magical and pseudonymous. Well, unless you fill out all the profile info with your real values, in which case, they know everything about you, but that's an "out of band" problem.

What's to come

Honestly, quite a bit. While a lot has been nailed down (both Push and Bipostal are on Github), but that doesn't mean we don't want to hear folks comments and ideas. i've included two of the ways you can provide feedback on the Notifications main page. Likewise, you can comment here and i'll try to respond both here and via email.

Likewise, we'd really love for other companies to help us work out the details to provide a cost effective, light weight platform for this sort of thing. (Websockets and SIP are neat, but require persistent connections which can be costly. We have the option to do message encryption, which would allow the server to not know the content of the message being transmitted, but it would be neat to use non invasive encryption validation to see if we can prevent bogus messages from being delivered.) It's always good to have bigger brains helping out. There's a lot we can do and a lot we're trying to make sure we don't mess up.

Now more than ever, What do you think?

Blogs of note
personal Christopher Conlin USMC memoirs of hydrogen guy rhapsodic.org Henriette's Herbal Blog
geek ultramookie

Powered by WordPress
Hosted on Dreamhost.