Oops! Something went sideways.

Looks like the styling got goofed up. Sorry about that, unless it's what you wanted. If this isn't what you were looking for, try force refreshing your page. You can do that by pressing Shift + F5, or holding Shift and clicking on the "reload" icon. (It's the weird circle arrow thing "⟳" just above this page, usually next to where it says https://blog.unitedheroes.net...)

isn't quite ashamed enough to present

jr conlin's ink stained banana

:: Don’t Craft Solutions, Solve Problems

Back in the last millennium, when i was in college, i had a systems architecture prof that established my love of simplicity. He would frequently state the title of this post to the class, and we, young formulating minds would have only the barest inkling of what the hell he was talking about.

He had another statement that he made the very first day of class “There will be some day when you walk in on a workplace that does business with shoeboxes full of index cards. You’ll do your analysis and discover that for that business, the best method is to use shoeboxes full of index cards. Walk away.”

There’s a lot of wisdom in that statement. Yes, there may be more efficient methods from an objective point of view, but that’s missing the subjective point that if it’s not what the customer wants, they’re not going to use it.

That gets me back to the “Don’t craft solutions” bit. Basically, don’t become focused on the solution. It may technically solve a customer’s problem, but it may be more complicated, or require more time, or not quite work with all the bits that the customer didn’t talk about or think was necessary. You’d think that would be common knowledge, but it’s not.

It’s surprisingly seductive to become enthralled with what you believe to be the solution. Often, these solutions are so wondrous, that the folks crafting them never bothered to actually talk to the people having the problem. Hacker News is full of folks who have these kinds of solutions, and the rich history of failed startups they created. Heck, spend some time in the medical or manufacturing industry and you’ll see dozens of these “crafted solutions”. Any time you’re wondering why you have to give your information to someone multiple times within minutes, you’re experiencing one “crafted solution” after another.

And the poor soul on the other side is just as annoyed as you are about it, and you’re pride and joy of digital craftsmanship is going into the can as soon as possible.

So, yeah, solve problems. Understand the actual flow and requirements of a customer. Spend time talking to the folks who will be using your system and understand what they need and how they are planning on using it. Yes, it takes time. It means crafting those ancient relics called “requirements”. It also means that it may be weeks or months before you can start the fun of actually building something. Yes, there’s the possibility that someone else may craft a solution, but you’ll be able to point out how it’s deficient.

And, yeah, maybe they’ll actually use your system and wonder why they ever used shoeboxes before.

:: Sending WebPush Notifications via Mozilla Push Service, Autopush

The following is a draft i’m working on that describes how to do WebPush data encryption along with VAPID signing. It’s a bit deep, particularly for the normal fare here. If you want a glimpse of what i do, feel free to dig in


Introduction

The Web Push API provides the ability to deliver real time events (including data) from application servers (app servers) to their client-side counterparts (applications), without any interaction from the user. In other parts of our Push documentation we provide general a reference for the API and a basic usage tutorial. This document addresses the server-side portion in detail, including integrating Push into your server effectively, and how to avoid common issues.

Note: Much of this document presumes you’re familiar with programming as well as have done some light work in cryptography. Unfortunately, since this is new technology, there aren’t many libraries available that make sending messages painless and easy. As new libraries come out, we’ll add pointers to them, but for now, we’re going to spend time talking about how to do the encryption so that folks who need it, or want to build those libraries can understand enough to be productive.

Bear in mind that Push is not meant to replace richer messaging technologies like Google Cloud Messaging (GCM), Apple Push Notification system (APNs), or Microsoft’s Windows Notification System (WNS). Each has their benefits and costs, and it’s up to you as developers or architects to determine which system solves your particular set of problems. Push is simply a low cost, easy means to send data to your application.


Push Summary


The Push system looks like:
push

Application — The user facing part of the program that interacts with the browser in order to request a Push Subscription, and receive Subscription Updates.
Application Server — The back-end service that generates Subscription Updates for delivery across the Push Server.
Push — The system responsible for delivery of events from the Application Server to the Application.
Push Server — The server that handles the events and delivers them to the correct Subscriber. Each browser vendor has their own Push Server to handle subscription management. For instance, Mozilla uses autopush.
Subscription — A user request for timely information about a given topic or interest, which involves the creation of an Endpoint to deliver Subscription Updates to. Sometimes also referred to as a “channel”.
Endpoint — A specific URL that can be used to send a Push Message to a specific Subscriber.
Subscriber — The Application that subscribes to Push in order to receive updates, or the user who instructs the Application to subscribe to Push, e.g. by clicking a “Subscribe” button.
Subscription Update — An event sent to Push that results in a Push Message being received from the Push Server.
Push Message — A message sent from the Application Server to the Application, via a Push Server. This message can contain a data payload.

The main parts that are important to Push from a server-side perspective are as follows — we’ll cover all of these points below in detail:


Identifying Yourself


Mozilla goes to great lengths to respect privacy, but sometimes, identifying your feed can be useful.

Mozilla offers the ability for you to identify your feed content, which is done using the Voluntary Application server Identification for web Push VAPID specification. This is a set of header values you pass with every subscription update. One value is a VAPID key that validates your VAPID claim, and the other is the VAPID claim itself — a set of metadata describing and defining the current subscription and where it has originated from.

VAPID is only useful between your servers and our push servers. If we notice something unusual about your feed, VAPID gives us a way to contact you so that things can go back to running smoothly. In the future, VAPID may also offer additional benefits like reports about your feeds, automated debugging help, or other features.

In short, VAPID is a bit of JSON that contains an email address to contact you, an optional URL that’s meaningful about the subscription, and a timestamp. i’ll talk about the timestamp later, but really, think of VAPID as the stuff you’d want us to have to help you figure out something went wrong.

It may be that you only send one feed, and just need a way for us to tell you if there’s a problem. It may be that you have several feeds you’re handling for customers of your own, and it’d be useful to know if maybe there’s a problem with one of them.

Generating your VAPID key

The easiest way to do this is to use an existing library for your language. VAPID is a new specification, so not all languages may have existing libraries.
Currently, we’ve collected several libraries under https://github.com/web-push-libs/vapid and are very happy to learn about more.

Fortunately, the method to generate a key is fairly easy, so you could implement your own library without too much trouble

The first requirement is an Elliptic Curve Diffie Hellman (ECDH) library capable of working with Prime 256v1 (also known as “p256” or similar) keys. For many systems, the OpenSSL package provides this feature. OpenSSL is available for many systems. You should check that your version supports ECDH and Prime 256v1. If not, you may need to download, compile and link the library yourself.

At this point you should generate a EC key for your VAPID identification. Please remember that you should NEVER reuse the VAPID key for the data encryption key you’ll need later. To generate a ECDH key using openssl, enter the following command in your Terminal:

openssl ecparam -name prime256v1 -genkey -noout -out vapid_private.pem

This will create an EC private key and write it into vapid_private.pem. It is important to safeguard this private key. While you can always generate a replacement key that will work, Push (or any other service that uses VAPID) will recognize the different key as a completely different user.

You’ll need to send the Public key as one of the headers . This can be extracted from the private key with the following terminal command:

openssl ec -in vapid_private.pem -pubout -out vapid_public.pem

Creating your VAPID claim

VAPID uses JWT to contain a set of information (or “claims”) that describe the sender of the data. JWTs (or Javascript Web Tokens) are a pair of JSON objects, turned into base64 strings, and signed with the private ECDH key you just made. A JWT element contains three parts separated by “.”, and may look like:

eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiJ9.eyJzdWIiOiAibWFpbHRvOmFkbWluQGV4YW1wbGUuY29tIiwgImV4cCI6ICIxNDYzMDg3Njc3In0.uyVNHws2F3k5jamdpsH2RTfhI3M3OncskHnTHnmdo0hr1ZHZFn3dOnA-42YTZ-u8_KYHOOQm8tUm-1qKi39ppA

  1. The first element is a “header” describing the JWT object. This JWT header is always the same — the static string {typ:"JWT",alg:"ES256"} — which is URL safe base64 encoded to eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiJ9. For VAPID, this string should always be the same value.

  2. The second element is a JSON dictionary containing a set of claims. For our example, we’ll use the following claims:
    {
    "sub": "mailto:[email protected]",
    "exp": "1463001340"
    }

    The claims are as follows:

    1. sub : The “Subscriber” — a mailto link for the administrative contact for this feed. It’s best if this email is not a personal email address, but rather a group email so that if a person leaves an organization, is unavailable for an extended period, or otherwise can’t respond, someone else on the list can. Mozilla will only use this if we notice a problem with your feed and need to contact you.
    2. exp : “Expires” — this is an integer that is the date and time that this VAPID header should remain valid until. It doesn’t reflect how long your VAPID signature key should be valid, just this specific update. Normally this value is fairly short, usually the current UTC time + no more than 24 hours. A long lived “VAPID” header does introduce a potential “replay” attack risk, since the VAPID headers could be reused for a different subscription update with potentially different content.

    Feel free to add additional items to your claims. This info really should be the sort of things you want to get at 3AM when your server starts acting funny. For instance, you may run many AWS S3 instances, and one might be acting up. It might be a good idea to include the AMI-ID of that instance (e.g. “aws_id”:”i-5caba953″). You might be acting as a proxy for some other customer, so adding a customer ID could be handy. Just remember that you should respect privacy and should use an ID like “abcd-12345” rather than “Mr. Johnson’s Embarrassing Bodily Function Assistance Service”. Just remember to keep the data fairly short so that there aren’t problems with intermediate services rejecting it because the headers are too big.

    Once you’ve composed your claims, you need to convert them to a JSON formatted string with no padding space between elements, for example:

    {"sub":"mailto:[email protected]","exp":"1463087677"}

    Then convert this string to a URL-safe base64-encoded string, with the padding ‘=’ removed. For example, if we were to use python:

    import base64
    import json
    # These are the claims
    claims = {"sub":"mailto:[email protected]",
    "exp":"1463087677"}
    # convert the claims to JSON, then encode to base64
    body = base64.urlsafe_b64encode(json.dumps(claims))
    print body

    would give us

    eyJzdWIiOiAibWFpbHRvOmFkbWluQGV4YW1wbGUuY29tIiwgImV4cCI6ICIxNDYzMDg3Njc3In0

    This is the “body” of the JWT base string.

    The header and the body are separated with a ‘.’ making the JWT base string. :

    eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiJ9.eyJzdWIiOiAibWFpbHRvOmFkbWluQGV4YW1wbGUuY29tIiwgImV4cCI6ICIxNDYzMDg3Njc3In0

  3. The final element is the signature. This is an ECDH signature of the JWT base string created using your VAPID private key. This signature is URL safe base64 encoded, “=” padding removed, and again joined to the base string with an a ‘.’ delimiter.

    Generating the signature depends on your language and library, but is done by the ecdsa algorithm using your private key. If you’re interested in how it’s done in Python or Javascript, you can look at the code in https://github.com/web-push-libs/vapid.

    Since your private key will not match the one we’ve generated, the signature you see in the last part of the following example will be different.

    eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiJ9.eyJzdWIiOiAibWFpbHRvOmFkbWluQGV4YW1wbGUuY29tIiwgImV4cCI6ICIxNDYzMDg3Njc3In0.uyVNHws2F3k5jamdpsH2RTfhI3M3OncskHnTHnmdo0hr1ZHZFn3dOnA-42YTZ-u8_KYHOOQm8tUm-1qKi39ppA

Forming your headers

The VAPID claim you assembled in the previous section needs to be sent along with your Subscription Update as an Authorization header Bearer token — the complete token should look like so:

Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiJ9.eyJzdWIiOiAibWFpbHRvOmFkbWluQGV4YW1wbGUuY29tIiwgImV4cCI6ICIxNDYzMDg3Njc3In0.uyVNHws2F3k5jamdpsH2RTfhI3M3OncskHnTHnmdo0hr1ZHZFn3dOnA-42YTZ-u8_KYHOOQm8tUm-1qKi39ppA

(Note: the header should not contain line breaks. Those have been added here to aid in readability)

The Authorization header ID will be changing soon from "Bearer" to "WebPush". Some Push Servers may accept both, but if your request is rejected, you may want to try changing the tag. This, of course, is part of the fun of working with Draft specifications

You'll also need to send a Crypto-Key header along with your Subscription Update — this includes a p256ecdsa element that takes the VAPID public key as its value — formatted as a URL safe, base64 encoded DER formatted string of the raw keypair.

An example follows:
Crypto-Key: p256ecdsa=BA5vkyMXVfaKuehJuecNh30-NiC7mT9gM97Op5d8LiAKzfIezLzCZMwrY7OypBBNwEnusGkdg9F84WqW1j5Ymjk
Note: If you like, you can cheat here and use the content of “vapid_public.pem”. You’ll need to remove the “-----BEGIN PUBLIC KEY------” and “-----END PUBLIC KEY-----” lines, remove the newline characters, and convert all “+” to “-” and “/” to “_”.

Note: You can validate your work against the VAPID test page — this will tell you if your headers are properly encoded. In addition, the VAPID repo contains libraries for JavaScript and Python to handle this process for you.

We’re happy to consider PRs to add libraries covering additional languages.


Receiving Subscription Information


Your Application will receive an endpoint and key retrieval functions that contain all the info you’ll need to successfully send a Push message. See Using the Push API for details about this. Your application should send this information, along with whatever additional information is required, securely to the Application Server as a JSON object.

Such a post back to the Application Server might look like this:

{"customerid": "123456",
"subscription": {"endpoint": "https://updates.push.services.mozilla.com/push/v1/gAAAA…",
"keys": {"p256dh": "BOrnIslXrUow2VAzKCUAE4sIbK00daEZCswOcf8m3TF8V…",
"auth": "k8JV6sjdbhAi1n3_LDBLvA"}},
"favoritedrink": "rainbow bowl"}

In this example, the “subscription” field contains the elements returned from a fulfilled PushSubscription. The other elements represent additional data you may wish to exchange.

How you decide to exchange this information is completely up to your organization. You are strongly advised to protect this information. If an unauthorized party gained this information, they could send messages pretending to be you. This can be made more difficult by using a “Restricted Subscription”, where your application passes along your VAPID public key as part of the subscription request. A restricted subscription can only be used if the subscription carries your VAPID information signed with the corresponding VAPID private key. (See the previous section for how to generate VAPID signatures.)

Subscription information is subject to change and should be considered “opaque”. You should consider the data to be a “whole” value and associate it with your user. For instance, attempting to retain only a portion of the endpoint URL may lead to future problems if the endpoint URL structure changes. Key data is also subject to change. The app may receive an update that changes the endpoint URL or key data. This update will need to be reflected back to your server, and your server should use the new subscription information as soon as possible.


Sending a Subscription Update Without Data


Subscription Updates come in two varieties: data free and data bearing. We’ll look at these separately, as they have differing requirements.
Data Free Subscription Updates
Data free updates require no additional App Server processing, however your Application will have to do additional work in order to act on them. Your application will simply get a “push” message containing no further information, and it may have to connect back to your server to find out what the update is. It is useful to think of Data Free updates like a doorbell — “Something wants your attention.”

To send a Data Free Subscription, you POST to the subscription endpoint. In the following example, we’ll include the VAPID header information. Values have been truncated for presentation readability.


curl -v -X POST\
-H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiJ9.eyJhdWQiOiJodHR…"\
-H "Crypto-Key: p256ecdsa=BA5vkyMXVfaKuehJuecNh30-NiC7mT9gM97Op5d8LiAKzfIezLzC…"\
-H "TTL: 0"\
"https://updates.push.services.mozilla.com/push/v1/gAAAAABXAuZmKfEEyPbYfLXqtPW…"

This should result in an Application getting a “push” message, with no data associated with it.

To see how to store the Push Subscription data and send a Push Message using a simple server, see our Using the Push API article.

Data Bearing Subscription Updates


Data Bearing updates are a lot more interesting, but do require significantly more work. This is because we treat our own servers as potentially hostile and require “end-to-end” encryption of the data. The message you send across the Mozilla Push Server cannot be read. To ensure privacy, however, your application will not receive data that cannot be decoded by the User Agent.

There are libraries available for several languages at https://github.com/web-push-libs, and we’re happy to accept or link to more.

The encryption method used by Push is Elliptic Curve Diffie Hellman (ECDH) encryption, which uses a key derived from two pairs of EC keys. If you’re not familiar with encryption, or the brain twisting math that can be involved in this sort of thing, it may be best to wait for an encryption library to become available. Encryption is often complicated to understand, but it can be interesting to see how things work.

Note: If you’re familiar with Python, you may want to just read the code for the http_ece package. If you’d rather read the original specification, that [is available](https://tools.ietf.org/html/draft-ietf-webpush-vapid-00). While the code is not commented, it’s reasonably simple to follow.

Data encryption summary

  • Octet — An 8 bit byte of data (between \x00 and \xFF)
  • Subscription data — The subscription data to encode and deliver to the Application.
  • Endpoint — the Push service endpoint URL, received as part of the Subscription data.
  • Receiver key — The p256dh key received as part of the Subscription data.
  • Auth key — The auth key received as part of the Subscription data.
  • Payload — The data to encrypt, which can be any streamable content between 2 and 4096 octets.
  • Salt — 16 octet array of random octets, unique per subscription update.
  • Sender key — A new ECDH key pair, unique per subscription update.

Web Push limits the size of the data you can send to between 2 and 4096 octets. You can send larger data as multiple segments, however that can be very complicated. It’s better to keep segments smaller. Data, whatever the original content may be, is also turned into octets for processing.

Each subscription update requires two unique items — a salt and a sender key. The salt is a 16 octet array of random octets. The sender key is a ECDH key pair generated for this subscription update. It’s important that neither the salt nor sender key be reused for future encrypted data payloads.

The receiver key is the public key from the client’s ECDH pair. It is base64, URL safe encoded and will need to be converted back into an octet array before it can be used. The auth key is a shared “nonce”, or bit of random data like the salt.

Emoji Based Diagram

Subscription Data Per Update Info Update
🎯 Endpoint 🔑 Private Server Key 📄Payload
🔒 Receiver key (‘p256dh’) 🗝 Public Server Key
💩 Auth key (‘auth’) 📎 Salt
🔐 Private Sender Key
✒️ Public Server Key

Encryption uses a fabricated key and nonce. We’ll discuss how the actual encryption is done later, but for now, let’s just create these items.
Creating the Encryption Key and Nonce
The encryption uses HKDF (Hashed Key Derivation Function) using a SHA256 hash very heavily.
Creating the secret
The first HKDF function you’ll need will generate the common secret (🙊), which is a 32 octet value derived from a salt of the auth (💩) and run over the string “Content-Encoding: auth\x00”.
So, in emoji =>
🔐 = 🔑(🔒);
🙊 = HKDF(💩, “Content-Encoding: auth\x00”).🏭(🔐)

An example function in Python could look like so:


# receiver_key must have "=" padding added back before it can be decoded.
# How that's done is an exercise for the reader.
receiver_key = subscription['keys']['p256dh']
server_key = pyelliptic.ECC(curve="prime256v1")
sender_key = server_key.get_ecdh_key(base64.urlsafe_b64decode(receiver_key))

#🙊
secret = HKDF(
hashes.SHA256(),
length = 32,
salt = auth,
info = "Content-Encoding: auth\0").derive(sender_key)

The encryption key and encryption nonce
The next items you’ll need to create are the encryption key and encryption nonce.

An important component of these is the context, which is:

  • A string comprised of ‘P-256’
  • Followed by a NULL (“\x00”)
  • Followed by a network ordered, two octet integer of the length of the decoded receiver key
  • Followed by the decoded receiver key
  • Followed by a networked ordered, two octet integer of the length of the public half of the sender key
  • Followed by the public half of the sender key.

As an example, if we have an example, decoded, completely invalid public receiver key of ‘RECEIVER’ and a sample sender key public key example value of ‘sender’, then the context would look like:


⚓ = "P-256\x00\x00\x08RECEIVER\x00\x06sender"

The “\x00\x08” is the length of the bogus “RECEIVER” key, likewise the “\x00\x06” is the length of the stand-in “sender” key. For real, 32 octet keys, these values will most likely be “\x00\x20” (32), but it’s always a good idea to measure the actual key rather than use a static value.

The context string is used as the base for two more HKDF derived values, one for the encryption key, and one for the encryption nonce. In python, these functions could look like so:

In emoji:
🔓 = HKDF(💩 , “Content-Encoding: aesgcm\x00” + ⚓).🏭(🙊)
🎲 = HKDF(💩 , “Content-Encoding: nonce\x00” + ⚓).🏭(🙊)


#🔓
encryption_key = HKDF(
hashes.SHA256(),
length=16,
salt=salt,
info="Content-Encoding: aesgcm\x00" + context).derive(secret)

# 🎲
encryption_nonce = HDKF(
hashes.SHA256(),
length=12,
salt=salt,
info="Content-Encoding: nonce\x00" + context).derive(secret)

Note that the encryption_key is 16 octets and the encryption_nonce is 12 octets. Also note the null (\x00) character between the “Content-Encoding” string and the context.

At this point, you can start working your way through encrypting the data 📄, using your secret 🙊, encryption_key 🔓, and encryption_nonce 🎲.

Encrypting the Data

The function that does the encryption (encryptor) uses the encryption_key 🔓 to initialize the Advanced Encryption Standard (AES) function, and derives the Galois/Counter Mode (G/CM) Initialization Vector (IV) off of the encryption_nonce 🎲, plus the data chunk counter. (If you didn’t follow that, don’t worry. There’s a code snippet below that shows how to do it in Python.) For simplicity, we’ll presume your data is less than 4096 octets (4K bytes) and can fit within one chunk.
The IV takes the encryption_nonce and XOR’s the chunk counter against the final 8 octets.


def generate_iv(nonce, counter):
(mask,) = struct.unpack("!Q", nonce[4:]) # get the final 8 octets of the nonce
iv = nonce[:4] + struct.pack("!Q", counter ^ mask) # the first 4 octets of nonce,
# plus the XOR'd counter
return iv

The encryptor prefixes a “\x00\x00” to the data chunk, processes it completely, and then concatenates its encryption tag to the end of the completed chunk. The encryptor tag is a static string specific to the encryptor. See your language’s documentation for AES encryption for further information.


def encrypt_chunk(chunk, counter, encryption_nonce, encryption_key):
encryptor = Cipher(algorithms.AES(encryption_key),
modes.GCM(generate_iv(encryption_nonce,
counter))
return encryptor.update(b"\x00\x00" + chunk) +
encryptor.finalize() +
encryptor.tag

def encrypt(payload, encryption_nonce, encryption_key):
result = b""
counter = 0
for i in list(range(0, len(payload)+2, 4096):
result += encrypt_chunk(
payload[i:i+4096],
counter,
encryption_key,
encryption_nonce)


Sending the Data

Encrypted payloads need several headers in order to be accepted.

The Crypto-Key header is a composite field, meaning that different things can store data here. There are some rules about how things should be stored, but we can simplify and just separate each item with a semicolon “;”. In our case, we’re going to store three things, a “keyid”, “p256ecdsa” and “dh”.

“keyid” is the string “p256dh”. Normally, “keyid” is used to link keys in the Crypto-Key header with the Encryption header. It’s not strictly required, but some push servers may expect it and reject subscription updates that do not include it. The value of “keyid” isn’t important, but it must match between the headers. Again, there are complex rules about these that we’re safely ignoring, so if you want or need to do something complex, you may have to dig into the Encrypted Content Encoding specification a bit.

“p256ecdsa” is the public key used to sign the VAPID header (See [Forming your Headers]). If you don’t want to include the optional VAPID header, you can skip this.

The “dh” value is the public half of the sender key we used to encrypt the data. It’s the same value contained in the context string, so we’ll use the same fake, stand-in value of “sender”, which has been encoded as a base64, URL safe value. For our example, the base64 encoded version of the string ‘sender’ is ‘c2VuZGVy’

Crypto-Key: p256ecdsa=BA5v…;dh=c2VuZGVy;keyid=p256dh

The Encryption Header contains the salt value we used for encryption, which is a random 16 byte array converted into a base64, URL safe value.

Encryption: keyid=p256dh;salt=cm5kIDE2IGJ5dGUgc2FsdA

The TTL Header is the number of seconds the notification should stay in storage if the remote user agent isn’t actively connected. “0” (Zed/Zero) means that the notification is discarded immediately if the remote user agent is not connected; this is the default. This header must be specified, even if the value is “0”.

TTL: 0

Finally, the Content-Encoding Header specifies that this content is encoded to the aesgcm standard.

Content-Encoding: aesgcm

The encrypted data is set as the Body of the POST request to the endpoint contained in the subscription info. If you have requested that this be a restricted subscription and passed your VAPID public key as part of the request, you must include your VAPID information in the POST.

As an example, in python:
headers = {
'crypto-key': 'p256ecdsa=BA5v…;dh=c2VuZGVy;keyid=p256dh',
'content-encoding': 'aesgcm',
'encryption': 'keyid=p256dh;salt=cm5kIDE2IGJ5dGUgc2FsdA',
'ttl': 0,
}
requests.post(subscription_info['subscription']['endpoint'],
data=encrypted_data,
headers=headers)

A successful POST will return a response of 201, however, if the User Agent cannot decrypt the message, your application will not get a “push” message. This is because the Push Server cannot decrypt the message so it has no idea if it is properly encoded. You can see if this is the case by:

  • Going to about:config in Firefox
  • Setting the dom.push.loglevel pref to “debug”
  • Opening the Browser Console (located under Tools > Web Developer > Browser Console menu.

When your message fails to decrypt, you’ll see a message similar to the following
console

You can use values displayed in the Web Push Data Encryption Page to audit the values you’re generating to see if they’re similar. You can also send messages to that test page and see if you get a proper notification pop-up, since all the key values are displayed for your use.

You can find out what errors and error responses we return, and their meanings by consulting our server documentation.


Subscription Updates


Nothing (other than entropy) lasts forever. There may come a point where, for various reasons, you will need to update your user’s subscription endpoint. There are any number of reasons for this, but your code should be prepared to handle them.

Your application’s service worker will get a onpushsubscriptionchanged event. At this point, the previous endpoint for your user is now invalid and a new endpoint will need to be requested. Basically, you will need to re-invoke the method for requesting a subscription endpoint. The user should not be alerted of this, and a new endpoint will be returned to your app.

Again, how your app identifies the customer, joins the new endpoint to the customer ID, and securely transmits this change request to your server is left as an exercise for the reader. It’s worth noting that the Push server may return an error of 410 with an errno of 103 when the push subscription expires or is otherwise made invalid. (If a push subscription has expired several months ago, the server may return a different errno value.

Conclusion

Push Data Encryption can be very challenging, but worthwhile. Harder encryption means that it is more difficult for someone to impersonate you, or for your data to be read by unintended parties. Eventually, we hope that much of this pain will be buried in libraries that allow you to simply call a function, and as this specification is more widely adopted, it’s fair to expect multiple libraries to become available for every language.

See also

  1. WebPush Libraries: A set of libraries to help encrypt and send push messages.
  2. VAPID lib for python or javascript can help you understand how to encode VAPID header data.

:: Failure Favor

Dear Web Devs,

Hi, can i ask you a favor?

Web pages are complicated beasts. There’s lots of moving parts as things get pulled in from all over. Which means, there’s more than a fleeting chance that for random reasons and efficiency bits of the page may “go missing”. A broken image may be one thing, but a missing CSS file is quite another.

Can i ask y’all a favor? Turn off “style.css” for this page.

Can we make that a thing? Can we use CSS for good and help the rather high number of folks who don’t just know to force refresh a page (or even what that means) from thinking their computer is broken or been hacked? Basically, consider your page’s failure modes, and help out folks that may not understand what to do. Honestly, it’s not hard.

Let’s treat our users like we’d like to be treated when we’re not fully versed on automotive repair, furnace maintenance, municipal transit regulations, or all the stuff that they’re far smarter about than us. That’d be swell.

:: eBiking

Let me start by saying i’m damn fortunate. i make enough and work somewhere that makes it possible to even consider buying a $1000-$3000 bicycle. i’m also fortunate enough that i live about 10 miles from work and am able to rationally consider spending 40-60 minutes getting to the office. i’m totally, and painfully aware that this is a deeply first world problem, and in fact a problem for a very small percentage of the general population.

So, yeah, much like watching Top Gear drone on about the values and faults of a car greater than the yearly income of most folks, here goes.

i’m now officially on my second eBike, and want to pass along some of the lessons i’ve learned.

The First Bike

The first bike was a Riide electric, which i got off their kickstarter for $1500. This was an experimental bike for a number of reasons, and while i was more than a little leary about giving a random bunch of strangers a sizeable chunk of money with only the promise that a bike would show up, i was able to consider that an acceptable risk.

The bike was very much version 1, but a pretty good v1. The bike has some fairly top notch gear on it, including disc brakes, and rugged tires. E-Bikes tend to be either throttle based (kind of like a motorcycle, but with the option to pedal), pedal assist (where there’s a motor that only runs when you’re pedaling), or a hybrid of the two. Riide is throttle based, with a reported range of about 25 miles and a top speed of about 20 mph. It’s a single gear bike, that gear being equivalent of being about gear “8” of a 10 speed. The battery is replaceable, but not considered user serviceable since Riide wanted to make the bike as simple as possible.

As i’ve learned, that’s also kind of a problem. There’s no battery level indicator, so you have no idea how much charge is left on your bike. The throttle tends to be “all” or “nothing” which is fine for starting off, but odd when you’re at full speed. i usually throttled back, but i have no idea what that does to performance or battery life. Top speed really depends on terrain and how much you had for lunch (the spandex guys will still fly by, but you’ll breeze by most everyone else).

i also have no idea how far you have to really ride before you can get any appreciable charge back onto the battery. i had it die on me once about 2.5 miles into a 10 mile ride, and it never fired up again until i put a full charge on it, so not quite sure about the recharging capacity. The bike is surprisingly easy to work on, which turned out to be a good thing. i had to swap the proprietary battery out which required a lot of creative thought and use of one of those combination bottle/paint can lid openers you get at the paint shop.

The Riide folk are EXCEPTIONALLY GOOD at customer service. They were more than willing to cover costs of proper repairs and adjustments to the bike should i have wanted them. This is why i’ve given my Riide to a friend because i think he’ll enjoy it. Might as well spread the addiction around.

Still, i needed to upgrade to something i can absolutely rely on, and matched my actual use more.

The New Bike

The Second bike was a bit more than the Riide, and it shows in a lot of ways. It’s a 2016 iZip Dash e3, and it has a lot of things the Riide doesn’t, like a swappable battery, built in speedometer/odometer with range and power gauge, front fork shocks, kick stand, fenders and back rack. It also costs roughly twice what the Riide cost.

The Dash can do a top speed of 28mph or it has a range of 36+ miles. High speed burns battery pretty fast, so it’s ok if you’re going for a 8 mile total trip. The battery takes 4-6 hours to recharge which is 2-3 times as long as the Riide took. There’s no throttle, only pedal assist, but it’s geared so riding without power doesn’t require standing on the pedals to get rolling. Honestly, unlike the Riide, you can use the Dash without power and not give yourself a heart attack going uphills or doing standing starts. i’ve actually ridden the Dash without power and lived to tell the tale.

Granted, the bike does look a bit less “cool” than the ride. Honestly, it has an odd “PeeWee Herman” vibe to it, and i’m not really sure why. The built in lights are mostly for show, so i’ve had to strap on some LED lights to keep from getting murdered. The grips don’t really allow me to mount a mirror easily, so i’ve had to fit one that uses a velcro strap. The owners manual spends a LOT of time carefully pointing out the various ways you can die and be horribly mutilated while riding a bike in general rather than go into detail about any differences riding this one. Feel free to laugh at me. i’ll be giggling too as the shocks smooth out the crap roads around here and i blow by folks sitting in traffic.

What i Learned

The ultimate thing i learned? Go try a few bikes. The place i bought from had dozens of models available and folks willing to answer questions and offer good suggestions. There are all sorts of options out there, from the sane to the insane, and like any vehicle, you should get one that meets what you’re actual needs are rather than what you think you need.

And yeah, they’re not cheap. You’re not going to be able to pick one up at a WalMart black friday sale with a 50% off coupon. That’s actually kind of a good thing. Eventually, they may get down to prices comparable to a used Chevy.

Still, they’re a freaking blast to ride.

:: Black Electrical Tape Based Security

So, both Apple and Google have decided to be quiet about letting you know a page is not as secure as it should be. Instead of showing you a warning, they’ve opted to just show the page as insecure and not raise any concerns. It’s a fair point. Most folks STILL don’t know to look for the “lock icon” showing that a site is running a secure connection. Heck, this block runs in the clear currently. Why bother the user’s pretty little head with scary symbols?

Well, probably for the same reason that if your Factory Authorized Vehicle Service told you “Oh that? Yeah, just put some black tape over that “Check Engine” light. It’ll be fine!” you’d probably consider going somewhere else to get your brakes checked.

The problem is that the page has said “i’m going to be secure. Everything we talk about is going to be encrypted. It’s safe here, so you can talk about anything.”

Only it’s not. It’s invited friends, some of which can’t keep their mouths shut.

So the browser has instead said “Yeah, no, this isn’t a safe spot, so no encryption. No lock for you, but it’s a normal page.”

But it’s not. The site is going to do things based on the idea that the page is safe, like ask your for passwords or personal info. Sure, you may realize that it’s a bad idea, but the folks far less familiar with security (that would be the VAST MAJORITY of people online) will look at the happy plain-gray icon and feel it’s A-OK to type in their credentials, because there’s nothing to scare them off from doing it.

Thing is, i get some of the complaints. Sure, it’s annoying that you’ve got some jpgs on a CDN, and sure, it’s hard to make a page that doesn’t specify scheme, but yeah, no. You might feel a tad differently if there’s a rogue bit of javascript reporting back keystrokes.

Yes, having that odd looking icon is troubling and confusing to users. That’s kind of the point. It’s the proverbial “Check Engine” light of the internet and yes, there’s going to be some users that happily ignore it and horrible things will happen to them. Those folk are doomed to their gleeful ignorance regardless.

i’d rather not doom the small percentage of folks that have Darwin-like evolved to look for those warning signs.

Oh, and by the way? Go secure your site. It’s free now.

And then this happens:

Blogs of note
personal Christopher Conlin USMC memoirs of hydrogen guy rhapsodic.org Henriette's Herbal Blog
geek ultramookie

Powered by WordPress
Hosted on Dreamhost.