Best guidance for allowing users to connect via HTTP in case of a certificate error

Question

I've coded my app to use https, but if a https transaction fails for any reason, I assume it's because the server isn't configured for https, and thereafter start all transactions with http. Seems like that's a vulnerability. Likewise, a script kiddie using a proxy to intercept the traffic on his client hardware would be able to make all https transactions fail.

I'm told that if someone tries to MITM your app's HTTPS request then the request should fail (invalid certificate) and your app should fail with an error, not fallback to HTTP. In a world where SSL is reliably available, sure, but maintaining valid SSL certs is a task in itself. For example, letsencrypt recently revoked some of their certificates and forced renewal of same because of some security problem. Aside from revocations, certs are short term and have to be renewed, and the renewal process involves a lot of stitchware, and can fail. If SSL goes down, I don't want my site to go dark.

What is the best guidance for either:

1. More reliably maintaining certificates (such that if they do fail, the resulting downtime falls within the "five nines" SLA unavailability window) without it being such a manual headache, or

2. Allowing the site to continue to work if SSL has failed? Is it easy to allow most activity to proceed using http, but allow known-critical transactions to require https.

Note that no browsers are involved in the scenarios that concern me.

Answer 1

I've coded my app to use https, but if a https transaction fails for any reason, I assume it's because the server isn't configured for https, and thereafter start all transactions with http. Seems like that's a vulnerability. Likewise, a script kiddie using a proxy to intercept the traffic on his client hardware would be able to make all https transactions fail.

Well, do you control the server or not ? It either is configured for https, or it is not. SSL could fail, but it's more likely that you will encounter other issues like the httpd service crashing for some reason, or the server being overloaded. The result is the same: interruption of service.

I'm told that if someone tries to MITM your app's HTTPS request then the request should fail (invalid certificate) and your app should fail with an error, not fallback to HTTP. In a world where SSL is reliably available, sure, but maintaining valid SSL certs is a task in itself.

You should automate the deployment and renewal of certificates. Automate as much as you can, always. If the task is boring, that's one more reason to automate it.

But also make sure that your script will send an alert if the process goes wrong. Since you mentioned Let's encrypt they do provide some scripts if I'm not wrong.

If you have only one server and want to do it manually, at a minimum add the renewal date to your agenda. Don't wait until the last minute to renew your certificate.

If you chose to use https then you should stick to it and not devise workarounds to defeat your own security measures. If service availability is important you can improve redundancy by adding more servers/endpoints to your network architecture. Then your app can try another server if the chosen one is unavailable.

Downgrading the transport layer is not a good move and there are many more problems that could occur, that are not related to SSL. Your effort should be focused on making the service more resilient.

Suggestion: log webserver errors such as SSL negotiation failures (very likely that your webserver software already does that), then make sure critical errors are reported to you asap. Set up a log collector or something. Logs are useless if nobody is paying attention.

Allowing the site to continue to work if SSL has failed? Is it easy to allow most activity to proceed using http, but allow known-critical transactions to require https.

You could, but you have to segregate 'sensitive' traffic from casual traffic. That probably means adding some application logic. I don't see much value in it for the reasons described above.

Answer 2

I'm providing three different answers to this question with different approaches - click here and here for the other answers. I'm doing this to allow all answers to be voted on, commented on, and accepted independently.

maintaining valid SSL certs is a task in itself

This actually sounds like a really good case for hiring somebody to do it for you. Much effort has been put into making SSL as simple to use as possible, so that everyone gets to using it already, but I get it. For some people, some tasks are just impenetrable. They get all ready to do it, having just downed a fresh coffee and sat down at their desk, but they can hardly get a keystroke in before they're right back up getting another cup of coffee. (I should know - I'm this way around online dating.)

Fortunately, not everyone is like that with any given task. Some people can set up reliable SSL certificate flows in their sleep. Many others find the task difficult, but love getting into the weeds of that sort of thing anyway (myself included). If you can find someone in your friend group or professional network who can do a bit of SSL configuration for you (either as a full-fledged employee working on other stuff too, or just as a few hour session), then you can find yourself with reliable HTTPS without you having had to ever touch the thing, especially if what they do for you is fully automated.

EDIT: To clarify, this would likely be realized in practice by just having another developer on your team that can help with other stuff too. Having another dev would also make you more resilient against site outages, since it's more likely one of you would be available to get up and manually address them.

Answer 3

I'm providing three different answers to this question with two different approaches - click here and here for the other answers. I'm doing this to allow all answers to be voted on, commented on, and accepted independently.

Let's assume that certificate errors happen too often for everything you've tried (perhaps because you've tried setting up certificates, going all over the place with them, and still can't get them to work reliably enough). Let's also assume that the data you are sending is not critical enough to absolutely require HTTPS encryption (e.g. you don't do anything with real money) and you are OK with sending it unencrypted. In this case, it probably is safe for you to proceed with HTTP downgrade under one condition: imformed user consent. The user must explicitly affirm that they data they are sending is not critical enough to absolutely require HTTPS encryption and that they are OK with sending their data (or receiving data that will become theirs) unencrypted.

What that essentially means for you is that your HTTP downgrade should not be automatic, behind the user's back. Instead, if you encounter a certificate error with the website, you should follow the example of modern browsers encountering such an error and inform the user, perhaps with some kind of dialog box. Let them know that the SSL certificate has failed and that, while they can technically still continue, it will be at the cost of encryption - they then can make the choice between what option they want, and proceed under their own consent, with both of you now having the knowledge that your connection is vulnerable.

Of course, when asking users to make a decision like this, there is an ethics issue of whether or not it is right to ask users to open up the data they send, and that will depend on the kind of data you expect to deal with. In general, the more free-form the data the user can reasonably send, the less ethical this becomes, since free-form data can very easily become secret data. This gets especially hairy in international markets, where the sensitive data the user is sending might include the fact that they're even using the app at all, depending on the user's jurasdiction.

Even if you can determine that the choice might be an ethical one to allow, it should not be a mindless one - most users are trained to just click "yes" if you let them, even if they're binding themselves to sell their firstborn baby. Most browsers, when offering the cert bypass choice, tend not to mention it at all initially, and instead hide the option behind some ominous sounding "Advanced Options" button that only technical people will willingly click.

And, after all that, once you have allowed the insecure HTTP connection, you may want to treat it differently now that it has been compromised, perhaps outright refusing certain transactions and limiting the effects of others. For instance, one could reasonably limit HTTP transactions to be read-only and not allow them to affect any state.

Answer 4

Just don't. Trying to get your https connection to fallback to http is a basic attack, and after that all your traffic is accessible to that same mitm.

There's no difference between what you are trying to do and going through http systematically.

Just because https is - in your opinion - occasionally a bit complicated to maintain is not a good enough reason to throw it out the window.

Answer 5

I'm providing three different answers to this question with different approaches - click here and here for the other answers. I'm doing this to allow all answers to be voted on, commented on, and accepted independently.

Is it easy to allow most activity to proceed using http, but allow known-critical transactions to require https?

If you are really committed to this idea, there actually is a way that you can limit this set of "known-critical transactions" down to merely the act of logging in, and get the security benefits of encryption and signing for everything else even if your SSL certificate fails.

The primary insight here is that the signing certificate and the Diffie-Hellman key exchange is only used to establish the secure communication channel - past that point, the channel is maintained via a symmetric AES key (exchanged at the start) and the certificate and public/private keys are no longer used until the connection ends and a new one is made. Therefore, if you provide an additional AES key to a user upon logging in (via HTTPS), and ask that that user use that key to encrypt their HTTP request bodies (or, more likely, their raw TCP/UDP connections), then all existing users will continue to be able to use your app without so much as blinking an eye - they just keep using the AES key you gave them when HTTPS was working.

To lay out what you would do step by step:

1. When the user logs in (over HTTPS), send them a session token as normal, and also send them a true-random generated AES key (which will be securely sent because you're using HTTPS). Store that AES key along with the session token in your database and any other information stored next to it (such as user ID, expiry time, etc.).
2. For each subsequent request after that, read the HTTP/other headers for the session token being used to contact them, and then assume that everything that follows (request body, rest of the UDP packet, rest of the TCP stream) is encrypted with the matching AES key, and decrypt it. If the resulting message is gibberish (perhaps not including the correct 16-byte magic number at the beginning), throw it out - the key is wrong.
3. When the user finally explictily signs out (as opposed to just closing the app - you could technically keep the AES key between app runs), or the session expires, the AES key is deleted from the server along with it. In order to log in again, the client will request a new one (again over HTTPS), and encrypted communication can resume from there.

Now, this system does have a few disadvantages, that taken together may well mean that figuring out reliable SSL is a better/easier option:

• At least for HTTP endpoints, this isn't exactly industry standard - SSL is the standard, and takes care of this encryption for you. Therefore, in order to do this correctly, you must be vigilant and only accept encrypted payloads for your HTTP requests, either by wrapping your request handler library (e.g. Express) or by adding boilerplate to each request.
• You of course also have to make sure that the encryption is done correctly, which isn't always obvious. Many standard libraries come with cryptography primitives, but you should make sure you're handling the keys correctly and not exposing side-channel attacks or anything like that.
• In HTTP requests, your headers, URL, query string, and method remain unencrypted, which means you can't send any important data in them. This isn't as much of a problem for you since you're not using the browser and thus have far more control over all that, but it would mean that browser use is pretty much out of the question.
• This all won't help you if the server gets pwned - indeed, all of these AES keys lying around can give a false sense of security until you take back the server and wipe all the sessions off.

It does, however, provide an advantage you may find appealing: the maintainence that you do have to do is entirely within your control, and you don't rely as heavily on the CA to get everything working right. Indeed, taken to its extreme, if you have some other way of getting login authorization to users (such as secure invites or mailed/store-bought discs), you don't need a CA or HTTPS at all. And even if you are more assuredly using HTTPS, this AES scheme also allows you to more easily get encryption on raw TCP streams or UDP datagrams, which may well be a better fit for your use case.

Answer 6

As mentioned in a comment, if you fully control both the server and the client, using HTTPS doesn't have to mean using the same public key infrastructure as the open web. In short, you can issue and trust your own certificates without any third-party involvement.

In HTTPS, a certificate isn't directly involved in the encryption of the connection; what it does is attach an identity to the key pair the server is using. The infrastructure around Certificate Authorities is built to solve the key distribution problem: the browser can't obtain every possible public key in advance, so servers send proof that one of a limited set of third-parties has verified their relationship to a domain name.

It's therefore very simple in a single-use client to ship a single public key, corresponding to the private key on your server. Rather than a set of root certificates for CAs as its "trust anchor", the client would ship with a single certificate containing this public key.

On the face of it, this solves a lot of maintenance problems: the certificate needn't ever expire, or be checked for revocation, and the connection remains secure.

But there's a reason expiry dates, revocation lists, etc, exist - if anyone obtains that one private key, you can never stop them connecting to your client. And the private key has to exist on every server that can terminate the HTTPS connections from your app.

So in practice, even without a CA involved, you would want to:

• Use a separate "root certificate" corresponding to a private key that is never deployed anywhere. The client trusts this root certificate, and the servers have separate certificates signed by it.
• Give the individual server certificates an expiry date (as short as possible), to limit the damage if someone gets hold of a private key.
• Possibly have the client perform checks against a revocation list (which itself needs to be trusted).

In other words, you'll have to do all the same things you would with a"normal" certificates, plus some of the things a CA would do for you.

In summary, the complexities you're trying to avoid are there for a reason, so you can't avoid doing them at all and have the security that they provide. What you can do is avoid doing them manually - seek out tools to automate everything, and treat them as just vital as any other component of your infrastructure.