Danger: Bypass SSL Interception Proxy Server Certificate Validation in .NET or Python HTTP Requests

In enterprise environment, it is common to setup SSL interception via a proxy server. SSL interception is a technique used by a proxy server to decrypt and inspect the encrypted traffic between a client (like a web browser) and a server (like a website), before re-encrypting it and sending it on its way. This is often referred to as a “man-in-the-middle” attack.

The problem

In most setup, the proxy server will return an internal certificate to the client that is usually trusted already by the client user/machine. In some cases, the client machines might not be setup correctly, i.e. not added into trust store of the machines. This causes SSL connection fails in many applications or code.

For example, in Python, it may generate the following error (routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed):

import requests

url = 'https://example.com'
response = requests.get(url)

# This will raise an error if the SSL certificate is not trusted:
# requests.exceptions.SSLError: [Errno 1] _ssl.c:503: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

In .NET, the following code can raise similar exception if the intercepted public cert from proxy server is not trusted.

using System;
using System.Net.Http;
using System.Threading.Tasks;

public class Program
{
    public static async Task Main(string[] args)
    {
        var client = new HttpClient();

        try
        {
            var response = await client.GetAsync("https://example.com");
            response.EnsureSuccessStatusCode();
            var content = await response.Content.ReadAsStringAsync();
            Console.WriteLine(content);
        }
        catch (HttpRequestException ex)
        {
            Console.WriteLine($"Request exception: {ex.Message}");
        }
    }
}

Snowflake error about SSL

These problems can occur in other packages that are built on top http client in Python or .NET or any other programming languages. For example, Snowflake python connector can raise issues like the following if the cert is not trusted:

snowflake.connector.errors.OperationalError: 25003: Failed to get the response. Hanging? method: post, url: https://***

In Snowflake .NET driver, similar exception can be raised:

Unhandled exception. Snowflake.Data.Client.SnowflakeDbException : Error: Snowflake Internal Error: Unable to connect. One or more errors occurred.

When looking into details, it usually involves errors like RemoteCerticateChainErrors.

Fix the problems

There are different ways to address these issues.

The right approach

The right approach is to get in touch your company IT network team and system administrators to use a valid certificate that is trusted by root so that as a system user you don't need to use workarounds as bypass SSL certificate verification can be very dangerous.

Tactical solutions

Sometimes, it might not be easy to apply a system wide fix or it takes time to implement. Hence you can use some workarounds to fix them in your application temporarily.

Consider the risk first

Before you take these tactical solutions, evaluate the risks and make sure you absolute trust the certificate returned from the proxy. Please note that it’s generally not recommended to disable SSL verification due to the security risks involved.

If you are comfortable with the cert, download and save it to your system. The following code snippets assume the cert is saved in path /path/to/public-key.pem.

Use environment variable REQUESTS_CA_BUNDLE

For Python, you can use environment variable REQUESTS_CA_BUNDLE to add this custom cert into Python's trusted certificate list.

import os
import requests

# Set the REQUESTS_CA_BUNDLE environment variable
os.environ['REQUESTS_CA_BUNDLE'] = '/path/to/your/certificate.pem'

# Now, requests will use the specified CA bundle for SSL verification
response = requests.get('https://example.com')

For all HTTPS requests, Python will trust this customized certificate.

Alternatively you can set it up when you run the scripts:

export REQUESTS_CA_BUNDLE=/path/to/your/certificate.pem
python script.py

For Python requests package

For requests initiated using Python requests package, you can directly specify it too instead of using the above environment variable.

import requests

url = 'https://example.com'
response = requests.get(url, verify='/path/to/public-key.pem')

For .NET applications

For .NET applications, there are no equivalent environment variables you can use. You have to install the cert into root trust store. If you cannot do that and if the cert itself cannot be verified or is invalid, you will have to bypass it using the following approach. 

You can bypass SSL certificate validation in .NET by using the HttpClientHandler.ServerCertificateCustomValidationCallback property. This property allows you to provide a custom validation callback that can ignore certificate errors. Here’s an example:

using System;
using System.Net.Http;
using System.Threading.Tasks;

public class Program
{
    public static async Task Main(string[] args)
    {
        var handler = new HttpClientHandler();
        handler.ServerCertificateCustomValidationCallback = 
            (message, cert, chain, errors) => { return true; };

        var client = new HttpClient(handler);

        try
        {
            var response = await client.GetAsync("https://example.com");
            response.EnsureSuccessStatusCode();
            var content = await response.Content.ReadAsStringAsync();
            Console.WriteLine(content);
        }
        catch (HttpRequestException ex)
        {
            Console.WriteLine($"Request exception: {ex.Message}");
        }
    }
}

In the code above, the ServerCertificateCustomValidationCallback is set to a lambda function that always returns true, effectively ignoring any SSL certificate errors.

Please note that bypassing SSL certificate validation can pose a security risk, as it allows connections to servers with potentially untrusted or compromised certificates. This approach should only be used in controlled environments or for testing purposes.

Other thoughts

For certificates returned by SSL interception proxy server, they are usually have a short valid period. Hence, you will need to apply these solutions regularly until the proxy server can issue public certs that are automatically trusted by your applications.