Linux SSL/TLS

From Ever changing code
Jump to navigation Jump to search

The main SSL tools on Linux to manage certificates are


Terminology

DER|PEM
These are encoding formats. The DER option uses an ASN1 DER encoded form compatible with the PKCS#10. The PEM form consists of the DER format base64 encoded with additional header and footer lines.
.p12|.pfx

Search installed certificates

Example searches 

find -H / -name "*.crt" | while read line ; do echo $line ; openssl x509 -in certificate.crt -text | egrep -i 'Algorithm|Issuer' ; done
find -H / -name "*.crt" | while read line ; do echo $line ; openssl x509 -in $line -text | egrep -i 'Algorithm|Issuer|Not After' ; done
find -H / -name cacerts | while read line ; do echo $line ; /usr/bin/keytool -list -v -keystore $line -storepass changeit -noprompt | egrep -i 'Alias|Owner|algorithm|SHA1' ; done;
find -H / -path /usr/share/doc -prune -o -name "*.pem" | while read line ; do echo $line ; openssl x509 -in $line -text | egrep -i 'Algorithm|Issuer|After' ; done
find -H / -path "/usr/share/doc /proc" -prune -o -name "*.pem" | while read line ; do echo $line ; openssl x509 -in $line -text | egrep -i 'Algorithm|Issuer|After' ; done
 
# -H do not follow symbolic links
# -path "pattern" file name matches shell pattern
# -prune -o ignores a whole directory tree if the pattern matches a directory path

Apache - SSL configuration in httpd.conf

$ grep SSLCertificate /etc/httpd/conf.d
SSLCertificateChainFile "/etc/httpd/conf.d/ssl.pem/example.pem" # certificate chain
SSLCertificateFile      "/etc/httpd/conf.d/ssl.crt/exmple.crt"  # domain certificate (private key signed by 3rd party CA)
SSLCertificateKeyFile   "/etc/httpd/conf.d/ssl.key/exmple.key"  # private key file

OpenSSL

Generate self-signed certificate

# generate the RSA private key
openssl genpkey -outform PEM -algorithm RSA -pkeyopt rsa_keygen_bits:4096 -out cert.key

# Create the CSR
openssl req -new -nodes -key cert.key -config csr.cfg -out cert.csr

# Self-sign your CSR
openssl req -x509 -nodes -in cert.csr -days 365 -key cert.key -config cert.cfg -extensions req_ext -out cert.crt
Generate self-signed certificate
csr.cfg cert.cfg 
cat > csr.cfg << EOF
[ req ]
default_md = sha256
prompt = no
req_extensions = req_ext
distinguished_name = req_distinguished_name
[ req_distinguished_name ]
commonName          = acme.ae
countryName         = AE
stateOrProvinceName = Abu Dhabi
localityName        = JVC
organizationName    = Acme Inc
[ req_ext ]
keyUsage=critical,digitalSignature,keyEncipherment
extendedKeyUsage=serverAuth,clientAuth
basicConstraints=critical,CA:false
subjectAltName = @alt_names
[ alt_names ]
DNS.0 = acme.ae
EOF
 
cat > cert.cfg << EOF
[ req ]
default_md = sha256
prompt = no
req_extensions = req_ext
distinguished_name = req_distinguished_name
[ req_distinguished_name ]
commonName          = acme.ae
countryName         = AE
stateOrProvinceName = Abu Dhabi
localityName        = JVC
organizationName    = Acme Inc
[ req_ext ]
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
keyUsage=critical,digitalSignature,keyEncipherment
extendedKeyUsage=serverAuth,clientAuth
basicConstraints=critical,CA:false
subjectAltName = @alt_names
[ alt_names ]
DNS.0 = acme.ae
EOF

Renew a certificate based on the current certificate

openssl x509 -x509toreq -in current_certificate.crt -out request.csr -signkey current_certificate_private.key

Verify a Certificate was Signed by a CA

Use this command to verify that a certificate (domain.crt) was signed by a specific CA certificate (ca.crt). This can be a chain certificate that contains signing certificate. 

$ openssl verify -verbose -CAFile ca.crt domain.crt 
domain.crt: OK

Verify a certificate from a CLI

Insecure option tells libcurl to not verify the peer. 

curl --insecure -v https://www.example.com/mgm-your-account 2>&1 \
  | awk 'BEGIN { cert=0 } /^\* Server certificate:/ { cert=1 } /^\*/ { if (cert) print }'


If your webserver uses SNI scheme is not IP based, specify the vhost server using -servername option, otherwise can be skipped 

echo \
  | openssl s_client -connect <web_server_IP>:443 -servername www.example.com 2>/dev/null \
  | openssl x509 -inform pem -noout -text
                                                          Hosted_website
                              Apache_server          in_virtual_host_directive
                                      |                           |
server1# openssl s_client -connect 10.0.0.1:443 -servername www.example.com

Verify a Private Key Matches a Certificate

The private key contains a series of numbers. Two of those numbers form the public key, the others are part of your private key. The public key bits are also embedded in your Certificate (we get them from your CSR). To check that the public key in your cert matches the public portion of your private key, you need to view the cert and the key and compare the numbers. To view the Certificate and the key run the commands: 

$ openssl x509 -noout -text -in server.crt
$ openssl rsa -noout -text -in server.key

The modulus and the public exponent portions in the key and the Certificate must match. But since the public exponent is usually 65537 and it's bothering comparing long modulus you can use the following approach: 

$ openssl x509 -noout -modulus -in server.crt | openssl md5
$ openssl rsa -noout -modulus -in server.key | openssl md5

To check to which key or certificate a particular CSR belongs you can compute 

$ openssl req -noout -modulus -in server.csr | openssl md5

OpenSSL test modules

With Server Name Indication (SNI), a web server can have multiple SSL certificates installed on the same IP address. SNI-capable browsers will specify the hostname of the server they’re trying to reach during the initial handshake process. This allows the web server to determine the correct SSL certificate to use for the connection. If you try to connect to SmallSites server with s_client, you’ll find that you receive the default SSL certificate installed on the server and not the one for this site. Therefore we need to use the -servername argument and s_client will do the additional SNI negotiation step.


OpenSSL provides three modules that allow you to test SSL connections: s_client, s_server and s_time. The first two, as the names suggest, are for simulating a client and a server in an SSL connection. The third one is for connection timing tests. s_server use to setup a test backend server to send our test client requests to 

# Certificates
## Create a root certificate and private key to sign the certificate for your services
openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example.com.key -out example.com.crt
## Create a certificate and a private key
openssl req -out sni_value.example.com.csr -newkey rsa:2048 -nodes -keyout sni_value.example.com.key -subj "/CN=sni_value.example.com/O=some organization"
openssl x509 -req -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 0 -in sni_value.example.com.csr -out sni_value.example.com.crt.pem

# Simplify names in the next command
cp sni_value.example.com.crt.pem server.pem
cp sni_value.example.com.key     server.key

# Simulate webserver
openssl s_server -cert server.pem -key server.key -tlsextdebug

# Simulate web client '-servername' specifies server we want to use/connect to
openssl s_client -connect localhost:4433 -servername sni_value.example.com

# Alternative 'curl' command test to set specific SNI server value
curl -k https://sni_value.example.com:4433/ --resolve sni_value.example.com:4433:127.0.0.1


s_client Simulate TLS connection, is particularly useful for checking which protocols and which ciphers the server agrees to use. This information is useful in security and functionality audits. For example, you could use this protocol information to find servers that don’t accept a legitimate protocol or cipher, thus preventing a legitimate client from connecting. You could also locate servers that accept weak protocols or ciphers and could thus allow a malicious attack.


s_client - output interpretation - https://www.feistyduck.com/library/openssl-cookbook/online/ch-testing-with-openssl.html

Preview a certificate from CLI

openssl x509 -in /etc/httpd/conf/ssl.crt/certificate.crt -text -noout
# -text 	Prints out the certificate in text form.
# -noout 	Prevents output of the encoded version of the request.
# -subject 	Outputs the subject name.
# -issuer 	Outputs the issuer name.
# -dates 	Prints out the start and expiry dates of a certificate.
# -fingerprint 	Prints out the digest of the DER encoded version of the whole certificate.

man x509 # see the all available options

Preview CSR request from CLI

openssl req -in certificate.csr -text -noout

Disabling Diffie-Hellman on Apache Servers

On each web server, in the ssl.conf file or, in some cases, the main Apache conf file, add the !DH: identifier to the start of the SSLCipherSuite config option string.

Procedure In Apache's conf directory, locate file: ssl.conf or httpd.conf

Look for the SSLCipherSuite keyword, whose string value must be similar to the following string: 

"ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"

Add !DH: after the ALL: list so that the line looks like the following string: 

"ALL:!DH:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"

Note: The !ADH: string in the above string is now redundant and can be removed. Repeat this edit in every SSL config section, if you are not using one global section. Save the file. Restart the web server for the changes to take effect.

Control Apache server

ssh apache.server.com "apachectl configtest"
ssh apache.server.com "apachectl graceful"
ssh apache.server.com "service httpd status"

Chain certificate structure

Apache ChainSSL is in PEM format containing all certificates from domain one to Root CA 

# cat www_example_com_chain_cert-PEM.crt

-----BEGIN CERTIFICATE-----          #Domain certificate, eg. www_example_com.crt
MIIHkTCCBnmgAwIBAgIUZDLrn0fQy7z62YTRnsd/BsvWI1swDQYJKoZIhvcNAQEL
   < output obmitted >
TlXvxzWXBaAm4YzdMQt0owFUbeItrgn8IcfotEesUU6OMSGOSFi+WTlSC6rJmZM+
eNyDv54nHqqlCpbvZI3umxDx015/
-----END CERTIFICATE-----

 -----BEGIN CERTIFICATE-----          #Intermitennt CA, eg. QuoVadis_EV_SSL_ICA_G1.crt
MIIFeTCCA2GgAwIBAgIUc9pa+iPZP7qELgog9AHJ2G4k/F0wDQYJKoZIhvcNAQEL
   < output obmitted >
OUKBHeWRPmCuv59UkTZXA/LT/tHt2JC6htu2pYislARuJGsnq4Bgrw76c0AS3UF8
wxZChJIYvI+fVDAKwQ==
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----          #Root certificate, eg. QuoVadis_Root_CA_2.crt
MIIFtzCCA5+gAwIBAgICBQkwDQYJKoZIhvcNAQEFBQAwRTELMAkGA1UEBhMCQk0x
   < output obmitted >
ohEUGW6yhhtoPkg3Goi3XZZenMfvJ2II4pEZXNLxId26F0KCl3GBUzGpn/Z9Yr9y
8eOx79+Rj1QqCyXBJhnEUhAFZdWCEOrCMc0u
-----END CERTIFICATE-----

Java Keytool certificate management

SAN Names

Oracle Java Keytool internally uses the class sun.security.x509.DNSName to check the input. DNSName enforces the syntax specified in RFC 1034. That's why its complaining when we use * wildcard in the DNS. Instead you can use openssl to create Microsoft .p12 keystore or java KeyExplorer gui tool.

Quote from its Javadoc comment:

The name MUST be in the "preferred name syntax," as specified by RFC 1034.

<domain> ::= <subdomain> | " "
<subdomain> ::= <label> | <subdomain> "." <label>
<label> ::= <letter> [ [ <ldh-str> ] <let-dig> ]
<ldh-str> ::= <let-dig-hyp> | <let-dig-hyp> <ldh-str>
<let-dig-hyp> ::= <let-dig> | "-"
<let-dig> ::= <letter> | <digit>
<letter> ::= any one of the 52 alphabetic characters A through Z in upper case and a through z in lower case
<digit> ::= any one of the ten digits 0 through 9


So according to this syntax, domain names have to begin with a letter (A-Z, a-z). Newer RFCs (e.g. RFC 2181, RFC 1123) are relaxing these restrictions, so this can be considered a bug in Java. There are already several related bug reports: JDK-8016345, JDK-8007706

Default password

changeit

Types of Java keystores

Terms like keyStore and trustStore are often used interchangeably and the same file can act as keystore as well as trustStore it just matter of pointing javax.net.ssl.keyStore and javax.net.ssl.trustStore properties to that file but there is a slight difference between keystore and trustStore.

  • Trustedstore - a keystore containing system wide available Root certificates
  • Java keystore - a keystore used by an application

Typical locations of a keystores

  • located in server.xml file of Tomcat / Jboss server
  • /etc/pki/tls/certs/ca-bundle.crt
  • Trustedstore - JAVA_HOME/jre/lib/security/cacerts

Keytool useful commands

List a keystore 

keytool -list -v -keystore tomcat.jks

Import Trusted Certificate into keystore 

keytool -import -trustcacerts -file GoDaddy_rootCA.cer -alias rootCA -keystore cacerts.jks

Import Signed Certificate into keystore 

keytool -importcert -file <signed cert file> -alias <alias> -keystore <key store file>

Remove 

keytool -delete -alias godaddy_rootCA -keystore cacerts.jks

Preview a trusted store of different format than JKS. You can also add -storetype parameter for pkms7, 10 or 12 stores listing 

keytool --printcert -file /etc/pki/tls/certs/ca-bundle.crt

Create CSR with SANs names. Note that Java 7+ does not allow for wildcards '*' in the names adhering strictly to one of latest RFC standards. 

keytool -certreq -file test.csr -keystore keystore.jks -alias wso2carbon -storepass PASS -ext SAN=dns:*.dev.company.com,dns:*.test.company.com

Generate self signed keystore with KeyUsage extension

This example can be used for WSO2 Secure Vault implementation where Key Usage: keyEncipherment,dataEncipherment is mandatory. 

keytool -genkey -alias wso2carbon -keyalg RSA -keysize 2048 -noprompt \
-dname "CN=wso2carbon-passwords-encryption,OU=IT Department,O=CompanyPLC,L=City,S=Shire,C=GB" \
-ext KeyUsage=digitalSignature,keyCertSign,keyEncipherment,dataEncipherment \
-keystore wso2carbon-data-encryption.jks -keypass "password1" -storepass "password1"

Renew a cert in .jks based on the current private key

Convert .jks to .pkcs12. This is required as in essence .jks architecture does not allow for the private key to be moved. 

keytool -importkeystore -srckeystore keystore.jks -destkeystore keystore.p12 -destkeypass PASSWORD -srcstorepass PASSWORD -deststoretype PKCS12

Export private key using openssl in PKCS8 format. Openssl by default uses .PEM format. 

openssl pkcs12 -in keystore.p12 -nodes -nocerts -out keystore_pkcs8_format.key

#Example beginning of the key file
Bag Attributes
    friendlyName: mycertificate
    localKeyID: 54 69 6D 65 20 31 35 32 38 30 35 35 31 31 31 34 33 34 
Key Attributes: <No Attributes>
-----BEGIN PRIVATE KEY-----

Export private key using openssl in .PEM format this is the same format as KSE below. 

openssl pkcs12 -in keystore.p12 -nocerts -nodes | openssl rsa > keystore_pem_format.key

#Example beginning of the key file
-----BEGIN RSA PRIVATE KEY-----

Export private key using KSE in .PEM format that can be used by openssl. 

java -jar kse.jar
#Export Private Key > No encryption > PEM format > keystore_pem_format_from_kse.key

#Example beginning of the key file
-----BEGIN RSA PRIVATE KEY-----

Generate CSR using openssl 

openssl req -new -key keystore_pem_format.key -config openssl.cnf -out keystore.csr

Sign the certificate, using internal CA, external CA, free CA like Let's Encrypt 

#<... more steps to follow ...>

Generate .p12 keystore from the key (keystore_pem_format.key) and signed certificate 

openssl pkcs12 -export -in signedcertificate.cer -inkey keystore_pem_format.key -out new.p12 -name mycertificate

Convert .p12 keystore into .jks keystore 

keytool -importkeystore -deststorepass PASSWORD -destkeypass PASSWORD -destkeystore new.jks -srckeystore keystore.p12 -srcstoretype PKCS12 -srcstorepass PASSWORD -alias mycertificate

SSLPoke to test Java certificates

Once you compiled a source code into jar, run as: 

# Usage: SSLPoke <host> <port>

java -Djavax.net.ssl.trustStore=OLD-client-truststore.jks -jar SSLPoke.jar app.appspot.com 443
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

java -Djavax.net.ssl.trustStore=NEW-client-truststore.jks -jar SSLPoke.jar app.appspot.com 443
Successfully connected

Windows keystore .p12

When working with Windows .p12 is a keystore format that can hold multiple cryptographic objects. It's Microsoft's successor of .pfx format. Both .p12 and .pfx are interchangeable. Example usage is for:

  • Octopus Deploy, requires to import .key and cert into own hosting application. Therefore it needs to be .p12 (with a private key and cert) imported into Local Computer 'Web Hosting' or 'Personal' store first.
  • Strongswan VPN private identity certificate to be copied into /etc/ipsec.d/private/

Convert .cer to .p12

#convert x509(DER format) -> PEM. It'll fail if wrong format
openssl x509   -out cert.pem -outform PEM    -in cert.cer -inform DER
#convert x509 -> PEM
openssl x509   -out cert.pem -outform PEM    -in cert.cer 
#create .p12 keystore            
openssl pkcs12 -out cert.p12 -inkey cert.key -in cert.pem -export

CFSSL - CloudFlare's PKI toolkit

Install 

wget -q --show-progress --https-only --timestamping \
  https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 \
  https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64

chmod +x cfssl_linux-amd64 cfssljson_linux-amd64
sudo mv cfssl_linux-amd64     /usr/local/bin/cfssl
sudo mv cfssljson_linux-amd64 /usr/local/bin/cfssljson

cfssl version

Resources

Build your own CA / PKI

Follow this blog to create own CA PKI infrastructure based on openssl openssl-certificate-authority

Certificates atributes

Key usage extension

Digital signature
Use when the public key is used with a digital signature mechanism to support security services other than non-repudiation, certificate signing, or CRL signing. A digital signature is often used for entity authentication and data origin authentication with integrity.
Non-repudiation
Use when the public key is used to verify digital signatures used to provide a non-repudiation service. Non-repudiation protects against the signing entity falsely denying some action (excluding certificate or CRL signing).
Key encipherment
Use when a certificate will be used with a protocol that encrypts keys. An example is S/MIME enveloping, where a fast (symmetric) key is encrypted with the public key from the certificate. SSL protocol also performs key encipherment.
Data encipherment
Use when the public key is used for encrypting user data, other than cryptographic keys.
Key agreement
Use when the sender and receiver of the public key need to derive the key without using encryption. This key can then can be used to encrypt messages between the sender and receiver. Key agreement is typically used with Diffie-Hellman ciphers.
Certificate signing
Use when the subject public key is used to verify a signature on certificates. This extension can be used only in CA certificates.
CRL signing
Use when the subject public key is to verify a signature on revocation information, such as a CRL.
Encipher only
Use only when key agreement is also enabled. This enables the public key to be used only for enciphering data while performing key agreement.
Decipher only
Use only when key agreement is also enabled. This enables the public key to be used only for deciphering data while performing key agreement.

Mozilla SSL Configuration Generator

ClipCapIt-200202-150209.PNG

SSL Scanners

curl

Get certificate details 

echo | openssl s_client -showcerts -servername gnupg.org -connect gnupg.org:443 2>/dev/null | openssl x509 -inform pem -noout -text

That command connects to the desired website and pipes the certificate in PEM format on to another openssl command that reads and parses the details. redundant -servername parameter is necessary to make openssl do a request with SNI support.


Display only certificate information 

curl --insecure -v https://www.google.com 2>&1 | awk 'BEGIN { cert=0 } /^\* SSL connection/ { cert=1 } /^\*/ { if (cert) print }'


Display certificate information with all chain 

keytool -printcert -sslserver gnupg.org:443


Test SSL cert connection 

$> curl https://$(dig +short api.example.com @8.8.8.8 | tail -1) -kv
* Rebuilt URL to: https://52.222.111.111/
*   Trying 52.222.111.111...
* TCP_NODELAY set
* Connected to 52.222.111.111 (52.222.111.111) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.example.com
(...)

nmap

$> nmap -p 443 --script ssl-cert $(dig +short api.example.com @8.8.8.8 | tail -1)

Starting Nmap 7.60 ( https://nmap.org ) at 2020-02-04 12:23 GMT
Nmap scan report for ec2-52-222-111-111.eu-west-1.compute.amazonaws.com (52.222.111.111)
Host is up (0.021s latency).

PORT    STATE SERVICE
443/tcp open  https
| ssl-cert: Subject: commonName=api.example.com/organizationName=ACME Ltd /stateOrProvinceName=London/countryName=GB
| Subject Alternative Name: DNS:api.acme.com
| Issuer: commonName=QuoVadis Global SSL ICA G3/organizationName=QuoVadis Limited/countryName=BM
| Public Key type: rsa
| Public Key bits: 2048
| Signature Algorithm: sha256WithRSAEncryption
| Not valid before: 2018-01-19T09:20:50
| Not valid after:  2020-01-19T09:30:00
| MD5:   1111 2222 60be f6f4 a4a9 c971 1111 2222
|_SHA-1: 1111 2222 cea1 e3c1 e1cc 8f01 c08b 29be 1111 2222

Nmap done: 1 IP address (1 host up) scanned in 5.89 seconds


Deeper scan 

$> nmap -sV -sC google.com -p 443
Starting Nmap 7.60 ( https://nmap.org ) at 2020-02-07 14:40 UTC
Nmap scan report for google.com (216.58.198.174)
Host is up (0.012s latency).
rDNS record for 216.58.198.174: lhr25s10-in-f14.1e100.net

PORT    STATE SERVICE   VERSION
443/tcp open  ssl/https gws
| fingerprint-strings: 
|   GetRequest: 
|     HTTP/1.0 200 OK
|     Date: Fri, 07 Feb 2020 14:41:10 GMT
|     Expires: -1
|     Cache-Control: private, max-age=0
|     Content-Type: text/html; charset=ISO-8859-1
|     P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
|     Server: gws
|     X-XSS-Protection: 0
|     X-Frame-Options: SAMEORIGIN
|     Set-Cookie: 1P_JAR=2020-02-07-14; expires=Sun, 08-Mar-2020 14:41:10 GMT; path=/; domain=.google.com; Secure
|     Set-Cookie: NID=197=SdLatzxeD4ojxetCRjndLij4h3qknBL8XSR1Gw0_4WPizXArCPs_izyI5eYu1eOff16grZcpxgPrLIacuVLwJrWEcJZqmTfqN4kdHgoHHnLMsiVA-ig9RiUErKjXgLfRHkLqMmVKT-ZbHKU0lzX8-mljT8kSbxv1at7snnCwoKw; expires=Sat, 08-Aug-2020 14:41:10 GMT; path=/; domain=.google.com; HttpOnly
|     Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
|     Accept-Ranges: none
|     Vary: Accept-Encoding
|     <!doctype html><
|   HTTPOptions: 
|     HTTP/1.0 405 Method Not Allowed
|     Allow: GET, HEAD
|     Date: Fri, 07 Feb 2020 14:41:10 GMT
|     Content-Type: text/html; charset=UTF-8
|     Server: gws
|     Content-Length: 1592
|     X-XSS-Protection: 0
|     X-Frame-Options: SAMEORIGIN
|     Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
|     <!DOCTYPE html>
|     <html lang=en>
|     <meta charset=utf-8>
|     <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
|     <title>Error 405 (Method Not Allowed)!!1</title>
|     <style>
|_    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11p
|_http-server-header: gws
|_http-title: Did not follow redirect to https://www.google.com/
| ssl-cert: Subject: commonName=*.google.com/organizationName=Google LLC/stateOrProvinceName=California/countryName=US
| Subject Alternative Name: DNS:*.google.com, DNS:*.android.com, DNS:*.appengine.google.com, DNS:*.cloud.google.com, DNS:*.crowdsource.google.com, DNS:*.g.co, DNS:*.gcp.gvt2.com, DNS:*.gcpcdn.gvt1.com, DNS:*.ggpht.cn, DNS:*.gkecnapps.cn, (...)
| Not valid before: 2020-01-21T08:16:06
|_Not valid after:  2020-04-14T08:16:06
|_ssl-date: 2020-02-07T14:42:02+00:00; 0s from scanner time.
| tls-nextprotoneg: 
|   grpc-exp
|   h2
|_  http/1.1
1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at https://nmap.org/cgi-bin/submit.cgi?new-service :
(...)
SF:}p{margin:11p");

Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 65.24 seconds

sslscan and sslscan2

Not available any more in Ubuntu 20.04 repositories. 

sudo apt-get install sslscan # Ubuntu <=18.04
Ssltest.sh screenshot


New sslscan2 is out, but requires you to compile. 

git clone https://github.com/rbsec/sslscan.git
cd sslscan
apt-get install build-essential git zlib1g-dev
apt-get build-dep openssl
make static

testssl.sh

git clone --depth 1 https://github.com/drwetter/testssl.sh.git
./testssl.sh -p <https://$URL>
# -p display only offered protocols

Documentation

Trusted Certificates list - cacert.pem

Server Name Indication (SNI)

Server Name Indication (SNI) is the clear text part of TLS connection that allows a proxy to select the correct certificate or send a connection to the correct backend resource. It helps with multi-tenant hosting of sites on single ip.


Basics for hosting multiple websites on a same host(ip)

  • For HTTP connections we use -H "Host: api.example.com" but this part is data portion of the protocol encrypted and not available during TLS handshake
  • To host over Https:// multiple-websites on single ip, wildcard certificates could help
  • For HTTPs we use SNI that allows to select specific certificate


Client Hello with SNI. Note that with HTTP1.1 Client Hello there is no SNI included.

ClipCapIt-210518-000458.PNG


With and without SNI with TCP forward

ClipCapIt-210517-232631.PNG


Limitations and challenges of SNI:

  • sends a hostname in a clear text. It's part of TLS including v1.3 but "client hello" is not encrypted because is the first request
ClipCapIt-210517-235239.PNG


ESNI (Encrypted-SNI) is SNI extension for TLSv1.3+ protocol aiming at preventing server name leakage.
ClipCapIt-210517-235341.PNG

It's proposal to encrypt "client hello" already implemented by Firefox with DoH (DNS over HTTPs) or DoT(DNS over TLS). The latter allows metrics but requires a custom port. The concept is that the public key is distributed over DNS using TXT record. Then only private key matching at the server side can decipher the request destination.

But DNS is over UDP, so unless you use DNS over TCP the whole effort of ESNI is lost as eve droppers can learn destination listening DNS traffic.


Public key is distributed over DNS using TXT record

The server publishes a public key on a well-known DNS record, which can be fetched by the client before connecting (as it already does for A, AAAA and other records). The client then replaces the SNI extension in the ClientHello with an “encrypted SNI” extension, which is none other than the original SNI extension, but encrypted using a symmetric encryption key derived using the server’s public key, as described below. The server, which owns the private key and can derive the symmetric encryption key as well, can then decrypt the extension and therefore terminate the connection (or forward it to a backend server). Since only the client, and the server it’s connecting to, can derive the encryption key, the encrypted SNI cannot be decrypted and accessed by third parties.

It’s important to note that this is an extension to TLS version 1.3 and above, and doesn’t work with previous versions of the protocol. The reason is very simple: one of the changes introduced by TLS 1.3 (not without problems) meant moving the Certificate message sent by the server to the encrypted portion of the TLS handshake (before 1.3, it was sent in plaintext). Without this fundamental change to the protocol, an attacker would still be able to determine the identity of the server by simply observing the plaintext certificate sent on the wire.


Enable ESNI
  1. Open about:config in Firefox
  2. Set network.security.esni.enabled to true
  3. Set network.trr.mode to 3
  4. Set network.trr.uri to https://1.1.1.1/dns-query
  5. Check if it works: https://www.cloudflare.com/ssl/encrypted-sni

TLS Passthrough

In this mode, HAProxy does not decipher the traffic. It simply opens a TCP tunnel between the client and the server to let them negotiate and handle the TLS traffic. HAProxy simply runs in mode tcp. The sample fetch methods that apply to this mode are those whose names starts with req.ssl_.

ClipCapIt-210518-010047.PNG

TLS 1.2 vs 1.3

ClipCapIt-210518-010445.PNG

ALPN and H2

HTTP/2 relies on TLS extension ALPN (Application-Layer Protocol Negotiation) that includes the protocol negotiation within the exchange of TLS ClientHello messages. ALPN is able to negotiate which protocol (IANA registered proto Ids) should be handled over a secure connection, such as 'http/1.1' for HTTP/1.1 and 'h2' for HTTP/2. It works seamlessly using the same https port that provides HTTP/1.1 service over TLS. To give an example, if client sends 'h2' in its protocol list and server responds with 'h2', client should send HTTP/2 requests to server and not HTTP/1.1 requests, ie., even though https port is used.

ClipCapIt-210518-081936.PNG

Resources

Retrieved from ‘http://wiki.ciscolinux.co.uk/index.php?title=Linux_SSL/TLS&oldid=6989’