Difference between revisions of "Linux SSL/TLS"
Line 581: | Line 581: | ||
<source lang="bash"> | <source lang="bash"> | ||
git clone --depth 1 https://github.com/drwetter/testssl.sh.git | git clone --depth 1 https://github.com/drwetter/testssl.sh.git | ||
./testssl.sh -p <https://$URL> | |||
# -p display only offered protocols | |||
</source> | </source> | ||
[https://testssl.sh/ Documentation] | [https://testssl.sh/ Documentation] |
Latest revision as of 09:38, 21 July 2023
The main SSL tools on Linux to manage certificates are
- keytool
- openssl
- insec pki from strongswan
- https://certificatetools.com online tool to generate certificates with extensions
- Mozilla ssl server config generator
- keystore-explorer
- crt.sh Search public certificates history - CT Log
Terminology
- DER|PEM
- These are encoding formats. The DER option uses an ASN1 DER encoded form compatible with the PKCS#10. The PEM form consists of the DER format base64 encoded with additional header and footer lines.
- .p12|.pfx
Search installed certificates
Example searches
find -H / -name "*.crt" | while read line ; do echo $line ; openssl x509 -in certificate.crt -text | egrep -i 'Algorithm|Issuer' ; done find -H / -name "*.crt" | while read line ; do echo $line ; openssl x509 -in $line -text | egrep -i 'Algorithm|Issuer|Not After' ; done find -H / -name cacerts | while read line ; do echo $line ; /usr/bin/keytool -list -v -keystore $line -storepass changeit -noprompt | egrep -i 'Alias|Owner|algorithm|SHA1' ; done; find -H / -path /usr/share/doc -prune -o -name "*.pem" | while read line ; do echo $line ; openssl x509 -in $line -text | egrep -i 'Algorithm|Issuer|After' ; done find -H / -path "/usr/share/doc /proc" -prune -o -name "*.pem" | while read line ; do echo $line ; openssl x509 -in $line -text | egrep -i 'Algorithm|Issuer|After' ; done # -H do not follow symbolic links # -path "pattern" file name matches shell pattern # -prune -o ignores a whole directory tree if the pattern matches a directory path
Apache - SSL configuration in httpd.conf
$ grep SSLCertificate /etc/httpd/conf.d SSLCertificateChainFile "/etc/httpd/conf.d/ssl.pem/example.pem" # certificate chain SSLCertificateFile "/etc/httpd/conf.d/ssl.crt/exmple.crt" # domain certificate (private key signed by 3rd party CA) SSLCertificateKeyFile "/etc/httpd/conf.d/ssl.key/exmple.key" # private key file
OpenSSL
Generate self-signed certificate
# generate the RSA private key openssl genpkey -outform PEM -algorithm RSA -pkeyopt rsa_keygen_bits:4096 -out cert.key # Create the CSR openssl req -new -nodes -key cert.key -config csr.cfg -out cert.csr # Self-sign your CSR openssl req -x509 -nodes -in cert.csr -days 365 -key cert.key -config cert.cfg -extensions req_ext -out cert.crt
csr.cfg | cert.cfg |
---|---|
cat > csr.cfg << EOF [ req ] default_md = sha256 prompt = no req_extensions = req_ext distinguished_name = req_distinguished_name [ req_distinguished_name ] commonName = acme.ae countryName = AE stateOrProvinceName = Abu Dhabi localityName = JVC organizationName = Acme Inc [ req_ext ] keyUsage=critical,digitalSignature,keyEncipherment extendedKeyUsage=serverAuth,clientAuth basicConstraints=critical,CA:false subjectAltName = @alt_names [ alt_names ] DNS.0 = acme.ae EOF |
cat > cert.cfg << EOF [ req ] default_md = sha256 prompt = no req_extensions = req_ext distinguished_name = req_distinguished_name [ req_distinguished_name ] commonName = acme.ae countryName = AE stateOrProvinceName = Abu Dhabi localityName = JVC organizationName = Acme Inc [ req_ext ] subjectKeyIdentifier = hash authorityKeyIdentifier = keyid:always,issuer keyUsage=critical,digitalSignature,keyEncipherment extendedKeyUsage=serverAuth,clientAuth basicConstraints=critical,CA:false subjectAltName = @alt_names [ alt_names ] DNS.0 = acme.ae EOF |
Renew a certificate based on the current certificate
openssl x509 -x509toreq -in current_certificate.crt -out request.csr -signkey current_certificate_private.key
Verify a Certificate was Signed by a CA
Use this command to verify that a certificate (domain.crt) was signed by a specific CA certificate (ca.crt). This can be a chain certificate that contains signing certificate.
$ openssl verify -verbose -CAFile ca.crt domain.crt domain.crt: OK
Verify a certificate from a CLI
Insecure option tells libcurl to not verify the peer.
curl --insecure -v https://www.example.com/mgm-your-account 2>&1 \ | awk 'BEGIN { cert=0 } /^\* Server certificate:/ { cert=1 } /^\*/ { if (cert) print }'
If your webserver uses SNI scheme is not IP based, specify the vhost server using -servername
option, otherwise can be skipped
echo \ | openssl s_client -connect <web_server_IP>:443 -servername www.example.com 2>/dev/null \ | openssl x509 -inform pem -noout -text Hosted_website Apache_server in_virtual_host_directive | | server1# openssl s_client -connect 10.0.0.1:443 -servername www.example.com
Verify a Private Key Matches a Certificate
The private key contains a series of numbers. Two of those numbers form the public key, the others are part of your private key. The public key bits are also embedded in your Certificate (we get them from your CSR). To check that the public key in your cert matches the public portion of your private key, you need to view the cert and the key and compare the numbers. To view the Certificate and the key run the commands:
$ openssl x509 -noout -text -in server.crt $ openssl rsa -noout -text -in server.key
The modulus and the public exponent portions in the key and the Certificate must match. But since the public exponent is usually 65537 and it's bothering comparing long modulus you can use the following approach:
$ openssl x509 -noout -modulus -in server.crt | openssl md5 $ openssl rsa -noout -modulus -in server.key | openssl md5
To check to which key or certificate a particular CSR belongs you can compute
$ openssl req -noout -modulus -in server.csr | openssl md5
OpenSSL test modules
With Server Name Indication (SNI), a web server can have multiple SSL certificates installed on the same IP address. SNI-capable browsers will specify the hostname of the server they’re trying to reach during the initial handshake process. This allows the web server to determine the correct SSL certificate to use for the connection. If you try to connect to SmallSites server with s_client, you’ll find that you receive the default SSL certificate installed on the server and not the one for this site. Therefore we need to use the -servername
argument and s_client
will do the additional SNI negotiation step.
OpenSSL provides three modules that allow you to test SSL connections: s_client
, s_server
and s_time
.
The first two, as the names suggest, are for simulating a client and a server in an SSL connection. The third one is for connection timing tests.
s_server
use to setup a test backend server to send our test client requests to
# Certificates ## Create a root certificate and private key to sign the certificate for your services openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example.com.key -out example.com.crt ## Create a certificate and a private key openssl req -out sni_value.example.com.csr -newkey rsa:2048 -nodes -keyout sni_value.example.com.key -subj "/CN=sni_value.example.com/O=some organization" openssl x509 -req -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 0 -in sni_value.example.com.csr -out sni_value.example.com.crt.pem # Simplify names in the next command cp sni_value.example.com.crt.pem server.pem cp sni_value.example.com.key server.key # Simulate webserver openssl s_server -cert server.pem -key server.key -tlsextdebug # Simulate web client '-servername' specifies server we want to use/connect to openssl s_client -connect localhost:4433 -servername sni_value.example.com # Alternative 'curl' command test to set specific SNI server value curl -k https://sni_value.example.com:4433/ --resolve sni_value.example.com:4433:127.0.0.1
s_client
Simulate TLS connection, is particularly useful for checking which protocols and which ciphers the server agrees to use. This information is useful in security and functionality audits. For example, you could use this protocol information to find servers that don’t accept a legitimate protocol or cipher, thus preventing a legitimate client from connecting. You could also locate servers that accept weak protocols or ciphers and could thus allow a malicious attack.
s_client
- output interpretation - https://www.feistyduck.com/library/openssl-cookbook/online/ch-testing-with-openssl.html
Preview a certificate from CLI
openssl x509 -in /etc/httpd/conf/ssl.crt/certificate.crt -text -noout # -text Prints out the certificate in text form. # -noout Prevents output of the encoded version of the request. # -subject Outputs the subject name. # -issuer Outputs the issuer name. # -dates Prints out the start and expiry dates of a certificate. # -fingerprint Prints out the digest of the DER encoded version of the whole certificate. man x509 # see the all available options
Preview CSR request from CLI
openssl req -in certificate.csr -text -noout
Disabling Diffie-Hellman on Apache Servers
On each web server, in the ssl.conf file or, in some cases, the main Apache conf file, add the !DH: identifier to the start of the SSLCipherSuite config option string.
Procedure In Apache's conf directory, locate file: ssl.conf or httpd.conf
Look for the SSLCipherSuite keyword, whose string value must be similar to the following string:
"ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
Add !DH:
after the ALL:
list so that the line looks like the following string:
"ALL:!DH:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
Note: The !ADH:
string in the above string is now redundant and can be removed. Repeat this edit in every SSL config section, if you are not using one global section. Save the file. Restart the web server for the changes to take effect.
Control Apache server
ssh apache.server.com "apachectl configtest" ssh apache.server.com "apachectl graceful" ssh apache.server.com "service httpd status"
Chain certificate structure
Apache ChainSSL is in PEM format containing all certificates from domain one to Root CA
# cat www_example_com_chain_cert-PEM.crt -----BEGIN CERTIFICATE----- #Domain certificate, eg. www_example_com.crt MIIHkTCCBnmgAwIBAgIUZDLrn0fQy7z62YTRnsd/BsvWI1swDQYJKoZIhvcNAQEL < output obmitted > TlXvxzWXBaAm4YzdMQt0owFUbeItrgn8IcfotEesUU6OMSGOSFi+WTlSC6rJmZM+ eNyDv54nHqqlCpbvZI3umxDx015/ -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- #Intermitennt CA, eg. QuoVadis_EV_SSL_ICA_G1.crt MIIFeTCCA2GgAwIBAgIUc9pa+iPZP7qELgog9AHJ2G4k/F0wDQYJKoZIhvcNAQEL < output obmitted > OUKBHeWRPmCuv59UkTZXA/LT/tHt2JC6htu2pYislARuJGsnq4Bgrw76c0AS3UF8 wxZChJIYvI+fVDAKwQ== -----END CERTIFICATE----- -----BEGIN CERTIFICATE----- #Root certificate, eg. QuoVadis_Root_CA_2.crt MIIFtzCCA5+gAwIBAgICBQkwDQYJKoZIhvcNAQEFBQAwRTELMAkGA1UEBhMCQk0x < output obmitted > ohEUGW6yhhtoPkg3Goi3XZZenMfvJ2II4pEZXNLxId26F0KCl3GBUzGpn/Z9Yr9y 8eOx79+Rj1QqCyXBJhnEUhAFZdWCEOrCMc0u -----END CERTIFICATE-----
Java Keytool certificate management
SAN Names
Oracle Java Keytool internally uses the class sun.security.x509.DNSName to check the input. DNSName enforces the syntax specified in RFC 1034. That's why its complaining when we use * wildcard in the DNS. Instead you can use openssl to create Microsoft .p12 keystore or java KeyExplorer gui tool.
Quote from its Javadoc comment:
The name MUST be in the "preferred name syntax," as specified by RFC 1034.
<domain> ::= <subdomain> | " " <subdomain> ::= <label> | <subdomain> "." <label> <label> ::= <letter> [ [ <ldh-str> ] <let-dig> ] <ldh-str> ::= <let-dig-hyp> | <let-dig-hyp> <ldh-str> <let-dig-hyp> ::= <let-dig> | "-" <let-dig> ::= <letter> | <digit> <letter> ::= any one of the 52 alphabetic characters A through Z in upper case and a through z in lower case <digit> ::= any one of the ten digits 0 through 9
So according to this syntax, domain names have to begin with a letter (A-Z, a-z). Newer RFCs (e.g. RFC 2181, RFC 1123) are relaxing these restrictions, so this can be considered a bug in Java. There are already several related bug reports: JDK-8016345, JDK-8007706
Default password
changeit
Types of Java keystores
Terms like keyStore and trustStore are often used interchangeably and the same file can act as keystore as well as trustStore it just matter of pointing javax.net.ssl.keyStore and javax.net.ssl.trustStore properties to that file but there is a slight difference between keystore and trustStore.
- Trustedstore - a keystore containing system wide available Root certificates
- Java keystore - a keystore used by an application
Typical locations of a keystores
- located in server.xml file of Tomcat / Jboss server
- /etc/pki/tls/certs/ca-bundle.crt
- Trustedstore - JAVA_HOME/jre/lib/security/cacerts
Keytool useful commands
List a keystore
keytool -list -v -keystore tomcat.jks
Import Trusted Certificate into keystore
keytool -import -trustcacerts -file GoDaddy_rootCA.cer -alias rootCA -keystore cacerts.jks
Import Signed Certificate into keystore
keytool -importcert -file <signed cert file> -alias <alias> -keystore <key store file>
Remove
keytool -delete -alias godaddy_rootCA -keystore cacerts.jks
Preview a trusted store of different format than JKS. You can also add -storetype parameter for pkms7, 10 or 12 stores listing
keytool --printcert -file /etc/pki/tls/certs/ca-bundle.crt
Create CSR with SANs names. Note that Java 7+ does not allow for wildcards '*' in the names adhering strictly to one of latest RFC standards.
keytool -certreq -file test.csr -keystore keystore.jks -alias wso2carbon -storepass PASS -ext SAN=dns:*.dev.company.com,dns:*.test.company.com
Generate self signed keystore with KeyUsage extension
This example can be used for WSO2 Secure Vault implementation where Key Usage: keyEncipherment,dataEncipherment is mandatory.
keytool -genkey -alias wso2carbon -keyalg RSA -keysize 2048 -noprompt \ -dname "CN=wso2carbon-passwords-encryption,OU=IT Department,O=CompanyPLC,L=City,S=Shire,C=GB" \ -ext KeyUsage=digitalSignature,keyCertSign,keyEncipherment,dataEncipherment \ -keystore wso2carbon-data-encryption.jks -keypass "password1" -storepass "password1"
Renew a cert in .jks based on the current private key
Convert .jks to .pkcs12. This is required as in essence .jks architecture does not allow for the private key to be moved.
keytool -importkeystore -srckeystore keystore.jks -destkeystore keystore.p12 -destkeypass PASSWORD -srcstorepass PASSWORD -deststoretype PKCS12
Export private key using openssl in PKCS8 format. Openssl by default uses .PEM format.
openssl pkcs12 -in keystore.p12 -nodes -nocerts -out keystore_pkcs8_format.key #Example beginning of the key file Bag Attributes friendlyName: mycertificate localKeyID: 54 69 6D 65 20 31 35 32 38 30 35 35 31 31 31 34 33 34 Key Attributes: <No Attributes> -----BEGIN PRIVATE KEY-----
Export private key using openssl in .PEM format this is the same format as KSE below.
openssl pkcs12 -in keystore.p12 -nocerts -nodes | openssl rsa > keystore_pem_format.key #Example beginning of the key file -----BEGIN RSA PRIVATE KEY-----
Export private key using KSE in .PEM format that can be used by openssl.
java -jar kse.jar #Export Private Key > No encryption > PEM format > keystore_pem_format_from_kse.key #Example beginning of the key file -----BEGIN RSA PRIVATE KEY-----
Generate CSR using openssl
openssl req -new -key keystore_pem_format.key -config openssl.cnf -out keystore.csr
Sign the certificate, using internal CA, external CA, free CA like Let's Encrypt
#<... more steps to follow ...>
Generate .p12 keystore from the key (keystore_pem_format.key) and signed certificate
openssl pkcs12 -export -in signedcertificate.cer -inkey keystore_pem_format.key -out new.p12 -name mycertificate
Convert .p12 keystore into .jks keystore
keytool -importkeystore -deststorepass PASSWORD -destkeypass PASSWORD -destkeystore new.jks -srckeystore keystore.p12 -srcstoretype PKCS12 -srcstorepass PASSWORD -alias mycertificate
SSLPoke to test Java certificates
Once you compiled a source code into jar, run as:
# Usage: SSLPoke <host> <port> java -Djavax.net.ssl.trustStore=OLD-client-truststore.jks -jar SSLPoke.jar app.appspot.com 443 javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target java -Djavax.net.ssl.trustStore=NEW-client-truststore.jks -jar SSLPoke.jar app.appspot.com 443 Successfully connected
Windows keystore .p12
When working with Windows .p12 is a keystore format that can hold multiple cryptographic objects. It's Microsoft's successor of .pfx format. Both .p12
and .pfx
are interchangeable. Example usage is for:
- Octopus Deploy, requires to import
.key
andcert
into own hosting application. Therefore it needs to be .p12 (with a private key and cert) imported into Local Computer 'Web Hosting' or 'Personal' store first. - Strongswan VPN private identity certificate to be copied into
/etc/ipsec.d/private/
Convert .cer to .p12
#convert x509(DER format) -> PEM. It'll fail if wrong format openssl x509 -out cert.pem -outform PEM -in cert.cer -inform DER #convert x509 -> PEM openssl x509 -out cert.pem -outform PEM -in cert.cer #create .p12 keystore openssl pkcs12 -out cert.p12 -inkey cert.key -in cert.pem -export
CFSSL - CloudFlare's PKI toolkit
Install
wget -q --show-progress --https-only --timestamping \ https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 \ https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 sudo mv cfssl_linux-amd64 /usr/local/bin/cfssl sudo mv cfssljson_linux-amd64 /usr/local/bin/cfssljson cfssl version
Resources
Build your own CA / PKI
Follow this blog to create own CA PKI infrastructure based on openssl openssl-certificate-authority
Certificates atributes
Key usage extension
- Digital signature
- Use when the public key is used with a digital signature mechanism to support security services other than non-repudiation, certificate signing, or CRL signing. A digital signature is often used for entity authentication and data origin authentication with integrity.
- Non-repudiation
- Use when the public key is used to verify digital signatures used to provide a non-repudiation service. Non-repudiation protects against the signing entity falsely denying some action (excluding certificate or CRL signing).
- Key encipherment
- Use when a certificate will be used with a protocol that encrypts keys. An example is S/MIME enveloping, where a fast (symmetric) key is encrypted with the public key from the certificate. SSL protocol also performs key encipherment.
- Data encipherment
- Use when the public key is used for encrypting user data, other than cryptographic keys.
- Key agreement
- Use when the sender and receiver of the public key need to derive the key without using encryption. This key can then can be used to encrypt messages between the sender and receiver. Key agreement is typically used with Diffie-Hellman ciphers.
- Certificate signing
- Use when the subject public key is used to verify a signature on certificates. This extension can be used only in CA certificates.
- CRL signing
- Use when the subject public key is to verify a signature on revocation information, such as a CRL.
- Encipher only
- Use only when key agreement is also enabled. This enables the public key to be used only for enciphering data while performing key agreement.
- Decipher only
- Use only when key agreement is also enabled. This enables the public key to be used only for deciphering data while performing key agreement.
Mozilla SSL Configuration Generator
SSL Scanners
curl
Get certificate details
echo | openssl s_client -showcerts -servername gnupg.org -connect gnupg.org:443 2>/dev/null | openssl x509 -inform pem -noout -text
That command connects to the desired website and pipes the certificate in PEM format on to another openssl
command that reads and parses the details. redundant -servername
parameter is necessary to make openssl
do a request with SNI
support.
Display only certificate information
curl --insecure -v https://www.google.com 2>&1 | awk 'BEGIN { cert=0 } /^\* SSL connection/ { cert=1 } /^\*/ { if (cert) print }'
Display certificate information with all chain
keytool -printcert -sslserver gnupg.org:443
Test SSL cert connection
$> curl https://$(dig +short api.example.com @8.8.8.8 | tail -1) -kv * Rebuilt URL to: https://52.222.111.111/ * Trying 52.222.111.111... * TCP_NODELAY set * Connected to 52.222.111.111 (52.222.111.111) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/certs/ca-certificates.crt CApath: /etc/ssl/certs * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server accepted to use h2 * Server certificate: * subject: CN=*.example.com (...)
nmap
$> nmap -p 443 --script ssl-cert $(dig +short api.example.com @8.8.8.8 | tail -1) Starting Nmap 7.60 ( https://nmap.org ) at 2020-02-04 12:23 GMT Nmap scan report for ec2-52-222-111-111.eu-west-1.compute.amazonaws.com (52.222.111.111) Host is up (0.021s latency). PORT STATE SERVICE 443/tcp open https | ssl-cert: Subject: commonName=api.example.com/organizationName=ACME Ltd /stateOrProvinceName=London/countryName=GB | Subject Alternative Name: DNS:api.acme.com | Issuer: commonName=QuoVadis Global SSL ICA G3/organizationName=QuoVadis Limited/countryName=BM | Public Key type: rsa | Public Key bits: 2048 | Signature Algorithm: sha256WithRSAEncryption | Not valid before: 2018-01-19T09:20:50 | Not valid after: 2020-01-19T09:30:00 | MD5: 1111 2222 60be f6f4 a4a9 c971 1111 2222 |_SHA-1: 1111 2222 cea1 e3c1 e1cc 8f01 c08b 29be 1111 2222 Nmap done: 1 IP address (1 host up) scanned in 5.89 seconds
Deeper scan
$> nmap -sV -sC google.com -p 443 Starting Nmap 7.60 ( https://nmap.org ) at 2020-02-07 14:40 UTC Nmap scan report for google.com (216.58.198.174) Host is up (0.012s latency). rDNS record for 216.58.198.174: lhr25s10-in-f14.1e100.net PORT STATE SERVICE VERSION 443/tcp open ssl/https gws | fingerprint-strings: | GetRequest: | HTTP/1.0 200 OK | Date: Fri, 07 Feb 2020 14:41:10 GMT | Expires: -1 | Cache-Control: private, max-age=0 | Content-Type: text/html; charset=ISO-8859-1 | P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info." | Server: gws | X-XSS-Protection: 0 | X-Frame-Options: SAMEORIGIN | Set-Cookie: 1P_JAR=2020-02-07-14; expires=Sun, 08-Mar-2020 14:41:10 GMT; path=/; domain=.google.com; Secure | Set-Cookie: NID=197=SdLatzxeD4ojxetCRjndLij4h3qknBL8XSR1Gw0_4WPizXArCPs_izyI5eYu1eOff16grZcpxgPrLIacuVLwJrWEcJZqmTfqN4kdHgoHHnLMsiVA-ig9RiUErKjXgLfRHkLqMmVKT-ZbHKU0lzX8-mljT8kSbxv1at7snnCwoKw; expires=Sat, 08-Aug-2020 14:41:10 GMT; path=/; domain=.google.com; HttpOnly | Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000 | Accept-Ranges: none | Vary: Accept-Encoding | <!doctype html>< | HTTPOptions: | HTTP/1.0 405 Method Not Allowed | Allow: GET, HEAD | Date: Fri, 07 Feb 2020 14:41:10 GMT | Content-Type: text/html; charset=UTF-8 | Server: gws | Content-Length: 1592 | X-XSS-Protection: 0 | X-Frame-Options: SAMEORIGIN | Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000 | <!DOCTYPE html> | <html lang=en> | <meta charset=utf-8> | <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width"> | <title>Error 405 (Method Not Allowed)!!1</title> | <style> |_ *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11p |_http-server-header: gws |_http-title: Did not follow redirect to https://www.google.com/ | ssl-cert: Subject: commonName=*.google.com/organizationName=Google LLC/stateOrProvinceName=California/countryName=US | Subject Alternative Name: DNS:*.google.com, DNS:*.android.com, DNS:*.appengine.google.com, DNS:*.cloud.google.com, DNS:*.crowdsource.google.com, DNS:*.g.co, DNS:*.gcp.gvt2.com, DNS:*.gcpcdn.gvt1.com, DNS:*.ggpht.cn, DNS:*.gkecnapps.cn, (...) | Not valid before: 2020-01-21T08:16:06 |_Not valid after: 2020-04-14T08:16:06 |_ssl-date: 2020-02-07T14:42:02+00:00; 0s from scanner time. | tls-nextprotoneg: | grpc-exp | h2 |_ http/1.1 1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at https://nmap.org/cgi-bin/submit.cgi?new-service : (...) SF:}p{margin:11p"); Service detection performed. Please report any incorrect results at https://nmap.org/submit/ . Nmap done: 1 IP address (1 host up) scanned in 65.24 seconds
sslscan and sslscan2
Not available any more in Ubuntu 20.04 repositories.
sudo apt-get install sslscan # Ubuntu <=18.04
New sslscan2 is out, but requires you to compile.
git clone https://github.com/rbsec/sslscan.git cd sslscan apt-get install build-essential git zlib1g-dev apt-get build-dep openssl make static
testssl.sh
git clone --depth 1 https://github.com/drwetter/testssl.sh.git ./testssl.sh -p <https://$URL> # -p display only offered protocols
Trusted Certificates list - cacert.pem
- Curl CA bundle
Server Name Indication (SNI)
Server Name Indication (SNI) is the clear text part of TLS connection that allows a proxy to select the correct certificate or send a connection to the correct backend resource. It helps with multi-tenant hosting of sites on single ip.
Basics for hosting multiple websites on a same host(ip)
- For HTTP connections we use
-H "Host: api.example.com"
but this part is data portion of the protocol encrypted and not available during TLS handshake - To host over Https:// multiple-websites on single ip, wildcard certificates could help
- For HTTPs we use SNI that allows to select specific certificate
Client Hello with SNI. Note that with HTTP1.1 Client Hello there is no SNI included.
With and without SNI with TCP forward
Limitations and challenges of SNI:
- sends a hostname in a clear text. It's part of TLS including v1.3 but "client hello" is not encrypted because is the first request
- ESNI (Encrypted-SNI) is SNI extension for TLSv1.3+ protocol aiming at preventing server name leakage.
It's proposal to encrypt "client hello" already implemented by Firefox with DoH (DNS over HTTPs) or DoT(DNS over TLS). The latter allows metrics but requires a custom port. The concept is that the public key is distributed over DNS using TXT record. Then only private key matching at the server side can decipher the request destination.
But DNS is over UDP, so unless you use DNS over TCP the whole effort of ESNI is lost as eve droppers can learn destination listening DNS traffic.
- Public key is distributed over DNS using TXT record
The server publishes a public key on a well-known DNS record, which can be fetched by the client before connecting (as it already does for A, AAAA and other records). The client then replaces the SNI extension in the ClientHello with an “encrypted SNI” extension, which is none other than the original SNI extension, but encrypted using a symmetric encryption key derived using the server’s public key, as described below. The server, which owns the private key and can derive the symmetric encryption key as well, can then decrypt the extension and therefore terminate the connection (or forward it to a backend server). Since only the client, and the server it’s connecting to, can derive the encryption key, the encrypted SNI cannot be decrypted and accessed by third parties.
It’s important to note that this is an extension to TLS version 1.3 and above, and doesn’t work with previous versions of the protocol. The reason is very simple: one of the changes introduced by TLS 1.3 (not without problems) meant moving the Certificate message sent by the server to the encrypted portion of the TLS handshake (before 1.3, it was sent in plaintext). Without this fundamental change to the protocol, an attacker would still be able to determine the identity of the server by simply observing the plaintext certificate sent on the wire.
- Enable ESNI
- Open about:config in Firefox
- Set network.security.esni.enabled to true
- Set network.trr.mode to 3
- Set network.trr.uri to https://1.1.1.1/dns-query
- Check if it works: https://www.cloudflare.com/ssl/encrypted-sni
TLS Passthrough
In this mode, HAProxy does not decipher the traffic. It simply opens a TCP tunnel between the client and the server to let them negotiate and handle the TLS traffic. HAProxy simply runs in mode tcp
. The sample fetch methods that apply to this mode are those whose names starts with req.ssl_
.
TLS 1.2 vs 1.3
ALPN and H2
HTTP/2 relies on TLS extension ALPN (Application-Layer Protocol Negotiation) that includes the protocol negotiation within the exchange of TLS ClientHello messages. ALPN is able to negotiate which protocol (IANA registered proto Ids) should be handled over a secure connection, such as 'http/1.1' for HTTP/1.1 and 'h2' for HTTP/2. It works seamlessly using the same https port that provides HTTP/1.1 service over TLS. To give an example, if client sends 'h2' in its protocol list and server responds with 'h2', client should send HTTP/2 requests to server and not HTTP/1.1 requests, ie., even though https port is used.
Resources
- Working with Certificates and SSL Oracle documentation, keytool
- Handy OpenSSL Commands spin.atomicobject.com, includes SAN .cnf
- Java certs explained Keytool Example
- Difference between trustedStore and keyStore in Java - SSL
- Excellent examples of openssl usage dynacont.net
- https://chainchecker.certifytheweb.com/