Motivation From time to time I find myself in the need to troubleshoot an issue where network communications is involved.
Although all sounds good, TLS (former SSL) can be a headache when you need to find a needle in the your haystack. TLS cause everything to be transferred in gibberish.
Background TLS is an important layer in network communications. This layer prevents attackers (and ISPs) from viewing or tampering with data exchanged between two nodes—typically a user’s web browser and a web/app server.
TLS uses both asymmetric and symmetric encryption to protect the confidentiality and integrity of data-in-transit. Asymmetric encryption is used to establish a secure session between a client and a server, and symmetric encryption is used to exchange data within the secured session.
Using PKI , TLS negotiates a symmetric key (session key) to encrypt data in-transit. That bits that we are looking for to make our life easier.
SSLKEYLOGFILE When you are trying to check your web browser’s encrypted communications you may start your web browser with the SSLKEYLOGFILE environment variable set to a writable file.format .
Actually, the data written to this file is provided by TLS library used by your application or web browser.
Issue Not everything is a web browser and we may want this exact same feature on other applications.
Workaround Luckily, it is quite common to find those applications using OpenSSL as TLS library. The library itself does not have this feature out-of-the-box for security reasons, but there is nothing holding us from adding it there and creating custom binaries of OpenSSL.
Building custom OpenSSL I’ll be building this on Debian 20.04. Should work on Debian 11 too.
1 2 3 4 Distributor ID: Ubuntu Description: Ubuntu 20.04.3 LTS Release: 20.04 Codename: focal
First, we will need the source package .
Enable source code repo Enable source code repo on apt on /etc/apt/sources.list.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 deb http://archive.ubuntu.com/ubuntu/ focal main restricted deb-src http://archive.ubuntu.com/ubuntu/ focal main restricted deb http://archive.ubuntu.com/ubuntu/ focal-updates main restricted deb-src http://archive.ubuntu.com/ubuntu/ focal-updates main restricted deb http://archive.ubuntu.com/ubuntu/ focal universe deb-src http://archive.ubuntu.com/ubuntu/ focal universe deb http://archive.ubuntu.com/ubuntu/ focal-updates universe deb-src http://archive.ubuntu.com/ubuntu/ focal-updates universe deb http://archive.ubuntu.com/ubuntu/ focal multiverse deb-src http://archive.ubuntu.com/ubuntu/ focal multiverse deb http://archive.ubuntu.com/ubuntu/ focal-updates multiverse deb-src http://archive.ubuntu.com/ubuntu/ focal-updates multiverse deb http://archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse deb-src http://archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse deb http://security.ubuntu.com/ubuntu focal-security main restricted deb-src http://security.ubuntu.com/ubuntu focal-security main restricted deb http://security.ubuntu.com/ubuntu focal-security universe deb-src http://security.ubuntu.com/ubuntu focal-security universe deb http://security.ubuntu.com/ubuntu focal-security multiverse deb-src http://security.ubuntu.com/ubuntu focal-security multiverse
Don’t forget to sudo apt update.
Download Do not run the following with sudo
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 # apt source openssl Reading package lists... Done NOTICE: 'openssl' packaging is maintained in the 'Git' version control system at: https://salsa.debian.org/debian/openssl.git Please use: git clone https://salsa.debian.org/debian/openssl.git to retrieve the latest (possibly unreleased) updates to the package. Need to get 10.0 MB of source archives. Get:1 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu focal-updates/main openssl 1.1.1f-1ubuntu2.16 (dsc) [2470 B] Get:2 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu focal-updates/main openssl 1.1.1f-1ubuntu2.16 (tar) [9793 kB] Get:3 http://eu-west-1.ec2.archive.ubuntu.com/ubuntu focal-updates/main openssl 1.1.1f-1ubuntu2.16 (diff) [210 kB] Fetched 10.0 MB in 0s (36.6 MB/s) dpkg-source: info: extracting openssl in openssl-1.1.1f dpkg-source: info: unpacking openssl_1.1.1f.orig.tar.gz dpkg-source: info: unpacking openssl_1.1.1f-1ubuntu2.16.debian.tar.xz dpkg-source: info: using patch list from debian/patches/series dpkg-source: info: applying 0001-s390x-assembly-pack-add-OPENSSL_s390xcap-environment.patch dpkg-source: info: applying 0002-s390x-assembly-pack-add-OPENSSL_s390xcap-man-page.patch dpkg-source: info: applying 0003-s390x-assembly-pack-perlasm-support.patch dpkg-source: info: applying 0004-crypto-chacha-asm-chacha-s390x.pl-add-vx-code-path.patch dpkg-source: info: applying 0005-crypto-poly1305-asm-poly1305-s390x.pl-add-vx-code-pa.patch dpkg-source: info: applying 0006-s390x-assembly-pack-fix-formal-interface-bug-in-chac.patch dpkg-source: info: applying 0007-s390x-assembly-pack-import-chacha-from-cryptogams-re.patch dpkg-source: info: applying 0008-s390x-assembly-pack-import-poly-from-cryptogams-repo.patch dpkg-source: info: applying 0009-s390x-assembly-pack-allow-alignment-hints-for-vector.patch dpkg-source: info: applying 0010-s390x-assembly-pack-update-perlasm-module.patch dpkg-source: info: applying 0011-s390x-assembly-pack-remove-chacha20-dependency-on-no.patch dpkg-source: info: applying 0012-s390x-assembly-pack-remove-poly1305-dependency-on-no.patch dpkg-source: info: applying 0013-fix-strict-warnings-build.patch dpkg-source: info: applying 0014-s390x-assembly-pack-add-support-for-pcc-and-kma-inst.patch dpkg-source: info: applying 0015-Place-return-values-after-examples-in-doc.patch dpkg-source: info: applying 0016-s390x-assembly-pack-update-OPENSSL_s390xcap-3.patch dpkg-source: info: applying 0017-s390xcpuid.pl-fix-comment.patch dpkg-source: info: applying 0018-s390x-assembly-pack-accelerate-scalar-multiplication.patch dpkg-source: info: applying 0019-Enable-curve-spefific-ECDSA-implementations-via-EC_M.patch dpkg-source: info: applying 0020-s390x-assembly-pack-accelerate-ECDSA.patch dpkg-source: info: applying 0021-OPENSSL_s390xcap.pod-list-msa9-facility-bit-155.patch dpkg-source: info: applying 0022-s390x-assembly-pack-fix-msa3-stfle-bit-detection.patch dpkg-source: info: applying 0023-s390x-assembly-pack-fix-OPENSSL_s390xcap-z15-cpu-mas.patch dpkg-source: info: applying 0024-s390x-assembly-pack-accelerate-X25519-X448-Ed25519-a.patch dpkg-source: info: applying 0025-Add-self-generated-test-vector-for-x448-non-canonica.patch dpkg-source: info: applying debian-targets.patch dpkg-source: info: applying man-section.patch dpkg-source: info: applying no-symbolic.patch dpkg-source: info: applying pic.patch dpkg-source: info: applying tests-use-seclevel-1.patch dpkg-source: info: applying tls1.2-min-seclevel2.patch dpkg-source: info: applying CVE-2020-1967-1.patch dpkg-source: info: applying CVE-2020-1967-2.patch dpkg-source: info: applying CVE-2020-1967-3.patch dpkg-source: info: applying CVE-2020-1967-4.patch dpkg-source: info: applying CVE-2020-1971-1.patch dpkg-source: info: applying CVE-2020-1971-2.patch dpkg-source: info: applying CVE-2020-1971-3.patch dpkg-source: info: applying CVE-2020-1971-4.patch dpkg-source: info: applying CVE-2020-1971-5.patch dpkg-source: info: applying CVE-2020-1971-6.patch dpkg-source: info: applying CVE-2021-23840.patch dpkg-source: info: applying CVE-2021-23841.patch dpkg-source: info: applying CVE-2021-3449-1.patch dpkg-source: info: applying CVE-2021-3449-2.patch dpkg-source: info: applying CVE-2021-3449-3.patch dpkg-source: info: applying CVE-2021-3449-4.patch dpkg-source: info: applying lp-1926254-1-Allow-certificates-with-Basic-Constraints-CA-fa.patch dpkg-source: info: applying lp-1926254-2-Set-X509_V_ERR_INVALID_EXTENSION-error-for-inva.patch dpkg-source: info: applying lp-1926254-3-Add-test-cases-for-the-non-CA-certificate-with-.patch dpkg-source: info: applying lp1931994-s390x-evp-init-fix.patch dpkg-source: info: applying CVE-2021-3711-1.patch dpkg-source: info: applying CVE-2021-3711-2.patch dpkg-source: info: applying CVE-2021-3711-3.patch dpkg-source: info: applying CVE-2021-3712.patch dpkg-source: info: applying CVE-2021-3712-2.patch dpkg-source: info: applying CVE-2021-3712-3.patch dpkg-source: info: applying CVE-2021-3712-4.patch dpkg-source: info: applying CVE-2021-3712-5.patch dpkg-source: info: applying CVE-2021-3712-6.patch dpkg-source: info: applying CVE-2021-3712-7.patch dpkg-source: info: applying CVE-2021-3712-8.patch dpkg-source: info: applying CVE-2021-3712-9.patch dpkg-source: info: applying CVE-2021-3712-10.patch dpkg-source: info: applying CVE-2021-3712-11.patch dpkg-source: info: applying CVE-2021-3712-12.patch dpkg-source: info: applying CVE-2021-3712-13.patch dpkg-source: info: applying CVE-2021-3712-14.patch dpkg-source: info: applying lp-1940656-1-srp-fix-double-free.patch dpkg-source: info: applying lp-1940656-2-ts-fix-double-free-on-error-path.patch dpkg-source: info: applying lp-1940656-3-engine-fix-double-free-on-error-path.patch dpkg-source: info: applying lp-1940656-4-Prevent-use-after-free-of-global_engine_lock.patch dpkg-source: info: applying double-engine-load.patch dpkg-source: info: applying more-testing-dgst.patch dpkg-source: info: applying double-engine-load-test.patch dpkg-source: info: applying lp-1960863-crypto-poly1305-asm-fix-armv8-pointer-authenticat.patch dpkg-source: info: applying CVE-2022-0778-1.patch dpkg-source: info: applying CVE-2022-0778-2.patch dpkg-source: info: applying CVE-2022-0778-3.patch dpkg-source: info: applying lp1978093/0001-Update-expired-SCT-certificates.patch dpkg-source: info: applying lp1978093/0002-ct_test.c-Update-the-epoch-time.patch dpkg-source: info: applying lp1978093/0003-Update-further-expiring-certificates-that-affect-tes.patch dpkg-source: info: applying lp1947588.patch dpkg-source: info: applying CVE-2022-1292.patch dpkg-source: info: applying CVE-2022-2068.patch dpkg-source: info: applying c_rehash-compat.patch dpkg-source: info: applying CVE-2022-2097-1.patch dpkg-source: info: applying CVE-2022-2097-2.patch
Install build dependencies 1 2 sudo apt build-dep openssl sudo apt install quilt
Build package as-is Just to ensure everything is in place and working as expected we should build before applying any patches.
1 2 cd openssl-1.1.1f dpkg-buildpackage -us -uc
After dpkg-buildpackage is complete you should have the following files:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # find . -name '*.so' ./build_shared/libssl.so ./build_shared/libcrypto.so ./build_shared/engines/ossltest.so ./build_shared/engines/afalg.so ./build_shared/engines/padlock.so ./build_shared/engines/dasync.so ./debian/libssl1.1/usr/lib/x86_64-linux-gnu/engines-1.1/afalg.so ./debian/libssl1.1/usr/lib/x86_64-linux-gnu/engines-1.1/padlock.so ./debian/tmp/usr/lib/x86_64-linux-gnu/libssl.so ./debian/tmp/usr/lib/x86_64-linux-gnu/engines-1.1/afalg.so ./debian/tmp/usr/lib/x86_64-linux-gnu/engines-1.1/padlock.so ./debian/tmp/usr/lib/x86_64-linux-gnu/libcrypto.so ./debian/libssl-dev/usr/lib/x86_64-linux-gnu/libssl.so ./debian/libssl-dev/usr/lib/x86_64-linux-gnu/libcrypto.so
Apply SSLKEYLOGFILE patch You may get the patch file here .
1 2 3 # wget "https://gist.githubusercontent.com/jrocha/239d0d2afca3c43023a8cedaf140b4dd/raw/906edca056590f60bb32a72d2e646f8fed21a60c/sslkeylogfile.patch" -P ~ # cp ~/sslkeylogfile.patch debian/patches/ # echo "sslkeylogfile.patch" >> debian/patches/series
Rebuild with the new patch 1 dpkg-buildpackage -us -uc
If you prefer to skip tests just set environment variable DEB_BUILD_OPTIONS=nocheck before dpkg-buildpackage
Using new build Now we have our new openssl shared lib ready to be used.SSLKEYLOGFILE to create the key log file. If this SSLKEYLOGFILE is not set it will default to /tmp/sslkeylogfile.log
Loading curl with custom OpenSSL build 1 $ SSLKEYLOGFILE=mylog.log LD_PRELOAD=/home/jrocha/work/openssl-work/openssl-1.1.1f/build_shared/libssl.so curl https://www.google.com
You should now have a file mylog.log with something like this:
1 2 3 4 5 6 7 8 $ cat mylog.log CLIENT_RANDOM 72f3934b15eec4e268cefa9d6d5c664545c0284f67baed8980b6011013b35df4 c599d439787b594cf9e89ecb70d24c9448c50c9bbdf74d53a3c4ed63b35cb763386b21097e475979c811c4fc725b67af CLIENT_RANDOM 0775c44f83977d12aac2dd128be15ebb740035291d79e6eb7781ca6b7eddcabd f52b93e8c120fea0656b76d916d4450f499063fce71bdb57d8e2fe58f8f1cd16afabf2f2e18865853f72dc21a2a381d3 SERVER_HANDSHAKE_TRAFFIC_SECRET b7a8d0cbf660f56cc8f70c38d1158121d13f5489ccdb70a398f31bb24adcaa0c 2468a6cbab0463978808d23a8d605c326225115211ab0907ef6bb697f398d2a35d87bb4f3d9d56fb8f6b07698e335c5f EXPORTER_SECRET b7a8d0cbf660f56cc8f70c38d1158121d13f5489ccdb70a398f31bb24adcaa0c 8d9e9697949990b7bb401291a8b8a28f4de36bd9345cc40b95390c67dae0da95b66607579f401eec931cf27f1b8a25dd SERVER_TRAFFIC_SECRET_0 b7a8d0cbf660f56cc8f70c38d1158121d13f5489ccdb70a398f31bb24adcaa0c 81c0fa1cab9456f1391cbb64ddb615ce49f82b7e213c1b69c33ee885f7b0a7927157dcfd5a6d9bdb4025737eaa738c99 CLIENT_HANDSHAKE_TRAFFIC_SECRET b7a8d0cbf660f56cc8f70c38d1158121d13f5489ccdb70a398f31bb24adcaa0c 819a2f2ce659d9c09c8993d333dea745006eef1e71fb3a95e6e6a87b866b20c68894403e2ed601fe05457663afc9ed11 CLIENT_TRAFFIC_SECRET_0 b7a8d0cbf660f56cc8f70c38d1158121d13f5489ccdb70a398f31bb24adcaa0c 1e4ceb72be7a588d107f25d8c8eb589d5d56030d6076817ecdae270ed5497cb9357845e9491da487b03e5dd2d6ad67da
Things to consider
When you use this sort of technique you are weakening your security. Ensure you understand what you are doing.
Don’t replace your certified OpenSSL binaries with your custom one. Use your custom build only when needed in a per application basis.
Delete key files after you are done.
Conclusion Now we have a working setup to perform a network capture and later decrypt it using stored keys.
References https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_keylog_callback.html https://wiki.wireshark.org/TLS https://firefox-source-docs.mozilla.org/security/nss/legacy/key_log_format/index.html
