Mind Chasers Inc.
Mind Chasers Inc.

Build, Debug, and Experiment with Master Branch of OpenSSL (3.0 Beta) on Ubuntu Linux 20.04

A brief review of cloning, building, debugging, and experimenting with the master branch of the OpenSSL project. Includes stepping into the crypto library with GDB.

X-ray Engineering Services

Overview

OpenSSL is the de facto, open source cryptography software package for Linux and other operating systems. It consists of both a command line tool and libraries for cryptography. It has cutting edge support of cryptographic algorithms and an experienced team of developers. However, there seems to be a dearth of recent documentation and examples regarding developing with OpenSSL, especially with the latest API. Therefore, this article is a brief summary on getting started with OpenSSL development.

The openssl executable itself is a command line tool that can be used interactively. A very simple example of using the openssl executable packaged with Ubuntu 20.04 is shown below:

$ which openssl
/usr/bin/openssl

$ openssl
OpenSSL> version
OpenSSL 1.1.1f  31 Mar 2020

The version numbering system for OpenSSL can be confusing and is explained in Release Strategy. In this article, we work with the master branch, which tracks 3.0 development.

Note: Version 1.1.1 will be supported by OpenSSL until 2023-09-11 (LTS)

OpenSSL Components

OpenSSL relies on two important libraries that are part of the OpenSSL project:

  • libssl provides the client and server-side implementations for SSLv3 and TLS.
  • libcrypto provides general cryptographic and X.509 support needed by SSL/TLS but not logically part of it

We have found that the best way to learn how to work with the OpenSSL libraries is to look at how the openssl application makes use of them itself, and this is shown by example below.

Build Master Branch on Ubuntu 20.04

Below we show steps to clone, build, and use the master branch of the OpenSSL repository. We install it to /opt/openssl and make use of it in such a way that it does not collide with the version installed & supported by Ubuntu 20.04. You may wish to build and install your own version of OpenSSL in a virtual environment or container.

$ cd /build/  # /build is where we build our code

$ git clone https://github.com/openssl/openssl.git
$ cd /build/openssl

$ find . -name tls1.h | xargs grep TLS1_3_RFC
# define TLS1_3_RFC_AES_128_GCM_SHA256                   "TLS_AES_128_GCM_SHA256"
# define TLS1_3_RFC_AES_256_GCM_SHA384                   "TLS_AES_256_GCM_SHA384"
# define TLS1_3_RFC_CHACHA20_POLY1305_SHA256             "TLS_CHACHA20_POLY1305_SHA256"
# define TLS1_3_RFC_AES_128_CCM_SHA256                   "TLS_AES_128_CCM_SHA256"
# define TLS1_3_RFC_AES_128_CCM_8_SHA256                 "TLS_AES_128_CCM_8_SHA256"

$ mkdir build; cd /build/openssl/build

$../config -v --prefix=/opt/openssl --openssldir=$HOME/openssl --debug
C compiler: clang
C compiler vendor: clang
C compiler version: 1000
Configuring OpenSSL version 3.0.0-beta2-dev for target linux-x86_64
Using os-specific seed configuration
Creating configdata.pm
Running configdata.pm
Creating Makefile.in
Creating Makefile

**********************************************************************
***                                                                ***
***   OpenSSL has been successfully configured                     ***
***                                                                ***
***   If you encounter a problem while building, please open an    ***
***   issue on GitHub   ***
***   and include the output from the following command:           ***
***                                                                ***
***       perl configdata.pm --dump                                ***
***                                                                ***
***   (If you are new to OpenSSL, you might want to consult the    ***
***   'Troubleshooting' section in the INSTALL.md file first)      ***
***                                                                ***
**********************************************************************

config options used (see INSTALL for extensive list of supported options):

  • --prefix: top of the installation directory tree
  • --openssldir: directory for OpenSSL configuration files, and also the default certificate and key store.
  • --debug: build with debugging symbols

Note that OpenSSL doesn't use autotools for its build system but instead uses perl (scripts). The following is from the OpenSSL FAQ: "For OpenSSL 1.1, we decided to base our build system on perl, information files and build file (Makefile) templates, thereby covering all the systems we support. Perl was the base language of choice because we already use it in diverse scripts, and it's one of the most widely spread scripting languages."

Let's look a little further into how we configured the build. First, let's make sure we configured properly for debugging by looking at CFLAGS and CXXFLAGS

$ perl configdata.pm -m

Makevars:

    AR              = ar
    ARFLAGS         = qc
    CC              = gcc
    CFLAGS          = -Wall -O0 -g
    CPPDEFINES      = 
    CPPFLAGS        = 
    CPPINCLUDES     = 
    CXX             = g++
    CXXFLAGS        = -Wall -O0 -g
    HASHBANGPERL    = /usr/bin/env perl
    LDFLAGS         = 
    LDLIBS          = 
    PERL            = /usr/bin/perl
    RANLIB          = ranlib
    RC              = windres
    RCFLAGS         = 
...

Let's also look at what's enabled and disabled:

$ perl configdata.pm -o

Enabled features:

    afalgeng
    aria
    asm
    async
    autoalginit
    autoerrinit
    autoload-config
    bf
    blake2
    bulk
    cached-fetch
    camellia
    capieng
    cast
    chacha
    cmac
    cmp
    cms
    comp
    ct
    deprecated
    des
    dgram
    dh
    dsa
    dso
    dtls
    dynamic-engine
    ec
    ec2m
    ecdh
    ecdsa
    engine
    err
    filenames
    gost
    idea
    legacy
    loadereng
    makedepend
    md4
    mdc2
    module
    multiblock
    nextprotoneg
    ocb
    ocsp
    padlockeng
    pic
    pinshared
    poly1305
    posix-io
    psk
    rc2
    rc4
    rdrand
    rfc3779
    rmd160
    scrypt
    secure-memory
    seed
    shared
    siphash
    siv
    sm2
    sm3
    sm4
    sock
    srp
    srtp
    sse2
    ssl
    ssl-trace
    static-engine
    stdio
    tests
    threads
    tls
    ts
    ui-console
    whirlpool
    tls1
    tls1-method
    tls1_1
    tls1_1-method
    tls1_2
    tls1_2-method
    tls1_3
    dtls1
    dtls1-method
    dtls1_2
    dtls1_2-method

Disabled features:

    acvp-tests          [default]        OPENSSL_NO_ACVP_TESTS
    asan                [default]        OPENSSL_NO_ASAN
    buildtest-c++       [default]        
    crypto-mdebug       [default]        OPENSSL_NO_CRYPTO_MDEBUG
    devcryptoeng        [default]        OPENSSL_NO_DEVCRYPTOENG
    ec_nistp_64_gcc_128 [default]        OPENSSL_NO_EC_NISTP_64_GCC_128
    egd                 [default]        OPENSSL_NO_EGD
    external-tests      [default]        OPENSSL_NO_EXTERNAL_TESTS
    fips                [default]        
    fips-securitychecks [cascade]        OPENSSL_NO_FIPS_SECURITYCHECKS
    fuzz-afl            [default]        OPENSSL_NO_FUZZ_AFL
    fuzz-libfuzzer      [default]        OPENSSL_NO_FUZZ_LIBFUZZER
    ktls                [default]        OPENSSL_NO_KTLS
    md2                 [default]        OPENSSL_NO_MD2 (skip crypto/md2)
    msan                [default]        OPENSSL_NO_MSAN
    rc5                 [default]        OPENSSL_NO_RC5 (skip crypto/rc5)
    sctp                [default]        OPENSSL_NO_SCTP
    trace               [default]        OPENSSL_NO_TRACE
    ubsan               [default]        OPENSSL_NO_UBSAN
    unit-test           [default]        OPENSSL_NO_UNIT_TEST
    uplink              [no uplink_arch] OPENSSL_NO_UPLINK
    weak-ssl-ciphers    [default]        OPENSSL_NO_WEAK_SSL_CIPHERS
    zlib                [default]        
    zlib-dynamic        [default]        
    ssl3                [default]        OPENSSL_NO_SSL3
    ssl3-method         [default]        OPENSSL_NO_SSL3_METHOD

Let's build it.

$ make

$ make install

$ ls ~/openssl
certs  ct_log_list.cnf  ct_log_list.cnf.dist  misc  openssl.cnf  openssl.cnf.dist  private
	
$ export PATH=/opt/openssl/bin:$PATH
$ export LD_LIBRARY_PATH=/opt/openssl/lib/

$ which openssl
/opt/openssl/bin/openssl

$ openssl version
OpenSSL 3.0.0-beta2-dev  (Library: OpenSSL 3.0.0-beta2-dev 

Note that the two exports above are temporary. It is probably most conveinent to add these two lines to a file (e.g., /opt/openssl/env-openssl) that you would then source:

$ which openssl
/usr/bin/openssl

$ source /opt/openssl/env-openssl

$ which openssl
/opt/openssl/bin/openssl

$ ldd /opt/openssl/bin/openssl 
	linux-vdso.so.1 (0x00007ffe90bc4000)
	libssl.so.3 => /opt/openssl/lib/libssl.so.3 (0x00007fd87e5bc000)
	libcrypto.so.3 => /opt/openssl/lib/libcrypto.so.3 (0x00007fd87e0c2000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd87e07b000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd87de89000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd87de83000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fd87e7b0000)

Now you're set up to experiment with the master branch of OpenSSL in your current shell whenever you wish on your Ubuntu 20.04 system.

Debugging genrsa

Generating an RSA private key is a common and interesting function for the openssl application. Below we'll set up a breakpoint in an RSA crypto library function, which will be called when we invoke genrasa:

$ gdb -v
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2
...

$ gdb --args openssl genrsa
...
Reading symbols from openssl...done.

(gdb) break main
Breakpoint 1 at 0x75411: file ../apps/openssl.c, line 234.

(gdb) run
Starting program: /opt/openssl/bin/openssl genrsa
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, main (argc=999273375, argv=0x64f79d1c20) at ../apps/openssl.c:234
234	{

(gdb) break rsa_keygen
Breakpoint 2 at 0x7ffff7cb224e: file ../crypto/rsa/rsa_gen.c, line 426.

(gdb) cont
Continuing.

Breakpoint 2, rsa_keygen (libctx=0x555555718b70, rsa=0x555555718b70, bits=21845, primes=1433504768, e_value=0x7fffffffdef0, cb=0x7ffff7c5750b <CRYPTO_new_ex_data+46>, 
    pairwise_test=0) at ../crypto/rsa/rsa_gen.c:426
426

(gdb) backtrace
#0  rsa_keygen (libctx=0x555555718b70, rsa=0x555555718b70, bits=21845, primes=1433504768, e_value=0x7fffffffdef0, cb=0x7ffff7c5750b <CRYPTO_new_ex_data+46>, pairwise_test=0)
    at ../crypto/rsa/rsa_gen.c:426
#1  0x00007ffff7cb12e5 in RSA_generate_multi_prime_key (rsa=0x555555718b70, bits=2048, primes=2, e_value=0x5555556b2ff0, cb=0x5555556bc370) at ../crypto/rsa/rsa_gen.c:71
#2  0x00007ffff7d94841 in rsa_gen (genctx=0x5555556b2f90, osslcb=0x7ffff7c3e34a <ossl_callback_to_pkey_gencb>, cbarg=0x5555556b2ed0)
    at ../providers/implementations/keymgmt/rsa_kmgmt.c:585
...

(gdb) list
421	}
422	#endif /* FIPS_MODULE */
423	
424	static int rsa_keygen(OSSL_LIB_CTX *libctx, RSA *rsa, int bits, int primes,
425	                      BIGNUM *e_value, BN_GENCB *cb, int pairwise_test)
426	{
427	    int ok = 0;
428	
429	    /*
430	     * Only multi-prime keys or insecure keys with a small key length will use

Now at this point, you can step through the RSA key generation routine, print variables, dump memory, etc. If you need help with GDB, then consult its documentation page.

Troubleshooting

Keep in mind that OpenSSL includes a test suite:

$ cd /build/openssl/build/
$ make tests
...
01-test_abort.t .................... ok   
01-test_fipsmodule_cnf.t ........... skipped: Test only supported in a fips build
01-test_sanity.t ................... ok   
01-test_symbol_presence.t .......... ok   
01-test_test.t ..................... ok   
02-test_errstr.t ................... ok     
02-test_internal_context.t ......... ok   
02-test_internal_ctype.t ........... ok   
02-test_internal_keymgmt.t ......... ok   
02-test_internal_provider.t ........ ok   
02-test_lhash.t .................... ok   
02-test_ordinals.t ................. ok   
02-test_sparse_array.t ............. ok   
02-test_stack.t .................... ok   
03-test_exdata.t ................... ok   
03-test_fipsinstall.t .............. skipped: Test only supported in a fips build
03-test_internal_asn1.t ............ ok   
03-test_internal_asn1_dsa.t ........ ok   
03-test_internal_bn.t .............. ok   
03-test_internal_chacha.t .......... ok   
03-test_internal_curve448.t ........ ok   
03-test_internal_ec.t .............. ok   
03-test_internal_ffc.t ............. ok   
03-test_internal_mdc2.t ............ ok   
03-test_internal_modes.t ........... ok   
03-test_internal_namemap.t ......... ok   
03-test_internal_poly1305.t ........ ok   
03-test_internal_rsa_sp800_56b.t ... ok   
03-test_internal_siphash.t ......... ok   
03-test_internal_sm2.t ............. ok   
03-test_internal_sm3.t ............. ok   
03-test_internal_sm4.t ............. ok   
03-test_internal_ssl_cert_table.t .. ok   
03-test_internal_x509.t ............ ok   
03-test_params_api.t ............... ok   
03-test_property.t ................. ok   
03-test_ui.t ....................... ok   
04-test_asn1_decode.t .............. ok   
04-test_asn1_encode.t .............. ok   
04-test_asn1_string_table.t ........ ok   
04-test_bio_callback.t ............. ok   
04-test_bio_core.t ................. ok   
04-test_bioprint.t ................. ok   
04-test_conf.t ..................... ok   
04-test_encoder_decoder.t .......... ok   
04-test_encoder_decoder_legacy.t ... ok   
04-test_err.t ...................... ok   
04-test_hexstring.t ................ ok   
04-test_param_build.t .............. ok   
04-test_params.t ................... ok   
04-test_params_conversion.t ........ ok   
04-test_pem_read_depr.t ............ ok   
04-test_pem_reading.t .............. ok    
04-test_provfetch.t ................ ok   
04-test_provider.t ................. ok   
04-test_provider_fallback.t ........ ok   
04-test_upcalls.t .................. ok   
05-test_bf.t ....................... ok   
05-test_cast.t ..................... ok   
05-test_cmac.t ..................... ok   
05-test_des.t ...................... ok   
05-test_hmac.t ..................... ok   
05-test_idea.t ..................... ok   
05-test_pbe.t ...................... ok   
05-test_rand.t ..................... ok   
05-test_rc2.t ...................... ok   
05-test_rc4.t ...................... ok   
05-test_rc5.t ...................... skipped: rc5 is not supported by this OpenSSL build
06-test_algorithmid.t .............. ok    
06-test_rdrand_sanity.t ............ ok   
10-test_bn.t ....................... ok   
10-test_exp.t ...................... ok   
15-test_dh.t ....................... ok   
15-test_dsa.t ...................... ok   
...

Also, the OpenSSL FAQ page includes a lot of troubleshooting information under "Questions on Building and Testing OpenSSL"

Simple TLS Server

The OpenSSL wiki provides a "complete implementation of a minimal TLS server". Let's copy it to tlsserver.c in our home directory and try it.

We'll make two small changes to tlsserver.c to make it easier to understand what's happening:

-    const char reply[] = "test\n";
+    const char reply[] = "Hello from Simple TLS Server\n";
        else {
+           printf("served\n");
            SSL_write(ssl, reply, strlen(reply));
        }

Note that the below example is kept as simple as possible and is just for testing and experimenation. Do not use to pass private information.

$ source /opt/openssl/env-openssl

$ gcc -O0 -g tlsserver.c -o tlsserver -L/opt/openssl/lib -lcrypto -lssl

$ ldd tlsserver
	linux-vdso.so.1 (0x00007ffd5cdeb000)
	libcrypto.so.3 => /opt/openssl/lib/libcrypto.so.3 (0x00007f9dc7aca000)
	libssl.so.3 => /opt/openssl/lib/libssl.so.3 (0x00007f9dc79f6000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9dc77e0000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9dc77da000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f9dc77b7000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9dc7fcb000)


./tlsserver 
8002A8D8A77F0000:error:80000002:system library:file_ctrl:No such file or directory:../crypto/bio/bss_file.c:297:calling fopen(cert.pem, r)
8002A8D8A77F0000:error:10080002:BIO routines:file_ctrl:system lib:../crypto/bio/bss_file.c:300:
8002A8D8A77F0000:error:0A080002:SSL routines:SSL_CTX_use_certificate_file:system lib:../ssl/ssl_rsa.c:291:

It probably comes as no surprise that we need to set up a private key and certificate. Do the following in the same directory as tlsserver:

$ openssl genrsa -out key.pem 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.
e is 65537 (0x010001)

$ openssl req -new -x509 -key key.pem -out cert.pem -days 7

Try it again:

$ ./tlsserver

And from a different shell:

$ curl -vv  --insecure https://localhost:4433
*   Trying 127.0.0.1:4433...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 4433 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=US; ST=NJ; O=Mind Chasers; CN=Charlie; emailAddress=chaz@example.com
*  start date: Jul  3 06:06:36 2021 GMT
*  expire date: Jul 10 06:06:36 2021 GMT
*  issuer: C=US; ST=NJ; O=Mind Chasers; CN=Charlie; emailAddress=chaz@example.com
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET / HTTP/1.1
> Host: localhost:4433
> User-Agent: curl/7.68.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
...

In the other shell, we see "served".

Note that we pass curl the --insecure option since we are using a self signed certificate.

Another good thing to do at this point is to run either Wireshark or tcpdump and watch the exchange of the TLS handshake.

References

Terms / Acronyms

  • ALPN: Application-Layer Protocol Negotiation
  • CMVP: Cryptographic Module Validation Program
  • FIPS: Federal Information Processing Standards
  • PEM: Privacy-enhanced Electronic Mail is the origin of this acronym, but it doesn't convey that it is a file format for OpenSSL
  • RSA: Popular Asymmetric / Private Key Crypto Algorithm, named after its inventors: Rivest–Shamir–Adleman
  • SSL: Secure Socket Layer
  • TLS: Transport Layer Security

Didn't find an answer to your question? Post your issue below or in our new FORUM, and we'll try our best to help you find a solution.

And please note that we update our site daily with new content related to our open source approach to network security and system design. If you would like to be notified about these changes, then please follow us on Twitter and join our mailing list.

Related articles on this site:

share
subscribe to mailing list:

Please help us improve this article by adding your comment or question:

your email address will be kept private
authenticate with a 3rd party for enhanced features, such as image upload
previous month
next month
Su
Mo
Tu
Wd
Th
Fr
Sa
loading