Mind Chasers Inc.
Mind Chasers Inc.

Build, Debug, and Experiment with Master Branch of OpenSSL on Ubuntu Linux 18.04

A brief review of cloning, building, debugging, and experimenting with the master branch of the OpenSSL project. Includes stepping into the crypto library with GDB.

Overview

OpenSSL is the de facto, open source cryptography software package for Linux and other operating systems. It consists of both a command line tool and libraries for cryptography. It has cutting edge support of cryptographic algorithms and an experienced team of developers. However, there seems to be a dearth of recent documentation and examples regarding developing with OpenSSL, especially with the latest API. Therefore, this article is a brief summary on getting started with OpenSSL development.

The openssl executable itself is a command line tool that can be used interactively. A very simple example of using the openssl executable packaged with Ubuntu 18.04 is shown below:

# make sure we have the latest package installed on Ubuntu:
$ sudo apt upgrade openssl
Reading package lists... Done
Building dependency tree       
Reading state information... Done
openssl is already the newest version (1.1.1-1ubuntu2.1~18.04.6).
...

$ which openssl
/usr/bin/openssl

$ $ openssl
OpenSSL> version
OpenSSL 1.1.1  11 Sep 2018
OpenSSL> quit

Note that Ubuntu 20.04 (Focal) makes use of OpenSSL 1.1.1f.

The version numbering system for OpenSSL can be confusing and is explained in Release Strategy. In this article, we work with the master branch, which tracks 3.0 development.

Note: Version 1.1.1 will be supported by OpenSSL until 2023-09-11 (LTS)

OpenSSL Components

OpenSSL relies on two important libraries that are part of the OpenSSL project:

  • libssl provides the client and server-side implementations for SSLv3 and TLS.
  • libcrypto provides general cryptographic and X.509 support needed by SSL/TLS but not logically part of it

We have found that the best way to learn how to work with the OpenSSL libraries is to look at how the openssl application makes use of them itself, and this is shown by example below.

Build Master Branch on Ubuntu 18.04

Below we show steps to clone, build, and use the master branch of the OpenSSL repository. We install it to /opt/openssl and make use of it in such a way that it does not collide with the version installed & supported by Ubuntu 18.04. You may wish to build and install your own version of OpenSSL in a virtual environment or container.

$ cd /build/  # /build is where we build our code

$ git clone https://github.com/openssl/openssl.git
$ cd /build/openssl

$ find . -name tls1.h | xargs grep TLS1_3_VERSION
# define TLS1_3_VERSION                  0x0304
# define TLS_MAX_VERSION                 TLS1_3_VERSION

$ mkdir build; cd /build/openssl/build

$ ../config -v --prefix=/opt/openssl --openssldir=$HOME/openssl --debug
C compiler: gcc
C compiler vendor: gnu
C compiler version: 700
Configuring OpenSSL version 3.0.0-alpha9-dev for target linux-x86_64
Using os-specific seed configuration
Creating configdata.pm
Running configdata.pm
Creating Makefile

**********************************************************************
***                                                                ***
***   OpenSSL has been successfully configured                     ***
***                                                                ***
***   If you encounter a problem while building, please open an    ***
***   issue on GitHub   ***
***   and include the output from the following command:           ***
***                                                                ***
***       perl configdata.pm --dump                                ***
***                                                                ***
***   (If you are new to OpenSSL, you might want to consult the    ***
***   'Troubleshooting' section in the INSTALL.md file first)      ***
***                                                                ***
**********************************************************************



config options used (see INSTALL for extensive list of supported options):

  • --prefix: top of the installation directory tree
  • --openssldir: directory for OpenSSL configuration files, and also the default certificate and key store.
  • --debug: build with debugging symbols

Note that OpenSSL doesn't use autotools for its build system but instead uses perl (scripts). The following is from the OpenSSL FAQ: "For OpenSSL 1.1, we decided to base our build system on perl, information files and build file (Makefile) templates, thereby covering all the systems we support. Perl was the base language of choice because we already use it in diverse scripts, and it's one of the most widely spread scripting languages."

Let's look a little further into how we configured the build. First, let's make sure we configured properly for debugging by looking at CFLAGS and CXXFLAGS

$ perl configdata.pm -m

Makevars:

    AR              = ar
    ARFLAGS         = r
    CC              = gcc
    CFLAGS          = -Wall -O0 -g
    CPPDEFINES      = 
    CPPFLAGS        = 
    CPPINCLUDES     = 
    CXX             = g++
    CXXFLAGS        = -Wall -O0 -g
    HASHBANGPERL    = /usr/bin/env perl
    LDFLAGS         = 
    LDLIBS          = 
    PERL            = /usr/bin/perl
    RANLIB          = ranlib
    RC              = windres
...

Let's also look at what's enabled and disabled:

$ perl configdata.pm -o

Enabled features:

    acvp_tests
    afalgeng
    aria
    asm
    async
    autoalginit
    autoerrinit
    autoload-config
    bf
    blake2
    camellia
    capieng
    cast
    chacha
    cmac
    cmp
    cms
    comp
    ct
    deprecated
    des
    dgram
    dh
    dsa
    dso
    dtls
    dynamic-engine
    ec
    ec2m
    ecdh
    ecdsa
    engine
    err
    filenames
    fips
    fips-securitychecks
    gost
    idea
    legacy
    makedepend
    md4
    mdc2
    module
    multiblock
    nextprotoneg
    pinshared
    ocb
    ocsp
    padlockeng
    pic
    poly1305
    posix-io
    psk
    rc2
    rc4
    rdrand
    rfc3779
    rmd160
    scrypt
    secure-memory
    seed
    shared
    siphash
    siv
    sm2
    sm3
    sm4
    sock
    srp
    srtp
    sse2
    ssl
    static-engine
    stdio
    tests
    threads
    tls
    ts
    ui-console
    whirlpool
    tls1
    tls1-method
    tls1_1
    tls1_1-method
    tls1_2
    tls1_2-method
    tls1_3
    dtls1
    dtls1-method
    dtls1_2
    dtls1_2-method

Disabled features:

    asan                [default]        OPENSSL_NO_ASAN
    buildtest-c++       [default]        
    crypto-mdebug       [default]        OPENSSL_NO_CRYPTO_MDEBUG
    devcryptoeng        [default]        OPENSSL_NO_DEVCRYPTOENG
    ec_nistp_64_gcc_128 [default]        OPENSSL_NO_EC_NISTP_64_GCC_128
    egd                 [default]        OPENSSL_NO_EGD
    external-tests      [default]        OPENSSL_NO_EXTERNAL_TESTS
    fuzz-libfuzzer      [default]        OPENSSL_NO_FUZZ_LIBFUZZER
    fuzz-afl            [default]        OPENSSL_NO_FUZZ_AFL
    ktls                [default]        OPENSSL_NO_KTLS
    md2                 [default]        OPENSSL_NO_MD2 (skip crypto/md2)
    msan                [default]        OPENSSL_NO_MSAN
    rc5                 [default]        OPENSSL_NO_RC5 (skip crypto/rc5)
    sctp                [default]        OPENSSL_NO_SCTP
    ssl-trace           [default]        OPENSSL_NO_SSL_TRACE
    trace               [default]        OPENSSL_NO_TRACE
    ubsan               [default]        OPENSSL_NO_UBSAN
    unit-test           [default]        OPENSSL_NO_UNIT_TEST
    uplink              [no uplink_arch] OPENSSL_NO_UPLINK
    weak-ssl-ciphers    [default]        OPENSSL_NO_WEAK_SSL_CIPHERS
    zlib                [default]        
    zlib-dynamic        [default]        
    ssl3                [default]        OPENSSL_NO_SSL3
    ssl3-method         [default]        OPENSSL_NO_SSL3_METHOD

Let's build it.

$ make

$ make install

$ export PATH=/opt/openssl/bin:$PATH
$ export LD_LIBRARY_PATH=/opt/openssl/lib/

$ which openssl
/opt/openssl/bin/openssl

$ openssl version
OpenSSL 3.0.0-alpha9-dev  (Library: OpenSSL 3.0.0-alpha9-dev )

Note that the two exports above are temporary. It is probably most conveinent to add these two lines to a file (e.g., /opt/openssl/env-openssl) that you would then source:

$ which openssl
/usr/bin/openssl

$ source /opt/openssl/env-openssl

$ which openssl
/opt/openssl/bin/openssl

$ ldd /opt/openssl/bin/openssl 
	linux-vdso.so.1 (0x00007fffe8f90000)
	libssl.so.3 => /opt/openssl/lib/libssl.so.3 (0x00007f67f6850000)
	libcrypto.so.3 => /opt/openssl/lib/libcrypto.so.3 (0x00007f67f6199000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f67f5f7a000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f67f5b89000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f67f5985000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f67f6e1e000)

Now you're set up to experiment with the master branch of OpenSSL in your current shell whenever you wish on your Ubuntu 18.04 system.

Debugging genrsa

Generating an RSA private key is a common and interesting function for the openssl application. Below we'll set up a breakpoint in an RSA crypto library function, which will be called when we invoke genrasa:

$ gdb -v
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1

$ gdb --args openssl genrsa
...
Reading symbols from openssl...done.

(gdb) break main
Breakpoint 1 at 0x6c8a5: file ../apps/openssl.c, line 232.

(gdb) run
Starting program: /opt/openssl/bin/openssl genrsa
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, main (argc=2, argv=0x7fffffffe478) at ../apps/openssl.c:232
232	{

(gdb) break rsa_keygen
Breakpoint 2 at 0x7ffff76eb072: file ../crypto/rsa/rsa_gen.c, line 424.

(gdb) cont
Continuing.

Breakpoint 2, rsa_keygen (libctx=0x7ffff7b0f8e0 , rsa=0x5555558c3d80, bits=2048, primes=2, e_value=0x5555558b0600, 
    cb=0x5555558af4c0, pairwise_test=0) at ../crypto/rsa/rsa_gen.c:424
424	{

(gdb) backtrace
#0  rsa_keygen (libctx=0x7ffff7b0f8e0 , rsa=0x5555558c3d80, bits=2048, primes=2, e_value=0x5555558b0600, 
    cb=0x5555558af4c0, pairwise_test=0) at ../crypto/rsa/rsa_gen.c:424
#1  0x00007ffff76ea0f0 in RSA_generate_multi_prime_key (rsa=0x5555558c3d80, bits=2048, primes=2, e_value=0x5555558b0600, 
    cb=0x5555558af4c0) at ../crypto/rsa/rsa_gen.c:71
#2  0x00007ffff77d62fe in rsa_gen (genctx=0x5555558afac0, osslcb=0x7ffff7680e99 , cbarg=0x5555558b0680)
    at ../providers/implementations/keymgmt/rsa_kmgmt.c:565
...

(gdb) list
419	}
420	#endif /* FIPS_MODULE */
421	
422	static int rsa_keygen(OSSL_LIB_CTX *libctx, RSA *rsa, int bits, int primes,
423	                      BIGNUM *e_value, BN_GENCB *cb, int pairwise_test)
424	{
425	    int ok = 0;
426	
427	    /*
428	     * Only multi-prime keys or insecure keys with a small key length will use



Now at this point, you can step through the RSA key generation routine, print variables, dump memory, etc. If you need help with GDB, then consult its documentation page.

Troubleshooting

Keep in mind that OpenSSL includes a test suite:

$ make tests
make depend && make _tests
make[1]: Entering directory '/build/openssl/build'
make[1]: Leaving directory '/build/openssl/build'
make[1]: Entering directory '/build/openssl/build'
( SRCTOP=.. \
  BLDTOP=. \
  PERL="/usr/bin/perl" \
  FIPSKEY="f4556650ac31d35461610bac4ed81b1a181b2d8a43ea2854cbae22ca74560813" \
  EXE_EXT= \
  /usr/bin/perl ../test/run_tests.pl  )
01-test_abort.t .................... ok   
01-test_sanity.t ................... ok   
01-test_symbol_presence.t .......... ok   
01-test_test.t ..................... ok   
02-test_errstr.t ................... ok     
02-test_internal_context.t ......... ok   
02-test_internal_ctype.t ........... ok   
02-test_internal_keymgmt.t ......... ok   
02-test_internal_provider.t ........ ok   
02-test_lhash.t .................... ok   
02-test_ordinals.t ................. ok   
02-test_sparse_array.t ............. ok   
02-test_stack.t .................... ok   
03-test_exdata.t ................... ok   
03-test_fipsinstall.t .............. ok    
03-test_internal_asn1.t ............ ok   
03-test_internal_asn1_dsa.t ........ ok   
03-test_internal_bn.t .............. ok   
03-test_internal_chacha.t .......... ok   
03-test_internal_curve448.t ........ ok   
03-test_internal_ec.t .............. ok   
03-test_internal_ffc.t ............. ok   
03-test_internal_mdc2.t ............ ok   
03-test_internal_modes.t ........... ok   
03-test_internal_namemap.t ......... ok   
03-test_internal_poly1305.t ........ ok   
03-test_internal_rsa_sp800_56b.t ... ok   
03-test_internal_siphash.t ......... ok   
03-test_internal_sm2.t ............. ok   
03-test_internal_sm4.t ............. ok   
03-test_internal_ssl_cert_table.t .. ok   
03-test_internal_x509.t ............ ok   
03-test_params_api.t ............... ok   
03-test_property.t ................. ok   
03-test_ui.t ....................... ok   
...
90-test_threads.t .................. ok   
90-test_time_offset.t .............. ok   
90-test_tls13ccs.t ................. ok   
90-test_tls13encryption.t .......... ok   
90-test_tls13secrets.t ............. ok   
90-test_v3name.t ................... ok   
95-test_external_boringssl.t ....... skipped: No external tests in this configuration
95-test_external_gost_engine.t ..... skipped: No external tests in this configuration
95-test_external_krb5.t ............ skipped: No external tests in this configuration
95-test_external_pyca.t ............ skipped: No external tests in this configuration
99-test_ecstress.t ................. ok   
99-test_fuzz_asn1.t ................ 1/? 
99-test_fuzz_asn1.t ................ ok   
99-test_fuzz_asn1parse.t ........... ok   
99-test_fuzz_bignum.t .............. ok   
99-test_fuzz_bndiv.t ............... ok   
99-test_fuzz_client.t .............. ok   
99-test_fuzz_cmp.t ................. ok   
99-test_fuzz_cms.t ................. ok   
99-test_fuzz_conf.t ................ ok   
99-test_fuzz_crl.t ................. ok   
99-test_fuzz_ct.t .................. ok   
99-test_fuzz_server.t .............. ok   
99-test_fuzz_x509.t ................ ok   
All tests successful.
Files=224, Tests=3604, 318 wallclock secs ( 6.13 usr  0.38 sys + 291.98 cusr 21.30 csys = 319.79 CPU)
Result: PASS

Also, the OpenSSL FAQ page includes a lot of troubleshooting information under "Questions on Building and Testing OpenSSL"

Simple TLS Server

The OpenSSL wiki provides a "complete implementation of a minimal TLS server". Let's copy it to tlsserver.c in our home directory and try it.

We'll make two small changes to tlsserver.c to make it easier to understand what's happening:

-    const char reply[] = "test\n";
+    const char reply[] = "Hello from Simple TLS Server\n";
        else {
+           printf("served\n");
            SSL_write(ssl, reply, strlen(reply));
        }

Note that the below example is kept as simple as possible and is just for testing and experimenation. Do not use to pass private information.

$ source openssl-env

$ gcc -O0 -g tlsserver.c -o tlsserver -L/opt/openssl/lib -lcrypto -lssl

$ ldd tlsserver
	linux-vdso.so.1 (0x00007ffe80e8d000)
	libcrypto.so.3 => /opt/openssl/lib/libcrypto.so.3 (0x00007f20e157c000)
	libssl.so.3 => /opt/openssl/lib/libssl.so.3 (0x00007f20e12c7000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f20e0ed6000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f20e0cd2000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f20e0ab3000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f20e1cec000)


 ./tlsserver 
140681987502528:error:02001002:system library:fopen:No such file or directory:../crypto/bio/bss_file.c:290:fopen('cert.pem','r')
140681987502528:error:20074002:BIO routines:file_ctrl:system lib:../crypto/bio/bss_file.c:292:
140681987502528:error:140AD002:SSL routines:SSL_CTX_use_certificate_file:system lib:../ssl/ssl_rsa.c:390:

It probably comes as no surprise that we need to set up a private key and certificate. Do the following in the same directory as tlsserver:

$ openssl genrsa -out key.pem 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.
.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.
e is 65537 (0x010001)

$ openssl req -new -x509 -key key.pem -out cert.pem -days 7

Try it again:

$ ./tlsserver

And from a different shell:

$ curl -vv  --insecure https://localhost:4433
* Rebuilt URL to: https://localhost:4433/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 4433 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=US; ST=NJ; O=mind chasers; CN=mindchasers.com
*  start date: Nov 22 18:22:55 2020 GMT
*  expire date: Nov 29 18:22:55 2020 GMT
*  issuer: C=US; ST=NJ; O=mind chasers; CN=mindchasers.com
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET / HTTP/1.1
> Host: localhost:4433
> User-Agent: curl/7.58.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 104
* stopped the pause stream!
* Closing connection 0
* TLSv1.3 (OUT), TLS Unknown, Unknown (21):
curl: (56) OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 104

In the other shell, we see "served", but you can see that curl experienced an error - let's try this again with TLSv1.2:

curl --insecure --tlsv1.2  https://localhost:4433
Hello from Simple TLS Server

Note that we pass curl the --insecure option since we are using a self signed certificate.

Another fun thing to do at this point is to run either Wireshark or tcpdump and watch the exchange of the TLS handshake.

References

Terms / Acronyms

  • CMVP: Cryptographic Module Validation Program
  • FIPS: Federal Information Processing Standards
  • PEM: Privacy-enhanced Electronic Mail is the origin of this acronym, but it doesn't convey that it is a file format for OpenSSL
  • RSA: Popular Asymmetric / Private Key Crypto Algorithm, named after its inventors: Rivest–Shamir–Adleman
  • SSL: Secure Socket Layer
  • TLS: Transport Layer Security

Didn't find an answer to your question? Post your issue below or in our new FORUM, and we'll try our best to help you find a solution.

And please note that we update our site daily with new content related to our open source approach to network security and system design. If you would like to be notified about these changes, then please follow us on Twitter and join our mailing list.

Related articles on this site:

share
subscribe to mailing list:

Please help us improve this article by adding your comment or question:

your email address will be kept private
authenticate with a 3rd party for enhanced features, such as image upload
previous month
next month
Su
Mo
Tu
Wd
Th
Fr
Sa
loading