Skip to content

Lab 7 - Transport Layer Security (TLS)

Nowadays, everything is on the internet. Not only cat pictures but also financial institutions, mail services, schools, and even the government.

The problem is, that as a person, you usually do not want someone else to read your private emails or access your bank account. Especially if that someone has malicious intent.

This is entirely the job of System Administrators. They try to protect the privacy of data. Thankfully, to make their job easier, mathematicians have come up with a technology called asymmetric cryptography.

Asymmetric cryptography allows you to generate a pair of certificates - usually called certificate and key. The key is secret, you never let anyone know the key. But you do let them know the certificate.

Then, when anyone wants to send you anything private, they take your certificate and encrypt the data they want to send with this certificate. Now, after encryption, the only way to decrypt this data is using your key. This is the same idea SSH keys use.

The same concept can be used, when you are hosting your services. You can take a certificate/key pair, and tell the Apache web server to use this pair. This means, that any time someone connects to Apache, they are given a public certificate to encrypt their information with. When they encrypt and send their data to the server, the server takes the private key and decrypts the data. This allows you to exchange data with a server while remaining completely safe from prying eyes.

For example, you can see Google's public certificate by doing: openssl s_client -connect google.com:443 Under the server certificate section, you can see the public certificate.

Now that certificates have entered the picture, there is yet another security issue. If everyone could generate a certificate/key pair for any web service, then it would be very easy to fake a web server, for example, a bank. To prevent this, these certificate/key pairs are made in the following steps:

  • Generate a private key
  • Create a certificate request based on this key
  • Send the certificate request to a Certificate Authority
  • Validate why you should be able to generate a certificate for this domain
  • Done over email, phone, DNS secret, or HTTP secret
  • Wait until you receive your email and start using it

This process allows a client to check if the certificate/key pair is trusted. Most computer systems have a built-in Certificate Authority list with their keys, and can cryptographically check the validity of all certificate/key pairs. Even your browser has one built-in, and you can check it.

Because this process takes time, and/or costs money (CAs like Digicert, Namecheap, etc.) or requires a service on the public web (Lets Encrypt), then we are going to use a CA made by the teachers, and we will sign our own certificates.

Signing a certificate in the scoring server

For our own little Certificate Authority, we have set up a service named vault in our scoring system.

Vault is a secrets' engine, capable of securely storing secrets and generating them if necessary.

The main access point for vault is the web GUI - https://scoring.sa.cs.ut.ee:8200/ui/

For the authentication method, choose LDAP and log in with your University of Tartu central credentials.

You should see that a secrets engine named pki_int (PKI: Public Key Infrastructure, INT: Shorthand for intermediate) is available to you, and under that field, you can see a field for signing certificates under the domain sa.cs.ut.ee.

Complete

To sign a certificate, you need to provide some additional data

  • Under Common name provide the Fully Qualified Domain Name(FQDN) for your certificate.
    • For ex. for the web domain we would provide www.<vm-name>.sa.cs.ut.ee
  • Under Format make sure it is set to pem

There is nothing for you to change under the extra options part, the default TTL of the certificate is set to 1 year and should be sufficient for this course.

After generating the certificate, make sure you save each part - once you close the window, vault does not display the data for you anymore.

If you loose the data or forget to save it, just generate a new certificate - in this course they are free.

For saving the certificate, key, and chain certificate individually copy-pasting is probably easiest. For this, you can copy the data with the small clipboard icon (next to the eye). In the bottom there is also a button 'Copy credentials' that copies the data in json format, but then you'll have to format each piece yourself.

The most common location for saving files related to certificates is /etc/pki/tls/ and usually filenames are prepended with the domain of the certificate

  • Certificate file /etc/pki/tls/certs/www_server.crt
  • Private key is the SSLCertificateKeyFile file /etc/pki/tls/private/www_server.key
  • Since the CA chain is the same for each domain (we only have 1 CA in this lab), you can keep a generic name
    • SSLCACertificateFile /etc/pki/tls/certs/cacert.crt
    • NB! The chain Vault gives back has the certificates in the wrong format (separated by a comma).
    • Find out what the correct format is. The name of the format is .pem.
  • You won't need the issuing CA certificate.

You can do this for every virtual host (there are 3 of them not including postfix). But you can also make a general name certificate bundle using *.<vm-name>.sa.cs.ut.ee for the Common name .

TLS in Apache (HTTPS)

Using certificate/key pair in Apache is fairly straightforward, but first, we have to generate our certificates.

The thing to remember is that the HTTPS uses port 443 instead of 80. You should also verify that your firewall rules reflect this.

TLS related site configuration directives can initially be found in the /etc/httpd/conf.d/ssl.conf file. We will use this as the template and create the configuration for the virtual hosts in their appropriate site configuration files.

You should familiarize yourself with the following Apache httpd configuration directives before moving to the next task

Now, let's enable TLS with our web services.

Complete

  • Install the mod_ssl package using the package manager.
    • This package is an httpd web server module, that allows it to deal with HTTPS connections.
    • For each of the virtual hosts you made in the webserver lab, do the following:
  • In the virtual host file, duplicate the listening virtual host.
  • Instead of the new virtual host listening on port 80, set it to listen on port 443 (HTTPS port).
  • Add an SSLEngine directive, and set it to on.
  • Add the following SSL settings:
    SSLCertificateFile /etc/pki/tls/certs/www_server.crt
    SSLCertificateKeyFile /etc/pki/tls/private/www_server.key
    SSLCACertificateFile /etc/pki/tls/certs/cacert.crt
    
  • Repeat this for each of the 3 virtual hosts.
  • Make sure to open port 443 on all firewalls.
  • Restart your webserver.

Verify

Now you should be able to access your HTTPS virtual host in the web browser:

  • Open for example https://www.<vm_name>.sa.cs.ut.ee with your browser
    • you must create a security exception for your self-signed certificate
    • This is intended, as we have not imported our private CA's root certificate to our browsers yet, so it does not know to trust this web page.
    • Nevertheless, you should see the same content as on the http://www.<your-domain>.sa.cs.ut.ee when you accept the exception.
  • To remove the security exception, you must download cacert.crt to your host machine (not your VM) and then import it into a browser.

  • In Firefox import cacert.crt by Edit -> Preferences -> Advanced -> Certificates -> View Certificates -> Authorities -> Import -> Select File -> ...

  • In Chrome Setting -> Show advanced settings ... -> Manage certificates -> trusted Root Certificate Authorities -> Import -> Next -> File name: cacert.crt ; type: All Files -> Next -> Place all certificates in the following store; Certificate store: Trusted Root Certification Authorities -> Finish -> Yes -> OK

  • Close browser and try again (error "Not Secure" should be removed.

    • In order to see Certificate info in Chrome press CTRL+SHIFT+i and in the windows that opens choose security tab and View certificate -> Details.
  • Now try accessing the HTTPS site again, it should not throw any errors since we added our CA into the browser to recognize.
    • If the other virtual hosts also work like https://proxy.<vm_name>.sa.cs.ut.ee and https://wordpress.<vm_name>.sa.cs.ut.ee, then you have configured them correctly.

TLS in Postfix

Last week we set up Postfix in a non-secure way. Anybody either in the client's network, or your personal VM's network, can eavesdrop on network traffic, and see any passwords and/or emails your email server sends or receives.

To remediate that, the solution is, again, using TLS. Enabling TLS means the following in Postfix:

Complete

Generate a new certificate for postfix mail server mail.<vm_name>.sa.cs.ut.ee. Add TLS for the mail server virtualhost the same way it was added for other virtual hosts in the previous task.

In postfix main.cf configuration file, change the following:

smtpd_tls_security_level = may
smtpd_tls_cert_file=/etc/pki/tls/certs/postfix.pem
smtpd_tls_key_file=/etc/pki/tls/private/postfix.key
smtpd_tls_loglevel = 1
smtp_tls_loglevel = 1

Also, copy the certificate and key in place of the files we specified above, or sign a new one and copy that.

Complete

In postfix master.cf configuration file: - Find the ''submission'' port declaration, it should look something like this:

submission inet n       -       n       -       -       smtpd
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_tls_auth_only=yes
-o smtpd_reject_unlisted_recipient=no
-o smtpd_client_restrictions=$mua_client_restrictions
-o smtpd_helo_restrictions=$mua_helo_restrictions
-o smtpd_sender_restrictions=$mua_sender_restrictions
-o smtpd_recipient_restrictions=
-o smtpd_relay_restrictions=permit_sasl_authenticated,reject
-o milter_macro_daemon_name=ORIGINATING
(Some of it might be disabled from the previous lab.)

  • Enable the ''submission'' block (uncomment the header line)
  • In the options list (lines starting with -o) in the submission block make sure the following options are enabled:
    -o syslog_name=postfix/submission
    -o smtpd_tls_security_level=encrypt
    -o smtpd_sasl_auth_enable=yes
    -o smtpd_sasl_path=private/auth
    -o smtpd_sasl_security_options=noanonymous
    -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
    -o milter_macro_daemon_name=ORIGINATING
    
    (In case some of the options are missing, you should add them)

Now let's also enable secure submission over explicit TLS (SMTPS). We will use the same limitation policies as in the ''submission'' block

Complete

Again in the master.cf file. - Find the ''smtps'' port declaration, it should look something like this:

#smtps     inet  n       -       n       -       -       smtpd
#  -o syslog_name=postfix/smtps
#  -o smtpd_tls_wrappermode=yes
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING

(It is disabled by default)

  • Enable the ''smtps'' block (uncomment the header line)
  • In the options list (lines starting with -o) in the submission block make sure the following options are enabled:
  • -o syslog_name=postfix/smtps
  • -o smtpd_tls_wrappermode=yes
  • '''SMTPS connection runs in TLS pipe (See submission block options) so we do not have to enroll any in-line TLS encryption, therefore we omit the tls_security_level option here.'''
  • The rest of the options you may configure similarly to the submission block

    -o smtpd_sasl_auth_enable=yes
    -o smtpd_sasl_path=private/auth
    -o smtpd_sasl_security_options=noanonymous
    -o smtpd_client_restrictions=permit_sasl_authenticated,reject
    -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
    -o milter_macro_daemon_name=ORIGINATING
    

  • Save the file

  • To fix a new issue caused by a postfix update, install the package cyrus-sasl-plain
  • Restart the postfix service
  • Make sure to open ports 465 and 587 in all the firewalls.

Verify

You can check whether your changes work, by doing the following:

  • openssl s_client -connect mail.<vm_name>.sa.cs.ut.ee:587 -starttls smtp -CAfile <path_to>/cacert.crt
  • openssl s_client -connect mail.<vm_name>.sa.cs.ut.ee:465 -CAfile <path_to>/cacert.crt

The most important part is the Verify return code field. If that is 0, everything is working, and CA validation also works.

If it is complaining about "self signed certificate", then either the CA certificate is not working, you're using the wrong certificate, or something else is wrong.

TLS in Dovecot

Encrypting the traffic of the mail server itself is useless if the authentication credentials can be still seen by malicious people when logging into the mail server.

This is why we also need to encrypt traffic in Dovecot.

Complete

  • In the /etc/dovecot/conf.d/10-ssl.conf file, set the following parameters:
    ssl = yes
    ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
    ssl_key = </etc/pki/dovecot/private/dovecot.pem
    
  • Also make sure to copy the certificate into those files, again, or generate a new certificate.
  • Restart your dovecot service.
  • Open port 993 in firewalls.

Verify

You can check whether everything works again, by utilizing openssl command:

  • openssl s_client -connect mail.<vm_name>.sa.cs.ut.ee:993 -CAfile <path_to>/cacert.crt

The most important part is the Verify return code field. If that is 0, everything is working, and CA validation also works.

If it is complaining about "self signed certificate", then either the CA certificate is not working, you're using the wrong certificate, or something else is wrong.

Self-signed certificates in a nutshell Self-signed certificates in a nutshell

Source for the image. Published on 11.01.2017 by E.William in The Barbed Wire.

Wireshark

This part is not mandatory, but it will help you to understand why we were doing what we were doing.

We will try to see if we can sniff some traffic.

First, go install yourself a tool called wireshark. (To your personal computer)

  • install it from your package manager
  • e.g. yum install wireshark
  • brew install wireshark

Once you have it installed, start it up with the highest permissions in your machine (Administrator in Windows, root in Linux and Mac).

You should get a window like this:

Wireshark interface

From there, you need to choose the proper interface. This depends on which interface your University VPN tunnel is working. If you hit the correct interface and double-click it, you should be getting a lot of traffic on the screen. If not, you're using the wrong interface.

Wireshark traffic

Then, instead of the "Apply a display filter" line, write the following: ip.dst == <your_vm_ip> and (http or tls) And press enter.

After having done that, go and access the web pages on your machine. Use both http:// and https:// pages. You should be getting traffic on Wireshark.

After having done that, go and click on the traffic lines. See how different is TLS and HTTP information. With HTTP, you can see all the information that was exchanged. With TLS you cannot.

Just HTTP traffic: Wireshark traffic

Versus TLS traffic: Wireshark traffic

You can also try it with other pages on the world wide web, just remove the ip.dst filter.

Ansible and TLS

Putting the whole lab into Ansible is not something that would result in an idempotent playbook. So we will list the things that can be automated in an idempotent way, but some parts of the labs will require manual intervention by the user. The following suggestions and steps are just guidelines, if you think you can do better, feel free to do so.

Complete

  • Creating directories CA, certs, crl, newcerts, and private can and should be written into ansible.
  • Creating the file serial with content can be copied over with copy module same goes for index.txt file.
  • openssl.cnf is a file that should be templated with Jinja2.
  • For signing the certificate in vault, you could try playing around with the Ansible module hashi_vault, or use the HTTPS API on https://scoring.sa.cs.ut.ee:8200/v1, but it is most likely easier to do it manually.

Just because it is reasonable to do some bits manually doesn't mean you can't still use the playbook even if manual intervention is expected in the middle of the playbook. Read about the pause module from ansible. This allows you to stop your playbook execution, do things manually on the host in the meantime and then continue executing the playbook by simply pressing Enter in the terminal. This pause module is an option for issuing the openssl commands from the lab manual.

Complete

  • After the manual part one can use the Copy module to copy the newly created certificates and keys around. Figure out how to move files inside the remote host.
  • Installing a package with ansible is given of course.
  • Use the file module (or template) for the Virtualhost files for Httpd. Restarting a service with Ansible is also a given.
  • Using file or template module on main.cf and master.cf for Postfix.
  • Using file or template module for 10-ssl.conf for Dovecot.
  • Opening necessary ports with ansible.
  • Use the pause module for the openssl check commands.

Keep your playbook safe

As per usual always push your playbook to the course's Gitlab.

Complete

  • In your ansible playbook directory:
  • git add .
  • git commit -m "TLS lab"
  • git push -u origin master

Verify

Go to your GitLab page to see whether all of your latest pushes reached the git repository. If you play around with your repository and have made changes to the ansible that you wish to utilize also in the future, always remember to commit and push them.