Editing Installing Lets Encrypt Security Certificates In Elastic Beanstalk Environments

<p class="alert alert-warning">'''The information on this page is outdated.''' Let's Encrypt is no longer needed to install security certificates in Elastic Beanstalk environments. They used to offer free certificates, but the built-in AWS ACM is now free. See [[Elastic_Beanstalk_Security_Certificates|Elastic Beanstalk Security Certificates]] for the latest instructions.</p>

== Overview ==

Instructions on installing and maintaining SSL for AWS Elastic Beanstalk web apps.

Amazon offers its own security certificates for load balanced EC2 instances, but not for smaller stand-alone instances.

[https://letsencrypt.org/ Let's Encrypt] offers free security certificates.

The instructions for installing Let's Encrypt change depending on the platform. E.g. 

* Amazon Linux AMI vs Amazon Linux 2
* nginx server vs Apache server
* Web server for python vs web server for PHP. 

It is important to determine which platform is needed, what its limitations are in terms of supporting Let's Encrypt certificates before proceeding.

== Prerequisites ==

=== Enable SSH on the server ===

It is useful, even''essential'', to be able to issue commands on the server from the command line,

See [[Enabling SSH Connections to an Elastic Beanstalk Environment]]

=== Open HTTPS port (443) ===

See [[Open HTTPS Port For Elastic Beanstalk Environments]]

=== Enable SSL on Elastic Beanstalk environment ===

There are (at least) two major types of EBS platforms Amazon Linux AMI vs Amazon Linux or Amazon Linux 2. 

* TODO: Confirm if PHP is limited to one of these two platforms.
* TODO: Confirm if Python is limited to one of these two platforms.

==== Amazon Linux 2 ====

* SSL library is `mod_ssl`.
* Default web server is nginx? (<-- TODO: confirm)
* PHP platform options appear to be limited to PHP for EBS. (<-- TODO: Confirm)

==== Amazon Linux AMI ====

* SSL library is `mod24_ssl`.

== Installing LE Certificates Manually ==

=== Unresolved questions ===

* Does uploading a new application version for an Elastic Beanstalk environment cause the security certificates and/or HTTPS configurations for that environment to be destroyed?
* Does updating the platform for an Elastic Beanstalk environment cause the security certificates and/or HTTPS configurations for that environment to be destroyed?

=== Process ===

Manually run `certbot` to install Let's Encrypt security certificates for the first time.<ref>[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SSL-on-amazon-linux-2.html#letsencrypt Let's Encrypt with Certbot on Amazon Linux 2] - AWS Eleastic Beanstalk documentation</ref>

<p class="alert alert-warning">`eb deploy` commands will fail if `.ebextensions` commands contain any reference to the certificates and the security certificates have not been installed with Certbot.</p>

==== Prepare to install ====

Download the Extra Packages for Enterprise Linux (EPEL) 7 repository packages. These are required to supply dependencies required by Certbot.

<pre>
$ cd ~
$ sudo wget -r --no-parent -A 'epel-release-*.rpm' https://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/
</pre>

Install the repository packages:

<pre>
$ sudo rpm -Uvh dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-*.rpm
</pre>

Enable EPEL:

<pre>
$ sudo yum-config-manager --enable epel*
</pre>

Now you can confirm that EPEL is enabled:

<pre>
$ sudo yum repolist all

...
epel/x86_64                          Extra Packages for Enterprise Linux 7 - x86_64                               enabled: 12949+175
epel-debuginfo/x86_64                Extra Packages for Enterprise Linux 7 - x86_64 - Debug                       enabled:      2890
epel-source/x86_64                   Extra Packages for Enterprise Linux 7 - x86_64 - Source                      enabled:         0
epel-testing/x86_64                  Extra Packages for Enterprise Linux 7 - Testing - x86_64                     enabled:    778+12
epel-testing-debuginfo/x86_64        Extra Packages for Enterprise Linux 7 - Testing - x86_64 - Debug             enabled:       107
epel-testing-source/x86_64           Extra Packages for Enterprise Linux 7 - Testing - x86_64 - Source            enabled:         0
...
</pre>

==== Install and run Certbot ====

1. Install Certbot packages and dependencies:

<pre>
$ sudo yum install -y certbot python2-certbot-nginx
</pre>

2. Run Certbot

<pre>
$ sudo certbot
</pre>

3. Follow the instructions and enter the appropriate responses at the prompts.

==== Configure automated certificate renewal ====

<syntaxhighlight lang="sh">
    systemctl start crond
    systemctl enable crond
    
    LINE="15 1 */29 * * certbot -q renew; service nginx reload; service nginx restart"
    
    (
        crontab -u root -l
        echo "$LINE"
    ) | crontab -u root -
</syntaxhighlight>

* `15 1 */29` defines the times at which to run the command, e.g. 1:15 am every 29 days
* The tail end uses Certbot to renew the existing cerificates and restarts nginx

2. Restart the cron deamon.

<pre>
$ sudo systemctl restart crond
</pre>

== Installing LE certificates with eb deploy ==

=== Overview ===

* Download and install `certbot` if necessary.
** `certbot` is a utility script that installs Let's Encrypt certificates.
* Check if security certificates are currently installed.
** Not installed.
*** Use `certbot` to install the LE certficates.
*** As part of the installation, `certbot` updates the nginx configuration files to allow SSL and link it to the LE certificates.
** Already installed.
*** Do not run `certbot`.
*** Before deploying the application, save the current nginx configuration file containing certificate configuration.
*** After deploying the application, restore the nginx configuration file.

=== Implementation ===

[https://github.com/HausCloud/AWS-ElasticBeanstalk-SSL This repo] contains EB platform hook scripts that will perform the actions described above. 

There is a discussion about this process in [https://gist.github.com/tony-gutierrez/198988c34e020af0192bab543d35a62a this thread] that started with scripts that used `certbot-auto` which is now deprecated. The repo above is referenced further down in the thread.

The script in `/.platform/hooks/` runs the `certbot` command.

The scripts in `/.platform/confighooks/` save the nginx configuration. If these scripts are not executed then any redeployment of the EB application will overwrite the nginx SSL configuration, causing the web application to be unavailable due to the missing certficates.

== Other Platforms ==

=== Apache ===

See [[Converting Amazon Linux nginx to Apache]]

=== Python ===

[https://blog.lucasferreira.org/howto/2017/07/21/set-up-let-s-encrypt-ssl-certificate-with-aws-elastic-beanstalk-single-instance.html Set Up Let's Encrypt SSL Certificate With AWS Elastic Beanstalk Single Instance] has good clear instructions on how to accomplish this. Also see [https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/https-singleinstance-python.html Terminating HTTPS on EC2 Instances Running Python] in the AWS documentation, which is the basis for that blog post.

Create a config file in `.ebextensions` for SSL and use `eb deploy` to update the environment (this should also work for PHP environments).

With this method the domain names and admin email address are specified with environment variables. This is nice because staging and production can have different secure domain names while sharing the same code base.

To set the values for these variables: '''AWS Management Console''' > '''Elastic Beanstalk''' > ''choose application'' > ''choose environment'' > '''Configuration''' > '''Software''' > '''Modify'''

* `LE_DOMAIN_ARGS` - List of all domains to be included in the certificate. Each domain name should be preceded by the `-d` flag, e.g. `-d mydomain.com -d www.mydomain.com`
* `LETSENCRYPT_DOMAIN` - Primary domain for the purposes of creating a symlink between /etc/letsencrypt/live/ebcert/ and the directory where Let's Encrypt actually places the certificate files.
* `LETSENCRYPT_EMAIL` - Contact email

Reference the [https://github.com/dbarchowsky/north-rose north-rose] project for a working example of this configuration.

== Troubleshooting ==

=== Installation ===

==== Confirm that certificates are installed ====

<pre>
$ ls /etc/letsencrypt/
$ sudo ls /etc/letsencrypt/live/[DOMAIN]
</pre>

==== Confirm certificate expiration date ====

<pre>
$ LETSENCRYPT_DOMAINS=$(/opt/elasticbeanstalk/bin/get-config environment -k LETSENCRYPT_DOMAINS)
$ echo | openssl s_client -connect $LETSENCRYPT_DOMAINS:443 -servername $LETSENCRYPT_DOMAINS 2>/dev/null | openssl x509 -noout -dates
</pre>

Note that the command above only works with a single domain name. Use only one of the domains in `$LETSENCRYPT_DOMAINS` if it contains multiple domains separated by commas.

==== Confirm crontab schedule ====

<pre>
$ sudo crontab -u root -l 
</pre>

See also [[Troubleshooting Elastic Beanstalk Deployment]]

=== Certificates ===

==== URLs for evaluating a domain's SSL ====

* [https://www.ssllabs.com/ssltest/analyze.html Qualys SSL Labs]
* [https://www.sslshopper.com/ssl-checker.html SSL Shopper]
* [https://www.htbridge.com/ssl/ High-Tech Bridge]
* [https://www.digicert.com/help/ digicert]

==== Let's Encrypt security certificates ====

It may be necessary to [[EC2 Security Certificates#Re-installing certificates after upgrading an Elastic Beanstalk instance platform|reinstall any Let's Encrypt security certificates]] if the upgrade process has not copied the `/opt/letsencript/` directory.

==== Error running certbot: "Sorry, I don't know how to bootstrap Certbot on your operating system!" ====

There can be an issue running the `certbot` script (and `certbot-auto`) on Amazon Linux 2 where a line in the script does not correctly identify the OS. <ref>[https://medium.com/@andrenakkurt/great-guide-thanks-for-putting-this-together-gifford-nowland-c3ce0ea2455 Cerbot on Amazon Linux 2] - Medium</ref> <ref>[https://github.com/certbot/certbot/issues/5455 certbot-auto should support Amazon Linux 2] - GitHub</ref>

Edit `/opt/certbot/certbot-auto` to replace this line 

<syntaxhighlight lang="sh">
elif [ -f /etc/redhat-release ]; then
</syntaxhighlight>

with 

<syntaxhighlight lang="sh">
elif [ -f /etc/redhat-release ] || grep 'cpe:.*:amazon_linux:2' /etc/os-release > /dev/null 2>&1; then
</syntaxhighlight>

On the command line this can be achieved with:

<syntaxhighlight lang="sh">
$ sed -i '/elif \[ -f \/etc\/redhat-release \];[[:space:]]*then/c\elif [ -f /etc/redhat-release ] || grep '"'"'cpe:.*:amazon_linux:2'"'"' /etc/os-release > /dev/null 2>&1; then' /opt/certbot/certbot-auto
</syntaxhighlight>

In yaml this can be achieved with:

<syntaxhighlight lang="yaml">
    30_certbot_os_test_fix:
        command: "sed -i '/elif \[ -f \/etc\/redhat-release \];[[:space:]]*then/c\elif [ -f /etc/redhat-release ] || grep '\"'\"'cpe:.*:amazon_linux:2'\"'\"' /etc/os-release > /dev/null 2>&1; then' /opt/certbot/certbot-auto"
</syntaxhighlight>

==== Digicert reports certificate issuer as "Fake LE Intermediate X1" ====

One symptom of this situation are testing the domain's SSL with [https://www.digicert.com/help/ digicert] will result in a report stating that "Certificate Name matches domaininquestion.com", however the issuer will be listed as "Fake LE Intermediate X1". A valid production certificate will have "Let's Encrypt Authority X3" as the issuer.

Another symptom is that the browser will state that the domain's certificate is not from a trusted source, even though everything will look as expected in the `ssl.conf` file and in `/etc/letsencrypt/live/ebcert/`.

==== Certificates are not renewed automatically ====

* SSH to EC2 server hosting the domain covered by the certificate.
* Confirm that `certbot` with `which certbot` on the command line.
* Confirm that the renewal task has been added as a cron tab. View the contents of `/etc/crontab`. See [[#Renewing_certificates| Renewing Certificates]] for the certbot renew command.
* Confirm that nginx configuration has not been removed by an application platform update. See [[#Installing_LE_Certificates_with_nginx|Installing LE Certificates with nginx]].

== Reference ==
=== See also ===

* [https://letsencrypt.org/docs/integration-guide/ Let's Encrypt Integration Guide]
* [[Force_HTTPS_Requests#nginx|Force HTTPS requests]] - Wiki

=== Notes ===
<references />

[[Category:AWS]][[Category:Elastic Beanstalk]][[Category:Web Development]]