Elastic Beanstalk Security Certificates: Difference between revisions

From Littledamien Wiki
Jump to navigation Jump to search
No edit summary
Tag: wikieditor
 
(33 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Overview ==
== Overview ==


Instructions on installing and maintaining SSL for AWS Elastic Beanstalk web apps.
Instructions on how to install security certificates for websites hosted in AWS Elastic Beanstalk environments.


Amazon offers its own security certificates for load balanced EC2 instances, but not for smaller stand-alone instances.
== AWS Certificate Manager ==


[https://letsencrypt.org/ Let's Encrypt] offers free security certificates.
* In the AWS management console, go to '''AWS Certificate Manager (ACM)'''
* Click '''Request a Certificate'''
** '''Certificate Type''': "Request a public certificate"
** '''Fully qualified domain name:''' ''Enter domain name''
** Click ''Add another name to this certificate'' to add any additional subdomains or wildcards to the certificate
** '''Select validation method:''' DNS validation
** Click '''Request''' button
*  The new certificate will be displayed in a list along with its associated domain name.  


The instructions for installing Let's Encrypt change depending on the platform. E.g.
== Validate the certificate’s domains ==


* Amazon Linux AMI vs Amazon Linux 2
Each domain listed on the certificate must be validated to prove the person creating the certificate has control of the domain.
* nginx server vs Apache server
* Web server for python vs web server for PHP.  


It is important to determine which platform is needed, what its limitations are in terms of supporting Let's Encrypt certificates before proceeding.
* Click on the certificate in the ACM list.
* Under '''Domains''', a “Create records in Route 53” button is displayed if the Route 53 is used to manage the domain name.
* Click that button.
* Select the domains to validate.
* Click '''Create Records'''.


== Prerequisites ==
It can take up to 30 minutes for the status of the domain to change from “pending validation” to “issued.”


=== Enable SSH on the server ===
== Apply the certificate to load balancers ==


It is useful, even''essential'', to be able to issue commands on the server from the command line,
A certificate is used to add a "listener" to the load balancer associated with an Elastic Beanstalk instance. The listener routes HTTPS requests to the EBS instance.


See [[Enabling SSH Connections to an Elastic Beanstalk Environment]]
* Go to the '''Elastic Beanstalk management console'''.
* Select an EBS environment.
* Click '''Configuration''' from the menu on the left.
* Select '''Load Balancer''' > '''Edit'''
* Under '''Listeners''', click '''Add Listener'''
** '''Port:''' 443
** '''Protocol:''' HTTPS
** '''SSL Certificate:''' Select the certificate created in ACM
** '''SSL Policy:''' (blank)
** '''Default process:''' (default)
** Click '''Add'''


=== Open HTTPS port (443) ===
<span style="color:red;">Make sure to scroll down to the bottom of the page</span> to click '''Apply'''. I didn’t notice this at first and was wondering why the new listener was disappearing.


See [[Open HTTPS Port For Elastic Beanstalk Environments]]
== Notes ==
=== See Also ===


=== Enable SSL on EC2 instance ===
* Legacy page: [[Installing Lets Encrypt Security Certificates In Elastic Beanstalk Environments]]


There are (at least) two major types of EBS platforms Amazon Linux AMI vs Amazon Linux or Amazon Linux 2.
=== References ===
 
* TODO: Confirm if PHP is limited to one of these two platforms.
* TODO: Confirm if Python is limited to one of these two platforms.
 
==== Amazon Linux 2 ====
 
* SSL library is `mod_ssl`.
* Default web server is nginx? (<-- TODO: confirm)
* PHP platform options appear to be limited to PHP for EBS. (<-- TODO: Confirm)
 
==== Amazon Linux AMI ====
 
* SSL library is `mod24_ssl`.
 
==== Apache server ====
 
The Amazon documentation instructs you to install `mod_ssl` with the following command which should create a file `/etc/httpd/conf.d/ssl.conf` when it completes.
 
Note that applies to Apache servers. Confirm that Apache is serving web requests with
 
<pre>
$ sudo systemctl is-enabled httpd
</pre>
 
If the command returns `disabled` then another server, most likely nginx, is already serving web requests.
 
<syntaxhighlight lang="sh">
$ sudo yum install mod_ssl
</syntaxhighlight>
 
This did not work for me for `dbarchowsky.com` which was on a t1.micro instance, Amazon Linux AMI version 2018.03. What worked instead was:
 
<syntaxhighlight lang="sh">
$ sudo yum install mod24_ssl
</syntaxhighlight>
 
These commands don't need to be entered manually. Instead they should be put into an `.ebextensions` config file so that the site can be deployed without manual configuration.
 
<syntaxhighlight lang="yaml">
packages:
  yum:
    mod_ssl : []
</syntaxhighlight>
 
=== Installing `certbot` ===
 
[https://certbot.eff.org/docs/using.html#certbot-commands Certbot Commands] documentation
 
`certbot` is a command line script that will install Let's Encrypt certificates and configure the current web server to recognize the certificates in order to serve https requests.
 
Let's encrypt certificates typically are stored in `.pem` files located in `/etc/letsencrypt/live/mydomain.com`.
 
Also the server's config file must be updated in order store the location of the certificate files. A typical path for this file is `/etc/httpd/conf.d/ssl.conf`.
 
==== Installing `certbot` from the command line ====
 
<pre>
$ mkdir -p /opt/certbot
$ wget https://dl.eff.org/certbot-auto -O /opt/certbot/certbot-auto
$ chmod a+x /opt/certbot/certbot-auto
</pre>
 
==== Installing `certbot` in `.ebextensinos` ====
 
The above commands likely would require `sudo` on the command line. The best location for this configuration is in a `.ebextensions` config file with something like this:
 
<syntaxhighlight lang="yaml">
  20_install_certbot:
    command: "mkdir -p /opt/certbot &amp;&amp; wget https://dl.eff.org/certbot-auto -O /opt/certbot/certbot-auto &amp;&amp; chmod a+x /opt/certbot/certbot-auto"
</syntaxhighlight>
 
== Generating LE Certificates ==
 
=== Running certbot-auto ===
 
The following is an example `certbot-auto` command.
 
<pre>
$ sudo /opt/certbot/certbot-auto certonly --standalone --debug --non-interactive --email ${LETSENCRYPT_EMAIL} --agree-tos -d ${LETSENCRYPT_DOMAIN} -d www.${LETSENCRYPT_DOMAIN} --expand --renew-with-new-domains --pre-hook "service nginx stop"
</pre>
 
Different authentication methods can be specified. The method above creates a challenge file in the web server directory, then makes a http request for that file in order to confirm there is legitimate access to the website. This means that the domain name must point to the server where `certbot-auto` is running.
 
=== Installing LE Certificates with nginx ===
 
The following instructions originate from [https://piratefache.ch/lets-encrypt-with-amazon-elastic-beanstalk/ Let's Encrypt with AWS Elastic Beanstalk] by “PirateFaché”. This installs LE certificates on nginx.
 
<p class="alert alert-warning">The following directives in the .ebextensions file will ''NOT'' change the configuration of the nginx server. See [[#Cannot_create_files_in_.2Fetc.2Fnginx.2Fconf.d_with_.ebextensions|Troubleshooting]] for details.</p>
 
Creating `/etc/nginx/conf.d/https_custom.conf` allows requests to port 443 to access the LE certificates
 
<syntaxhighlight lang="yaml">
# HTTPS server
server {
    listen      443 default ssl;
    server_name  localhost;
    error_page  497 https://$host$request_uri;
 
    ssl_certificate      /etc/letsencrypt/live/ebcert/fullchain.pem;
    ssl_certificate_key  /etc/letsencrypt/live/ebcert/privkey.pem;
 
    ssl_session_timeout  5m;
    ssl_protocols  TLSv1.1 TLSv1.2;
    ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
    ssl_prefer_server_ciphers  on;
 
    # Include the Elastic Beanstalk generated locations
    include conf.d/elasticbeanstalk/*.conf;
}
</syntaxhighlight>
 
It’s necessary to create a symbolic link from the “ebcert” directory specified in the file above with the actual directory containing the LE certificates:
 
<pre>
$ sudo ln -sf /etc/letsencrypt/live/wiki.dbarchowsky.com /etc/letsencrypt/live/ebcert
</pre>
 
Restart nginx
 
<pre>
$ sudo systemctl restart nginx
</pre>
 
At this point it should be possible to make https requests to the site:
 
<pre>
$ curl https://mydomain.com
</pre>
 
=== TLS 1.2 ===
 
Note that this line in `/etc/nginx/conf.d/https_custom.conf`
 
<pre>
ssl_protocols  TLSv1.1 TLSv1.2;
</pre>
 
Allows for both TLS 1.1 and TLS 1.2. TLS is the most current protocol (as of mid-2020) and addresses security issues with TLS 1.1. When confirming the certificate with [https://www.ssllabs.com/ssltest/ SSL Labs] this will result in a “B” grade only because TLS 1.1 is allowed. If you scroll down on the report TLS 1.2 is still supported. It looks like requests should still be handled if the client is limited to TLS 1.2.
 
=== Other Reference ===
 
* [https://pyliaorachel.github.io/blog/tech/system/2017/07/14/nginx-server-ssl-setup-on-aws-ec2-linux-with-letsencrypt.html Nginx Server SSL Setup on AWS EC2 Linux with Letsencrypt]<br />This covers some of the same territory as the link at the top of this section. Some configuration examples using Let’s Encrypt and Amazon Linux 2
* [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SSL-on-amazon-linux-2.html Tutorial: Configure SSL/TLS on Amazon Linux 2]<br />This steers you to using apache, which is not running by default on an Amazon Linux 2 server by default. There is some information that helps understand how these servers are set up to process security certificates and https requests.
* [https://www.tutcodex.com/ssl-on-single-instance-elastic-beanstalk-tutorial/  SSL On Single Instance Elastic Beanstalk Tutorial]<br />Ultimately wasn’t helpful as it was configuring an Apache server. The basic steps are covered with working examples.
* [https://undebugable.blogspot.com/2019/07/setup-aws-amazon-elastic-beanstalk-to.html Setup (AWS) Amazon Elastic Beanstalk to work with Let's Encrypt]<br />Another Apache example]
 
== Other Platforms ==
 
=== Converting EBS nginx platform to apache ===
 
[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SSL-on-amazon-linux-2.html Tutorial: Configure SSL/TLS on Amazon Linux 2] - AWS Documentation
 
The page above links to Tutorial: [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SSL-on-amazon-linux-2.html Install a LAMP Web Server on Amazon Linux 2] which describes installing an Apache web server on Amazon Linux 2.
 
As a part of those instructions, this diagnostic was provided to confirm the state of the httpd service:
 
<pre>
$ systemctl status -l httpd.service
</pre>
 
Which returned these lines:
 
<pre>
May 30 04:12:27 ip-172-31-39-99.us-west-2.compute.internal httpd[16139]: AH00526: Syntax error on line 10 of /etc/httpd/conf.d/ssl.conf:
May 30 04:12:27 ip-172-31-39-99.us-west-2.compute.internal httpd[16139]: SSLCertificateFile: file '/etc/letsencrypt/live/LETSENCRYPT_DOMAIN/fullchain.pem' does not exist or is empty
</pre>
 
“LETSENCRYPT_DOMAIN” should be the actual name of the domain that was used to create the security certificate. See below, [#Accessing_EBS_environment_variables_from_the_command_line]
 
This page, [https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.migration-al.html Migrating your Elastic Beanstalk Linux] application to Amazon Linux 2, contains information about the difference between Amazon Linux and Amazon Linux 2, and some of the properties of Amazon Linux 2 that impact SSL.
 
<pre>
$ sudo systemctl stop nginx
</pre>
 
Then try running Apache.
 
Ok, after stopping nginx and then starting apache, it’s possible to get a response making a https request to the server.
However, now in the EBS console the server’s status is listed as “degraded”.
 
Next question: how to put all of this into .ebextensions config files.
 
Also: make sure that the Let’s Encrypt certificate renews automatically.
 
=== Python ===
 
[https://blog.lucasferreira.org/howto/2017/07/21/set-up-let-s-encrypt-ssl-certificate-with-aws-elastic-beanstalk-single-instance.html Set Up Let's Encrypt SSL Certificate With AWS Elastic Beanstalk Single Instance] has good clear instructions on how to accomplish this. Also see [https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/https-singleinstance-python.html Terminating HTTPS on EC2 Instances Running Python] in the AWS documentation, which is the basis for that blog post.
 
Create a config file in `.ebextensions` for SSL and use `eb deploy` to update the environment (this should also work for PHP environments).
 
With this method the domain names and admin email address are specified with environment variables. This is nice because staging and production can have different secure domain names while sharing the same code base.
 
To set the values for these variables: '''AWS Management Console''' > '''Elastic Beanstalk''' > ''choose application'' > ''choose environment'' > '''Configuration''' > '''Software''' > '''Modify'''
 
* `LE_DOMAIN_ARGS` - List of all domains to be included in the certificate. Each domain name should be preceded by the `-d` flag, e.g. `-d mydomain.com -d www.mydomain.com`
* `LETSENCRYPT_DOMAIN` - Primary domain for the purposes of creating a symlink between /etc/letsencrypt/live/ebcert/ and the directory where Let's Encrypt actually places the certificate files.
* `LETSENCRYPT_EMAIL` - Contact email
 
Reference the [https://github.com/dbarchowsky/north-rose north-rose] project for a working example of this configuration.
 
== Renewing certificates ==
 
=== Cron job ===
 
Let's Encrypt certificates expire every 90 days. Create a cron job (in `/etc/crontab`) that will run once per day to check and renew the certificates as necessary.
 
Follow the renewal of the certificate with a restart of the Apache server in order to have the server recognize any renewed certificates.
 
<pre>
# Renew SSL Certs
0  1  *  *  *  ec2-user  /opt/letsencrypt/letsencrypt-auto --no-bootstrap renew
 
# Refresh Server
10 1  *  *  *  root      apachectl -k restart > /dev/null 2>&1
</pre>
 
=== Manually ===
 
Certificates can be manually forced to renew with 
 
<syntaxhighlight lang="sh">
$ /opt/certbot/certbot-auto renew --pre-hook "service nginx stop" --post-hook "service nginx start"
</syntaxhighlight>
 
== Troubleshooting ==
 
=== Installation ===
 
==== eb deploy failure ====
 
View the tail of the file `/var/log/cfn-init-cmd.log`. This file will list all commands in `.ebextensions` and whether they executed successfully or not.
 
Also can check the `.ebextensions` config files on the server in `/var/app/staging/`
 
==== Directives in .ebextensions config files aren't executed ====
 
AWS eb cli uses git HEAD to create zip file to upload to the server.
 
Confirm that zip files have been added and committed to the repo.
 
Or to deploy changes before they are committed:
 
<pre>
$ eb deploy --staged
</pre>
 
==== Cannot create files in /etc/nginx/conf.d with .ebextensions ====
 
According to this Stackoverflow thread, [https://stackoverflow.com/questions/24812375/websockets-on-elastic-beanstalk-with-docker WebSockets on Elastic Beanstalk with Docker], it seems that when EBS creates an application it basically clears out the nginx configuration after the .ebextensions commands are run. So any custom ngnix configuration done through .ebextensions would be overwritten.
 
I have confirmed this insofar as I put my nginx configuration in a file and uploaded it successfully to the ec2-user home directory. I put in another command to move that file to the nginx configuration directory, and after the application was successfully deployed, the custom nginx configuration file was gone.
 
There were some solutions offered on the Stackoverflow thread above. They involved moving Python scripts to an EBS “hooks” directory which would be executed after the application is deployed. There is no “hooks” directory in that location on my EBS server.
For the time being, I am manually creating the nginx config file on the command line on the server after the application is deployed. This will allow the server to use the Let’s Encrypt certificates to serve https requests, and should stay in place through LE certificate renewals until the next application deployment.
 
The AWS documentation assumes that you generate the certificates manually and insert the contents of the certificate in the `.ebextensions` config file. The alternative example above places the command to generate the certificates in the `.ebextensions` config file. There is a flag that is incompatible with production environments: `--staging`. This will cause the Let's Encrypt staging server to issue the certificates. The server address will also get stored in a local config file, so subsequent attempts to reissue the certificates without the `--staging` flag will still invoke that staging server. <ref>[https://community.letsencrypt.org/t/cn-fake-le-intermediate-x1/13437/4 CN=Fake LE Intermediate X1] - Let's Encrypt forums</ref>
 
==== systemctl command not found ====
 
The AWS documentation uses `systemctl` to restart the Apache server. If this command is not available use the `service` command instead.
 
==== Cannot find SSLCertificateFile directive ====
 
When running `letsencrypt-auto` or `certbot-auto`
 
<pre>
Cannot find an SSLCertificateFile directive in /files/etc/httpd/conf/httpd-le-ssl.conf/IfModule/VirtualHost. VirtualHost was not modified
Unable to find an SSLCertificateFile directive
</pre>
 
This was fixed by successfully installing `mod_ssl`
 
==== Re-installing certificates after upgrading an Elastic Beanstalk instance platform ====
 
When [[AWS Elastic Beanstalk#Upgrading the platform of an Elastic Beanstalk instance|updating the platform of an Elastic Beanstalk instance]] (e.g. Linux 2.0.x > Linux 2.9.x), the `/opt/letsencript/` directory is not copied over to the new EB instance.
 
It is necessary to go through all the installation steps for [[EC2_Security_Certificates#Prerequisites|Let's Encrypt]] and a [[EC2_Security_Certificates#Installing_certificates|new Let's Encrypt certificate]].
 
=== Connectivity ===
 
==== ERR_CONNECTION_REFUSED in Chrome ====
 
Attempting to load the site using https protocol in Chrome results in ERR_CONNECTION_REFUSED error.
 
Check the security certificates in `/etc/letsencrypt/live/`. There should be a directory with the name of the domain, and another directory named `ebcert` that is a symbolic link to `/etc/letsencrypt/live/securedomainname.com`
 
Check that the server is configured to accept requests on port 443, e.g. in `/etc/nginx/conf.d/https_custom.conf`
 
<p class="alert alert-warning">nginx configuration is set back to defaults during `eb deploy`, meaning the certificates configuration is removed from the server. Also, haven't figured out how to insert custom configuration on the server via .ebextensions configuration directives. It may be necessary to copy this https configuration file manually after running `eb deploy`.</p>
 
=== Certificates ===
 
==== URLs for evaluating a domain's SSL ====
 
* [https://www.ssllabs.com/ssltest/analyze.html Qualys SSL Labs]
* [https://www.sslshopper.com/ssl-checker.html SSL Shopper]
* [https://www.htbridge.com/ssl/ High-Tech Bridge]
* [https://www.digicert.com/help/ digicert]
 
==== Let's Encrypt security certificates ====
 
It may be necessary to [[EC2 Security Certificates#Re-installing certificates after upgrading an Elastic Beanstalk instance platform|reinstall any Let's Encrypt security certificates]] if the upgrade process has not copied the `/opt/letsencript/` directory.
 
==== Error running certbot: "Sorry, I don't know how to bootstrap Certbot on your operating system!" ====
 
There can be an issue running the `certbot` script (and `certbot-auto`) on Amazon Linux 2 where a line in the script does not correctly identify the OS. <ref>[https://medium.com/@andrenakkurt/great-guide-thanks-for-putting-this-together-gifford-nowland-c3ce0ea2455 Cerbot on Amazon Linux 2] - Medium</ref> <ref>[https://github.com/certbot/certbot/issues/5455 certbot-auto should support Amazon Linux 2] - GitHub</ref>
 
Edit `/opt/certbot/certbot-auto` to replace this line
 
<syntaxhighlight lang="sh">
elif [ -f /etc/redhat-release ]; then
</syntaxhighlight>
 
with
 
<syntaxhighlight lang="sh">
elif [ -f /etc/redhat-release ] || grep 'cpe:.*:amazon_linux:2' /etc/os-release > /dev/null 2>&1; then
</syntaxhighlight>
 
On the command line this can be achieved with:
 
<syntaxhighlight lang="sh">
$ sed -i '/elif \[ -f \/etc\/redhat-release \];[[:space:]]*then/c\elif [ -f /etc/redhat-release ] || grep '"'"'cpe:.*:amazon_linux:2'"'"' /etc/os-release > /dev/null 2>&1; then' /opt/certbot/certbot-auto
</syntaxhighlight>
 
In yaml this can be achieved with:
 
<syntaxhighlight lang="yaml">
    30_certbot_os_test_fix:
        command: "sed -i '/elif \[ -f \/etc\/redhat-release \];[[:space:]]*then/c\elif [ -f /etc/redhat-release ] || grep '\"'\"'cpe:.*:amazon_linux:2'\"'\"' /etc/os-release > /dev/null 2>&1; then' /opt/certbot/certbot-auto"
</syntaxhighlight>
 
==== Digicert reports certificate issuer as "Fake LE Intermediate X1" ====
 
One symptom of this situation are testing the domain's SSL with [https://www.digicert.com/help/ digicert] will result in a report stating that "Certificate Name matches domaininquestion.com", however the issuer will be listed as "Fake LE Intermediate X1". A valid production certificate will have "Let's Encrypt Authority X3" as the issuer.
 
Another symptom is that the browser will state that the domain's certificate is not from a trusted source, even though everything will look as expected in the `ssl.conf` file and in `/etc/letsencrypt/live/ebcert/`.
 
== Reference ==
=== See also ===
 
* [https://letsencrypt.org/docs/integration-guide/ Let's Encrypt Integration Guide]
* [[Force_HTTPS_Requests#nginx|Force HTTPS requests]] - Wiki
 
=== Notes ===
<references />
<references />


[[Category:AWS]][[Category:Elastic Beanstalk]][[Category:Web Development]]
[[Category:AWS]][[Category:Elastic Beanstalk]][[Category:Web Development]]

Latest revision as of 20:48, 21 August 2022

Overview[edit]

Instructions on how to install security certificates for websites hosted in AWS Elastic Beanstalk environments.

AWS Certificate Manager[edit]

  • In the AWS management console, go to AWS Certificate Manager (ACM)
  • Click Request a Certificate
    • Certificate Type: "Request a public certificate"
    • Fully qualified domain name: Enter domain name
    • Click Add another name to this certificate to add any additional subdomains or wildcards to the certificate
    • Select validation method: DNS validation
    • Click Request button
  • The new certificate will be displayed in a list along with its associated domain name.

Validate the certificate’s domains[edit]

Each domain listed on the certificate must be validated to prove the person creating the certificate has control of the domain.

  • Click on the certificate in the ACM list.
  • Under Domains, a “Create records in Route 53” button is displayed if the Route 53 is used to manage the domain name.
  • Click that button.
  • Select the domains to validate.
  • Click Create Records.

It can take up to 30 minutes for the status of the domain to change from “pending validation” to “issued.”

Apply the certificate to load balancers[edit]

A certificate is used to add a "listener" to the load balancer associated with an Elastic Beanstalk instance. The listener routes HTTPS requests to the EBS instance.

  • Go to the Elastic Beanstalk management console.
  • Select an EBS environment.
  • Click Configuration from the menu on the left.
  • Select Load Balancer > Edit
  • Under Listeners, click Add Listener
    • Port: 443
    • Protocol: HTTPS
    • SSL Certificate: Select the certificate created in ACM
    • SSL Policy: (blank)
    • Default process: (default)
    • Click Add

Make sure to scroll down to the bottom of the page to click Apply. I didn’t notice this at first and was wondering why the new listener was disappearing.

Notes[edit]

See Also[edit]

References[edit]

Retrieved from "https://wiki.dbarchowsky.com/index.php?title=Elastic_Beanstalk_Security_Certificates&oldid=3365"