Settings to deploy to AWS Elastic Beanstalk on CircleCi (EB Cli 3)

deploy_with_ebcli3_on_circleci.md

This is how I configured the deploy of my rails apps to AWS Elastic Beanstalk through CircleCI 1.0.

If you are using the Circle CI 2.0, take a look at this article from ryansimms

Configure Environments Variables

On Project Settings > Environment Variables add this keys:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
  The aws user must have the right permissions. This can be hard, maybe, this can help you.

Create a bash script to create the eb config file

./setup-eb.sh

set -x
set -e

mkdir /home/ubuntu/.aws
touch /home/ubuntu/.aws/config
chmod 600 /home/ubuntu/.aws/config
echo "[profile eb-cli]" > /home/ubuntu/.aws/config
echo "aws_access_key_id=$AWS_ACCESS_KEY_ID" >> /home/ubuntu/.aws/config
echo "aws_secret_access_key=$AWS_SECRET_ACCESS_KEY" >> /home/ubuntu/.aws/config

Configure circle.yml

Add the awsebcli dependency:

dependencies:
  pre:
    - sudo apt-get update
    - sudo apt-get install python-dev
    - sudo pip install awsebcli

Add the deployment config:

deployment:
  production:
    branch: master
    commands:
      - bash ./setup-eb.sh
      - eb deploy

• If your deploy user don't have the elasticbeanstalk:DescribeEvents permission, the eb deploy will run for ever. CircleCI will cancel it after 10 minutes and break the build with timeout.

Create the EB Cli config file

eb init will create this file for you. However, if you don't want to run it, you can simply create and configure this file manualy:

./elasticbeanstalk/config.yml

branch-defaults:
  master:
    environment: you-environment-name
global:
  application_name: your-application-name
  default_ec2_keyname: ec2-key-pair-name
  default_platform: 64bit Amazon Linux 2015.03 v1.4.3 running Ruby 2.2 (Puma)
  default_region: sa-east-1
  profile: eb-cli
  sc: git

Thanks for this! After adjusting the permissions on my policy for ages, I finally managed to get my builds passing using both codeship and Span CI, but they still just left EB hanging indefinitely. Your IAM policy fixed the issue on both.

Thanks for sharing this. Really helpful!

Thanks! Very useful! I realized that if you fill the AWS KEY and SECRET in the CircleCI site, you can remove the bash ./setup-eb.sh and just use the command eb deploy --profile default because CircleCI creates this profile automatically

deployment:
  production:
    branch: master
    commands:
      - eb deploy --profile default

Thanks for this gist - the most straightforward solution to simply having Circle automate the EB deploy. Also +1 for euharrison's comment about the KEYS in yoru circle setup.

tried to deploy using artifacts but the eb config couldn't understand the $CIRCLE_ARTIFACT environment variable. It would take it as a string instead of a variable

This has been working fine for a few months, but since yesterday evening fails on sudo pip install awsebcl', with the error OSError: [Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/setuptools-3.4.4-py2.7.egg'. Have any of you seen this problem?

I did, just replace sudo pip install awsebcli by sudo pip install awsebcli --ignore-installed setuptools

@tomw I only just started getting this issue, as well. I wonder what changed?

They released a new version and PIP is failing to fully uninstall the previous version.

sudo pip install 'awsebcli==3.7.4' --force-reinstall

Is what I use

Solution

1. Set Ubuntu 14.04 (Trusty) (Project Settings > Build Environment)
2. Update the awsebcli dependency

dependencies:
  pre:
    - pip install awsebcli

It seems to fix everything

This works for me:

dependencies:
  pre:
    - sudo pip install 'awsebcli==3.7.4' --force-reinstall
deployment:
  production:
    branch: master
    commands:
      - eb deploy --profile default

While @allanrr solution returns a fatal error pyconfig.h: No such file or directory, @LevBravE 's works for me too.

To install awsebcli I found that python-dev was required. Here's my dependencies snippet

dependencies:
  pre:
    - sudo apt-get install python-dev
    - sudo pip install awsebcli

After the below task get executed, i am getting this error message:-

eb deploy --profile default

ERROR: This directory has not been set up with the EB CLI
You must first run "eb init".

Worked flawlessly with the dependencies snippet from @tonyfruzza.

@anbestephen: I got the same error message, then noticed this snippet in my gitignore

# Elastic Beanstalk Files
.elasticbeanstalk/*
!.elasticbeanstalk/*.cfg.yml
!.elasticbeanstalk/*.global.yml

which was automatically generated when I ran eb init in my project folder locally, preventing the generated .elasticbeanstalk/config.yml from being commited to my repo. After adding an !.elasticbeanstalk/config.yml exception, Circle Ci ran eb deploy <my_environment> with no problem. Not sure if you're having a similar issue, but EB Cli config file must exist in order to run a deploy.

This is great, thank you! 👍 🌟

@justinthareja Why do you think those were in gitignore? Would tracking those files in git them break any recommended best-practices by EB?

@joemccall86 It's probably a good idea to leave those entries in gitignore. This lets you do things like eb ssh into different environments without changing the config in VC.

I got around this by storing a config.yml elsewhere in my codebase and copying into place for circleci:

release:
    tag: /release-.*/
    owner: me
    commands:
      - mkdir .elasticbeanstalk
      - cp resources/circle/eb/config.yml .elasticbeanstalk/config.yml
      - eb deploy --profile default

@euharrison
I am trying to follow your cue i.e " fill the AWS KEY and SECRET in the CircleCI site, you can remove the bash ./setup-eb.sh and just use the command eb deploy --profile default"
It asks for running eb init and
eb init is stuck without selecting any option, (new to eb commands). What needs to be done here?

@vivekna1
I got the same issue, i.e. I needed to put 'eb init' and the process was halted when it asked for an option.
The solution was to choose an option from the list (for me it was '15: eu-west-2') and then do the following: -
- echo 15 | eb init
I.e. pipe the selected response into the 'eb init' command.

  pre:
    - sudo apt-get install libyaml-dev libpython2.7-dev
    - sudo pip install 'awsebcli==3.7.4' --force-reinstall

This works for me

To use the current version of awsebcli on the 14.04, I had to install python-dev (solves the Python.H issue) and update pip (solves Import Error: No module named packaging.version):

pre:
    - sudo apt-get install python-dev
    - sudo pip install --upgrade pip
    - sudo pip install awsebcli

@jkaldenbach was the only solution that worked for me. Thanks guys

@osazemeu -- had to end up using easy install

pre:
    - sudo apt-get install python-dev
    - sudo easy_install --upgrade six
    - sudo pip install awsebcli

Thanks a lot! I had to use a slightly modified version:

#!/usr/bin/env bash
mkdir ~/.aws
touch ~/.aws/config
chmod 600 ~/.aws/config
echo "[profile eb-cli]" > ~/.aws/config
echo "aws_access_key_id=$AWS_ACCESS_KEY_ID" >> ~/.aws/config
echo "aws_secret_access_key=$AWS_SECRET_ACCESS_KEY" >> ~/.aws/config

Also, specifying the keys in the CircleCI UI made them available only as environment variables. Not sure why it didn't create these files!

Thanks for this, it was a big help for getting started with deploying to Elastic Beanstalk via CircleCi.

For those coming to this page that want to use CircleCi 2.0 (as I did), I wrote up the steps I took after hours of trial and error, hopefully they'll help someone:

https://gist.github.com/ryansimms/808214137d219be649e010a07af44bad

I am get this error
ERROR: InvalidProfileError - The config profile (default) could not be found

This error means that is need to set the Environment variables for AWS on CircleCI. Which i have done. I even went under Permissions to set my aws Permissions. I even created a script to copy my aws credentials but nothings seems to work

            - run: 
                name: Create aws credentials
                command: |
                    chmod +x ./setup-eb.sh
                    bash ./setup-eb.sh
            - run:
                name: Deploying
                working_directory: /
                command: |
                    eb deploy TcwlApp-env

script 
#!/usr/bin/env bash

set -x
set -e

mkdir ~/.aws
touch ~/.aws/config
chmod 600 ~/.aws/config
echo "[profile prod]" > ~/.aws/config
echo "aws_access_key_id=$AWS_ACCESS_KEY_ID" >> ~/.aws/config
echo "aws_secret_access_key=$AWS_SECRET_ACCESS_KEY" >> ~/.aws/config