delete all aws log groups

delete_all_awslogs.sh.md

specify the region

export AWS_DEFAULT_REGION=ap-northeast-1
aws logs describe-log-groups --query 'logGroups[*].logGroupName' --output table | \
awk '{print $2}' | grep -v ^$ | while read x; do  echo "deleting $x" ; aws logs delete-log-group --log-group-name $x; done

only delete loggroup name starting with /aws/lambda

export AWS_DEFAULT_REGION=ap-northeast-1
aws logs describe-log-groups --query 'logGroups[*].logGroupName' --output table | \
awk '{print $2}' | grep ^/aws/lambda | while read x; do  echo "deleting $x" ; aws logs delete-log-group --log-group-name $x; done

I ended up with this incantation to delete some unknown number (many thousands) of log groups using GNU parallel to speed things up.

parallel --delay=1sauto --retry-failed --verbose echo aws logs delete-log-group --log-group-name={} ::: \
    $(aws logs describe-log-groups --log-group-name-prefix=PREFIX --query='logGroups[].logGroupName' --output=text)

Use:

1. Replace PREFIX with your own, such as "/aws/lambda/ci"
2. Run and verify that it (eventually) prints some reasonable commands
3. Run without the echo safety catch

Features:

• The delay + retry uses exponential backoff, as recommended by AWS, slowing down and speeding up depending on whether the commands succeed. See man parallel for details.
• Verbose mode prints each command as it runs, so you can review them after the fact.

Notes:

• Log group names can only contain

  a-z, A-Z, 0-9, '_' (underscore), '-' (hyphen), '/' (forward slash), and '.' (period)

  according to the documentation, so it's safe to not deal with any special characters such as space in this command.