delete all aws log groups

delete_all_awslogs.sh.md

specify the region

export AWS_DEFAULT_REGION=ap-northeast-1
aws logs describe-log-groups --query 'logGroups[*].logGroupName' --output table | \
awk '{print $2}' | grep -v ^$ | while read x; do  echo "deleting $x" ; aws logs delete-log-group --log-group-name $x; done

only delete loggroup name starting with /aws/lambda

export AWS_DEFAULT_REGION=ap-northeast-1
aws logs describe-log-groups --query 'logGroups[*].logGroupName' --output table | \
awk '{print $2}' | grep ^/aws/lambda | while read x; do  echo "deleting $x" ; aws logs delete-log-group --log-group-name $x; done

Neat!

That's great thanks! Found that the grep pattern needed single quotes though: grep -v '^$'

To delete only certain groups, try this :

LOG_GROUP=MY_LOG_GROUP
aws logs describe-log-groups --query 'logGroups[*].logGroupName' --output table | awk '{print $2}' | grep $LOG_GROUP | grep -v ^$ | while read x; do aws logs delete-log-group --log-group-name $x; done

Be careful that LOG_GROUP doesn't partially match similar groups or they will all be deleted.

Very well thank you, you are amazing!

Get an error from trying to delete the table header rows..could exclude them, but text output does the job!

aws logs describe-log-groups --query 'logGroups[*].logGroupName' --output text | while read -a tmp_arr; do for x in "${tmp_arr[@]}"; do aws logs delete-log-group --log-group-name $x; done; done

logs are region-specific, so added a little something:

REGION=us-west-2 ; aws logs describe-log-groups --query 'logGroups[*].logGroupName' --output table --region "$REGION" | awk '{print $2}' | grep -v ^$ | while read x; do aws logs delete-log-group --log-group-name --region "$REGION" $x; done

thanks

This script is interactive. Is there a way to just delete all log groups regardless of anything?

@dhruvit01. I don't think there's a way to just delete everything without looping, but you can wrap the aws cli command in an execution to not get feedback:

aws logs describe-log-groups --query 'logGroups[*].logGroupName' --output table | \
awk '{print $2}' | grep ^/aws/lambda/ | while read x; do  echo "deleting $x" ; $(aws logs delete-log-group --log-group-name $x); done;

I ended up with this incantation to delete some unknown number (many thousands) of log groups using GNU parallel to speed things up.

parallel --delay=1sauto --retry-failed --verbose echo aws logs delete-log-group --log-group-name={} ::: \
    $(aws logs describe-log-groups --log-group-name-prefix=PREFIX --query='logGroups[].logGroupName' --output=text)

Use:

1. Replace PREFIX with your own, such as "/aws/lambda/ci"
2. Run and verify that it (eventually) prints some reasonable commands
3. Run without the echo safety catch

Features:

• The delay + retry uses exponential backoff, as recommended by AWS, slowing down and speeding up depending on whether the commands succeed. See man parallel for details.
• Verbose mode prints each command as it runs, so you can review them after the fact.

Notes:

• Log group names can only contain

  a-z, A-Z, 0-9, '_' (underscore), '-' (hyphen), '/' (forward slash), and '.' (period)

  according to the documentation, so it's safe to not deal with any special characters such as space in this command.