[amazon-web-services] Query EC2 tags from within instance

Amazon recently added the wonderful feature of tagging EC2 instances with key-value pairs to make management of large numbers of VMs a bit easier.

Is there some way to query these tags in the same way as some of the other user-set data? For example:

$ curl http://169.254.169.254/latest/meta-data/placement/availability-zone
us-east-1d

Is there some similar way to query the tags?

If you are not in the default availability zone the results from overthink would return empty.

ec2-describe-tags \
   --region \
     $(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone  | sed -e "s/.$//") \
   --filter \
     resource-id=$(curl --silent http://169.254.169.254/latest/meta-data/instance-id)

If you want to add a filter to get a specific tag (elasticbeanstalk:environment-name in my case) then you can do this.

ec2-describe-tags \
   --region \
     $(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone  | sed -e "s/.$//") \
   --filter \
     resource-id=$(curl --silent http://169.254.169.254/latest/meta-data/instance-id) \
   --filter \
     key=elasticbeanstalk:environment-name | cut -f5

And to get only the value for the tag that I filtered on, we pipe to cut and get the fifth field.

ec2-describe-tags \
  --region \
    $(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone  | sed -e "s/.$//") \
  --filter \
    resource-id=$(curl --silent http://169.254.169.254/latest/meta-data/instance-id) \
  --filter \
    key=elasticbeanstalk:environment-name | cut -f5

A variation on some of the answers above but this is how I got the value of a specific tag from the user-data script on an instance

REGION=$(curl http://instance-data/latest/meta-data/placement/availability-zone | sed 's/.$//')

INSTANCE_ID=$(curl -s http://instance-data/latest/meta-data/instance-id)

TAG_VALUE=$(aws ec2 describe-tags --region $REGION --filters "Name=resource-id,Values=$INSTANCE_ID" "Name=key,Values='<TAG_NAME_HERE>'" | jq -r '.Tags[].Value')

Download and run a standalone executable to do that.

Sometimes one cannot install awscli that depends on python. docker might be out of the picture too.

Here is my implementation in golang: https://github.com/hmalphettes/go-ec2-describe-tags

Once you've got ec2-metadata and ec2-describe-tags installed (as mentioned in Ranieri's answer above), here's an example shell command to get the "name" of the current instance, assuming you have a "Name=Foo" tag on it.

Assumes EC2_PRIVATE_KEY and EC2_CERT environment variables are set.

ec2-describe-tags \
  --filter "resource-type=instance" \
  --filter "resource-id=$(ec2-metadata -i | cut -d ' ' -f2)" \
  --filter "key=Name" | cut -f5

This returns Foo.

For Python:

from boto import utils, ec2
from os import environ

# import keys from os.env or use default (not secure)
aws_access_key_id = environ.get('AWS_ACCESS_KEY_ID', failobj='XXXXXXXXXXX')
aws_secret_access_key = environ.get('AWS_SECRET_ACCESS_KEY', failobj='XXXXXXXXXXXXXXXXXXXXX')

#load metadata , if  = {} we are on localhost
# http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AESDG-chapter-instancedata.html
instance_metadata = utils.get_instance_metadata(timeout=0.5, num_retries=1)
region = instance_metadata['placement']['availability-zone'][:-1]
instance_id = instance_metadata['instance-id']

conn = ec2.connect_to_region(region, aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key)
# get tag status for our  instance_id using filters
# http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DescribeTags.html
tags = conn.get_all_tags(filters={'resource-id': instance_id, 'key': 'status'})
if tags:
    instance_status = tags[0].value
else:
    instance_status = None
    logging.error('no status tag for '+region+' '+instance_id)

You can alternatively use the describe-instances cli call rather than describe-tags:

This example shows how to get the value of tag 'my-tag-name' for the instance:

aws ec2 describe-instances \
  --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) \
  --query "Reservations[*].Instances[*].Tags[?Key=='my-tag-name'].Value" \
  --region ap-southeast-2 --output text

Change the region to suit your local circumstances. This may be useful where your instance has the describe-instances privilege but not describe-tags in the instance profile policy

The following bash script returns the Name of your current ec2 instance (the value of the "Name" tag). Modify TAG_NAME to your specific case.

TAG_NAME="Name"
INSTANCE_ID="`wget -qO- http://instance-data/latest/meta-data/instance-id`"
REGION="`wget -qO- http://instance-data/latest/meta-data/placement/availability-zone | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`"
TAG_VALUE="`aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCE_ID" "Name=key,Values=$TAG_NAME" --region $REGION --output=text | cut -f5`"

To install the aws cli

sudo apt-get install python-pip -y
sudo pip install awscli

In case you use IAM instead of explicit credentials, use these IAM permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {    
      "Effect": "Allow",
      "Action": [ "ec2:DescribeTags"],
      "Resource": ["*"]
    }
  ]
}

Jq + ec2metadata makes it a little nicer. I'm using cf and have access to the region. Otherwise you can grab it in bash.

aws ec2 describe-tags --region $REGION \
--filters "Name=resource-id,Values=`ec2metadata --instance-id`" | jq --raw-output \
'.Tags[] | select(.Key=="TAG_NAME") | .Value'

No jq.

aws ec2 describe-tags --region us-west-2 \
--filters "Name=resource-id,Values=`ec2-metadata --instance-id | cut -d " " -f 2`" \
--query 'Tags[?Key==`Name`].Value' \
--output text

For those crazy enough to use Fish shell on EC2, here's a handy snippet for your /home/ec2-user/.config/fish/config.fish. The hostdata command now will list all your tags as well as the public IP and hostname.

set -x INSTANCE_ID (wget -qO- http://instance-data/latest/meta-data/instance-id)
set -x REGION (wget -qO- http://instance-data/latest/meta-data/placement/availability-zone | sed 's/.$//')

function hostdata
    aws ec2 describe-tags --region $REGION --filter "Name=resource-id,Values=$INSTANCE_ID" --output=text | sed -r 's/TAGS\t(.*)\t.*\t.*\t(.*)/\1="\2"/'
    ec2-metadata | grep public-hostname
    ec2-metadata | grep public-ipv4
end

Using the AWS 'user data' and 'meta data' APIs its possible to write a script which wraps puppet to start a puppet run with a custom cert name.

First start an aws instance with custom user data: 'role:webserver'

#!/bin/bash

# Find the name from the user data passed in on instance creation
USER=$(curl -s "http://169.254.169.254/latest/user-data")
IFS=':' read -ra UDATA <<< "$USER"

# Find the instance ID from the meta data api
ID=$(curl -s "http://169.254.169.254/latest/meta-data/instance-id")
CERTNAME=${UDATA[1]}.$ID.aws

echo "Running Puppet for certname: " $CERTNAME
puppet agent -t --certname=$CERTNAME 

This calls puppet with a certname like 'webserver.i-hfg453.aws' you can then create a node manifest called 'webserver' and puppets 'fuzzy node matching' will mean it is used to provision all webservers.

This example assumes you build on a base image with puppet installed etc.

Benefits:

1) You don't have to pass round your credentials

2) You can be as granular as you like with the role configs.

I have pieced together the following that is hopefully simpler and cleaner than some of the existing answers and uses only the AWS CLI and no additional tools.

This code example shows how to get the value of tag 'myTag' for the current EC2 instance:

Using describe-tags:

export AWS_DEFAULT_REGION=us-east-1
instance_id=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
aws ec2 describe-tags \
  --filters "Name=resource-id,Values=$instance_id" 'Name=key,Values=myTag' \
  --query 'Tags[].Value' --output text

Or, alternatively, using describe-instances:

aws ec2 describe-instances --instance-id $instance_id \
  --query 'Reservations[].Instances[].Tags[?Key==`myTag`].Value' --output text

Install AWS CLI:

curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
sudo apt-get install unzip
unzip awscli-bundle.zip
sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

Get the tags for the current instance:

aws ec2 describe-tags --filters "Name=resource-id,Values=`ec2metadata --instance-id`"

Outputs:

{
    "Tags": [
        {
            "ResourceType": "instance", 
            "ResourceId": "i-6a7e559d", 
            "Value": "Webserver", 
            "Key": "Name"
        }
    ]
}

Use a bit of perl to extract the tags:

aws ec2 describe-tags --filters \
"Name=resource-id,Values=`ec2metadata --instance-id`" | \
perl -ne 'print "$1\n" if /\"Value\": \"(.*?)\"/'

Returns:

Webserver

You can add this script to your cloud-init user data to download EC2 tags to a local file:

#!/bin/sh
INSTANCE_ID=`wget -qO- http://instance-data/latest/meta-data/instance-id`
REGION=`wget -qO- http://instance-data/latest/meta-data/placement/availability-zone | sed 's/.$//'`
aws ec2 describe-tags --region $REGION --filter "Name=resource-id,Values=$INSTANCE_ID" --output=text | sed -r 's/TAGS\t(.*)\t.*\t.*\t(.*)/\1="\2"/' > /etc/ec2-tags

You need the AWS CLI tools installed on your system: you can either install them with a packages section in a cloud-config file before the script, use an AMI that already includes them, or add an apt or yum command at the beginning of the script.

In order to access EC2 tags you need a policy like this one in your instance's IAM role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1409309287000",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeTags"
      ],
      "Resource": [
        "*"
      ]
    }
  ]
}

The instance's EC2 tags will available in /etc/ec2-tags in this format:

FOO="Bar"
Name="EC2 tags with cloud-init"

You can include the file as-is in a shell script using . /etc/ec2-tags, for example:

#!/bin/sh
. /etc/ec2-tags
echo $Name

The tags are downloaded during instance initialization, so they will not reflect subsequent changes.

The script and IAM policy are based on itaifrenkel's answer.