AWS EC2 API access with Python

AWS has always had a great GUI interface, but the API side of calls is the programmatic way to work with AWS. AWS prides itself on having a rich set of API’s, allowing most (if not all) operations to be created via API rather than having to head to the GUI to achieve the results.

I run a Windows 2K8 VM on AWS, which I use as a jumpbox to get to public services and clients public IPs (RDP/SSH/Teamviewer etc.). I had to set this up due to the very restrictive nature of the firewall I work behind and the corporate PC I work on blocking these services outbound – something not uncommon for corporate networks.

The instance I have setup for use is a standard Windows 2008 R2 AMI with 2 CPU, 4GB and a public IP (not an EIP – so not fixed as I don’t want to be paying for the EIP when I don’t have the instance on).

The following Python APP was built to support this via a script call rather than me having to log in to the GUI and manage the EC2 VM – I only start up the VM when I need to use it, so don’t want to leave it running 24/7 and just costing money for no use (thus the beauty of AWS !).

To use the following script, you will need Python installed (in my case it’s 3.6) and Boto3 (the Python SDK for the AWS CLI API access). Boto3 works with the AWS CLI, which is a command line way to run API calls.

The following guides are useful to setup these items ready to build the APP:

Python Installation:

https://wiki.python.org/moin/BeginnersGuide

PIP installation (used to install Python modules):

https://github.com/BurntSushi/nfldb/wiki/Python-&-pip-Windows-installation

BOTO3 install (simple with PIP):

https://pypi.org/project/boto3/

AWS CLI Install:

https://aws.amazon.com/cli/

From a user security point of view, you need to have an IAM user with an API key (programmatic) configured, who has the ability to manage EC2 instances. The API keys get stored via the AWS CLI program to C:\Users\[user]\.aws\ (for Windows), in 2 files called config and credentials.

Run ‘$ aws configure’  to set this up.

Once that is all built, you can create a new file called AWS-EC2.py (or a name of your choice – with the .py extension), and then use IDLE (right click on the file and chose Edit with IDLE) to put the code in. IDLE is a basic Python text editor that comes with the Python installer, with good recognition of code and structures for Python.

The APP has a few sections – whats imported (modules to be used), our variables, and the subroutines (def:). Firstly we need to import the required modules into Python:

# ————————–
import boto3
import time
import subprocess
# ————————–

This supports the AWS module, our retry timer for reading the public IP, and the command line process to open the remote desktop app in windows.

Next our variables:

# ————————–
myTag1 = “MyInstance”
myTag2 = “JJ-Jumpbox”

myRegion = “us-east-1”

cmdCommand = “C:\Windows\System32\mstsc.exe”
# ————————–

The myTag1/2 is a Tag I have on created my instance to uniquely identify it, the region is where it is hosted, and the command is the Windows Remote Desktop APP on my PC.

Now we have the first 2 routines – the start and stop of an EC2 instance calls:

# ————————–
def start_ec2(ec2_instance_id, client):
    response = client.start_instances(
        InstanceIds=[
            ec2_instance_id,
        ],
        DryRun=False # change to true to test only
    )
    return response

def stop_ec2(ec2_instance_id, client):
    response = client.stop_instances(
        InstanceIds=[
            ec2_instance_id,
        ],
        DryRun=False
    )
    return response
# ————————–

These routines use the boto3 API calls to start_instances and stop_instances, to manage the state of the EC2 instance.

Now for the main function, that runs through to read the current config, and uses the above routines to make changes to the instance.

# ————————–
def main():
    boto3.setup_default_session(profile_name=’default’)
    ec2client = boto3.client(‘ec2’, region_name=myRegion)
    response = ec2client.describe_instances()
    for reservation in response[“Reservations”]:
        for instance in reservation[“Instances”]:
            #print(instance) – use this if you want to see ALL the instance fields
            print(instance[“InstanceId”], instance[“InstanceType”], instance[“State”], instance[“Placement”], instance[“PrivateIpAddress”], instance[“KeyName”], instance[“Platform”])
            for mytags in instance[“Tags”]:
                if mytags[“Key”] == myTag1 and mytags[“Value”] == myTag2:
                    myInstanceId = instance[“InstanceId”]
                    myStatus = instance[“State”]
# ————————–

The first part of the main function creates a boto3 client with access to our AWS environment (as defined with keys in the AWS CLI). Note the profile_name option is present just so I can use different profiles when i need to change between customers for other uses of the code.

describe_instances() is called, to list all instances that are present in this regions EC2 instance list. We then go through that list to find the one with the JJ-Jumpbox TAG reference, and find what the current state of the instance is.

# ‘pending’|’running’|’shutting-down’|’terminated’|’stopping’|’stopped’
# ————————–
    if myStatus[‘Name’] == ‘stopped’:
        print (“Instance”, myTag1, myTag2)
        yesno = input (“Do you want to START the instance ? {y or n}”)
        if yesno == “y”:
            result = start_ec2(myInstanceId, ec2client)
            print (“\n”)
            print (result)
            print (“\n”)
            publicIP = {}
            publicIP[“ip”] = 0        
            while publicIP[“ip”] == 0:
                print (“Waiting for public IP (rechecking every 10s)”)
                time.sleep(10)
                response = ec2client.describe_instances(
                    InstanceIds=[
                        myInstanceId,
                    ],
                DryRun=False
                )
               # print (response) – use this if you want to see ALL the response fields
                for reservation in response[“Reservations”]:
                    for instance in reservation[“Instances”]:
                        myState = instance[“State”]
                        print (myState)
                        if  myState[“Name”] == “running”:
                            publicIP[“ip”] = instance[“PublicIpAddress”]
            print (“\n”)               
            print (“Public IP is:”, publicIP[“ip”])
            process = subprocess.Popen(cmdCommand.split(), stdout=subprocess.PIPE) 
# ————————–

If the state is stopped – we are given the choice to run it. The instance is then started, and a 10s while loop is started, checking each time to see if the instance state is changed to ‘running’, so we can find out the public IP of the instance to return to the user (to enter in to the RDP box). Once the IP is obtained, we exit the while loop, show the IP and open the mstsc.exe app which is Windows Remote Desktop.

# ————————–        
    elif myStatus[‘Name’] == ‘running’:
        print (“Instance”, myTag1, myTag2)
        yesno = input (“Do you want to STOP the instance ? {y or n}”)
        if yesno == “y”:
            result = stop_ec2(myInstanceId, ec2client)
            print (“\n”)
            print (result)
            print (“\n”)
    else:
        print (“\n”)
        print (“Status is:”, myStatus[‘Name’])
if __name__ == “__main__”:
    main()
# ————————–

The other options in this main routine are to stop the instance – which is displayed if it is found to be already running when the APP is run. The last option is to show its status (ie not running or stopped) – so you can see what it is doing (starting up, shutting down or terminated).

That’s it ! You should now have a working Python app that can talk to your AWS environment via IAM API key, and start/stop/review your EC2 instances. To make this work for your own EC2 units, just add a TAG to the instance, and change the myKey variable at the start of the script.

Where to from here ? The boto3 guide has a list of all the functions you can call via the API (there;s a lot !), so have a read through and see what else is in the list of things you would like to do with your EC2 (and other) AWS service.

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html

Its also possible to do this via AWS lambda… that’s for anther day of learning !