Post

Getting started with AWS open-source tools (1/3)

Posted
By Alejandro Lazaro Chueca 7 min read

This article is part of a series about open-source tools and AWS, in which I will share those tools that I have tested and that I think you may find useful.

1/3: Getting started: main open-source tools and extend AWS CLI (this article).

2/3: Analyze your AWS environment: focus on the inventory, analysis and security assessment of the AWS environment.

3/3: How to help you with your code: to generate it (IaC), validate it (policy as code and compliance) and analyze it (static analysis/credentials).

Introduction

The Open Source Initiative defines here the following criteria for considering software as open-source:

  1. Free Redistribution
  2. The source code must be accessible
  3. The license must allow modifications and derived works
  4. Integrity of The Author’s Source Code
  5. No Discrimination Against Persons or Groups
  6. No Discrimination Against Fields of Endeavour
  7. Distribution of License
  8. License Must Not Be Specific to a Product
  9. License Must Not Restrict Other Software
  10. License Must Be Technology-Neutral

Open source software is code designed to be accessible to the public: anyone can view, modify and distribute the code as they wish.

By the way, both terms are correct: open-source and open-source. I could use them interchangeably in this article.

What is the relationship between open-source and AWS?

I’m sure you already know that there are many open-source projects related to AWS, too many. This is because AWS is very popular, but also because AWS is very committed to the open-source community.

AWS claims that open-source is good for everyone and regularly develops open-source software and contributes to thousands of open-source communities on GitHub, Apache, and the Linux Foundation. More information can be found here

In this post, I will try to show you some open-source projects, since you probably don’t know all of them and you can surely start using some of them.

So, let’s start sharing public code!

Main AWS open-source tools

Let’s start with the most popular AWS open-source projects.

You are probably already using some of them and did not realize they are open-source. However, I am not going to explain them or give more information, I will just name them here:

Extend AWS CLI

My first idea was to name this section CLI tools but all the open-source tools listed here are CLI (Command Line Interface) tools, so this section is for the tools that you can use to improve/extend/replace your AWS CLI tool.

  • Security
    • aws-vault: a tool to securely store and access AWS credentials in a development environment
  • Extend AWS CLI
    • aws-shell: interactive productivity booster for the AWS CLI
    • awsls: a list command for AWS resources
    • steampipe: Use SQL to query cloud infrastructure, SaaS, code, logs, and more
    • ohmyzsh with the AWS plugin: provides completion support for AWS CLI and a few utilities to manage AWS profiles and display them in the prompt
  • Logs
    • awslogs: a simple command line tool for querying groups, streams and events from Amazon CloudWatch logs

aws-vault

aws-vault securely stores and accesses AWS credentials in a development environment.

AWS Vault stores IAM credentials in your operating system’s secure keystore and then generates temporary credentials from those to expose to your shell and applications. It’s designed to be complementary to the AWS CLI tools and is aware of your profiles and configuration in ~/.aws/config.

Why use it: Complementary tool for AWS CLI tools, for secure your connections protecting your credentials.

aws-vault

With the last command, the AWS Console will be open and you will be logged!

Is it popular? Yes, GitHub statistics: Watch 119; Fork 725; Stars 7.1k

Recently updated? Yes, 1096 commits, last 2 weeks ago

URL: https://github.com/99designs/aws-vault

In my opinion, this tool is a must-have for securing your credentials.

aws-shell

The interactive productivity booster for the AWS CLI

Why use it: AWS CLI is awesome but maybe you don’t know the commands. With aws-shell, you have a helper and as you type you can visually see the available options:

aws-shell

Is it popular? Yes, GitHub statistics: Watch 230; Fork 755; Stars 6.8k

Recently updated? No, the project seems abandoned (last commit Oct 7, 2020). Total commits 235

URL: https://github.com/awslabs/aws-shell

The tool is not working with AWS CLI v2 (here is the official Issue in github), and the project seems to have been abandoned (last commit July 10, 2020).

However, if you use the v1, the tool is worth it because it contains all the core AWS services.

awsls

A list command for AWS resources. More than 100 AWS resources are supported. The goal is to code-generate a list function for every AWS resource that is covered by the Terraform AWS Provider (currently over 500)

Why use it: If you want to search for resources across multiple regions and/or accounts and filter by any value using GREP, this is the tool for you!

awsls

Is it popular? Yes, GitHub statistics: Watch 10; Fork 51; Stars 763

Recently updated? No, one year from the last update (Feb 13, 2022), with 91 commits in total.

URL: https://github.com/jckuester/awsls

Although it has not been updated recently, it is worth using for its ability to search multiple accounts and filter using the GREP command.

steampipe

Steampipe is the universal interface to APIs. Use SQL to query cloud infrastructure, SaaS, code, logs, and more.

Why use it: Using SQL you can query AWS resources, perform join queries (same account, several accounts, between different sources), and you have a helper to perform the queries. All in one!

steampipe-1

Is it popular?: On Github: Watch 32; Fork 171; Stars 4.6k

Recently updated? Yes, last commit 2 days ago. Total commits 2007

URL: https://github.com/turbot/steampipe

More information (querying AWS resources):

I like this tool! If I want to get specific information in AWS using a CLI tool this is my first choice to do so, I recommend you to try it!

awslogs

awslogs is a simple command line tool for querying groups, streams and events from Amazon CloudWatch logs.

Why use it: If you want to review your logs with console this is your tool. You can filter start/end and also using GREP! An example:

awslogs

Is it popular?: On Github: Watch 61; Fork 326; Stars 4.5k

Recently updated? No, the project seems to have been abandoned. The last commit was July 10, 2020. Total commits 326

URL: https://github.com/jorgebastida/awslogs

The tool is not updated but if you want to query the CloudWatch logs, this is your tool!

ohmyzsh with the AWS plugin

If you are using ohmyzsh (framework for managing your zsh configuration), you can add this aws plugin to extend your CLI.

This plugin provides completion support for AWS CLI and a few utilities to manage AWS profiles and display them in the prompt.

Why use it: Useful if you are using different profiles and ohmyzsh.

ohmyzsh-aws

Is it popular?: ohmyzsh is a very popular framework for zsh (155k stars), and the AWS plugin is part of this framework.

Recently updated? Yes. The AWS plugin was updated 2 weeks ago

URL: https://github.com/ohmyzsh/ohmyzsh/tree/master/plugins/aws

Next steps

As I said in the introduction, there are 2 more articles in this series of open-source tools:

  • Analyze your AWS environment: focus on the inventory, analysis and security assessment of the AWS environment - here
  • How to help you with your code: to generate it (IaC), validate it (policy as code and compliance) and analyze it (static analysis/credentials) - here
General
getting-started open-source github level-200
This post is licensed under CC BY 4.0 by the author.