What I learnt from releasing an open source tool

Since its release s3audit has garnered a modest amount of interest from the wider AWS community. Following the release:

These figures are modest compared with the many popular open source projects which are available. But it’s fair to say my expectations of interest were much lower when we decided to make the tool public. And it’s certainly the first time something I’ve released publicly has achieved even modest interest on this level.

Here I’d like to talk a little bit about what I learnt from releasing s3audit; because it’s actually a perfect case study for the importance of lean principles and MVPs when building software and of overcoming the natural fear which often comes from releasing software.

I initially built s3audit because I wanted to get an overview of the settings for S3 buckets in an AWS account. So, wanting to be a responsible DevOps person, I thought I should automate this as much as possible.

I did a quick search for an existing tool and couldn’t find anything which entirely suited my needs. So I decided to quickly put something together.

The majority of the coding was done over about 2 working days. I decided to write it in Node.js because I’m familiar with JavaScript, and TypeScript, so I was confident I could get something working quickly. I wanted to be able to get on with the task in hand without being blocked by putting this together.

On the other hand, I’ve never written a CLI tool with Node.js before so there was an opportunity to learn something new as well. Importantly though, the learning curve wouldn’t take up too much time.

I also considered using Go because I’d like to learn that a bit more, and in some ways if feels like a more sensible option for a CLI tool. But the learning curve would have been too steep and I needed to avoid getting distracted from the actual task in hand.

At this point I started to panic. I’d expected this to go the same way as every other thing I’d made public, that is to disappear into GitHub oblivion. The problem with being a developer most of the time is that your mum doesn’t know what GitHub is, so you don’t even have her to cheer you on.

When I first hacked the tool together in a just over a day I wasn’t sure if it would be useful for anyone else, releasing it has been the last thing on my mind.

It didn’t have any tests because it hadn’t needed them. It was a simple tool to quickly get information from the AWS API that I wasn’t expecting to use again. It would be tested by verifying the information before acting on it. The code in general wasn’t exactly a work of art either, I certainly wasn’t going to be printing it out to put on the fridge door.

This worked well enough for me, but I was pretty worried that it would result in other people dismissing the tool as unusable. In the end that didn’t happen.

I had other concerns as well because of the limitations of using a task list library for something which wasn’t a task list — not everything I was adding would necessarily need a tick or a cross and I sometimes wanted to display a bit more context, for example which SSE algorithm was being used. I decided to ignore these problems for now.

Subsequently, I realised that I needed the output in a more usable format anyway. So I added a flag for outputting as CSV so that I could open it up in a spreadsheet and add conditional formatting to give me lots of lovely green and red boxes. I could also now include the extra information in the CSV output, a win-win. If it gets developed more in the future then the console output will probably be changed for something more appropriate.

Related posts:

Michael Purcell is currently a SiS engineer in the internal sales department of Cisco Systems. He obtained his first CCIE qualification in Routing and Switching in September 1998, and then obtained…

Walking the streets of London I see history preserved in the architecture, in the cobblestones and the Thames. I see the plaques and the monuments in remembrance of an empire that was once so…

Many people these days want to buy just about any new gadget that bursts onto the market. Indeed, so anxious are they to get their hands on the latest invention that they wait all night in long…