A CLI tool to analyze your Elastic Load Balancer logs from AWS.
The need for this tool came on a day a few years ago late at night. I was working for Onedio at the time. We would work a lot and we would work until late. One evening, probably around 7 or 8 pm, we started receiving a DDOS attack on critical levels. We didn't have Cloudflare turned on by default at the time so this DDOS took the website down pretty easily. We turned Cloudflare protection on. We also managed the identify some of the attacker IPs and started blocking them ourselves but they were changing IPs and the patterns they use to attack us. It was to the point that even after we got the attack under control, we couldn't recover the system because some microservices started misbehaving. It took until 3-4 am for us to revive the website. A hell of a night in the life of a software developer in a startup.
After going over what happened that night, we realized that even though the attack itself was tiresome, the real problem we had was with the recovery of the system. Particularly when finding out what went wrong with the microservices. We didn't want to have something like this again, so we decided that there was a tool for analyzing our loadbalancer logs better/faster to identify the problems.
There were already many SaaS products out there that collected your logs and analyzed them for you. We considered them but it seemed like an overkill considering we didn't need such analysis that often. Also, it was a startup. We didn't have thousands of dollars a month to splash on log analyzing. Therefore, the logical way seemed to assign a developer (me) to code a basic CLI tool to analyze big chunks of logs.
This tool was born from that event and was used by thousands of people over the years. Currently, it has about ~150 stars on GitHub. I've never marketed or promoted this product anywhere. GitHub analytics indicate that most people find this tool when they're Googling. It serves a niche with people who need a basic tool to analyze millions of lines of logs and do not have the time or the budget to set up complicated software solutions for it.
It was originally written in JavaScript (Node.js). I later migrated it to TypeScript. I still try to keep its dependencies up-to-date and address bugs and feature requests when I have time.
You can take a look at it here https://github.com/ozantunca/elb-log-analyzer.
