Go's finding its way into our Tech Stack

Go is becoming more and more a silver bullet in our arsenal of tools.

How Go came to Digitize

At Digitize we manage quite a lot of servers. Not all of these are inside our network, but the requirement of keeping them up, secure, and well maintained is the same.

With the flexibility of AWS, a lot of servers have either 1GB or 2GB of ram. The applications use most of this, and there isn’t much room for anything else.

We conducted a research project, testing different frameworks for writing an agent that can sit on each server.

The agent doesn’t need to be fast, any programming language can do what it does. Though reducing performance impact to the server is a top priority, what was more a concern, was memory usage.

After different teams produced sample implementations, we settled on Go. The memory usage was a few MB and the processing was the fastest. Sure C++/Rust could have beaten this, but we really didn’t want to have to work at such a low level for this kind of task.

Best of all, every other team could “just read” the code, without much priori Go exposure.

When a service needs some more performance

We use Ruby on Rails and Python for a lot of our projects. The speed of development is hard to beat, and we can deliver results quite fast. We accept they aren’t the “fastest” languages, but we argue they are the fastest to get results. 99% of the time, the programming language doesn’t affect the speed the client experiences when they are using the site.

Occasionally, we hit a hurdle where some data would just be processed a lot faster in parallel. Ruby and Python process them quite well, but the solution just needs to be simple and not add to the resource bill. We turn to Go.

Recent examples would be rewriting a service for parsing family relationship data, building out relationships and all the associations from a large family history. The Python implementation worked, but moving it to a Go service, with equally similar algorithm and simplicity, outperformed it by magnitudes.

Other examples have included working log parsing, where we want to ingest data as fast as possible for use later on. The ingestion API was a perfect fit for Go, but we used Ruby for the analytics and querying, as Ruby on Rails just provided the rapid development and performance that work load needed.

One off tools

Our developers work on different machines. From Windows to Mac & Linux. Go’s cross compilation to a native binary that just works on any platform is as simple as putting GOOS=windows in front of it, and you can distribute that to any Windows user. If for a Mac user, GOOS=darwin GOARCH=64. Then we can ship this to anyone running Mac. This works great when we need to send clients one-off tools for a quick task, as no other dependencies need installing.