-
Onboarding driven documentation
When starting a new software engineering job, it’s an exciting moment getting let loose on the code for the first time. I’ve been in the same job for about 3 years now but still remember the satisfaction of raising my first pull request there. However, whether or not a new starter is able to ship code in the first week on the job is dependant on several factors. Arguably the most important factor being good documentation.
Read more… -
Will generative AIs curate our history?
History is what humans have recorded over the last 6 millennia. For the majority of this timespan, these records are disparate and sparse: diaries, business records, funeral rites, censuses. Historians curate these snippets of evidence to create a picture of life in our past. However, the last few decades have ushered in the information age. The amount of data about our lives that is now recorded is staggering. It’s estimated we collectively produce trillions of megabytes of data every day.
Read more… -
Tips for writing long running data processing scripts
Data Engineering is a broad field with a large number of complex tools, but sometimes you find yourself needing to write a quick script to do a one-off ETL job or fix some data. For example, an initial population of a system with historical data before setting up a streaming pipeline, or repairing some data that was corrupted by a bug. Writing such scripts can feel like a trivial undertaking. After all, it’s easy enough to write something simple that transforms some data and moves it from one system to another.
Read more… -
Designing Iterable HTTP JSON APIs
Having worked on backend systems at a start up for over 3 years, I’ve learnt a fair bit about getting code out of the door quickly. A lot of my work involved building internal HTTP APIs producing and consuming JSON. These APIs were only used by internal developers, thus could be rapidly iterated on to add new features. However, it is easy to back yourself into a corner when designing APIs.
Read more… -
The value of quick start guides
I learn by doing. I believe this is true of many software engineers. The quickest way to get to grips with the latest programming language, library or tool is almost always to just get stuck in using it. It’s hard to know WHAT you need to learn about something until you’ve tried using it and realise all the things you could do with it that you don’t yet know how to do.
Read more…