I asked myself, am I overengineering my pet projects? Are you!?
I like solving problems! I like finding problems more...
Call me negative! That's how I work; like anything, it's both good and bad, depending on perspective.
Almost every time I find a problem, I also get ideas. Ideas on how that particular experience could be better! And my hands start itching. I've had plenty of these moments during my career—plenty of ideas seemingly simple to implement and turn into a successful product.
But in 17 years, I barely shipped something (outside of my day job, that is).
I often ask myself, WHY!?
Has it got to do with time? Sure, I have a full-time job that keeps me busy - I always had. And I also have small kids who take up a lot of my time and energy.
But I have some spare time; spare time can be a self-imposed constraint. It took me a while to realize the power of constraints - but I am here now!
Ok, so given some limited amount of time that I can put towards personal projects, what do I do?
Well, the thing is that I'm also indecisive. I have too many ideas, and it's not reasonable that I can do everything. I started many projects but finished none. This challenge took longer to grasp and digest, but I'm also here now!
Good! So I need to apply focus to limited availability.
This leads me to the topic at hand.
Am I building products or engineering for fun!?
Building products means moving fast and lean, releasing new software, testing it against your user base, and then rinsing and repeating.
(Cue your favorite buzzword here: Lean, Agile, SDLC, etc.)
So I know what to do, but I just don't do it!
I recently started thinking of a fun data project I'd like to build. It requires a few data collection jobs running in perpetuity.
I started thinking about where I would run these and how I could optimize my costs. Cloud providers are a no go since the costs scale up fast, and I also want to avoid vendor lock-in.
You get the most bang for the buck by going low-level. Rent a bare-metal server, deploy a few VMs (i.e., with the KVM hypervisor), and operate your software.
But this model is also very old-school, not to mention unreliable. What if your VMs crash?
It's better to use a scheduler such as Kubernetes. Installing it, however, is non-trivial; you need backups, automation, you need to factor in resiliency, control plane upgrades, etc. The list goes on and on!
It sounds like a fun project, and once I finished automating all of the above, I could safely deploy and operate my data collector jobs.
That I would have had not written a single line of code for...
If that's not crazy, I don't know what is!
This has been my existential crisis all along. I use my short and valuable time to solve the wrong problem and optimize because that's what I'd do professionally at scale, and that's what I am most comfortable with.
I'm starting to get it now. As the famous saying goes: "You have to spend money to make money!"
I have to find a way to start these projects more expensive than I'm comfortable with because the cost of missed opportunities is way higher than a few hundred coins a month!
Taking this approach is a forcing function of its own. I can easily afford to pay 50 quid a month for a server I don't use and will unlikely feel any pressure to deploy something on it instead of watching the next "promising" Netflix show...
This is a loser's game.
I can do better.