Do you know the maintenance man?
an appropriately unhinged plan to chip away at tech debt
I think I get injured so often doing sports because I was a lonely theater dweeb in high school. I adopted various fitness activities in college, and in adopting most of them, I frequently went way too hard. Theater rewards going hard, sports reward calculated try-hardness.
I ran too far too fast and gave myself a foot stress fracture. I lifted too heavy too quickly and displaced muscles in my back. I hiked too long and too steep and got achilles tendinitis. I boxed too aggressively and tore my labrum in my shoulder thanks to a mad dislocation.
I tend not to do anything halfway. I’m intense and focused and I make friends with the same type of person. It’s a personal quality that tends to either put people off or draw them in. When I get injured from sports, my failure mode is getting discouraged and deciding that not doing whatever injured me is better than going too hard.
It’s a tough mindset to shake in fitness and is one I struggle with all the time. I’ve been consistent in my fitness routine for over a decade and am proud of my 146-week streak on Peleton. But I don’t like how all-or-nothing my attitude feels about fitness sometimes.
Currently, I am trying to bat against that by doing a 100 days of pilates challenge. As of posting this, I’ll be well over 1/3 of the way through the challenge. I set some rules for myself with this challenge.
Each day of the 100 days needs to be consecutive. No skipping any days.
Each pilates class needs to be at least 10 minutes long.
The daily streak is more important than the difficulty or length of the class.
If I am sore or tired, a 10-minute beginner class is more than acceptable.
This challenge has been going well so far. I’ve had to take advantage of 10 mins being my minimum about once a week at this point. That’s about how often I feel sore and tired, and I barely get the 10 minute class in. For 4 of the days of the week, I’m usually doing a 20-min intermediate level sessions on the Peleton app. The remaining 2 days a week I’m doing 50-min reformer classes. This balance works well because I also cycle and go for walks throughout the week.
I’m proud of my streak so far. I don’t like Pilates that much (though it’s growing on me and I do enjoy reformer classes mostly), but I notice how good it is for my back and my balance, so sticking with it daily is a big deal for me. It occurred to me in late August my back had been bothering me almost every day, and the whole time during this challenge my back has not been bothering me at all. Pilates hasn’t given me a 6 pack or model legs or whatever the fuck bullshit fitness influencers tell you it will give you, but that’s because I’m built fairly thick and I was never gonna look like them anyways. I don’t look like a Pilates princess, but I’m still feeling the gains.
Ready for a hard left?!
I think about good data maintenance and tech debt care in the same way that I think about these incremental fitness gains. It makes no sense to me to only care about your technical debt backlog on a certain sprint cycle or to only care about well governed data once you have a full blown council and representatives and all that kind of stuff. Sure, dedicated tech debt sprint cycles are great and so are governance councils, but why wait for those things to be in place? Why not just start now?
Drawing from both my past experience as an analyst and my current experiences working with data teams all over the country, here’s a few ideas for adopting an incrementalist approach to better data maintenance and tech debt control.
Disclaimer that I do specifically mention dbt cloud tools here, and yes their payroll does pay my bills. They don’t specifically pay me for this blog though. I write this blog off the clock, and I do just genuinely like dbt enough to write about it off the clock. But I am on the payroll, so when i post this in data communities it’ll be in vendor content for that reason.
Anyways! Here’s an organized collection of thoughts around adopting a more incrementalist, some-is-better-than-none approach to taming your technical debt.
Part 1: State of the Union
My homie, there is no reason on this godforsaken planet that you shouldn’t start figuring out the exact scope of your technical debt and data stack maintenance needs. I know—your sprints never allow for it!! No one cares about tech debt, just shipping!! Well, you’re reading this article, so you care about it. You should start by building up a record of technical debt as you run into it. Try some of these ideas on for size.
You should use a task management software for this. I like Notion’s task management capabilities. They’re super simple, and this doesn’t need to be complicated.
Whenever you run into a piece of technical debt, write a short ticket and explanation for it. Make a “backlog” swimlane for it, and put it there.
Make a few other swimlanes. I would do “Scoping”, “in-progress”, “blocked”, and “complete”.
Make sure you’re sorting the tasks by date created. It’s helpful to see how long something has been an issue, too. If you’re gonna convince the powers that be to let you handle tech debt issues, come at them with the data to support it! Enumerate the problems and how long they’ve been an issue.
Every time you make a backlog tech debt ticket, tag it with either “affects no one” or “affects someone”.
For example: you are working on a column that has no documentation. You handling that yourself affects no one.
A counter example: you are working on a column whose documentation is detailed yet incomplete and misleading. If you change the docs on that column, it probably affects someone.
Try to find one of these tech debt items per week, and record them in your little tech debt database. If you run into a lot more than that, limit yourself to one tech debt ticket per day. Be chill about this. You will not solve your org’s tech debt problem all at once. You are merely taking a stab at it.
If you’re a dbt Cloud user, you should also take stock of dbt’s explorer recommendations. Some may not be relevant to you, but some might be! Give these a good shot.
Part 2: Things you don’t need permission for
Good news. If any of these technical debt items are things like you providing documentation where there was none before, or are otherwise in that “affects no one” category, you can probably just start knocking those out.
As you fix each tech debt ticket that affects no one (or is a no docs—>yes docs situation), update its ticket in your task manager with a date and what you did.
Once you’ve finished a unit of work, let the people or channel who would potentially interact with that item know that you fixed something up for them. Then move that ticket to completed.
Try making 10 minutes of progress per week on these tech debt tickets that you don’t need permission for. It’ll feel psychologically good to start chipping away at the tech debt, and as a bonus you will start seeding your coworker’s heads with the idea that you are in fact the maintenance man (gender neutral).
Part 3: Things you probably do need permission for
Okay, so not every item in your tech debt list is one you should responsibly just go and fix without telling anyone. That’s fine! You can still work on getting the right buy-in for items that will affect someone.
Before you can start working on your tech debt items that do affect other people, you should figure out how many people need to give you permission to fix whatever’s wrong. In my experience, with data stuff, this is usually at least 2 people. One person is someone else from the data team (or adjacent to it) who can double check for you that your change isn’t going to mess anything up downstream, or that your docs are accurate, or that the model you want to deprecate can get safely deprecated. The other is a business user who you know probably interacts with the downstream life of the table you’re fixing up.
Finding the right business user is probably going to be a matter of figuring out which group of business users actually interact with the table’s downstream life. That might be a marketing or sales channel in your company Slack (or, gag, Teams). Find at least one person from that business user category to make aware of your tech debt fix.
Once you’ve identified your two people to give you permission to fix up your tech debt item, you should probably double check these with your boss depending on your position and tenure at your job. If you’re more up there in terms of seniority, go on with your bad self and just fix it. But if you’re an IC like me with a boss and a grandboss and a great grandboss and a great great grandboss before you get to the CEO, you should probably double check your two mini-stakeholders.
(But also—good news here—identifying your mini stakeholders on one of these types of tech debt tickets counts as working on it! Update your ticket accordingly and FEEL THE SEROTONIN, BABY!)
Once you’ve gotten the upvote from your boss, work in another 10 minutes per week on fixing your current “need permission” tech debt ticket. Don’t forget your little dated updates on the task management card!
Being an absolutist is bad, actually
The idea here is to:
Have an accurate sense of the scope of tech debt problems at your job
Take some initiative and start fixing what you see
If you have to get someone else’s permission, document all of that too
Accept and celebrate that small amounts of progress on tech debt are better than no progress on tech debt.
A corporate truism that actually makes me want to vomit is “Don’t let perfect be the enemy of good.” It’s literally so dramatic why are we using the word “enemy” about our silly little jobs.
Unfortunately it’s a thousand percent right though. Your little 10-20 minutes a week of progress on your tech debt tickets is so much better than no progress on them at all. Plus, these things tend to be a bit infectious. If you start caring about tech debt, I’d be willing to bet your teammates will start to care too.
That’s because you’re, like, the maintenance man, dude. *extreme The Dude voice* You’re the devil-may-care maintenance man who gives a shit about the overall state of your tech stack because you think it’s cool to do good work for its own sake. You don’t get too caught up in drama around who should do the maintenance work, because you think it’s all our responsibilities. It’s like, vital energy, man. We all share the load and it all gets a little easier.
Thanks for reading. See you next week.