I was on data twitter when DuckDB first started making waves. It was fun to watch my data twitter friends lose their minds over it. I didn’t understand a whole lot of what was going on, but I grokked that it was easy to use, fast, and fun to mess around with. I’ve felt the joy of using a delightful technology before, and I wanted in!
I spent a weekend trying my damndest to get Meltano to load some CSVs into duckdb and it took all weekend to figure it out. I found Meltano really hard to use. It took so long to get the data into DuckDB. The whole process was so frustrating that I decided I wasn’t smart enough to finish the project, and I abandoned it. It’s especially embarrassing that I abandoned that project after posting about it on this blog, twice.
I wasn’t even trying to rig up anything particularly fancy or impressive. I love hockey, and I thought it would be fun to hook up a little data “pipeline” that read some static CSVs that held game data, dropped em in DuckDB, and then I could transform the data into some fun tables and graphs. My desire to do this project stemmed almost entirely from wanting to understand why data twitter was so excited about DuckDB.
I just wanted to be part of the fun online and enjoy the act of making little things with technology with my data twitter friends. That feels kind of embarrassing to admit because I am in no way as smart or experienced as my data twitter friends—they’ve all been in the game for many more years than I have. That’s not intended to be a grab for compliments or pity either—it’s just the truth. I am actually quite confident in the abilities I have and what I bring to professional settings, I just know when I’m around people who are smarter than me is all.
So, I abandoned that DuckDB project and I still haven’t come back to it since. I still spend a lot of time in online data spaces, though. I still see the same data twitter (now bluesky) friends tinkering and building stuff and showing each other what they’ve come up with. I know exactly what it feels like to make a computer do something that you asked it to do and how much fun that is, and how exciting that is. I just haven’t done it with anything that impressive.
When I have felt that “builder’s excitement” it’s been things like rigging up a dbt model that reads data from temperature sensors and flags when a temperature sensor is reading outlier ranges for more than a week. That was a model that I built at my last job, and we used it to proactively alert health workers in Cote d’Ivoire when their vaccine fridges were acting up and needed maintenance. It’s probably my favorite thing I built at my last gig.
I like to see my data friends tinkering and building stuff because I’m sure it feels good for them the same way it felt good for me to finally get that temperature alert model working.
But like, I want to be part of the fun too 🥹. I want to build something that isn’t behind an NDA or running on proprietary data or something I otherwise can’t show people but can only talk about. I haven’t done it before. It’s not a helpful feeling to want to be part of the fun and vibes of building stuff with code, but not know what to build. If you’re a bit of an overthinker (like me), you can get caught up in the whole question spiral of it all.
What if I run into a roadblock that I can’t pass again like the DuckDB project? What if I don’t like a certain tool and I abandon using it but the people who make that tool take that personally? What if the data I find is lame? What if the graphs or maps I make have been made a gazillion times already, what am I even contributing? What if my data friends don’t care that I’m trying to join the fun and I just get ignored trying to chime in? What if I am simply just too dumb and inexperienced to actually find some data from somewhere, dump it in a database, transform it a bit, and then use it to make something visual and fun? Do I belong in this industry at all? Everyone else seems like they get it so easily.
Well, there’s a lot of platitudes that you can throw at (extremely human and I’m hoping extremely common) sentiments like that.
⭐ Learning is more important than what other people think.
💫 No one is thinking about you as much as you are thinking about yourself.
💃🏽 Data people are nice and they will think it’s cool that you’re trying something. They respect effort.
👩🏽💻 No one is grading you on your little side project. It doesn’t need to be very good, you’ll still learn something along the way.
🤪 Comparison is the thief of joy!! (probs the one that makes me vom the most)
Unfortunately, most platitudes are at least a little bit true, and I think all of those are very true.
I think what they miss is a feeling of honor and respect to the fear that holds me back from participating. The fear is a natural part of human interaction and wanting to be part of a group. She’s like my own inner big sister, and she wants me to have fulfilling social relationships. She’s watched me experience social rejection many times in the past, and she sees how much that hurts. In bringing up the question spiral of “what-ifs”, my fear is trying to protect me from the pain of social rejection. I understand that she wants that for me, and I respect that she wants that for me. I want that for me too.
But, I also know that the presence of fear in something as low-stakes as this is (and it is low stakes to agonize over starting a personal project) probably means it’s worth leaning into and trying anyways. Fear does not just serve the purpose of protecting me from social rejection. She’s also very observant of how I change over time and how my abilities and knowledge grow over time. If I ask her to, and continually encourage her to, my fear will let go enough of her questions to let me try the thing anyways, and instead she will watch for how I change through the experience. She does this for me when I go to my improv class, and when I go swing dancing. Two other things I have no experience with but have been trying lately.
When my fear shifts into a more helpful, observational role instead of trying to suffocate me with what-ifs, I can see the truth of the matter a little more clearly.
It matters to feel socially accepted in your profession. I largely do feel that way, barring the odd bl;’’’’’’ips and run-ins with tech bros. It is also normal to not want to risk that social accept/ance by looking stupid.
Despite the unfo]rtunate emergence of influencer dynamics in the data world, people do not think about me in nearly as much depth as I do, if they think about me at all. A little healthy dose of “no one cares what you do in your side project” is a helpful balm against fear of judgment.
Deciding in advance that data friends will judge me for not being as smart as they are is denying them the opportunity to be kind. There are going to be shitty people in every industry. But deciding in advance that people I’m intimidated by are going to treat me poorly just because I’ve met people like them who did in the past isn’t fair. We owe it to each other to be generous and give each other the benefit of the doubt.
I do actually like just tinkering and building stuff for my own enjoyment alone. I’m not very good at it, but it is addictive and puzzle-like to fuck around with a python script until it spits out the data you asked it for. Doing exactly that has consumed my brain for a few weeks now. It’s normal for me to want social affirmation that the stuff I build (or write!) is cool, but it’s also true that I just do it for fun as well. Some of my best (in my opinion) posts on this substack and some of my favorite fic I’ve ever written are my least loved in terms of views and likes.
It’s giving myself a gift to just try to jump into the fray, consequences and questions be damned. If my worst fears come true and for some stupid reason my python script being shitty makes me lose all my data friends (as if that would ever happen), I can always delete the github repo and never come back to it.
But, if I happen to enjoy the process of building a little data pipeline, if I happen to learn a few things from chatGPT coaching me through the whole process (yes, using an LLM counts, remember how no one is grading me and this is just for fun and vibes?), then I have gained something really valuable. Feeling self-sufficient and watching myself try to learn something completely from scratch and then figure it out is free serotonin and a building block to more professional confidence. Isn’t that worth risking looking dumb online for?
I think so. I’ve been obsessing over some goofy little weather data for a few weeks and shitposting on Bluesky about the process, and my data friends have been really nice about it. It turns out the good outcome was just waiting for me there all along.
Thanks for reading. I need to go figure out how to get GitHub Actions to read a secret into my python script, wish me luck!
See you next week. Please subscribe if you haven’t and if you want to give me free serotonin.
I love the honesty here. I've often felt the same way, tbh.
I find there are also other fallacies on the mix affecting my own self-judgment, e.g.: 1. when I see lots of people making lots of cool things, I tend to think I should be making *everything* everyone else is building on top of my normal work, which is obviously impossible (some sort of twist on the composition fallacy), 2. you don't see how long it took them to build the project, only the output - it easily may have taken months for them to build something, and the social media post just makes it seem effortless
also, good to remember that for some people (esp those in dev rel or some founders), doing this well is sometimes their main job, period. so of course they'll be exceptional at it 🙂