Our first Hack Week produced great demos. Almost none shipped as built.
I ran both of Quantum Metric’s company-wide Hack Weeks, in spring 2022 and summer 2023. The first one didn’t work.
All of engineering and product blocked off the week. Pitch, build Monday through Thursday, demo on Friday. People built genuinely good things.
Then the good projects died after demo day. Fewer than 5% of the hacks shipped as built.
My read on why
In 2022, people who hadn’t taken part voted on the winners.
I can’t prove that’s why projects died. But it meant the prize went to whatever demoed best on Friday. Nothing in the week asked whether a customer would use it on Monday… and I wanted that question at the center.
What I changed
For 2023, I made two changes.
A judge panel instead of a popular vote. Judges scored every hack against four value categories. The question became “would this matter to a customer?”
An LLM track. Large language models had just gotten interesting, so I added a dedicated day for LLM training and pitching, plus an award for the best use of generative AI.
A hack week is the cheapest place there is to be bad at something new.
What happened
Since then, roughly 40% of the hack ideas have been adopted into the codebase.
That number needs a caveat. It’s not the same measure as the first one: “shipped as built” and “idea adopted later” are different things. And I changed two things at once, the same year AI took off.
So this isn’t a clean experiment, and I won’t pretend it is. What I’m confident of is the direction: more ideas outlived demo day.
If you run a hack week
- Measure what ships, and keep measuring. Adoption can take a long time to show up.
- Let judges score value instead of letting non-participants vote on the demo. That’s my bet, not a proven result.
- Point it at the thing you need to learn. For us in 2023, that was LLMs.
Under 5% is a fine place to start. It’s a bad place to stay.