Out of the Software Crisis: two-year project review
This is a part of a series where I review the work I’ve done over the past couple of years.
- Two-year review: to plan a strategy you must first have a theory of how the hell things work
- Out of the Software Crisis: two-year project review (this page)
- Sunk Cost Fallacy: chasing a half-baked idea for much too long
- The Intelligence Illusion: stepping into a pile of ‘AI’
- A print project retrospective: the biggest problem with selling print books is the software
- Thinking about print
- Disillusioned with Deno
- An Uncluttered retrospective: Teachable is a mess and I need to pick a lane
What it is and how it happened #
As I have mentioned before, one of my tools for navigating the world around me is research.
- If I have a problem, I research it.
- If I have an idea, I research it.
- If I have a task, I research it.
Over the years I’ve “collected” a number of references and concepts related to software development, and they’ve congealed in my brain as a single concept:
Software development is uniquely sensitive to common forms of mismanagement.
This isn’t to say that managers in the software industry are uniquely incompetent, merely that managers, as a class, seem to be universally disinclined to take their own field of expertise seriously and that software development is a complex combination of virtual (software) and non-virtual (hardware and wetware) objects, and multidisciplinary problems, which makes it more fragile than most other processes we encounter in the modern workplace.
I’d wanted for a long time to explain, in a single book:
- How software development is managed is a problem.
- There are ways to mitigate that problem.
What drove me to finally pull my research together and write the book was COVID.
I was hit hard by COVID in the summer of 2022. The illness itself was very easy to manage, but it left me with heavy brain fog that made focused work impossible. Coding was out of the question. Cohesive long term planning was not an option.
(My eyesight was also affected. It went blurry in one eye, but that only lasted a few days.)
For six weeks I could do little more than stare at the TV.
When the fog began to clear, I discovered that even though I still couldn’t code, I could write.
Not particularly well – extraordinarily shitty first drafts.
But with each shitty draft in hand, I could, through the fog, slowly rewrite each text into coherence.
As I got deeper and deeper into the book, the fog began to give way – like an exorcism through prose. Driving out the demons of incoherence through sheer force of will.
I know the brain fog lifted simply because I was lucky and time had passed, but writing – even incoherently – gave me the sense of control I desperately needed.
Then as I became sharper, I began to hone and direct the book through additional research. Even though the book itself wasn’t driven by market or audience research, its structure and editing very much was. I trawled through subreddits and forums looking for software development dysfunction and analysed each forum-explained disaster as best I could. (What Amy Hoy and Alex Hillman in 30x500 call a “Sales Safari”.)
It didn’t affect the actual content of the book at the time, but it definitely affected the final structure, tone, and presentation.
As I wrote, I had two target audiences in mind:
- It was the book I wished I had read when I was a younger developer.
- More importantly, when I was writing, I was speaking to every product and project manager I’ve ever had and telling them what I, as the lead dev they were working with, wished they understood about software development as a system.
I edited and rewrote the sales page for the book again and again based on the feedback I got from private Slacks (including the 30x500 one) and correspondences with friends and acquaintances who were willing to help.
The book is as relentlessly pragmatic as I could make it. Whenever I referred to a theory or concept from academia, I tried to immediately put it in a practical context and tie it to actual software development practice.
The driving thought behind every page was that it should feel real – it should be absolutely obvious to the reader that every opinion in it was formed in the fires of an actual software development disaster and isn’t just an academic analysis cast down from an ivory tower.
What worked #
It was my first genuine attempt at applying entrepreneurial thinking to my work and I think it worked quite well overall. Reviews from readers were very good and it has sold better than anything else I’ve done.
Not a bestseller, but respectable.
In terms of the principles I outlined in the first entry in this series, I think I did well.
1. Affordable Loss #
I was only spending my own time, much of which I scrounged together in the periods of cohesiveness I had between the fog. If it failed completely I wouldn’t have been any worse off than if I’d done nothing.
2. Co-Creation #
I restructured the book completely based on early feedback and the edits were based primarily on the research I did on the issues people seem to be facing in forums and subreddits. The sales page for the book was edited over several days with the extremely helpful feedback I got.
Even after I first published the book, I took some of the early feedback and people’s responses to the first mass lay-offs in tech, which were happening around that time, and added four bonus essays to the book that looked specifically at some of the higher level risks larger corporations were facing.
3. Means #
Like I mentioned above, I have no shortage of opinions that are grounded in both research and experience. I’ve worked in web development for over twenty-five years and have formed over-opinionated ideas for what does and does not work.
My PhD years left me with an incurable tendency towards structured academic research practices, which means those opinions are also informed by a heavy dose of theory and reading.
My time working in publishing I’m fairly adept at producing decent ebooks and PDFs.
This wasn’t even my first attempt at self-publishing. I experimented a bit with self-publishing back in the early 2010s with a series of fantasy fiction novellas.
It didn’t work out, but I also realise in hindsight that I was in a particularly unsupportive environment at the time. It might not have been the horrible ghastly mistake I felt I’d made at the time. For the longest time I felt deep shame for having even tried, but that feeling is slowly fading away. However I feel about it, the experience left me with a understanding of how self-publishing works that’s still relevant today.
I also helped publish the illustrated children’s book Von be don: Magnús og Malaika leysa málið a few years ago, so I have to face the fact that I have quite a bit of experience with making and publishing books.
I’ve also always enjoyed writing, which has resulted in quite a bit of writing practice over the years.
My means lent themselves to creating an ebook.
I was also writing exactly the book I wish somebody else had written for me years ago. I had good reason to believe that there was a market out there of people who had been in my position and would benefit from the book, and the research I did while editing the book supported that.
4. Lemonade #
The COVID disruption left my ability to program temporarily compromised, but I was still able to write shitty first drafts and then edit those texts to be more coherent. When the fog began to lift and my ability to do focused work slowly returned, I had the outline and the core substance of a book I could then finish properly.
I definitely managed to make lemonade out of the COVID lemon.
Control the Unpredictable #
More importantly, making something was a way for me to take control over what little I could in my circumstances. I was making new possibilities for myself out of thin air where none existed before.
The book did better than I expected. Over 90% the sales have been direct despite it being available on the Kindle. This tells me that it was a very good match for the audience I can reach. It hasn’t made me rich – I would have made much more money if I’d spent the same time working as a freelance consultant – but it’s a pretty solid accomplishment for a self-published title with no marketing budget.
It also continues to be useful to people. Every month somebody discovers it for the first time and finds it helpful enough to post quotes from it on social media.
Freelance projects never have that kind of longevity.
What didn’t work #
This being the project where I followed most of the principles of entrepreneurial thinking and the 30x500 model (for the most part), it’s also the one that seems to have mostly worked.
I probably priced it too low. I think I could have priced it at $45 USD at least.
Even though the reader I had in mind while writing was a product or project manager, the audience ended up being mostly mid to senior-level software developers. I like to think they’re using it as an aid to help them convey to their manager what their projects need.
A minor issue that sometimes comes up is a bit of an audience-author mismatch. I’m a web guy. I’m a big fan of the web as a platform and I think it’s very poorly used by modern web development. But based on the conversations I’ve had and the feedback I get, quite a few of the book’s readers think the web is itself a horrible mistake that is to blame for much of what has gone wrong in software development. I don’t know if that had a meaningful effect on my platform or the success of my later projects or not. I’m inclined to think that it didn’t.
Where things went wrong wasn’t in the project itself but in the follow-up.
The book would have been a good foundation to build on, but I got stuck right back into the outcome-oriented rut once it was out. Instead of looking at what I could accomplish in the near term with what I had at hand, I went back into the mode of picking a specific long term objective to work towards.
What happened after the book? #
I did one thing that was sensible. I created a new landing page for the newsletter that tied in with the book. There are a few things I’d like to fix with how I originally set that page up, but the idea is sound.
I should have built on that by continuing the audience research, writing focused essays that helped people solve specific problems, and maybe even put together simple tools that would help them manage specific software development problems.
Something along the lines of hillchart.co, which is a simple web app for making hill chart images, for example. Maybe not that idea specifically, but similar.
But, at the time, I didn’t see how that could lead to something that generated income. I didn’t see myself as getting into product or project management in my freelance work and I didn’t have the imagination to think of ways this could lead to books, courses, or other ways of paying the bills.
I was also quite stuck on continuing the research I had done on the Colophon Cards project, which has both led to a user-tested design for a bookmarking app and a more computer science-y prototype of a prose-oriented version control system for collaboration. I’ll go over that project, what went wrong, and how it ended in a later post in this series.
What matters here is that instead of spending my spare time building on the success of Out of the Software Crisis I spent it chasing funding for a research project that was unlikely to ever pan out.
The other time thief was my research into generative models such as ChatGPT, which resulted in my second ebook The Intelligence Illusion: a practical guide to the business risks of Generative AI. That book will be the third project I analyse in this series.
I also turned it into a print book late 2023, but that was involved enough for me to analyse that as a separate project in this series.
What should I do now? #
The book is still selling, which makes sense. There’s nothing in it that’s particularly likely to become outdated – at least, not in the space of fifteen months.
The thing I need to remember is that good ideas tend to remain good ideas.
That list above of what I should have done after the release? I can do that now, this year. Those ideas haven’t expired just because a year has passed.
I can get back into writing about software development as a system and how it can help you solve problems, both in web development and other kinds of software development. I can look at delivering simple but useful tools as PDFs, simple websites, or even simple Squoosh-style simple web apps.
It might be less effective than if I had acted right after publishing the book, but that doesn’t mean it’ll be ineffective.
It’s clear that the book’s existence opened up a number of options.
How they might generate income is still a little bit unclear to me, but even if the worst case scenario is that I might sell a few more copies of Out of the Software Crisis then that’s a win.
Next: that damned research project.