Are you a game master who dreads the tedious task of taking session notes? Introducing GM Assistant, a web app designed to transform your audio recordings into organized, accurate notes effortlessly. I'm Alex, a software engineer and fellow game master, and I'd like to share the story of how this tool came to life.
Back in August 2023, I quit my full-time job as a software engineer, partly due to a company reorg I was not happy with and partly due to burnout. After that, I went deep into being a house-husband, gardener, and gamer.
Once I got antsy and concerned about what I was gonna do with my life, I looked into a ton of business ideas from freeze-drying candy, 3d printer farm, and even power-washing. Those numbers were really depressing coming from a software engineering income and frankly would be harder jobs.
Once I got my head straight, I remembered I actually enjoy writing code and solving really hard problems. I also had everyone telling me I was a good engineer. So I applied to Recurse Center in NYC at the behest of a friend of mine. It's basically a 6-12 week retreat for programmers where you can work on anything you want with the support of a community.
Going there relit some serious fires I had deep down. I had no idea what I was going to work on when I got there but my "batch" of 40 or so super passionate and some super ambitious programmers had their ideas. A group of us ended up attending/forming an insane bootcamp with some ML/AI experts who were there.
The next two weeks were a roller coaster, we learned all about modern AI, LLMs, Diffusion Models, etc. We wrote our own code from scratch to create neural networks, we trained our own models with public data sets, we even got into mechanistic interpretability to try to *dissect* how these neural nets worked.
Coming out of those grueling two weeks, I still had 4 weeks to go in the Recurse Center. Every day I was listening to the latest AI news, being filled with dread and anxiety about the future. Not because I think AI is going to go skynet on us, but because it was impossible to understand what my career would look like going forward. AI was writing code for people. "Devon" was announced about this time which could write programs without hand-holding (turns out it was a lot of hype IMO).
The idea
In my remaining time at the retreat, I needed to work on something that I could point to and say "I made that". While burned out as a Dungeon Master at the time, I was also concerned I didn't want to do that anymore. So with AI simmering in my brain next to DMing, I had some ideas start forming.
The obvious things are character art generators, name generators, story generators, blah blah generators. I mean "generative" AI, it's in the name. I did actually write and train a name generator but it didn't take more than a day. I did it in a way where it made totally unique names that *sounded* like names and that was cool and different but still small potatoes. Here's a screenshot:
Learning AI at the lowest possible levels really gave me an understanding of what it can and can't do, and more importantly what it would be good at and what it wouldn't be. Then I had a random idea, what if I could remove the part of game mastering that I hate the most and am terrible at, taking session notes.
The Webapp
In my remaining weeks at the Recurse Center, I cranked out a very ugly proof-of-concept for GM Assistant.
Even though that was only about 7 months ago, transcription tools were way more expensive and AI models weren't nearly as good. I tried all sorts of things like GPT-3.5, Anthropic's models, AWS transcriptions, etc. The results were pretty good, but I did get a lot of hallucinations where all of a sudden my players were in some town that didn't exist in my world or the brief mention of South Park made "Mr. Mackey" an NPC. Even still, I was excited because it saved me a lot of time as a DM even if I had to heavily tweak the output by hand. That's how I knew I was solving a real problem that at least some other game masters must also face.
My gaming PC in my home office gave me access to a GPU I could use to develop my own "stack" without spending money. Some frustrating work with very nascent libraries and tools made it so I could finally get really, really good results in a reasonable amount of time for cheap enough I could turn it into a business.
Now we have users (!!!) and our tech stack is becoming pretty fancy and robust. Although in the near-term we are focused on our core feature of Audio → Notes, we have so many other ideas. I am super excited about what's to come.
Check out our plans here: Roadmap
Any new features we add will adhere to our AI Philosophy
I invite you to try out gmassistant.app, provide feedback, or share this post with others who might benefit.