Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

IIRC, Jeff Atwood of Discourse said it takes at least three years for a project to finally approach the objectives that it set out to achieve (or something). I'm sure he was referring to his experience with Discourse and Stack Exchange at the very least.

I might have butchered that, but three years has been my experience as well. I chuckled when I read that post because the three year mark was finally when I looked back on my forum, felt somewhat content with it, and went "holy shit, that took a while".

My forum had active users from day one since I was porting vBulletin to my own stack. I actually used Clojure + Datomic for the first iteration and rewrote it to Node + Postgres a year later.

As developers it's easy for us to go "3 years? Psh! What are you doing all that time!" because we like to pretend everything is just a pure technical problem and consistently underestimate the work. I certainly wasn't working on my forum for 3 years full time.

It was a hobby balanced on community building and creating a compelling experience for my users. It took a week to build the initial release where people could actually register and post. From there I'd burst a week of full-time development work into it every 1-3 months.

I'm glad I experienced this journey, though bless my poor users that had to wait on me to implement basic conveniences over the years. I can say with certainty that I will never go "hah, that looks easy!" ever again.



Curious - why did you move from Clojure/Datomic to Node/Postgres?


I had used Datomic on my previous hobby project, so when I had to pick a database for my forum rewrite, it was what I was most comfortable with and it was also just fun to use.

For example, I figured I could use its transaction querying for post edit history. Datalog made it trivial to jump arbitrarily through any relationship. And transactor functions were trivial to write at a time when I had never written a Postgres trigger function.

I didn't apply much professional due process here because I looked at my forum rewrite as a hobby project. It's not like I was being paid.

However, over the course of that year, my forum suffered due to my hobby-level understanding of Datomic.

For example, I had the forum application, the Datomic transactor, and the Datomic client running on one Linode machine. I told myself I'd figure out how to host Datomic on AWS in the future, but of course that never happened.

As the forum grew in size, I started dreading restarting my application because the Datomic client would have to sync with the database. It started to take many minutes. And I didn't have enough information to know why -- is this just I/O limitations of loading info into the process? Or was the fact that hundreds of users were F5'ing the server? I never made myself a real deploy system either. I would manually ssh in, `tmux a`, `git pull`, and manually restart my processes. Very amateurish, but it was so easy to say that I'd fix it in the future.

I also made mistakes like having zero insight into my application. My application was growing slower over time, but I didn't know why. I had some really slow database queries and I was still using LIMIT/OFFSET for pagination at this time.

I also found out that Datomic isn't made for massive blocks of text like forum posts. When I saw that Datomic had full-text search, I assumed that meant otherwise.

A year went by and I started to get stressed out about the forum. I had this crippling guilt that I even tried to build a forum from scratch instead of just using Xenforo. Felt bad for my users who were hanging in there or just leaving out of frustration.

One of the reasons I rolled my own forum aside from thinking that it would be easy was that it would also be cheaper. I was spending like $500/mo on the forum when I was using vBulletin and I was frustrated that I couldn't do much to change that. By my hand-rolled forum cost 1/5th of that, so at least I had that going for me.

During all this, I started working with a friend on a different project and he refused to use Clojure. We ended up with Node + Postgres and he was a Postgres pro who I learned a lot from.

Before long, I had a full Node + Postgres application under my belt and decided to rewrite my Clojure + Datomic forum. I could have swapped out Datomic for Postgres instead of throwing the baby away with the bathwater, but I wasn't convinced that that would be any less work.

I also decided to host my rewrite on Heroku this time to force myself to rewrite the forum in a more stateless way. For example, my Clojure forum had a hard dependency on the filesystem. Not to mention having the database on the same machine meant I'd never be able to spin up another application server.

Of course, any time you rewrite something, you have the opportunity to fix your legacy mistakes from day one which I did. For example, I finally optimized pagination away from LIMIT/OFFSET. I got to clean up a lot of cruft. I also used RDS for Postgres which solved all my database problems. I really had no business using Datomic since I had such a weak understanding of its trade-offs.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact