Ok, sorry, bitter old man coming through: this is Web 2.0, not 1.0. For all the buzzwords, Web 2.0 was defined by the dynamic interactive solicitation of user input as opposed to Web 1.0 being just static HTML. I don't think we've coined a good catchphrase for fat applications implemented in tons of Javascript with only lightweight AJAX calls to the backend.
Nah, this is as 1.0 as you can get, assuming you're not asking for completely unstyled text. Forum scripts with this sort of simplicity were around for decades, with UBB being created in 1996 (and having significantly more features than this software), and basic scripts like WWWBoard dating back to 1995 or earlier.
It was AJAX and processing data via JavaScript that was a web 2.0 thing (for the most part), not just submitting forms in general.
2.0 is not about a switchover date, it's functionality, or even philosophy. Mid-90s forums would be understood as visionary betas of Web 2.0 under this terminology. I concede that Orange Forum is definitely very early 2.0, possibly even very late 1.0, but it it most definitely not "as 1.0 as you can get". That would be static websites with at most a guest book and/or email form.
Nope. Flickr was the quintessential Web 2.0 app, and it wasn't very JavaScript heavy at the start. Web 2.0 was fundamentally about user-generated content.
"Web 2.0" came around about 2004ish with Digg and other sites using AJAX. Or at least static content was not the defining factor. There were plenty of forums around in the late 90s with dynamic content.
Yeah I was looking at that too. The definition is kind of nonsensical. Slashdot met that definition in 1997 but nobody was calling it that at the time. Web 2.0 became a very popular buzzword with Digg, Flickr etc. Or at least that's how I remember it, the author is not "wrong" for referring to it by a very popular definition.
What's nonsensical about it? Slashdot was very avant-garde, so fits well for a term coined in 1999. I agree that AJAX represented somewhat of a technological watershed, but wikis and blogs (which came of age pre-AJAX) represented a social watershed, a much better anchor for the democracy that PG seems to consider only from a technical perspective.
This is from the criticism section of your link, by a Brit who would know better than most:
Nobody really knows what it means... If Web 2.0 for you is blogs and wikis, then that is people to people. But that was what the Web was supposed to be all along... Web 2.0, for some people, it means moving some of the thinking [to the] client side, so making it more immediate, but the idea of the Web as interaction between people is really what the Web is. That was what it was designed to be... a collaborative space where people can interact.
He's also called it jargon. It's not a well-defined term (or at least there are multiple definitions for it).
You're wrong. Neither Digg nor Flickr had anything to do with AJAX. You're thinking of the "Web App" era precipitated by Google Maps. That's a whole 'nother thing.
"Web 2.0" and "crowdsourcing" were more or less one in the same.
Maybe people (including Graham) have been trying to revise "Web 2.0" to include more things, to simplify the taxonomy. But when Web 2.0 first started being discussed, it wasn't about JavaScript, it was just about interactivity, specifically writeability.
From what I recall AJAX wasn't for making things faster or for scaling like other siblings posts describe.
How I remember it, it was created for user experience reason. Not having to submit a form and refresh the entire page and cause the page to bounce back to the top of the page, etc.
AJAX (at least when I learned and implemented it) was for updating the user about an action they performed without refreshing the entire universe in front of their eyes, only updating the contents of a div or something else.
The whole web app as a javascript app.js blob idea took the AJAX idea to an extreme. In this extreme it often does slow down page load speeds and negatively effect the user experience when Javascript doesn't work, for whatever reason.
The idea of AJAX/SPA was to provide interactivity and a less technical UI that caters to the average Joe. AJAX/SPA moves some of the computation from the backend to the frontend so the service scales better for millions of users.
This forum is not going to be fast for millions of users (it might not even be for hundreds of simultaneous users) because the server has to render everything again for everyone.
Server side rendering is good for response times, that's why the SPA world went back to it (Google: react ssr). SPA is good for scalability and average Joe UI.
...and AJAX is there so we don't need to do a full page reload all the time. Again, not because of response times.
> AJAX/SPA was to provide [...] a less technical UI
I hadn't heard that reason before. Can you please talk more about this?
> This forum is not going to be fast for millions of users (it might not even be for hundreds of simultaneous users) because the server has to render everything again for everyone.
This theory doesn't match reality. In reality, multipage applications are faster than single-page ones. In fact, most things on the internet still are multipage, including web forums. Most web forums are powered by PHPBB, a multipage web app, and they were powered by PHPBB 15 years ago, when hardware and PHP were much slower. This very site is a multipage web app, and it holds up just fine, even though it has enough users to bring down other websites.
Single-page applications seemed like they would be faster, but usually they aren't.
First, let's look closer at a multipage application. All of the scripts, styles, and images should be cached after loading the first page, if you set the headers right (the Expires header, mainly). Therefore, what is left? The content. And I have found that the size of the HTML content is often close to the size of the same thing in JSON --- at least the way I write HTML (I try to keep it lean. Few classes, extraneous divs, etc.). This is because JSON has all those keys:
{
"color": "red"
}
while HTML has just the values:
<div>red</div>
That was a simple example, but I measured a bigger one, and the size turned out to be about the same, especially after compression. All of those HTML tags compress well, because of repetition.
Now if they are the same size, then they should load in the same time. Except they don't. The server-rendered page loads faster. Why? Because of progressive rendering. When just some of the HTML has come down the pipe, the page appears. But with AJAX, all of the JSON must first load, then be parsed, then wrapped in a template, then inserted into the DOM. Then it appears all at once.
Average Joes want spinners, buttons, draggable stuff, toggles etc. that don't lead to a page load everytime you press them. Sure, you can say that "well just make a multipage app with some jquery". But at some point you're just approaching a SPA with SSR support.
Imagine facebook's chat but everytime you hit enter you had a full page reload :D
> In reality, multipage applications are faster than single-page ones.
I agree. They are faster for a single user when the usecase is a semi static page like a forum. However, the SPA + API model makes stuff easier for the backend side and allows that average Joe UI stuff. It scales well to millions of users.
The thing is, it's not about you. It's about everyone. We want to build services that please the majority and backends that stay up when the majority is logged in.
Also, it really isn't possible to build realtime/dynamic stuff with a "let's render some HTML on the server" mindset. How do you do realtime chat? Realtime games? How do you do a collaborative text editor? and so on.
JSON/SPA/AJAX are not there to so your static forum page loads fast. They are there because of the next generation of the web.
Load and bloat? Well that depends. If you're doing a realtime collaborative texteditor with a realtime chat, I just don't see how you'd get anything even remotely nice without what you call "load and bloat".
However, to return to the original topic. Which is simpler:
a) creating a performant data API
b) creating a performant server that handles data stuff just like a) AND all the rendering & UI stuff
Surely the latter is more complex since it has more responsibilities?
No, but I've seen comments on here from people under the impression that rendering on the client is a more efficient use of CPU, or that rendering a page in HTML is significantly more resource intensive than rendering the data in json. There is a generation out there that doesn't seem to know that server side rendering exists.
>There is a generation out there that doesn't seem to know that server side rendering exists.
Oh, don't worry, they're reinventing it in JS. I've had a recent project proposal and one of the engineers said that to make the webpage faster we could prerender the pages on the server side with JS.
Like, at that point, you're just using PHP with a different name and worse performance (a small 4$ shared hoster can easily handle 1M users per month, I've yet to see a NodeJS app handle 1M users per month on a 5$ VPS without problems)
>Do you get why you'd prerender a JS app and why an isomorphic setup is fundamentally different than server side rendering with PHP?
Tbh, I'm not really interested. I can squeeze amazing performance out of a 10kB self-written JS library with noscript fallbacks than most JS-heavy apps out there. The sites I develop work with dialup connections or worse. Because my phone regularly has only such a connection. Modern JS apps are a pain to use under such conditions, I've not noticed much difference between prerendered JS and unprerendered JS, it's both crap under these conditions.
On the other hand, Orange Forum loaded almost instantly (less than 10 seconds) on a crappy GPRS 16kbps connection. Discourse or NodeBB don't load at all and if I'm lucky enough I might see some error message or crapped out webpage.
>Also, JS performs as well or better than PHP. Not sure why you guys get so ideological about this.
I have only ever seen evidence of the contrary.
I can run a 1M user/month website with 128MB of RAM on a shared hoster using PHP. If you get a good shared hosters you can probably hit 10M user/month.
I have not seend a nodejs app that can handle 1M user/month on a 5$ VPS, which has 512MB RAM and probably more CPU and disk than the shared hoster offering.
But I'm willing to rethink this if I see real world evidence that a comparable software set runs better and more efficiently in JS than PHP. I won't consider synthetic benchmarks since those rarely model real world usage and comparable means the software must be usuable with and without JS on the client enabled.
Discourse is a Rails forum with a JS frontend. It's bloated. I don't disagree with your PHP experiences, you're extrapolating from:
1) your personal experience
2) mature software vs immature software
3) bad software vs good software
You can't say you're "not interested" in understanding the other side, throw out your own anecdotal benchmarks, draw conclusions from that, and then demand that others provide "non-synthetic" benchmarks in order to prove you wrong. Well, you can, but it doesn't seem especially objective.
My point is, you can write any kind of app with almost any kind of server-side tech. If you don't like the culture, you may be right, but please be explicit about it.
Which is why I never bought into the JS framework fad of the month and keep happily using .NET/Java stacks with server side rendering and minimal JavaScript.
We should make the best use of pure HTML/CSS web pages, that is what the web was made for.
For anything else, better go native with Web APIs.
The benefits from client side Vs server side are really a matter of scale. If you're running a personal forum / whatever then you're not going to notice a whole lot. But when you start having several hundred thousand or more concurrent users then being able to cache your pages in a CDN and only generating JSON responses via APIs really can have a profound impact on your server side resources.
You can cache server rendered pages in a CDN or Varnish or whatever you like, and there's also the use of the proper http headers to drive client side cache control. All of this works for server side rendered content.
Obviously there are a great many layers to caching (there's a whole multitude of other solutions out there that you've also missed off :p). However with regards to the specific points you've raised:
1) You cannot cache server rendered pages in a CDN (see footnote) if those pages contain user specific information (like this topic does). Information such as a user name, message read status - even on public messages, etc. If you do CDN cache those pages you'll leak user-specific information to other users. This is why you need tools like Varnish that cache page fragments rather than whole pages; or why you serve up HTML templates and then populate user information on the client side via RESTful API calls.
2) For similar reasons as above and again very relevant to the project in this discussion, HTTP Cache-Control headers also wouldn't help with the HTML if your backend is generating the HTML. In fact in those instances you'd probably want to set your max-age to 0 (again, speaking strictly about dynamically generated HTML. Static assets like JS, CSS, and images are a different matter but they're not server generated dynamic content). Granted with browser caching there isn't the risk of leaking user information to other users; the risk is just the browser not fetching an updated forum view / whatever.
Caching is one of those things that are easy to set up but also very easy to get wrong.
Footnote:
Akamai might support it - if you're willing to pay their premium - as it's an immensely sophisticated product. However it's not an option I've seen when using Akamai and the other solutions I've used definitely don't support caching page fragments.
1) nor can you cache client-side rendered pages in a CDN ... you can only cache EMPTY information-free pages in a CDN
2) Agreed, but again, same problem as one
Truth of the matter: Javascript rendered pages
1) request(s) for the page
2) response for the page
3) at least one AJAX request (in practice: dozens) (cannot be akamai'ed)
4) waiting for all responses
Server side rendering without AJAX
1) request page
2) respond to page
Seems to me server side rendering is guaranteed to be faster, if you count the full cycle. But sure, with AJAX you can show your logo faster. With server side rendering only you can show the actual data people are looking for faster.
Again, this is where the question of scale comes in. Server side may well work out quicker when you only have a few hundred visitors. But when you've got hundreds of thousands, being able to offload > 90% of your content makes a massive difference. JASON APIs are less expensive to get generate than full HTML pages which include the same content. It might only be a fraction less work on the servers but multiply that by several hundred thousand and you quickly enable yourself to scale down your back end infrastructure.
This isn't stuff I'm blindly guessing on either, I've worked on several hugely busy UK and international services that started out as monolithic code bases pushing HTML and migrated them to APIs. Each time the language remained the same, the main body of the backend logic even remained the same, but instead of pulling HTML templated from the backend and pushing out the completed contents, the move was to push out the templates and have the client side render them. Each time that change was made the back end infrastructure could shrink. In some instances by a factor of 10!! The drawback is the client side took longer to render the page upon first impression, however the application ran faster from then on in and the hosting costs were reduced as well.
So this is why I keep saying scale matters when discussing APIs Vs server side HTML generation. For the vast majority of project that people build, there isn't going to be much in the way of difference (heck even my own personal project are built API-less unless I specifically need an API for other - none performance relates - purposes. But when you start talking about hundreds of thousands or millions of concurrent users, then even small changes can have a real impact and thus that's when you really start to appreciate some of the advantages API driven development can offer
If you lose performance on every single request, scale will only make you lose more. But yes, you can offload a tiny amount of server-side processing onto the client side. If that makes a difference in your costs ...
No I can't say then go for it. What you say is bullshit. Offloading string mangling to the client at the cost of doing extra requests is absurd.
We established that a single request (ie. directly filled in page) is going to be faster than javascript doing requests. You appear to agree. The only thing scale will bring is a bigger difference.
Javascript/AJAX is simply a manifestation of "Worse is better":
1) programmers start making websites. Good ones, and bad ones.
2) we see lots of examples of badly programmed server-side programming.
3) "we must fix this", and so now we do client side programming
4) incredible amounts of examples of bad client-side programming
I'd recommend you to use Argon2 instead of bcrypt for storing password. It has won the Password Hashing Competition last year and is the recommended way to store passwords. Bcrypt is not bad but it could be used with insecure parameters while Argon2 does not have insecure parameters.
The way you create cookies is also insecure, you should be using crypto/rand instead of math/rand AND rather hex.EncodeToString() the result instead of just generating random numbers in the alphanumeric range.
The hex.EncodeToString() point is a nit. Generate 128 bits of randomness, and then encode it however you'd like. The track record of people trying to get "generate random numbers in the alphanumeric range" isn't great; it's an opportunity to reintroduce bias. Start with a random token of sufficient size, then encode.
The Argon2 vs. bcrypt thing is unhelpful. It does not matter what password hash you use, so long as you use a hash designed for password storage (ie: not "salted SHA-2"). Bcrypt is fine. I prefer scrypt, for the obvious hardware tradeoff. I don't recommend Argon2 to people (or tell people to stop using it) because of the library support issues.
But I think it's specifically a bad idea to tell people to switch password hashes from bcrypt (or PBKDF2) to the trendy new hash. The security benefit of "upgrading" from one password hash to another is marginal.
(Obviously, the benefit of switching from "salted" hashes to real password hashes is not).
Where do you think Argon2 should be present before it is considered to have good library support? AFAIK, it is in libsodium, debian, ubuntu, and other distros.
And I think one can also make mistakes with scrypt when choosing parameters which Colin himself acknowledged. So isn't it time to go ahead with Argon2?
No. Use Argon2 if it's convenient to do so. Not using Argon2 isn't a security flaw.
People have weird ideas about the importance of picking password hashes. It's important not to use non-password-hashes. Other than that, which password hash you use? Not so important.
There are a few 3rd party implementations... But is it more secure to use a lesser known 3rd party package to have Argon2 support or is it more secure to use the more widely adopted bcrypt package supported by the Go dev community?
> The Argon2 vs. bcrypt thing is unhelpful. It does not matter what password hash you use, so long as you use a hash designed for password storage (ie: not "salted SHA-2"). Bcrypt is fine.
Does it not have the issue that it will silently truncate passphrases beyond 72 bytes? As far as I can tell OP does not check for that and the Go API they use makes no mention of it and was affected at some point[0].
72 bytes is not a very high limit when SEA abugida use roughly as many symbols as western alphabets per word/phrase but each symbol takes 3 bytes (assuming UTF-8) rather than one.
For instance "Gujarati script" is 15 codepoints and 15 bytes but and "ગુજરાતી લિપિ" ("Gujarati script" in Gujarati) is 12 codepoints and 34 bytes.
Even fairly trivial codephases are at risk, XKCD's "correct horse battery staple" is 28 bytes but translated to Gujarati (via gtrans) it's 60.
> This is the reason I usually recommend to pre-hash the password with something like SHA512 or SHA3. Dropbox takes this approach too.
Yes, the issue is that you have to remember to do that.
> On the other hand, 72 bytes is a lot
From my own comment:
> 72 bytes is not a very high limit when SEA abugida use roughly as many symbols as western alphabets per word/phrase but each symbol takes 3 bytes (assuming UTF-8) rather than one.
> For instance "Gujarati script" is 15 codepoints and 15 bytes but and "ગુજરાતી લિપિ" ("Gujarati script" in Gujarati) is 12 codepoints and 34 bytes.
In two words, Gujarati (and many other SEA/Brahmic abugida e.g. Tai) is halfway there, a decent passphrase in an SEA script will blow right through.
Interesting.. Looks sleek.. DLang forum [1] is similarly lightweight and it runs as a newsgroup, IIRC. Source code at [2] and previous discussions on HN [3]
The old style forums are showing their age and need to be modernized but not abandoned. See the Archlinux forums based on Fluxbb. It's fast and effective.
The newer ones led by Discourse, Nodebb and Flarum have completely gone in another direction in reinventing how discussion forums should be and perhaps gone too far. They feel strangely 'rootless' and completely lack the 'community feel' of user forums.
This looks promising for something fast, lightweight and easy to deploy.
The UIs on those three should be dialed back (animation, JS), but Discourse is pretty good otherwise.
I've looked at all three:
* Flarum had nausea-inducing animation, and now it overrides natural scroll behavior. (Please never do that to users.)
* NodeBB had some problems when I was using it. If JS is disabled, even the homepage links don't work. Forums should be server-rendered.
* Discourse could be improved by removing most of the animation and Material Design creep (bad for motion accessibility), but other than that, it's the best at the moment. It would also be nice to have easier, full theme customization. Maybe it's in there somewhere, but I haven't found it yet.
I would like to see forum software that has the feel of classic forum software (like Flux), not in PHP, that is server-rendered, with a very minimal default theme (no animation) and minimal JavaScript, and that has many of the modern features of Discourse.
I guess this is as good a time as any to mention that something like that exists (and to toot our team's work over the years):
I started the project years ago and in the last few have gotten really great support from glebm and some others to get it to the point where it's a really lovely, stable, and fast piece of work.
We consider it the best Ruby/Rails-based alternative to Discourse out there (as an aside - wewere honored that Ryan Bigg would point people our way when he stopped maintaining Forem[1])
You said:
√ I would like to see forum software that has the feel of classic forum software (like Flux)
√ not in PHP
√ that is server-rendered
√ with a very minimal default theme (no animation)
√ and minimal JavaScript
√ and that has many of the modern features of Discourse.
I think those check out. In any case I suggest you check out the website[2], the repo[3], and the demo[4].
Interesting. Looks as if you're two people developing it (you and 'glemb', right?). Can I ask is this a startup, or is it a side project? (or something else?) What are your long term goals?
(Your about page says a lot about the forum software & its features, but nothing about who you are or why you're donig this, e.g. vision & goals.)
What about providing hosting? (for people who don't want to / cannot install themselves.) And what about Gmail & FB login, what are your thoughts about supporting that?
Not a startup. It's just an open source project. I wrote it by myself at first but Gleb joined the project several years ago and has pretty much taken stewardship of the project. He's been the spark that the project needed.
There are others working on it as well (much more than I have, or will be) and they're all doing an excellent job.
As far as vision and goals, it depends on what's needed. It's pretty loose for the time being although I could see it maybe changing in the future depending on everyone's circumstances.
I have copied ideas from Discourse — so yes it has some of Discourse's features. Plus features from Slack (i.e. chat), HackerNews (best comments rise to the top) and StackOverflow (question-answers) & Disqus (embedded comments).
More things you mentioned: It's not PHP (it's Scala instead, in Docker containers), it is server side rendered (React.js), fairly minimal Javascript: 150 kb JS on page load.
Looks good. One suggestion: it doesn't work with JS off, so I don't see content when I arrive.
I made a personal commitment not to use Facebook's software (React) whenever there are suitable alternatives available, because I don't want to support that awful company in any way, but hope that your project goes well.
Interesting commitment. Can I ask which URL didn't work for you, & which browser?
For me, when I disable JS in Chrome Dev Tools, the website still loads properly, incl homepage, forum index and discussion topics. (but one cannot leave comments, that still requires JS). I tested a bit in Lynx also.
And regarding the commitment not to support FB, it's because using FB's software supports FB, and FB is terrible for society. I don't want to support them (wherever possible), even if just in a small way.
Discourse does not run on an Ipad 1. It uses a noticeable amount of memory and cpu on any machine. If it is bad for the first world, it is bad for everyone.
What happens on an iPad? Do you mean memory and CPU on the server or on the client?
I do wish that it were server-rendered by default. SPAs are not ideal for content websites for several reasons. And one shouldn't need a recent browser version to read and participate in the WWW.
Discourse uses too much memory on the client and crashes the browser.
As you said, someone shouldn't need latest gen hardware to use the WWW. I personally don't have a lot of control in what the predominate platforms are on the internet, but I have remarked to myself in my bunker that Discourse is a first world echo chamber because no-one else can run it.
Look at hacker news. It isn't pretty and it has lots of glaring usability problems that are trivial to fix, but it could literally run with Firefox 1.0 on a P3-500 just fine.
I love HN's design. It's the "Craig's List school of design" -- simple presentation of information with no distractions.
The WWW has been heading in a terrible direction (animation, over-saturated colors) over the past few years. People use too much JS and CSS3 because it's possible, not because it's a good idea.
Have you tried turning of JS on Discourse forums? It should render a minimal HTML page via the <noscript> elements. Maybe they should do feature detection -- if a browser isn't capable of running the application, turn off JS and serve the minimal HTML. If it works, you could open an issue in their Github repo or post on meta.discourse.org.
HN is no where close to Craig's List in terms of design. They are both minimal, fast and task driven, but Craig's is a work of art. HN is a roughshod hackers tool.
Frankly, it never occurred to me that Discourse could ever run w/o JS. Next time I am near my ipad, I'll definitely test it out. I'd use a combo of allow/deny. Remember when we used "gradual enhancement" ?
I think there should be a standard way for the client to request the low resource usage version of an app, the hack is say you are a phone or request the m.dot site, but it should just be in the accepts header.
The term "Web 2.0" is an unfortunate choice and as a consequence it has been rarely used correctly. Funny enough I've been to a Web 2.0 Conference about 10 years ago where almost every speaker used it incorrectly.
Web 2.0 has nothing to do with a technical revision or change in the Web. It was used by Tim O'Reilly back in 2004 (and became popular) and refers to the rapid change in the way the web is used, more specifically the switch from static web to user generated content.
I'm sorry but your forum is all about UGC, and AJAX has nothing to do with Web 2.0.
Philosophically wrong is using technology that limits the web from being what the interpreter expected, as in this example (there's still a rational reason in this case to do it the "wrong" way and it's based on personal morality).
Objectively wrong being trying to make a website out of custard.
This. If you want old school, go with 10px Verdana, and pad sparingly. I want information, not negative space dammit! With pine, my email editor in 1995, I could read 40 subject lines on a 640x480 screen. With Gmail (in compact mode) on an MBP retina, I get 36. Progress, indeed. Designers 1, Users 0.
The amount of padding in modern websites drives up the wall. The idea that I bought a 24" monitor just so I can read text in fullscreen is ridiculous.
One trick I often use is to have the window take half (or less) of the screen and spoof my browser to be Chrome on Android Kit Kat. I just wish there was a change to have a different browser spoofed per tab, or maybe a "whitelist" of websites that are designed well enough that I won't pretend I'm on a phone.
Initially, I thought of adding no style at all, like this blog[1]. But, without a max-width, I found it hard to read. I'd love to hear your suggestions. What content do you think should be put to reduce the whitespace?
Looks like you'll need to start moderating this already, as of an hour ago, at least.
The process of setting up a public sandbox for users to play with seems like it should be easy, but abusive/obscene posts by users make a testing sandbox unusable/NSFW very easily.
I really like this movement of going back to the basics, web 1.0 style web apps/sites.
I do feel however that there can be a compromise, I think we can build our web applications in the 1.0 style and power them up in the 2.0 style, allowing the capability of the client drive the presentation of the application.
I looked into your work - remarkbox and read through all the while hoping/wishing for there would be a self-hosted version that is not prohibitively expensive for a free personal blog. No luck for me. Back to tinkering with self-hosted isso[1] comment system to make it work for me.
https://posativ.org/isso/
Embedded comments requires JS, but ... the discussion forum part of ED (not embedded) works like Web 2.0, that can fallback to Web 1.0 (I mean, static HTML with no JS). Although this is only partly implemented.
If I had your requirements I would look into isso too! Tinkering is a great way to learn as well, keep up the the great work.
If you decide that you would rather tinker on something else, and just want a hosted comment service to be a solved thing for your personal blog, please reach back out and I'll try to work with you.
This is a nice goal, especially if you need portability all the way down (... to IE 5.5 or something).
However, for any larger web application, this increases the testing effort dramatically, as you need to test all possible sets of capabilities (i.e. all kinds and versions of browsers and other clients). This becomes quickly unbearable burden, unless all functionality is 100% provided and battle-tested by the framework.
Can I ask what (if any) are your future plans with this? And what's the reason you decided to create it? (only to showcase web 1.0 style forums, or ... other reasons too?)
For example are you planning to provide hosting?
Continue developing the open source version and add features like spam protection? Google and FB login?
I'd been working on this on-again off-again for a while after reading Dan Luu's post on page bloat[1].
I'll provide hosting if anyone wants it. That said, ease of deployment was a major consideration right from the start. That's why I chose golang over Django/Rails and decided to offer SQLite as an option to support quick deployment for internal / low traffic sites.
I think you guys are awesome. Keep up good work. Battery, CPU and RAM of my laptop are having the same feeling. Can I deploy it as fcgi script on cheap shared Apache hosting?
Thanks! FastCGI is currently not supported. If your shared hosting allows you to run a binary that accepts connections on some port, it should work with Apache as a reverse proxy. I'll add fast cgi soon. It should be easy enough since golang's net/http supports it.
I built a forum from scratch once and it is a comical amount of work.
The initial CRUD weekend-ware is straight forward. LIMIT/OFFSET for pagination. Throw in some Markdown support. Seems easy enough.
But the devil is in all the individual features that make a forum usable. Like getting notified when someone @mentions or replies to you, marking threads that you've posted in, tracking the high watermark per user per thread so you can create a "go to first unread post", implementing a decent search, making deep pagination fast, a PM system, trying to generalize it.
A serious amount of breadth between weekend #1 and production if your users want the feature set of Xenforo. The main positive I can say is that my forum is cheap to host.
I've had certain forum software sometimes as a side project, sometimes as my man full time project, during the last 7 years. So yes there's lots of work. But in my case lots of time was "lost", in learning web development / React.js / Angular / Dart / other stuff, and porting from the-wrong-technology, to another the-wrong-technology, and building the wrong things that no one wanted.
I think the Discourse (forum software) team were a few people, like, 3? and they spent some year(s) developing the initial version of Discourse. So ... yes some weeks/weekend? for the initial version, and ... >= 3 years ? for a "real" version :- P
IIRC, Jeff Atwood of Discourse said it takes at least three years for a project to finally approach the objectives that it set out to achieve (or something). I'm sure he was referring to his experience with Discourse and Stack Exchange at the very least.
I might have butchered that, but three years has been my experience as well. I chuckled when I read that post because the three year mark was finally when I looked back on my forum, felt somewhat content with it, and went "holy shit, that took a while".
My forum had active users from day one since I was porting vBulletin to my own stack. I actually used Clojure + Datomic for the first iteration and rewrote it to Node + Postgres a year later.
As developers it's easy for us to go "3 years? Psh! What are you doing all that time!" because we like to pretend everything is just a pure technical problem and consistently underestimate the work. I certainly wasn't working on my forum for 3 years full time.
It was a hobby balanced on community building and creating a compelling experience for my users. It took a week to build the initial release where people could actually register and post. From there I'd burst a week of full-time development work into it every 1-3 months.
I'm glad I experienced this journey, though bless my poor users that had to wait on me to implement basic conveniences over the years. I can say with certainty that I will never go "hah, that looks easy!" ever again.
I had used Datomic on my previous hobby project, so when I had to pick a database for my forum rewrite, it was what I was most comfortable with and it was also just fun to use.
For example, I figured I could use its transaction querying for post edit history. Datalog made it trivial to jump arbitrarily through any relationship. And transactor functions were trivial to write at a time when I had never written a Postgres trigger function.
I didn't apply much professional due process here because I looked at my forum rewrite as a hobby project. It's not like I was being paid.
However, over the course of that year, my forum suffered due to my hobby-level understanding of Datomic.
For example, I had the forum application, the Datomic transactor, and the Datomic client running on one Linode machine. I told myself I'd figure out how to host Datomic on AWS in the future, but of course that never happened.
As the forum grew in size, I started dreading restarting my application because the Datomic client would have to sync with the database. It started to take many minutes. And I didn't have enough information to know why -- is this just I/O limitations of loading info into the process? Or was the fact that hundreds of users were F5'ing the server? I never made myself a real deploy system either. I would manually ssh in, `tmux a`, `git pull`, and manually restart my processes. Very amateurish, but it was so easy to say that I'd fix it in the future.
I also made mistakes like having zero insight into my application. My application was growing slower over time, but I didn't know why. I had some really slow database queries and I was still using LIMIT/OFFSET for pagination at this time.
I also found out that Datomic isn't made for massive blocks of text like forum posts. When I saw that Datomic had full-text search, I assumed that meant otherwise.
A year went by and I started to get stressed out about the forum. I had this crippling guilt that I even tried to build a forum from scratch instead of just using Xenforo. Felt bad for my users who were hanging in there or just leaving out of frustration.
One of the reasons I rolled my own forum aside from thinking that it would be easy was that it would also be cheaper. I was spending like $500/mo on the forum when I was using vBulletin and I was frustrated that I couldn't do much to change that. By my hand-rolled forum cost 1/5th of that, so at least I had that going for me.
During all this, I started working with a friend on a different project and he refused to use Clojure. We ended up with Node + Postgres and he was a Postgres pro who I learned a lot from.
Before long, I had a full Node + Postgres application under my belt and decided to rewrite my Clojure + Datomic forum. I could have swapped out Datomic for Postgres instead of throwing the baby away with the bathwater, but I wasn't convinced that that would be any less work.
I also decided to host my rewrite on Heroku this time to force myself to rewrite the forum in a more stateless way. For example, my Clojure forum had a hard dependency on the filesystem. Not to mention having the database on the same machine meant I'd never be able to spin up another application server.
Of course, any time you rewrite something, you have the opportunity to fix your legacy mistakes from day one which I did. For example, I finally optimized pagination away from LIMIT/OFFSET. I got to clean up a lot of cruft. I also used RDS for Postgres which solved all my database problems. I really had no business using Datomic since I had such a weak understanding of its trade-offs.
It was other things than frameworks too — also databases, and (to some degree) programming languages. For example, I started with using only file based storage. Then I switched to Oracle — because I needed to learn Oracle better, beause of some contracting work I did, and then from Oracle to PostgreSQL. & I tested Dart (a programming language) too (and Angular-Dart).
Actually now in the end, I've found some frameworks that I like (love?) and ... without them, I'm afraid everything would instead have beeen a terrible mess. These frameworks help me structure the code and reduce my total amount of code: React.js, and Play Framework, and Nginx + some Lua (instead of doing everything in the app server).
My personal lesson is ... Do use frameworks but not the wrong ones :-) and it can take long to undo a use-the-wrong-stuff decision.
Well this @mention is a bit more complicated than what one might think. For example: If someone creates a post, without any @mention, then edits it, and adds a @mention — then, do you detect this, when looking at the edits, and send a @mention notification now? And what if s/he edits the post again, and removes the @mention, then, do you remove the notification? Cancel the email if it hasn't been sent yet? And if the email has been sent, and the user clicks the link, and views the post — but the @mentioned user sees no @mention because it was removed, then ... this is not good UX. Should you do something about this? Explain that the @mention got removed? Or just leave the user slightly confused?
Also, do you really want to send an email for a @mention immediately? What if the user who writes the post, keeps editing it, after having saved it? Should you wait for a minute or two, until s/he seems to be done editing it? And then generate the @mention email? Or directly, even if then an early out-of-date version of the post will be included in the email.
What if the @mentioned user is in do-not-disturb mode, then you need to schedule the notification for later.
And what if the @mentioned user is already online and sees the post with the @mention in, diriectly? Then maybe you needn't generate any notification at all. But how do you determine what s/he is reading currently?
There's a lot to think about (more than what I mentioned above) that one might not think about, ... before one starts implementing everything. — And this is just for @mentions. ... Kind of goes on like this, with every small seemingly simple feature you want to add.
It's a forum. Read the thread and don't have any such concept as a "mention". If you get quoted, you'll see it as you go through the pages. If you're really lazy just namesearch your handle (which may or may not change, may or may not be unique, etc) but basically participate don't just cherrypick.
Right, but you would just be one of the rare users that turns the system off. Which is fine, but don't let it blind you to what the typical user wants.
"Just use the /search feature to find replies" isn't something I can say with a straight face to the sort of users I have. I would just leak them to competing forums that have good UX. It's no ideal for someone to be unaware that a thoughtful reply was made to a post they invested in.
If they were so invested they'd be reading the thread for people who are communicating about the topic. For people who are referring to comments or quoting other posts or didn't specifically @mention to drag a person back to look STRAIGHT AWAY lest they lose engagement.
@mentions on forums are good UX in the way that twitter @mentions are good UX - they drive clicks and opens - not engagement. They don't push good content, good conversations, good experiences. They allow people to sandbox discussions, to talk across each other. To skip volumes of content to just get their little piece.
Having the feature degrades the forum community imo, irrespective of who does and does not use it.
I'm a guy who likes old school forums though, so, you know, my opinion is probably not worth that much ;)
Seems you've had a bad experience with mentions. Maybe people have been using them in the wrong way, where you've seen them.
At work, we were using Slack, and @mentioned each other "all the time", and I'm fairly positive we would have abandoned Slack if people couldn't mention each other. Mentions are like going to the other person, and saying, "Hi, look there, you knowledeg & help is needed over here a short while". And then the other person takes a look, replies with help or status update for example.
Our mentions were related to none of driving clicks or opens or engagemenet — instead, it was about getting work done, directly rather than ... some day later when the person happened to maybe see the relevant comments.
Can someone @mention (and thus notify) everyone on the forum in one post? Of course not, just take the first 6 unique mentions.
But what if they edit their post and @mention 6 different people? Easy, edits don't trigger notifications. Well, what if someone forgot to mention someone or typo'd someone's name and they just want to make that edit. Seems pretty lame that they can't fix their mentions.
So if you want to allow the latter but not the former, you need to start tracking historical mentions per post in the database. Which you needed anyway so that someone can't @mention, delete the @mention, and then @mention again to trigger dupe notifications.
And what does the notification actually look like if someone edits a one-week-old post to add a @mention? It's kinda confusing for a user to see "Foo mentioned you in post #123", click the link, and see that the post is a week old. They report it as a forum bug. So maybe you now have some logic that sends a notification with "Foo edited post #123 to mention you." Yet another if-statement.
We both could go on and on.
And this is how every feature goes. They are hard problems without a best solution, only trade-offs.
At least if you're building a generalized forum like phpbb, you're solving a problem that will compound across thousands of users deploying your forum software. Meanwhile my forum was completely bespoke for my community and I couldn't shake the feeling that I was wasting my time.
So then it was tempting to wonder if I should work on generalizing my forum so others could use it. Well, now you're even in the bigger hell of generalizing software in a pond with dozens of formidable competitors in a world where you're pretty certain that traditional forums are slowly dying. Good luck with that.
What if the @mention user decides to change his/her display name, now you need to parse @mention into a unique id and save that in the database with comment body and then translate it back when you serve it to the currently chosen @mention name...
Of course it's social media. The username and avatar is one's social identity on a forum. I think you have a very narrow experience with forums if you find it surprising that someone wants to change their username if you accept that they would also want to change their avatar.
A good example of "the devil's in the details" is the decision chart of how Slack decides whether to send a notification for a particular message.
I mean, how complicated could it be, if the user is in the channel and he doesn't have notifications turned off, then send it, it's just a simple nested if statement, right?
But, it turns out that when you map out all of the decision points, it's quite a bit more complicated.
If you count the situations that lead to "No" you'll realize that all the "No" cases can be easily checked for, and if you're not in a "No" case then you're in a "Yes" case.
Can probably be implemented in a regular function with about 30 lines of code?
You're glossing over the fact that in order to have all of those complicated notification rules that users expect, you have to allow the user to set all of those complicated notification settings. You have to give them the ability to subscribe to threads, set notification preferences for a variety of items (channels, username mentions, keyword mentions, @here mentions, etc), set DND and exceptions, etc.
> If you count the situations that lead to "No" you'll realize that all the "No" cases can be easily checked for,
I think you might be under-counting the "No" cases.
I counted at least 30 (and then stopped, with plenty still to go) so I don't think you'll get it in 30 lines of code unless you have some very long lines with rather impenetrable logic.
Counting paths from the first node to the "NO" node rather than just counting total edges leading to the "NO" node.
Many of the conditions you mentioned are themselves determined based on various other conditions, and/or only triggered if certain paths (but not others) are followed.
> You can check these conditions sequentially and if any of them match, return false.
Not so.
For example, you could have the 'notification preference for device set to "never"', but there are paths through the flow diagram that evaluate to "YES" without ever going through the path that checks for the global device notification preferences.
Therefore if you are just sequentially checking each of the conditions you mentioned, then you'd be returning false (because that one statement matched) instead of true (because the flow should never go down that path).
No matter. I transcribed that list in a matter of 5 minutes with no knowledge about the application (I haven't used slack in a long time). I'm not going to get it right in 5 minutes. It will obviously take time.
But, this is not the kind of thing that justifies, for example, having 20 engineers to work on this one problem for three weeks.
It's still a simple thing for one person to handle in a couple of days.
I don't get why people use feature lists as an excuse for why things take a long time and require many many engineers.
I guess it is complicated compared to all the features mentioned before: complicated is not the right term, but extending users sessions with notification/messaging support is probably more code than the rest of the forum combined (in every language).
Suddenly there are a lot of things (on top of user sessions, which most web languages/environments have plug and play solutions for) hitting your database and effecting your caching strategies.
A modern forum system is a culmination of all these features and many more.
And each feature always seems trivial on the surface. That you pick on this feature as an easy one makes me chuckle and shake my head slowly as I recall the hubris of my past self.
I once made the mistake of thinking that building a forum would be easy, so I can relate.
This is cool. I've been thinking about this for a while. The best way to write high quality web applications is to use compiled languages and minimize the complexity of the infrastructure by using e.g. SQLite instead of PostgreSQL.
I hope this trend starts to pick up more steam and become the sane default that everyone just assumes. Instead of the current mess where everyone assumes that it's "normal" to live in this messy world of countless abstractions and frameworks and micro servers, etc.
I don't see how this would reduce complexity. If you don't need to scale, postgre still better enforces correctness and uses almost no resources. When you do need to scale, you WILL need to switch to postgre. Postgre is also dead simple to install and use for the "simple personal blog" use-cases. The enourmous technical debt you're taking on to reduce the one time installation by 2-3 commands and a slightly quicker download is not worth it.
SQLite is ok if you don't have much traffic (for example personal blogs), but you can't replace a db like Postgresql with SQLite if you have lot of concurrent traffic.
Supposing that your server-side tech uses multiple workers to render pages, I suppose you have to serialize the access to the SQLite file. What is the best way to do this?
SQLite can handle concurrent reads, but needs to lock the entire DB when writing. Multiple workers shouldn't be a problem per se, but if you need multiple workers to support the traffic, then maybe a client-server db like postgres is a better choice.
And then, of course, there's Web 0.1: https://thedailywtf.com/articles/Web_0_0x2e_1