Gosora Forum Software: Supremely Fast and Feature Packed

[Azareal]

Site Name: Gosora Forum Software Project
URL: https://gosora-project.com
Repository: https://github.com/Azareal/Gosora
Download: https://gosora-project.com/topic/download-gosora.25
Type: Forum Software (with a forum attached 😉)
Site Info: A supremely fast and feature packed forum software written in Go (Golang).

Gosora is a little forum software I started working on as a project to familiarise myself with the Go Programming Language. It has four default themes to try to break up the monotony of just having one lol

Things aren't quite in the constant flux that they were a year ago, however many things are subject to change. You can hop from one version to another with the attached updater.
I'm also working on a two-way generic forum conversion framework called uniConverter, although it's largely on the back-burner right now.

Features: https://gosora-project.com/topic/features-introduction.72
Things are subject to change, the theme may be changed dramatically, and features are added all the time. Feedback is welcome.

 

[Page]

what kind of an forum is that? never seen that layout before. Looks nice.

 

[Azareal]

Since the opening post, I have implented global post counters (+ associated graphs), alerts in the Cosora Theme (they were in the others for like six months), bulk topic move, time range filtering for the routes / agents analytics lists (we already had it for the individual route / agent graphs), made some visual improvements and fixes, the other half of the topic list paginator, global topic counters (+ associated graphs), etc.

Thank you, Empire. It's the forum software Gosora, you can download it from Github and give it a play. Go is extremely easy to download and setup and I took efforts to make things particularly easy to get started.

Our goal is to build the fastest and most scalable forum software on the face of the planet, surpassing even AsmBB (due to architecture there, assembly is truly a monstrously fast language), but no matter how fast it is, it has the same bottlenecks as PHP.

I'll have to do some formal benchmarks at some point, but it can generate pages a hundred times faster than MyBB 1.8 and it lacks many edge cases which can rear their ugly head on big boards which can exponentially reduce performance as the activity increases.

I created Gosora in part to learn Go and in part because a friend of mine sadly had their forum thrashed by MyBB. I would estimate that Gosora is a hundred times faster, based on page generation times, but I'll have to do some more formal benchmarks.

I'm leaning towards something more traditional (as people put it) than the really alien players like Discourse / NodeBB while keeping things fairly modern. I want it to be something that someone who has used a regular forum (XenForo, etc.) can pick up without feeling like they're being thrown out of their depth.

 

[Azareal]

IP Search now searches profile comments, I don't know how I missed that, but it's a thing now. Also, Cosora no longer warps on large screens, I'm currently working on poll posts.

Unfortunately I've been finishing up some anime serieses (sorry lol), so that's been slightly delayed, but I think I can roll that out tomorrow. No commits for you today though.

 

[Lord Saru]

I am really impressed. How did you come up with the name 'Gosora'?

 

[Azareal]

Thank you very much, Lord Saru, I'm glad you like it. Random. It was originally Grosolo, but that wasn't too popular, so it got renamed to Gosora which is similar and has the unintended benefit of having Go at the front.

So, it's Go and Sky, I guess. Or Go and Space.

On a side note, slow week (x.x) but...

Topic polls are done, poll posts are part done. I've improved the suspicious requests thingymajig, plus continued moving the routes out of their current home and into the new one while refactoring tons of code in the process.

It's not intended to catch every suspicious request, although it'll be expanded as time goes by, it's just useful for providing analytics on how many... shady machines are crawling around and to remind admins that the internet is a dangerous place where their site will be attacked at every minute of every-day, so that they don't get complacent about security.

Also, the requests are logged, so you can see exactly what the little machines are doing. Or humans... Sometimes. We have stats for many bots including Googlebot (and human browsers), I plan to add more after refactoring the user agent parser as it's o(n) right now (gets slower with each entry, but it's Go so it's okay for now).

The UA parser isn't the only factor for detecting bad bots, actually ones which don't pass muster there show up as "unknown" as opposed to "suspicious". Ones which show up as malformed either botched the path or are providing the wrong host (usually when they hit every possible IP and don't know your domain).

 

[Azareal]

Word of advice, don't use Internet Explorer, it does not support CSS Variables.

I noticed a few IE UAs with referrers pointing back to here, I don't currently support IE, although I used to. I'll probably add some support at point, but I have my hands tied right now.

On a brighter note, as hinted, I added referrer tracking, only by domain though for performance reasons, I might expand it to full URLs once I'm a little more confident that it won't push things too far and I'm looking at a few options for doing so.

Poll posts are delayed. Implemented, but with no UI. Right now, you can only do per-topic polls. /topic/ is now 20% faster and I've removed redundant logging for debug mode which should improve throughput there too.

In addition to that, I've fixed the tests, so you can now run benchmarks on the routes again. I have a whole topic on my findings and I might be able to speed /topics/ up by a factor of 40: https://gosora-project.com/topic/mini-benchmarks-and-optimisation.37

Actual results may vary, but it should remove a considerable amount of load on database servers and significantly improve performance in that area. I'm also pushing a little fix to the Shadow theme with it, you've probably noticed the poll which shouldn't be there, or not.

If Shadow looks white instead of dark, you're either using IE or an old version of Edge, I'd advise upgrading or migrating. If things otherwise look broken, try a hard refresh, if you've been on the site before.

 

[Azareal]

The topic list is now (well, more like yesterday lol) zero query. Thanks to this, it's become 25 times faster, a little off our 40 times goal, but still brilliant, and frankly, I ran the benchmark while doing heavy stuff in the background lol

What do I mean by zero query? Well, you have to understand that a conventional forum software generally fires off a bunch of queries to fetch data to build the current page.

Templates, smilies, custom bbcode, topics, polls, replies, forums, forum permissions, groups, settings, and the list goes on ad infinitum. However, I found that a lot of this work is actually... Unnecessary, as they've noticed to some extent as-well and have a few mitigations for it (Redis, etc.), but that's still far from optimal.

When I say zero queries, I don't mean it quite as literally as you'd think, as described in the topic I linked I above, but rather... Each request has zero queries, or at-least the hot routes where 99% of things happen, while queries are fired off every now and then to synchronise the instance with changes in the database.

All writes go to the database except views, while reads are often pulled out of local instance memory.

Like this, building a software which is a hundred times faster than the baseline is actually quite easy, the fastest query is the query that never runs. Views are dealt with by collecting them up on the instance side and occasionally pushing them to the database, ditto for the other analytics stuff.

I've also added a little optimisation where if a topic doesn't have any replies, then it doesn't run the query to load the replies, you see, the opening post is stored on the topics table, so I don't need to consult the replies table for it.

And if we consider that eliminating two queries on the topics list caused it to jump up by 25 times, I think we should see near /forums/ performance on /topic/ for no reply topics, which are actually surprisingly common on forums.

Very common, how often have you seen a topic on a forum with no replies? I see them all the time, it won't work for everything, but it's a nice piece of low hanging fruit I fixed with two lines of code, and it should help sites out.

 

[Azareal]

I now track routes.StaticFile, I initially opted out of tracking this route as it would create a lot of noise and my CDN deflects virtually all of these requests, but it could be useful, so it's been added.

I've been fixing a lot of bugs, probably 50 or so bugs thus far, mostly in the side themes which have fallen behind Cosora a little, but also in Cosora. For instance, I made the Control Panel a little more tablet friendly.

I'm also adding per-forum analytics (like the others, you get a chart of the number of views over the specified time range), so you can see which forums are warmer than others rather than purely relying on per-topic stats or per-route stats to guess at which ones are warmer. This might help you identify areas which you might want to invest time in.

I also added better pagination to /forum/{id} to make it easier for to navigate through the pages and refactored a large portion of code to make development easier going forward.

 

[Azareal]

A bit of a shorty, but I've implemented per-forum analytics, one week analytic time ranges, fixed an annoying bug in the one month time range analytics labels, did some refactoring, etc.

 

[MrDangem]

I think the homepage is not that catchy. Though it has a modern and responsive style still, the design is too plain for me.
a WordPress blog with business/product Theme design as Homepage will make it cooler and attractive for users who want to start up making forum.

 

[GamingGuy]

Good luck with your project!

 

[Azareal]

Cosora has been promoted to the defaultest default theme for now, but it's probably going to give way to the next set of themes, Leta and Nox. There's always a new theme.

Tempra Simple was Gosora's first theme and it's evolved a lot, although it has a few fatal flaws I'll have to address, or end up superseding it with a new theme.

I've quite frankly added and changed so many things that I can't remember most of it, but lemme see if I can scribble down the main parts.

Firstly, I rediscovered that I have a feature which preloads the next topic page in a set in advance rather than waiting until you click it, bypassing the latency of hitting the server for it. I added it nine months ago and completely forgot about it o.o

I'll see about improving that soon-ish. Like I said to other people, I'm pushing it along to a numbered version, v0.1.0. One step will be the upgrader which will allow you to update Gosora via the command-line and will let you hop from build to build.

I've added 300 or so phrases as a gift to someone who was interested in translating the software. We also track the language which the end-user has set on their OS, including whether they sent a language header (usually a bot), I've discovered some very... interesting statistics with this.

The themes are largely on feature freeze until the upgrader is completed to avoid dividing efforts and more routes are being tracked now including robots.txt.

Fixing MSSQL and moving the routes to the new package have been bumped up to the highest priority, although moving the routes is non-trivial, so even with a few all-nighters, it'll be a very tricky one, I'll have to pace myself a little better with that one.

I've always been a fan of the unique layouts for Tempra Simple and Shadow, so I'll see what I can do to bring them inline with the expanded feature set, as they're a little stretched right now.

Tempra Conflux might be the next one to be slashed, the Tempra Cursive Theme will almost certainly be slashed, it was a nice experiment, but the time has come to put it to eternal rest like Cosmo.

Profile owners now get alerts for profile comments. The login, registration, IP Search, and error templates are now transpiled, so they should be around about sixty times faster than before.

According to Stack Overflow, not the most reliable source of information, but it'll do to give a bit of context, take it with a pinch of salt, the template interpreter is about PHP7 pinnacle speed, while the transpiler is far faster, and I'm going to fix a recent performance regression from adding the 400 or so phrases (nearly 2 times slower) to bring the themes back to where they were (essentially making them free, although it's going to take a bit of work).

I have a headache, so I might have missed a few things, but this should be the bulk of it. I've also been doing a ton of refactoring to make the code easier to maintain, and added support for Slackbot in the user agent analytics page.

Let's see if we can get that alpha (precursor to v0.1.0) out, that should be fun 🙂

 

[Azareal]

I had a bit of a migraine yesterday, so my post was a bit of a mess, but it should give you a general gist of things. I've been fixing up the README (the commit for this is coming up soon!), as the steps are slightly out-of-date, and I mentioned my plans for the upgrader + the current upgrade process here:
https://gosora-project.com/topic/fun-with-upgraders.40

Also, if you want some insights into how we do templates exactly, it's not described too well above, there's this:
https://gosora-project.com/topic/template-interpreter-vs-template-transpiler.41

The thing to remember when I mention the interpreter and the transpiler is that the interpreter is basically the old system from near Gosora's creation in Jan 2017, and the transpiler is about mid-2017.

In other words, there are two template systems, although it's usually only rarely hit routes with the interpreter which haven't been converted yet, the hot routes are hit by so many bots that I would be nervous to interpret it.

 

[Smokey]

Always nice to see new forum scripts popping up these days. 🙂 Good luck with it!

 

[Azareal]

Thank you very much, Smokey.

One of my dependencies imploded on me, so the upgrader is sort of crude and slightly belated, but it works, although you do need Git installed.
Run dev-update.bat or dev-update-linux, it's mentioned in the README.

I've also bumped the minimum requirement from Go 1.9 to Go 1.10 to allow me to use GopherJS which allows me to transpile Go to JavaScript, so that I can reuse some of the back-end validation logic on the client-side (e.g. for registrations), rather than doing a slower round-trip or implementing the same thing twice.

We may not need it for long however though, as WebAssembly seems to be in the works for Go 1.11 (or the version after that), so we may be able to compile to that in the coming future.

 

[Azareal]

Lots of things have been done, but right now, I'm working on a menu manager.

This is just the short, as the list of changes goes on and on and on, but.
We've got some patches running through the update pipeline which are working, yay.

I've also revamped the README somewhat, opened the software to outside contributions, refactored a great number of things, fixed a great number of bugs, and merged two phrase namespaces (among other things).

I'm probably going to put more time into Gosora now, so we should make a fair bit more progress than we've been making and any feedback is welcome.

 

[Azareal]

Finished the menu manager. You can drag and drop to reorder menu items, create menu items, delete menu items, and edit menu items.

There's only one menu, at the moment, the main one. Onto client-side alert rendering.

 

[Page]

I haven't check this site for an good amount of time and I See some changes 🙂 It's simple and fast. But still needs works, at least able to see more built into the system that member might like. Point system maybe.

https://gtmetrix.com/reports/gosora-project.com/qfEk4gq8

 

[Azareal]

GopherJS is out the window as a dependency, as the thing was pumping out 7.5MB JS files, but I did toss together a custom Go Template to Go to JS transpiler to render the alert templates out on the client-side instead of hard-coding them into the JS files.

I've also been doing some spring cleaning to get the code into order, to clean up some of the more grotesque looking parts of the visuals, and refactoring chunks of code which are a little unwieldy.

I have a couple of features in mind next, dunno which one I'll toss out first, but one is logging all the registration attempts, successful or not, so you can get an idea of what the dastardly little spambots are doing and which defences are thwarting them.

And I'm not sure if I'll do this before tossing out some sort of release (despite the thing being fairly stable a lot of the time, it has none of those and commits can be a little tricky to chase all the time), but the next major theme will likely be Nox which will be a dark theme, followed by at some point, Leta which will be light.

The hope is to alternate between light and dark to make things more interesting, and to explore what interesting design decisions I can take with each one. I'd like to have each theme looking fairly unique and distinct to stretch Gosora and to keep things fresh.

I haven't check this site for an good amount of time and I See some changes 🙂 It's simple and fast. But still needs works, at least able to see more built into the system that member might like. Point system maybe.

https://gtmetrix.com/reports/gosora-project.com/qfEk4gq8

I do have a level system however, it's a reflection of a bunch of hidden stats including topic counts, post counts, maybe likes?, etc. I've always been throwing around the idea of having an experience bar somewhere, but that might be a little overkill.

Achievements might be nice, or customisation options for profiles. I kinda miss some of the old school options in vB, etc. for making your profile your own, especially with some of the old social networks, although we will have to see what can be done.

Thanks for the feedback.