Gosora Forum Software: Supremely Fast and Feature Packed

https://gosora-project.com/topic/peeks-at-nox.53
Peeks at Nox. A little look at the Nox Theme, still more work to be done however.

There's also one screenshot which didn't make it into that post (Gosora has a five attachment per post limit, maybe I should tweak it up for staff), so that'll likely make it into the next, it was a fairly unrefined bit of Nox anyway.
 
Gzip has apparently been broken for a while due to a typo. Oops.
It's been fixed now though.

I've also been doing a bunch of things to improve the client-side performance of Gosora, including one I haven't pushed to the production site yet. This includes more aggressive caching, more asynchronous loading to cater to older browsers / mobile, and less render blocking.
 
Mostly technical post, but might be interesting:

Every now and then, I like to dig into the code and see how I'm doing in terms of performance and where my hotspots are, so I've dug pprof out of the cupboard and done some tests.

upload_2018-8-21_19-4-51.webp
Like with the previous benchmarks, I setup bombardier and pointed it at Gosora, but this time I did it while doing a CPU profile with pprof to see where the CPU hotspots are and as you can see, it spat out some largely byzantine results, although it's not that strange when you understand what it all means.

pprof is basically a tool which shows the functions which use up the most CPU time, in the cumulative mode, it also counts the CPU usage of the children of a function towards the parent's total. There is a mode which spits out a pretty diagram, but I can't get that to work, so I'll settle for the command-line interface.

As for fsnotify, it's a library which scans the filesystem for changes and reloads the CSS files, etc. but it really isn't relevant here as it ran far longer than the load test slightly skewing the results and it's really not what I care about here as it's not on a SSD.

As you may or may not know, Go is written in Go (plus assembly), so we usually wind up with some pretty low level functions you normally can never touch popping up here and there
We'll ignore most of that and focus on GenRouter, as that's the generated router which handles each and every request.

As we can see, each request flows from net/http through to my router's ServeHTTP method, and then, it calls routeSwitch, which in turn calls routes.TopicList. routes.TopicList calls GetListByGroup to pull the topic list out of memory and it sends the template system whirring into action.

Both of these actions make up about 50% of the program's CPU time (pretend fsnotify doesn't exist lol). But the times mentioned there are quite curious considering I only ran the load test for ten seconds, something strange is going on here.

upload_2018-8-21_19-27-8.webp
If I take a closer look with "list", it would seem that some things are slipping past the memory cache and hitting the database... That doesn't make any sense. I'll have to get to the bottom of that.

The other big problem is the template system which has been a tad on the slow side ever since I added the phrase system, I'll have to write some optimisations to halve the number of template fragments to halve the number of interface method calls.

I'm trying to finish up Nox and to create a quasi-release right now, so I don't want to get too distracted, but I'll see about dealing with these little gremlins as soon as possible as they're dragging the topic list down in contrast to the other routes.

Rapid Loader will also be a big help for bypassing the template system in some cases, although it's impossible to rely on it for every case.
 
For my earlier benchmarks, Nginx hits around 200 req/s on this machine, that is on both my SSD and HDD.

Keep in mind that that is for a blank page with just:
Code: 
<html><head><title>Hello Word</title></head><body></body></html>
So, it's doing essentially nothing while Gosora is doing a lot more work.

Now, let's take a look at how fast Gosora can serve static files:
upload_2018-8-22_15-15-15.webp
I disabled HTTPS here, but it's usually around a 10% deviation, it's notoriously fiddly on localhost as I have to play with self-signed certificates. I'll have to try it on a VPS too, as Windows tends to be notoriously slow for things like this.

So, how did I run circles around Nginx? Well, by cheating, I simply load all the static files into a giant map in memory and serve them from there. For avatars, etc. where I could have a possibly infinite number of items, I get Go's FileServer feature to handle it for me.

Also, I do a lot less work than Nginx, as I get to take a number of shortcuts and don't have to worry about the mountains of functionality it handles. Varnish on the other-hand might be a tough nut to beat.
 
Last edited:
If the site went down a load of times, I did a bunch of updates and tweaks.

I've also revamped the installation instructions, landed some compression related improvements, and simplified some bits of code.
 
As a side project while working on Nox, I'm trying to simplify the server side of installing and administrating Gosora, including taking steps towards making it compatible with Docker and adopting Go 1.11 modules (this does mean that the minimum requirements will probably be bumped up to Go 1.11 very, very soon).
 
I'm holding off on bumping things up to Go 1.11 for security reasons until the new version stabilises, it seems to be having some trouble with bounds checks, although not for Gosora specifically.

I also fixed that topic list bug and the topic list is two times faster for guests.
That's 2400 req/s vs the previous 1200 req/s (Windows).
P.S. I lifted staff activation, so you should be able to register without me activating you, but I'm keeping an eye on that user list for visitors who aren't human!
 
Last edited:
$3/month VPS

Topic List (I haven't patched the version on there yet for that topic list slowness bug, it should be waaay faster in the latest builds)
Code: 
Statistics        Avg      Stdev        Max
  Reqs/sec       444.59     169.43     803.54
  Latency         0.86s   766.88ms      6.94s

/static/nox/main.css
Code: 
Statistics        Avg      Stdev        Max
  Reqs/sec      6152.13     950.53   12308.58
  Latency       86.90ms      1.00s     17.31s

One of my announcements.
Code: 
Statistics        Avg      Stdev        Max
  Reqs/sec      1889.81     469.27    4938.32
  Latency      210.81ms   311.79ms      4.19s

It's oddly slow, I might have messed up the configuration somewhere, I will have to investigate o.o

There are a few possibilities, it might be that my current host really sucks (Go can get performance out of anything, so don't be fooled if the numbers look big) or there's some server or database configuration that aren't quite right.

Also, I screwed up quoting it from the console, but the /static/nox/main.css one is transferring 50MB/s.
 
Last edited:
The MySQL Adapter should now try to open a Unix socket, this should improve the throughput for the database somewhat, although I'm not quite sure by how much. This will only help if the database is on the same server as the application.

I've also exposed per-topic views on the Nox Theme (you might notice an announcement with 20 thousand views, that was the load tested one lol), added a link to take you back to the parent forum from a topic (only for Cosora and Nox, the simple layout themes don't have it, at the moment) and nearly finished the responsive topic pages for Nox.

I also added a content security policy header to hopefully upgrade image requests to HTTPS auto-magically to eliminate any mixed content warnings. I'm looking into other options too, but a proxy might be too open to abuse from bad actors.

As for the probably resolved issues with the topic list, I think a large portion of it is that I can't reuse prepared statements there (you can't parametrise arrays) and MySQL has a poor excuse for a query cache compared to more advanced databases like PgSQL, but luckily, I can just fire off a query each second and use the results of that for everyone.
 
Last edited:
I've revamped Cosora's Control Panel Menu, so it fits a bit better with the rest of the theme and more or less finished the Nox Theme, although I'm still improving it. There's a few things I'm revamping like the Account Manager / Control Panel Menus, Profiles, etc. there.

I'm thinking of making having a power saver mode or something. For instance, you could flip a config file flag to turn fsnotify (the file watcher) off, so that it doesn't reload the static resources (CSS, JS, etc. files) when they change. And maybe a different flag to throttle back the topic list rebuilding on inactive sites without at-least one request per five seconds.

The idea is so that Gosora will dial back it's usage to help reduce it's impact on the environment on little sites, basically doing our little bit to help curb the oceans of carbon dioxide and other fumes being pumped into the atmosphere, it wouldn't really have a real impact on actual performance or server load.

I don't know if there's interest in something like that?
 
https://gosora-project.com/topic/deployment-method-plans.54 Some vague plans for Docker. It'll enable some pie in the sky stuff like large scale deployment and automated deployment, useful for companies managing large clusters of instances.

Maybe webhosts? Dunno, Gosora isn't actually designed for webhosts, it might take some optimisations which assume that 90% of sites are dead 90% of the time, it would probably still be an order of magnitude more efficient than PHP, particularly when it comes to memory which is the real concern in those environments.

Although, to be honest, everything is more efficient than PHP, unless it's Rails.
I only mention it to give you a ballpark figure to compare things with, otherwise the statistics come off as fantastical and hard to wrap your head around.

Docker might also enable some things like Lilliput for faster image resizing including for animated GIFs, although I'll likely run that in it's own process, for security reasons.

https://gosora-project.com/topic/profile-fields.56 I'm also avoiding adding classical profile fields to the core for privacy reasons. Use a plugin or something. It also happens to be a serious case of feature bloat in a lot of software.

People really don't need to know where someone lives 90% of the time. I might do an introduce yourself block where someone has to get over the cognitive barrier of forming a paragraph to describe themselves.

Decisions are subject to change, but privacy matters are particularly an issue when entering this whole new world where people have suddenly started to care about privacy.
 
I like it. I would use this software with my wordpress sites. Great job.
 
I'm thinking of just taking a random commit and calling it v0.1, it's basically already at that point anyway.

One thing I'd like to do once I do that is to make per-topic views a little less useless, instead of just "all-time" views, I'd like to break it down by the last month, week and day.
 
I'm releasing v0.1.0 now, I'll add some instructions for pulling down tags specifically later or you could always pull master as the two are on the same commit right now (although, master will shift ahead of it fairly shortly).
 
Now that 0.1.0 is out of the way, we can get onto the fun stuff that might break people, although I'm trying to rein the destruction so that people don't notice.

I've rewritten the parser (not the pre-parser where most of the heavy logic is, but the just in time one which handles mentions, media and hash links), so that it uses Go 1.10 string builders and plugins can now add custom hash link types in a more efficient way.

# is allowed in URLs now, I'm not sure why it wasn't before, it must have slipped my mind when I wrote the URL parser. I'm also adding parser tests for mentions, URLs and hash links as my focus was more on the pre-parser for tests before.

And if anyone is bored and wants to try their hand at writing a string algorithm, there's always this little gem: https://github.com/Azareal/Gosora/issues/44

There is a CLA though which amounts to I can do whatever I want with contributions, mostly for legal reasons, so I can switch licenses (e.g. maybe to the MIT or some other license since people like that, although I worry about people running running off with the code and selling it, we don't need another IPB).

Also, businesses can sometimes be funny about using open source licenses or so I hear.
 
Git branches seem to be hip these days and it'll help me not to break people's sites while developing features, so I created a couple.
One called conversations which I've already merged changes into master from and tlistsort, so we can have more sort options for the topic list, e.g. the number of views, last updated, etc.

I've mostly held off on conversations, because well, I have Discord.
It's a far more streamlined and faster way of poking me, but I'll see about adding a live thingy in.
 
I fixed a couple of annoying bugs, one to do with the menu and the other to do with editing posts, so those should no longer be problems. I also added my Discord Chat to the menu as it seems like a good place to put it.

Asides from that, I added preliminary topic view counts to the Cosora Theme and added the Most Viewed sort filter to the topic list in the Nox Theme. All of the above should be deployed on the live site, although you might need to do a hard refresh, if you visited within the last day or so.

I've also added a few more phrases and added documentation on how to translate the interface, as I've noticed a few people from China, Japan, Germany, Spain, etc. trying out the software. I'll see about improving the internationalisation support very soon.
 
upload_2018-10-10_14-13-57.webp

I've been doing plenty of refactoring on the unit tests, including adding new ones, so we can automatically detect more bugs.

The current test suite helped out greatly with one issue someone reported which involved an exotic database configuration where I flipped a switch in the database configuration and ten bugs were discovered and fixed on the spot rather than having to prance around for a week or more.

One thing I'm expanding upon is Unicode tests, so we don't get too much MySQL stupidity or other such things messing people's posts, names, etc. Or really just being generally annoying.

I'm hoping we can reach 100% test coverage!

upload_2018-10-10_14-19-36.webp

I'm also working on adding more and more phrases, particularly for some more neglected portions like error messages (hopefully, they're not too much for translators to chew on!) and I've also de-duped a bunch of errors which essentially the same thing but in slightly different ways.

There is plenty to come here including more complex phrase handling. Yay.

upload_2018-10-10_14-12-2.webp

And this one is a fairly minor feature, but I'm improving the level UIs somewhat. Work in progress!
 
Do you have a demo for Gosora? I would like to try it out before installing it live. I'm going to start a new forum but still brainstorming on what forum software I should choose.
 
You must log in or register to reply here.

About Us

Since 2007, Forum Promotion has specialized in providing advertising solutions to webmasters looking to promote their communities. We pride ourselves in being the bridge that connects forum administrators, bloggers, and more.

Sponsors

Affiliates

    Universal Gaming The Coffee House

Stay Connected

Back
Top Bottom