Restoring this repo to a healthy state

[Sommerwiesel]

Like proposed here: #159 (comment) this issue will track the progress of restoring this repo to a clean and working state.

Tasks:

1. forking the current state in order to archive potential good commits
2. reverting this repo to 5ff2364 in order to get it clean again
3. go through all commits in the fork in order to filter out "the good ones". We should create a list of good commits here that we want to merge back into this repo
4. restore and run CI tests
5. run local tests

I will also lock-down this issue so that noone else can interfere.

[Sommerwiesel]

This is a list of good commits imo, in order:

f8de3af
7118683
91a6bc3
57b915f
7a8d44c
1d52ff4
c62a8b7
404a461
e96c9ee
f6975b9
541b6fc
36683fe
857ee32
a3d2b7c
8abc4d4
57e1fc3

---

Now comes a list of "problematic" commits we should check before merging them back

The CI tests broke after here since he made a mess of them, but there might be still valid commits, but we should double-check everything from here on.

---

rate-limit
I'm against implementing a rate-limit in Wikiless code, especially since this can easily break when used with a reverse proxy, this should solely be done by a hoster on the reverse proxy side, what are your thoughts about this @tiekoetter @rufatkiu. The commits are:
2629b0e
fac15cd

---

redis module change
He also apparently changed the node redis module after here, we should discuss whether this is something we want to adapt. I think maybe it's better to change to valkey, since redis is no longer foss:
dd531c9

---

csrf
There is also a new csrf token implementation we need to check and decide whether we want to keep it here:

3d2d841
4ca091c
8d39ec9
03f1bda
dc3d53d

His commits are however all over the place and a mess in general. I hope that I caught all csrf commits...

---

CodeQL
These are about CodeQL and should NOT be merged back imo, I think we should rewrite them from scratch since his commits are a complete mess. I think it's best if @tiekoetter looks at those and decides whether he wants to keep them since he created the CI tests in the first place:
877efb8
96b08b6
30df5c1

---

Everything else can be thrown out imo, but you should probably check for yourself.

---

bad commits
He also changed a lot about the config, docker compose and http server (like 4a50d16 4ca091c and 803855b and 4046ce9 and 6b99dab and daf3d85 and so on), but my guess is this was only done because his local setup on his server was broken and he couldn't get it to work. Consequently, he pushed broken commits, removed the CI and made a complete mess trying to fix it.

We all know that commit 5ff2364 was a perfectly fine working state, passing all CI tests and a clean local test I did on my server.
So for whatever reason his local setup broke, it must have been his fault and not ours.

Consequently, when we are finished re-merging all good commits, we should do a full test round to verify everything is in working order again.

[rufatkiu]

Vote: Should we use rate limits?

[Sommerwiesel]

Vote: Should we use rate limits?

I vote no.

Reasons:
Rate limit is better applied at the outer webserver level, i.e. a reverse proxy such as nginx or caddy. I personally think that basic bot/ai/bad actor protection should always take place at webserver level and not in the application itself, so that invalid or excessive requests never actually reach the application and cause more load for it.
Nginx especially is much more configurable when it comes to rate limits and can easily be combined with more protective layers such as fail2ban, ufw, etc...

Having this in the wikiless code would also mean:

• it's obsolute for any user who already does rate-limiting on the webserver level (like I do on my server)
• we have to make it optional and configurable because just hardcoding it to 100 requests / ip / 15 min will absolutely not do.
• there will potentially be problems with webservers / setups that do not properly forward the client ip (not really our problem but just fyi)
• We should disable it by default so that it does not surprise existing instances

So yeah, no from me.

[tiekoetter]

I am also against rate limiting in Wikiless. It currently isn't feasible to maintain a working rate limiter for this project internally. This can be easily delegated to the webserver for "dumb" limiting and to things like Anubis for a more aggressive and dynamic AI blocking mechanism.

[Sommerwiesel]

Yo @rufatkiu you haven't responded to my question from the other issue yet so I'm gonna ask it here again:

Who do you want to do the fork, revert of this repo?
Do you want to do this yourself?

@tiekoetter or I can probably also do that for you.
We should get that done asap imo, because @tiekoetter has also opened a new PR (#162) that fixes a longterm issue with images not loading and I would like to see that merged as well.

So if you're short on time, we can do all of the work, no problem.

[rufatkiu]

Yo @rufatkiu you haven't responded to my question from the other issue yet so I'm gonna ask it here again:

Who do you want to do the fork, revert of this repo? Do you want to do this yourself?

@tiekoetter or I can probably also do that for you. We should get that done asap imo, because @tiekoetter has also opened a new PR (#162) that fixes a longterm issue with images not loading and I would like to see that merged as well.

So if you're short on time, we can do all of the work, no problem.

Sorry for the delay, You are in charge of do the revert due to tons of work in my school recently and I can't manage this repo at all and the fork I prefer here: https://github.com/Metastem/Wikiless-Reborn/tree/main

[rufatkiu]

Yo @rufatkiu you haven't responded to my question from the other issue yet so I'm gonna ask it here again:

Who do you want to do the fork, revert of this repo? Do you want to do this yourself?

@tiekoetter or I can probably also do that for you. We should get that done asap imo, because @tiekoetter has also opened a new PR (#162) that fixes a longterm issue with images not loading and I would like to see that merged as well.

So if you're short on time, we can do all of the work, no problem.

The repo has been reverted. Now Im performing the fixes

[rufatkiu]

@tiekoetter @Sommerwiesel

[rufatkiu]

The @Jeyso215 fork https://github.com/V4NT-ORG/Wikiless-Reborn works correctly and fixed multiple bugs, I have merged it to the main branch.

[rufatkiu]

@Sommerwiesel lets remove the rate limit

[tiekoetter]

We didn't really get anywhere yet. We still have a lot of problematic stuff in the git log. We have his conspiratorial commit messages regarding the CIA running Wikipedia, the license change and not correctly credited (in this regard stolen) commits. In my opinion we should either rewrite a lot of his commits or drop them.