URL repositories

[thatfuckingbird]

Just like tag repositories, but this time with URLs.
We can call the public one PURR.

[Suika]

In that sense, IPFS hash repository would be most welcome since https://github.com/ipfs/go-ipfs/blob/v0.6.0/docs/config.md#peering is now a thing

[thatfuckingbird]

Don't think that needs a separate repo, you can handle those like a URL so can be added to the URL repo.

[bbappserver]

This repository and/or hydrus in general needs the concept of an ephemeral URL. Many URLs are not permalinks (for example discord attachments, or sadpanda content), but they end up in the known urls. These don't need to be propagated, and indeed could be deleted client side after a long enough period of time.

Simply having a few regexes serverside to check against would be a good start.

[ShadowJonathan]

An additional idea I had for this is that the PURR server promotes URLs it can independently verify as "canonical", by checking the links provided, and marking every one that (directly or through a parser) results in a file that has the same hash as provided.

No URLs will be gatekeeped, but URLs could be promoted to the "top of the list" if the server can independently verify them.

[Zweibach]

That feels really out of scope for almost any proposal for PURR ever. It would probably be better if you just had multiple clients indepedently verify an URL-hash association. Your comment did remind me that I want that association in Hydrus so I wrote #438 which could probably be used to do just that.

[DonaldTsang]

@Zweibach I think that his intention would be moderation and checking. Assuming PURR is trying to be a pure data repo, it is possible to have a separate mechanism for moderation. Just like how PTR does not moderate tag accuracy but people can add-on some extra stuff on top (private instance, not the one with our jannies since less bloat).

[Zweibach]

That is quite literally the exact opposite of what Jon is suggesting. PTR itself does no verification of the tags as you yourself say.

[DonaldTsang]

@Zweibach to clarify: I am saying even though Jon has good intentions, it is best to have separate software to handle verification.

[NyaaTell]

Why are URLs not stored in PTR already? What's the obstacle?
Since PTR knows all the files existing between every PTR user, why not keep track of ULRs while at it? Two birds with one stone.