FIP Discussion: Remove Fplus Notary Process

[Fatman13]

Summary

Remove centralized notary process and develop new decentrailized/algorithmic notary process

Motivation

Current datacap program not only disrupts a free storage market by creating manufactured demand but also hinders the actual adoption of filecoin as a viable alternative to traditional centralized storage services. Datacap program as it is implemented now is centralized, ineffective (at scale) and has no incentive/punitive framework in place for the governance body (i.e. notaries) rendering them unproductive in carrying out governance policies.

From the design of the filecoin storage market, SPs get portion of their revenue from providing retrieval services. As opposed to genuine market demand for storage, fabricated demand for storage won't incentivize SPs to optimize their retrieval capabilities as most data from datacap program has no commercial value at all. This stifles the development for building a competitive user experience (on-par with centralized services) for real-world storage use cases.

Ultimately, the program failed to align the interests of storage providers with the long-term goals of the network which is building a free storage market (free as in free market) over the span of a relatively long period of time (~1 year), during which 4PiB of datacap are sealed by storage providers according to fplus dashboard compared to 14EiB sealed by the whole network. I'd suggest cut the program loose or at least give it a thorough evaluation before it become "too big to fail" starting from removing centralized notary process.

As current LDN goes, verified deals basically made regular deals irrelavant. The network might as well just subsidize deal making directly on the protocol level instead of having applications go through a centralized and ineffective process.

Current Design

Current design of the notary process (centralized) fell short on the following

• Most Notaries are both player and referee
• Centrailized process either being slow and ineffective or fast with no audits (it doesn't scale well)
• No dispute/audit that I am aware of (Please correct me if I am wrong)
• Not helping real-world adoption as not many real-world clients would actually want to go through the datacap apllication process (It would be more useful for platform like nft.storage to get datacap and use it to subcidize storage but getting real world clients to do that is too ideal)
• No way to verify private datasets
• No way to determine if a dataset is useful or not
• No content moderation (NSFW)
• Rules around the centralised process that some may find hard to follow

Therefore, I propose allowing everyone any amount of datacap and maybe lowered the multiplier (10x right now), or develop other ways to subsidize datacap deals.

New Design

Premises

New design should adhere to the following premises...

• Free market principles (as it was presented in here)
• For real-world adoption
• Decentralized process

Ideas

New design could incorporate the following ideas...

• Remove current centralized datacap application process
• Reward regular deal making with a multiplier
• Reward "datacap" deal each time network successfully retrieve the deal data
  • something close to off-chain windowPost dispute logic but instead of slashing, it reward successful retrieve;
  • no slashing if fails;
  • facilitate for better user experience for storage client;

Consideration

Deal making is already bearing some extra cost associated with message gas and operation overhead. I think it is reasonable to remove current notary process and subsidize deal making directly until a decentralized process is developed, leaving most of it to the whim of the free market. If a dataset is useful then there should be a high market demand to retrieve such data as opposed to the usefulness of a dataset being determined by the personal preference of a notary.

[cryptowhizzard]

I don't think this is the way to go.

• Most notary' s i know work in Filecoin affiliated settings and voluntier to give back something usefull to the network.
• If there is a dispute there are 14 days recurring notary calls to discuss.
• We have had talks with University' s in our region. Non of them have opposed to do some kind of duedilligence.

In order to decide if a dataset is usefull for Filecoin i guess there have to be humans involved and not an automated process. At least a combination of both?

[Fatman13]

Most notary' s i know work in Filecoin affiliated settings and voluntier to give back something usefull to the network.

Yes, I believe most of notaries are do-gooders but in a centralized process there only need to be one bad actor to corrupt the whole system whereas a decentralized approach could be more resistant.

If there is a dispute there are 14 days recurring notary calls to discuss.

I didn't follow the process as closely. But my point is that there seems to be no audits by notary finding anything yet.

We have had talks with University' s in our region. Non of them have opposed to do some kind of duedilligence.

University datasets are good. But for real-world adoption that is not enough in my humble opinion.

In order to decide if a dataset is usefull for Filecoin i guess there have to be humans involved and not an automated process. At least a combination of both?

Likely. My point is current centralized approach is far from ideal and not really geared towards real-world adoption. That's what I am worrying about. Web3 only on paper not in practice...

[dkkapur]

@Fatman13 thanks for drafting this, good to have active discussion on how we can continue evolving the Fil+ system, adding a few of my thoughts on the points you raised.

Specifically, for your list of bullets on current status quo:

Most Notaries are both player and referee

Agree - though, even if we engage active community members that aren't storage providers or clients in the network, a majority of active participants are active because they also have a stake in the network. I don't think this is necessarily a problem -> if we have enough Notaries with varied interests/stakeholder positions in the network, we should theoretically be able to converge on rules and evolve them over time to ensure reasonable understanding of "fair" or "good" behavior vs. abuse of the system. However, building a system where we do engage enough notaries and enable them to help build this out is important, and is useful feedback for the current Fil+ setup.

Centrailized process either being slow and ineffective or fast with no audits (it doesn't scale well)

"Centralized" because its either one notary or a small group of notaries? Audits do happen - in the past this has usually been at notary elections, and I expect this to happen a lot more frequently with subsequent allocations in LDN applications coming through / being tracked better on dashboard and in analysis.

No dispute/audit that I am aware of (Please correct me if I am wrong)

There's been a couple, but the process is still young, most folks either flag issues in calls/Slack/or GitHub, and then work with the governance team to try and root cause it + identify and propose next steps to the rest of the group in the following governance call.

Not helping real-world adoption as not many real-world clients would actually want to go through the datacap apllication process (It would be more useful for platform like nft.storage to get datacap and use it to subcidize storage but getting real world clients to do that is too ideal)

Agree - the current client onboarding process is painful for clients. Approach IMO should be to improve this client onboarding UX, and this is a current priority for the program! On the second point, deal brokers like Estuary and Textile's bidbot have received DataCap and are using it in the manner you described.

No way to verify private datasets

Disagree on this - several notaries have methods for doing this to build confidence in the dataset / client. In the last governance call, a new method was also proposed to do this for LDN applications, so this is actively being thought about.

No way to determine if a dataset is useful or not

Would love to hear more on this one - is this more about there not being a consistent standard on what is worthy of DataCap or something else?

In general - I think that the approach should be more focused on:

• making the experience better for clients
• building better tooling that improves the due diligence / KYC process for clients and enables notaries to make more accurate decisions faster
• biasing towards given less DataCap faster to clients and doing analysis on actions on chain post DataCap received + deals being made, and using this as a method of validating client trustworthiness
• scaling up notaries and better defining the role as a "guardian" or "watchperson" rather than a gatekeeper of the system

These are aligned with our current goals in the Fil+ community and would result in a much more productive system to support clients in the network that establish the trust required to minimize the friction for them to onboard data onto Filecoin.

Would love to see you at the next governance call to discuss this further as well!

[Fatman13]

@dkkapur, Thank you for replying!

There's been a couple, but the process is still young, most folks either flag issues in calls/Slack/or GitHub, and then work with the governance team to try and root cause it + identify and propose next steps to the rest of the group in the following governance call.

I am not sure if the speed of governance call resolving issues could catch up the speed of LDN allocation. But we shall see.

Audits do happen - in the past this has usually been at notary elections, and I expect this to happen a lot more frequently with subsequent allocations in LDN applications coming through / being tracked better on dashboard and in analysis.

We see rules broken by projects in Slingshot and windowPost submitted on-chain disputed by other storage providers. Whether intentional or unintentional, there has to be some misbehaviour with PiBs of datacap allocated and we see none of such reports. That is concerning as to fairness of centralized notary process.

I don't think this is necessarily a problem -> if we have enough Notaries with varied interests/stakeholder positions in the network, we should theoretically be able to converge on rules and evolve them over time to ensure reasonable understanding of "fair" or "good" behavior vs. abuse of the system.

It becomes a problem: firstly, we see no reports on misbehavior; secondly, there is no clear exit route for notaries. When storage not proven for days, storage providers get slashed. But no clear punitive measures are implemented for a centralized notary process.

Disagree on this - several notaries have methods for doing this to build confidence in the dataset / client. In the last governance call, a new method was also proposed to do this for LDN applications, so this is actively being thought about.

If we have ways to verify private data then we can/should develop algorithmic ways to verify them, making notaries irrelevant.

Would love to hear more on this one - is this more about there not being a consistent standard on what is worthy of DataCap or something else?

I personally think whether a dataset is useful should be determined by the market not by notaries. One metric could be if a dataset is useful, a lot of people will retrieve such data on a regular basis.

• making the experience better for clients
• building better tooling that improves the due diligence / KYC process for clients and enables notaries to make more accurate decisions faster
• biasing towards given less DataCap faster to clients and doing analysis on actions on chain post DataCap received + deals being made, and using this as a method of validating client trustworthiness
• scaling up notaries and better defining the role as a "guardian" or "watchperson" rather than a gatekeeper of the system

All great points and totally agree with you. Given the premise that a decentralized process is better than centralized one, I think there should be rules that when A, B and C are realized, then current centralized notary process should be dissolved. Or within some time framework, A, B and C are not realized, current centralized notary process should also be dissolved or at least revaluated its effectiveness. One concern is that when instituion grow bigger, it exhaust means to perserve itself and it hunker down and defer progresses.