Skip to content

Latest commit

 

History

History
266 lines (164 loc) · 62 KB

5-privacy-2020-09-22.md

File metadata and controls

266 lines (164 loc) · 62 KB
Raw

Data. Together. Let's read about it

Private Data & Policies (September 22)

🎬 Recorded Call

Intro

How have particular implementations of data privacy policies impacted humans, economics, and legal systems? What are appropriate expectations around data privacy, and who should inform, create, or enforce policies?

Readings

Grounding

Attempted and proposed solutions

Other optional readings

Themes

  • Intersections with Trust conversation: does it make sense to apply the concept of trust to a corporation? and other questions
  • Intersections with Monopolies and Consent conversations: does "consent" really apply in situations where you have to click "ok" to access the one service you need?
  • Feasibility & implementation vs principles (esp with respect to GDPR reading)
  • Intersections with algorithmic racism: how do our policy choices (wrt protection of private data) potentially enable unintentional algorithmic racism

Notes

Grounding

  • Brookman & Hans, Center for Democracy & Technology (2013) Why Collection Matters: Surveillance as a De Facto Privacy Harm on why data collection matters

    • In terms of privacy protection, some theorists have insisted that advocates must articulate a concrete harm as a prerequisite for legislated rules

    • Others have argued that privacy protections should focus exclusively on curtailing controversial uses rather than on the collection of personal information.

    • This paper argues that consumers have a legitimate interest in the mere collection of data by third parties. That is, big data collection practices per se, rather than bad uses or outcomes, are sufficient to trigger an individual’s privacy interests.

  1. Data Breach
  2. Internal Misuse
  3. Unwanted Secondary Usage and Changes in Company Practices
  4. Government Access
  5. Chilling Effects
  • (optional) Hochfellner, Lane, and Kreuter, Responsible Data Science, NYU Center for Data Science (2019) Privacy and Confidentiality slides 1-9, 14, 18-19, 35-37 definitions and introductions to challenges and tools
    • Privacy includes the famous “right to be left alone,” and the ability to share information selectively but not publicly (White House 2014)
    • Confidentiality means “preserving authorized restrictions on information access and disclosure, including means for protecting personal privacy and proprietary information” (McCallister, Grance, and Scarfone 2010).

    • Why confidentality important

      • Promise, ethical, legal, practical implications
    • Attribute, identity, residual (combining to get confidential info) disclosure
    • Notification is either comprehensive or comprehensible, but not both. (Nissenbaum 2011)

    • Collection and analysis often no longer within same entity. Ownership of data less clear

    • The challenge in the case of big data is that data sources are often combined, collected for one purpose and used for another and users often have no good understanding of it or how their data will be used.

Attempted and proposed solutions

  • Sobers, Varonis (a cybersecurity company) (2020) A Year in the Life of the GDPR: Must-Know Stats and Takeaways a review of one year of GDPR implementation
    • Changing the landscape of data protection — The GDPR put a large spotlight on data protection and it’s being taken much more seriously across the board.
    • Greater reliance on third parties and data experts — There has been increased hiring around data protection and GDPR law advice.
    • Businesses were overall unprepared — Due to the strict penalties and open-ended nature of the legislation, very few companies felt confident in their level of compliance.
    • Fewer fines have been given than expected — It seems as though this first year has been somewhat of a grace period as everyone continues to adjust their practices.
    • Enforcement agencies overwhelmed with scope — There seem to be staffing shortages that hindered some agencies from keeping up with complaints and notifications.
    • Mixed feelings among consumers
    • Relatively minimal enforcement - seems like the success is largely self reported.
      • what are we gaining for the cost incurred?
    • Curious that some businesses chose to straight up leave the market
    • Mixed feelings about trust that this has been propely executed.
    • Notably many of the items that were supposed to be implemented seem quite fuzzy as a standard "wherever possible"
  • Office of the Press Secretary, The White House (2012) We Can’t Wait: Obama Administration Unveils Blueprint for a “Privacy Bill of Rights” to Protect Consumers Online Obama White House proposed approach
    • Transparency: Consumers have a right to easily understandable information about privacy and security practices.
    • Respect for Context: Consumers have a right to expect that organizations will collect, use, and disclose personal data in ways that are consistent with the context in which consumers provide the data.
    • Security: Consumers have a right to secure and responsible handling of personal data.
    • Access and Accuracy: Consumers have a right to access and correct personal data in usable formats, in a manner that is appropriate to the sensitivity of the data and the risk of adverse consequences to consumers if the data are inaccurate.
    • Focused Collection: Consumers have a right to reasonable limits on the personal data that companies collect and retain.
    • Accountability: Consumers have a right to have personal data handled by companies with appropriate measures in place to assure they adhere to the Consumer Privacy Bill of Rights.
  • O'Connor, Digital and Cyberspace Policy Program, Council on Foreign Relations (2018) Reforming the U.S. Approach to Data Protection and Privacy a critique of current U.S. approach and suggestion for path forward
    • Harmonization of data
    • Base-level privacy policies
    • Standardization across industry of application
  • (optional) Balkin & Zittrain, The Atlantic (2016) A Grand Bargain to Make Tech Companies More Trustworthy on applying the legal concept of a fiduciary to information as well as to finances (intersection with Trust conversation)
    • Information Fiduciary

Compliance with state legislation and common law—and the threat of class-action suits and actions by state attorneys general—have become sufficiently burdensome that some companies, such as Microsoft, already have indicated that they are open to comprehensive federal privacy legislation that would preempt conflicting state regulation. Congress could respond with a “Digital Millennium Privacy Act” that offers a parallel trade-off to that of the DMCA: accept the federal government’s rules of fair dealing and gain a safe harbor from uncertain legal liability, or stand pat with the status quo.

The DMPA would provide a predictable level of federal immunity for those companies willing to subscribe to the duties of an information fiduciary and accept a corresponding process to disclose and redress privacy and security violations. As with the DMCA, those companies unwilling to take the leap would be left no worse off than they are today—subject to the tender mercies of state and local governments. But those who accept the deal would gain the consistency and calculability of a single set of nationwide rules. Even without the public giving up on any hard-fought privacy rights recognized by a single state, a company could find that becoming an information fiduciary could be far less burdensome than having to respond to multiple and conflicting state and local obligations.

Other optional readings

Discussion

JONATHAN: I can get started with my initial gut reaction. One of the things I really liked was that first article. The short summary of it is the paper tries to reframe the question of, why do we care about private data, and why do we care about worrying about these policies, through the lens of not so much saying, do we need to prove that there's a harm, but then actually eliciting what are the things that are by default risk items that exist? I thought it was an interesting take. A thing that's a fact but not necessarily acknowledged, is that by virtue of how these systems are started constructed, and how we think about what is the default state, you can either assume harm is only introduced when a bad thing happens, or harm is introduced the moment that there is a risk of a bad thing happening. And it's interesting, tying that to some of the downstream things about, how do you think about proper regulation—it ties into what Jonathan Zittrains's things. The thing I thought was super interesting is, I don't think I've seen that framing before. Actually, you could probably comment better than I, I think a lot of the times when we think about environmental regulation, we think of it as, okay, after we've done a thing, how do we apologize? And figure out, Oh, yeah, that was truly a bad thing. Or you think about it in the context of Starlink, you shoot off all these satellites, which is theoretically a good, providing internet for everyone, but also really messing with astronomers and their ability to map out stars and planets and stuff.

KELSEY: It's a getting it under the wire before anybody tells us not to kind of feeling

JONATHAN: Which I think fundamentally highlights one of the ways that I think regulation is interesting, where a lot of times regulation is like, we don't want to stifle innovation. And so as a result, we tend to bias towards under-regulating. I think GDPR, this is really just tying all of them together. The thing I thought was interesting is having seen GDPR in practice at my old job, and then seeing it also in that other article, what is the impact after a year, you notice that when you're in the EU, you click through a lot and in theory, it does some stuff. But then it's not necessarily enforceed. They were saying in that other article, it's not like they necessarily are actually having people enforce all of it. And so there's still this question. 63% of Europeans still don't feel like their data is properly protected. And so I guess there's this question of, how do we think about regulation both from a, there is a harm that can be introduced, and in fact, we do materially see it when nothing is done, because there is a risk that we are just not acknowledging, as well as, how do you implement this in such a way that it doesn't actually stifle people trying to do stuff and it's actually easily applicable. I think it's an interesting tension.

One of the things that, again, from one of the other readings that I thought was super cool, is this idea that, maybe we just take a step back and realize, okay, there's a hodgepodge of things you've already applied here, we have HIPAA that thinks about things in a medical context, we have CAPA thinking about children. But truly, data is quite arbitrary. And you get these weird slices of things. And then maybe what we need is a meta model of these things to think about, so what do we define as PII. And if we started thinking from that lens, if you have this marking of PII on a specific column or data, then how do you need to treat that?

So I guess that loops through most of the articles way too quickly. But I feel like there's a really interesting theme there of, how do we actually write the regulation in such a way where it could be uniformly applied so there's actual confidence that this is doing the thing that we want it to do, which is providing some layer of security? How do we then make it not so hard for people to actually implement? And then also, how do we make sure that we're doing better than the status quo in the US?

KELSEY: Yeah, I think that's really interesting. And you looped through so many things that I couldn't even write down my thoughts fast enough. I'm sure we'll loop back around. These conversations are always big spirals. And I think that that's one of the things that's cool about them, is that somebody brings up a whole bunch of ideas, and then we rabbit hole on one or two of them for a little bit and figure out the next one. The last thing you said really reminded me of a quote from Nissenbaum that was quoted in that one slideshow: "notification is either comprehensive or comprehensible, but not both". It's very much along the lines of, how do we make this both functional and parseable? How can we have a regulation that actually works and that people understand and that they can implement? My product management professor had this concept of the Iron Triangle, where you have time, cost, and quality, and you can rotate it. But you can't have three fixed points, you only get to choose two of the points. There's some of that going on. The other thing that you were talking about was this tendency, and this is the one I want to drill into, this tendency to not regulate things if we don't have to. I come at this from a lot of different perspectives. One is that you were talking about that lens they were using of what is harm; does harm exist when the possibility for harm exists, or does harm exist when the harm actually occurs?

That touches on a lot of the different conversations we've had, perhaps the harm exists when the possibility for harm is likely. And it really ties into our trust conversation. I guess one of the things about regulation, I've set up a lot of different communities from scratch and gone into communities and tried to set them up to be healthier places. As an open source person, and as a community-oriented person, there are certain things that I do and don't do right at the beginning. One of the things I do right at the beginning is write a code of conduct. And that's less because I actually expect a code of conduct to come into play and more because I'm aware that that's a signal for people about whether or not this is the kind of space where they're likely to be welcome and listened to. But one of the things that I don't do and really try to explicitly not do is the kind of bikeshedding area of making rules for things that aren't problems yet.

That's kind of an interesting balance. It is a lot of work to pre create regulation, for problems that you don't yet have but can imagine being harmful. But I don't think that scales. And we always talk about what scales here. Because of course, you can have trust within a unit of five people, and you can't have the same kind of trust within a unit of 1000 people. And of course, a country is much, much bigger than that. So then you have this problem of, within a small community, within the kind of community that I'm likely to create, I think that you don't want to create a lot of regulations at the beginning, I think you want to leave it underdone. There's two main reasons. One is that it's a lot of work to make and try and enforce regulations. And the other is that it doesn't create the atmosphere that you're trying to create, where you actually depend on each other and trust each other.

But if you're scaling an organization, so that works up to 12 ish people. And this comes from startup. Once you get past about a dozen people, you don't necessarily know what everybody's doing anymore. And even if you actually are working in really good faith, you probably aren't going to understand, this is where you start needing some kind of daily scrum check in, this is where you might start needing a manager where somebody checks in and is like, Hey, is this stuff actually happening? are you communicating? Are you amassed in a community where you actually all have the same dream and intention? And so what usually happens with these organically growing communities is that you build regulations as you need them. And it can be a really communitarian exercise to start thinking about, what are the rules that we now need? And it's challenging.

But what we're talking about on a governmental level is always retrofitting, and it's weird to impose rules on an existing community at a grand scale, in ways that are definitely going to negatively impact businesses, because there's no new regulation that doesn't negatively impact some business, but that might negatively impact some individuals because of those businesses, or unrelatedly. But also not making it might also negatively impacted individuals. And so it's a really different question at that level of scale, what it is to create a regulation. And that's all a precursor to the other question that we've asked in the trust conversation of, well, are companies worthy of trust? Or does the concept even apply to a company, or is a corporate entity, something that if you don't regulate it, it will just grow? It will just do everything that it can to consume everything that it can.

JONATHAN: Two things, I think, are really interesting about that. Thing number one. I think it's interesting when we talk about retrofitting, because technology and the Internet has been a thing for a bit now. And we've seen the version of bad, so it's weird in the US. We seem not interested in getting a GDPR type version. So Obama administration put out some principles. Trump was like, why do we need this. But we're actually seeing I think what you're describing, California coming up with their own, we've seen Europe come up with their own. And so it's not even rooted in a reasonable principle. We also see data breaches and here are material bad things. It's not like it's a theoretical; it is a thing that is happening. Literally the tradeoff seems to be, so what is the sum of the total harm o- post harm thing, if we were to do aggregate number of people in the Equifax breach summed up, how bad is that versus the economic harm of companies implementing this? And I think this is where the question of, how hard is it actually to implement some of these things? And one of the questions I have is, is there a way that you could make this easier, if you were thinking about it from some of the technology side, I think about the large companies that are actually trying to do this stuff, they mostly contract from, there's open source things like Postgres or whatever. Could Postgres have specific markings that you can allocate to tables and stuff? I don't know. There might be something interesting about, how can we use technology to help scale from that perspective?

The other thing that I thought was interesting, Can we trust organizations or companies specifically? I'd be curious to know what your thoughts were on Zittrain's thing about privacy fiduciaries, because fiduciaries do exist. Generally, they are doing that for their clients, that is a legal requirement for them. I don't think I caught the trust conversation. I do think there is something about, how can we use the legal framework as a hammer or a stick to guarantee that whether you believe that they are completely rational, there is this other option of, you make it a thing that they could be designated if they want these rights, and there's a legal recourse if they don't honor that end of the bargain. Which, yeah, it doesn't have to be trustless. It could also be, it's "trustless", because there's a huge economic cameras that could in theory be swung against you, if you abuse that trust.

KELSEY: Yeah, ideologically, I really like it. Conceptually, I really like that there can be a legal thing. Like the legal responsibility that a corporation's people, I don't know if it's a corporation, or the CEO and board of directors, or whatever has to the shareholders to increase revenue.

That's just a regular fiduciary? Yeah. Well there we go. So I really like the idea of this counterpressure, or ideally counterpressure. I have a lot of cynicism about the idea that it could work, because it, has a lot of feeling to me—the phraseology used by companies like Microsoft have already indicated their interest in pre-complying with regulations that aren't yet imposed. And I look at that, and I kind of roll my eyes because I'm like, Okay, I'm very familiar with companies trying to ward off regulation by creating their own loophole-y version of it first, that says, you can't make a different one, this is covered.

JONATHAN: This isn't a complete parallel, but I have an example in this vein, too. I hear the same thing a lot about oil companies, especially now as oil companies in Europe are divesting more and more from oil as they invest in green tech. And they're selling all of their stuff to American oil companies. And so it seems weird to say, these are better oil companies than those. I do think there is something to this, as the dynamics of power change, you do actually see incentive alignment. And specifically, in this context, I would point to Apple, depending on how you feel about iOS 14, I do think that's an interesting incentive alignment; Apple is doing what it thinks is best for the privacy of its customers. And it's using that as a unique selling point. And there is this collapse of irrational and it doesn't mean that it's not opinionated, and there could be divergence of, how is Apple forming its opinion, what enforcement mechanisms do we have if we think that Apple is doing something that's not far enough? But I do think, especially as this becomes a hotter and hotter topic, there is more consumer power. And you see it also with Twitter, Jack Dorsey has talked about this too, not in the same context of data privacy, but as we think about algorithms and what is the role of social media, I know he's talked to a couple of times about, what is Twitter's role in that as, we want to keep you engaged in the short term, but the long term view of, can we do better? I will not say that I think Twitter is succeeding, but they're trying.

KELSEY: Well, there's a little bit of that, how much are they trying? I'm a cynic, but a human optimist. I'd like to believe that nobody's trying to do evil things, even within their corporate role. So I don't disbelieve that it could happen.

GREG: On that note, the phrase I usually use along those lines is Gramsci's: it's a skepticism of the intellect and optimism of the will. And I gotta say, the optimism of my will is really damp. It's really depressed right now. I'm not a technologist, but over the last five years, I've inserted myself into technology conversations, and I've been bringing up these questions about privacy, in a very specific cross section of health, human and social service, with the technology and innovation types. And nobody in these conversations were having conversations about harm. And for the first couple years, the response in these spaces was, Oh, the cybersecurity subcommittee is taking care of that, or, we have that all worked out, we have issues of consent worked out in the data use agreements, that's under legal. And I'm like, No, I don't know if you all are hearing me, I'm talking about harms that are lawful, and potentially from non-bad actors, and it just never occurred to them. And when I tried to learn from GDPR, and tried to bring some of those principles, of revocability, data transfer should be monitorable, thinking through Nissenbaum's distinction between comprehensive or comprehensible. And when I start bringing these things back, people get really quiet.

I haven't figured out how to stimulate the conversation, because it's so overwhelming. The technical people get quiet, because some of the things that I'm pointing out need to be accounted for, they're like, I don't know if that's possible. And the policy people get really quiet because the points that I'm makingt about the gaps between what's compliant with regulation versus what's ethical, they're like, I don't know what to say to that. And basically my sense of, what it is that we need, and maybe this relates to the conversation you're having about, is the harm the risk of harm? I don't know if I followed that. I might want to clarification on that. But my question in all these spaces is, who's going to be able to decide? Who's going to be able to evaluate and who's going to be able to decide? Because right now, there's some hand waving that goes on, behind the notion of individual consent, where it's like, oh, yeah, we'll ask for everybody's consent, but that just doesn't work at all as a method of giving people agency and thinking through the potential repercussions and the tradeoffs and unanticipated consequences. Individual consent, as a model, doesn't work. And also I don't see other models out there for how can communities make decisions about this stuff. The closest I get in these spaces is trying to get these—basically the issue in health human and social services in my field is, after Obamacare passed, hospitals and health insurance companies suddenly realized that people are sick because they're poor. And they suddenly cared about people not coming back to the hospital. They wanted them to stop getting sick, which apparently before Obamacare, actually it was fine if they kept coming back to the hospital because they kept getting sick because they were poor. And so now, health care is like, we've got to get everybody out of the hospital. We got to send them to social services. So we've got to get every community organization onto the same platform so that we can refer people directly to them and know exactly what happened with the social service organization and the case management system. And it all needs to be integrated. And when I come up, and I'm like, have you considered the harms of that, even though this is driven by healthcare, the prospect of do no harm as a first principle has never come up. But I've made the case, now people are turning to me, and they're saying, Okay, what should we do? And I don't know.

JONATHAN: I'm curious if you could maybe go into that a little bit more. Is it about data leaking? Or, in what contexts, or how does that manifest?

GREG: I think there's a range of possible harms. I might want to try to put this on the range of like, good and bad actors conscious conscious or just an unknowing, nature, because most people, when they think of harms, they think of like cyber hacking. But there's also deanonymization. Especially when we're talking about bringing data from all these different systems and linking them together, deanonymization seems to be a much greater risk than many of the people in the spaces seem to want to recognize. But I'm also thinking beyond deanonymization, of the tremendous potential harms that can come from the use of aggregate data from all these different systems in algorithmic decision making and regulation.

In the context of health care, these systems are building algorithms that decide who gets what kind of care. And they can make those decisions according to things like—most recently, Native American women coming in for COVID tests were separated from their children by an algorithm in New Mexico, by the New Mexico health system, because some algorithm decided that those children are at risk, because of some data that was fed to it. And so every time a Native American woman came in to get tested for COVID, they were separated from their child. And this might not have been a conscious intention; that policy might have emerged from just a bunch of decisions made by essentially machine learning and artificial intelligence that maybe nobody is specifically accountable for. And in other contexts, there's lots of talk in this space about improving health outcomes, but what it really means is saving money for the hospital system, and the proxy for, is something good or bad, is like, does it save money or not? And because poor people, especially in black people, in particular, have more health problems associated with themselves, they end up getting shunted by algorithms out of certain kinds of care contexts and into others, that the hospital system is just like, I don't have to deal with that, because that's going to be more expensive. And it's going to be a less valuable use of my resources. And also, the potential intervention is less impactful, because it might stack up against all these other problems that the person has, so they don't deserve to get it. There are all kinds of ways in which this data just serves as input into the system that yields all these inequitable outcomes. And so privacy doesn't really cut it.

It's also like, how is the aggregate set of this data being used to allocate resources in ways that might re-entrench existing patterns? And I don't know that people, certainly in the elite conversation, when you get people on a panel talking about how awesome healthcare interoperability is, these issues don't come up. And then when I ask these questions, and they're like, oh, gosh, we hadn't thought about that, it's the nurses who come up to me and say, thank you for asking that question, because I've been wondering about that. The technology innovators just don't really think about it, the healthcare executives just don't really think about the potential for these things to go wrong. It's the people who've seen things go wrong over and over again, who are the ones who might not know exactly what's gonna go wrong, but they know fuckery is underway. And so my question here is, how do we get those nurses into governing bodies? When when privacy comes up, that's what I'm wondering, is how do we get the people who actually deal with the shit to be involved in the process of making decisions about what should and should not happen?

And that is a very unpopular question, I'm finding.

KELSEY: Have you heard of Buurtzorg? This is a key example that is used in a book called Reinventing Organizations. I haven't read this in a while, but the example is that basically a group of health care workers working for a company and experiencing a lot of those issues, and also experiencing a lot of labor justice issues, on a personal level, kind of threw everything out and form this nurse cooperative, that's quite big. I think it covers a pretty large amount of a country, the Netherlands, maybe. I don't want to be the radical on the call who's like, cooperatize, it'll solve all your problems. But what they've done is create this really direct line of communication between actually doing the care and managing how care is done. It is a cooperative in this case,

GREG: You're not going to be the radical on this call if you start talking about cooperatives as the mode of solution for many of these problems. We might end up forming a cooperative.

You all talked about Ostrom, right?

KELSEY: We have before, not in a while.

GREG: Building off of Nissenbaum's work, there's a branch of Ostrom's common pool resource management school of academic thought that's specifically about knowledge, commons, I think y'all read some of those. Some of those folks have recently taken Nissenbaum's framework for contextual integrity as the important thing about privacy in this interconnected world, as opposed to, does the government know what's going on with me, it's more just, is information that I share in this specific context going to be appropriately translated or blocked from being used in a different context? The old mode of privacy doesn't really apply to that. And this actually does lend itself to thinking about privacy and trust, to both of these earlier points as a resource, and people's dignity as a resource. And the collective of that trust and dignity is a common pool resource of sorts, in that it can be easily squandered and polluted, and there are ways to potentially cope with the threats to that vulnerable resource, and those ways essentially entail institutional design. So maybe a company is capable of stewarding some piece of this puzzle, but that that steward needs to be monitored, based on what we know about vulnerable resources. You can have an appropriator who has the power to deal with this resource, but who's gonna monitor that appropriator? Who's gonna monitor the monitors? How are rules about what is being monitored set? Are those rules set by people whose stakes are involved in the management of the resource? And I appreciate having this frame. But the thing about common pool resources is, the more complex it gets, and the bigger the scale gets, and the more diverse interests are involved, the harder that shit is, and it's hard on simple scenarios, right? The more I learn about this stuff, the the less hope I have, which is a scary situation.

KELSEY: I think you hit it on the head earlier, when you were talking about, how do we get the nurses to be the ones making the decisions? I guess my point about cooperatives is I don't think that they're by themselves a panacea, I've definitely seen them done poorly. But I think that that's a big piece of what we're trying to reach for in a participatory democracy model. Maine has ranked choice voting, at least in theory, and that starts to get towards our ability to trust that our vote does something, and that starts to create this idea that a government might actually work for its people, it's very hard to not have the very American context centered right now of, we're about to go into this, what has already been a shitshow of an election cycle, and nothing is working and nobody trusts anybody. We used to get this lovely complacency of, well, we don't really have to worry about it, because it doesn't really impact us that much. And as untrue as that might have been, it's never been less true. Nobody's feeling that anymore.

One of the projects I'm working on right now is, the point of EPA is to enforce environmental regulations. And a big chunk of EDGI's work over the last few years has been showing that they basically just don't. My own research project that I didn't publish, because I'm a nervous data scientist, basically showed no correlation between violating a regulation and receiving enforcement action, nationwide.

And that just seems not good. We're doing a much more intensive reviewed process right now to get that much more specifically. But there is that problem of, you can make a regulation, but what happens next? You have actually yesterday to actually follow it. I don't even think these regulations are that good. They're literally permission to pollute. There's work around that. But even this very little bit that we have, there's not really any good reason to take it seriously.

JONATHAN: It feels like there's two separate threads that you could pick out. One is going back to Greg's scenario, this question of who designs the algorithms, what data is shared either willingly or not, and then, what are the conclusions and the facts, because there is a sort of arbiter of truthiness, that ends up coming in, as you decide, if we're all going to defer trust to the algorithm, we want to believe that the algorithm has fair inputs. We understand the caveats. Anyone who's tried to write an algorithm knows exactly how caveat intense that might be. But I think, especially in business contexts, when you get these layers of abstraction, that definitely falls away, and people just like, Oh, yeah, the thing spit out the score, and therefore, we do the thing. The other thing that Kelsey touched on, I think there's this other question of, weirdly, I feel like this comes into the immigration debate too, where you can have this policy, and there's no way that the policy is actually enforced, and so what is the point of the policy? And it leads to this question of, what is the policy doing? In the environmental context, it might be, you have this regulation, or even taxes, if you have all these rules for taxes, and you defund the IRS, no one's like, gonna go audit anyone. So then what is the point of the rule? It effectively is undercut by the fact that we can't actually monitor this stuff.

GREG: In my field, basically, I have to start at the remedial place of helping people think about the difference between infrastructure and application. Facebook, 15 years of Web 2.0 has has polluted an already dull American mind that thinks software applications are infrastructure. I guess there are some contexts in which you can make that argument but, what we need in this field of health, human and social services is infrastructure on which various applications can work, but people are seriously just stuck in, what will the software look like that everybody will use? That's their level. And so I'm trying to make the point of, infrastructure, you don't know what it looks like, I know you all want the solution in your hands, but we've got to actually build the things that stand behind the things that people use that enable those things to work. So finally, I'm on the path. And I'm helping people understand, what does that mean? And I had to get down to the level of, the data exchange pipes, the data lake, all that shit is infrastructure, but also the meetings where you review what's happening in the pipes and in the lake, that's infrastructure, and the process of making decisions over what should happen in the lake, and what should be able to go through the pipe, that process is infrastructure. Understanding that it's not just the thing itself, but the way we use the thing that's really at stake here is a level of education that I'm exhausted to have to get down and basically be a schoolmarm about. People have just been mystified by this Silicon Valley culture of, it just works, and they're not able to think in terms of complex systems, which seems like a priority.

I think that's also reflected in this notion that individuals will consent once to something that's spelled out according to some contract that was signed five years ago when the software was procured, and then that's it.

KELSEY: Which you didn't read in the first place, probably.

GREG: But this notion of individual consent, they take it as a given that people should own their own data, but I'm like, think about it for a second, man, a woman going into a social service provider has three kids, an ex-husband, the kid has a boyfriend, there's a caregiver involved. And her data is tied up in all those people's data, right? If she's going to talk to her social worker about this stuff, or her health care provider, they're going to ask her these questions, because if they want to address her social determinants of health, which is what it's all about, they need to know all this information about her home situation, her family life. So she's sharing all this data about other people. That's her data. She consented to share it, but what about them? We have no framework for thinking about how do you protect people whose data is entangled with other people?

KELSEY: Jonathan was talking earlier about that positioning of harm. Does the harm exist when it occurs? Or does it exist when the opportunity for harm is first created?

GREG: Explain this to me a little bit more? Because you said it a couple of times, and I don't know if I get it.

JONATHAN: It's very simple. Imagine you have a store. You get a bunch of credit card information, because people buy stuff from you, you have all their personal, Where are you shipping it to, their full name, all that good stuff. Is the harm introduced at the moment when Kelsey hacks me and that data is leaked? Or is the harm introduced the moment I didn't encrypt your data, such that even if she hacked my system, she wouldn't be able to read anything?

GREG: Right. So it's, is the bad thing, making the harm possible?

JONATHAN: Yeah. And I think the point that Kelsey is making is, there is this interesting question of especially when, even as a social thing, you think about what data permissions do we give to each other? Kelsey volunteered Cameron's email to me when I needed a contact. And there is some sort of social trust that we imbue on people. In a human context, it feels normal. If I was to look at Kelsey's contact list and see every person she's emailed in the last year, clearly very different. And so there's a version of, what are the socially acceptable versions of what we share, and then also, what right and in what contexts do I have the right of veto. Even think of Facebook, when back in 2011, or 2008 or whatever they were trying to really make the social graphs an API that anyone could plug into, that's an interesting example of the same issue of, do I get to volunteer the fact that we are friends to the world or some application that you may not want to know?

GREG: Yeah, and how do we navigate that tension between comprehensiveness and comprehensibility? I want people to have tools so that they can gradually think through the implications of different things. I think about this stuff all the time, and when I'm presented with a consent form, do you agree to these terms of service, I'm like, fuck this, if I really don't trust this place, I'm not going to agree. But if I feel like I need to get in there, then even if I only distrust it a little bit, I'm still gonna agree, because it's like, I gotta get in there. I'm presented with this binary choice, and it's disempowering. So, are there examples of methods that enable people to navigate between what they can immediately comprehend and the broader comprehensive universe of potential implications?

KELSEY: Well that's kind of fun, too. If you're asking me to consent to a data service, and it's a new niche one, and I'm picking one among many, I'm less likely to read the whole Terms of Service and more likely to read the founder bios. That's how I'm going to know whether I trust them.

JONATHAN: That's super interesting.

GREG: But you're super savvy, though, so what about other people?

JONATHAN: I do think there is an interesting model where something kind of similar has happened before. I think about open source licensing. And you think about how, especially, companies have different policies about the types of things that they can use. And so the tools that have been written to automatically flag when certain types of policies are embedded in dependencies or other things inside of projects. It does feel like there is an analogous thing that you want, where, I don't know how one actually goes about enforcing, but something to the effect of, you want some sort of general framework that can be applied over and over again, so I'm not trying to understand the 15 different flavors of Microsoft's version versus Facebook's, or whatever. There's a standard thing that I sort of know. I can then more explicitly give consent, because I know what I'm signing up for. It feels like that well-trodden path, and it also gives you the ability to, you can even imagine, in a browser, this is getting way too specific about a technical solution, but you can imagine the sorts of things that you may be able to configure, and say, I want to, for certain types of applications, enable these things, maybe by default, these things not by default, and then explicitly be able to, one, be able to review, who have you given whatever to, and be able to revoke those permissions.

GREG: Isn't this what Solid does? It's Tim Berners Lee's new modular approach to web browsing, where I think Solid you're able to, I think it gives you those kinds of granular controls over what's happening with any given site that you go to, is it? Is that right, Kelsey, are you familiar with this?

KELSEY: I feel like I ought to know. I've quoted him on the subject. But I still don't have a great understanding of it. It's very hard to read good explanations of this decentralized web stuff, because it's like, what level do you want it on? The basic "you can get it" level or the level where you still won't understand even when I told you everything it does?

JONATHAN: Is this the thing that's like everyone has like their own personal data capsule or whatever?

GREG: Yep, I think so. Yeah.

I've signed up for a whole hour and a half workshop tomorrow at one of these Internet symposiums about design patterns for decentralized technologies. So I'm not a designer or technologist, but I'm ready to hear what's up. I downloaded Mastodon, I looked at that shit, and I had no idea what to do.

KELSEY: Yeah, I mean, you should drop those notes in the chat if you take any tomorrow, because I'm really curious.

I wanted to circle back on, Jonathan, really early on the conversation, you brought up this idea. I had talked about how, for example, GDPR, or any retrofit regulation that applies very broadly, is going to negatively and positively impact people and companies. And you talked about, let's try to enumerate the harms, maybe we could do some version of looking at how much it hurts various communities. I'm curious what you think in terms of where the concept of equity fits into that?

JONATHAN: I don't know if I was making a comment about how things should be, more an observation how they are. I think when we talk about, the harm imposed, it tends to be this balance of some probability. If I was to imagine how Equifax prior to them getting hacked talked about things, I assume it was something on the order of like, we have this list of priorities. I'm sure it was on someone's to do list, whatever security stuff they need to build. But it was lower priority. And the reason it was lower priority is there's some sort of tradeoff between, what is the cost of this thing getting leaked, versus actually doing the fix? And the math balanced out to—and maybe that can also be, humans are bad at gauging risk—but clearly, there was, somehow, in whatever manager's function, some sort of discounting that was happening.

I do think there is this question about if we think about what is the harm that's introduced, I think it can really vary depending on what data and in what context, what conclusions that leads to, which can also make it really hard to make an argument from trying to quantify harm to even begin with, because it really does matter how that data ends up getting leaked and what it can be used again for, which is why I don't know if it leads to a convincing argument. Just because you could really go deep, or you could be like, Oh, yeah, but what's the probability that actually happens? So I don't know if I have a specific thought there, other than I don't know if that actually leads to a good. It feels like the wrong path to try to convince someone of why they should do a thing.

KELSEY: Yeah, there's something—Jon and I were both talking earlier about how ridiculously busy we are right now, while in the process of implementing technological infrastructure. I took a antiracism workshop this summer, that was really, really good, by the Adaway Group. And one of the things that it said that I hadn't heard anywhere before, was this concept that busyness is a tool of white supremacy. And there's a lot of different nuances to this. But one of the ways that I think that that can be true, is, if you're the person in the position of implementing the change, you're worried about you, and you're worried about what your boss is going to say if you don't get the change done on time, and that's one of the ways that we have these data vulnerabilities. Somebody's just trying to get stuff done in a system that says, sooner is better, no matter what the tradeoff is, as long as we don't notice a big gaping hole, quick merge.

JONATHAN: Do you wonder if—part of this is maybe a broader question about policy, it does feel like one of the most useful things that government can do, I think there's many useful things it could do, but one is, how you explicitly rebalance an equation that is unbalanced? How do you make this a higher priority, explicitly make it much more expensive if it goes wrong? I mean, in theory, this is how things work. You want this in many forms, but what is the right check and balance? You could argue, in theory that is the EPA. But what point does that get undercut, depending on who's in power and their actual commitment to trying to get into a specific outcome?

The economist in me really loves the idea that the thing that the government is doing is somehow collapsing the rational with the moral to just make it so incentivized or deincentivized that, the good thing is the thing that you get. But it's really hard to predict all the ways people will figure out how to do the bad thing that is still okay.

KELSEY: Where we got to in our trust conversation is mostly agreeing that trust is a human-to-human thing. One of the things that's kind of interesting, and this comes up in environmental conversations all the time is, what if there was personal rather than corporate responsibility for violation of laws and regulations?

GREG: You mean like, shouldn't Mark Zuckerberg go to jail?

KELSEY: Yeah. If it can be proven that he had control over the thing, and didn't fix it, and/or didn't anticipate the problem, or if he was the expert who should have known? Shouldn't he be personally vulnerable in the same way as his users are personally vulnerable when our data is used?

GREG: And his board members, that's when it would get really interesting.

KELSEY: Even just saying that, it's not like he made the code. It's not like he was on the process of auditing it.

JONATHAN: I do think there is an organizational thing that's hard to deal with, too. To pick an example, I don't know if you guys watched the congressional things where they interviewed all the CEOs. Bezos was asked point blank, does Amazon use pricing data from the website, to do something. And he was like, "it is a matter of policy that we don't," but he wouldn't explicitly say, "I know for a fact that we don't".

Maybe this is exactly the point. You may have a policy, but if there's no consequence for violating the policy, then you're not actually implementing. If someone discovered it, yes, they would be reprimanded or whatever. But no one has their neck on the line, because Bezos is worried about him personally being responsible. I do wonder what negative ramifications that could have. When you're talking about these super large actors, it's very clear and obvious, but I actually think they're the least likely, the very, very large ones, to have the same sort of issues as, think about Clearview, well, Clearview is also maybe a very bad example because they seem very intentionally trying to go into a specific area. Clearview, if you're not familiar, scraped a ton of data off the internet for facial recognition. I think it tends to be smaller companies that end up being the ones that—there's this counterbalance of, to what degree are we okay just solidifying Facebook's lead here, and just being like, Facebook is the one that will be the arbiter of power, because the new hurdles for anyone else is gonna be quite high. And part of me is like, we need to figure out how to both make it so that we can make it cheaper for people to be compliant and do the right thing, and also make it more expensive to not do that right thing.

KELSEY: Right in the beginning, you were talking about Starlink, different ways in which we try something in order to find out if it works, or in order to find out if it's viable, I guess. I read a book of environmentalist essays, back before I was really in this scene. Actually, at the time, I was very, very much in the tech startup scene, and was also very much on that train of: throw stuff out there. Make your name as fast as you can. Whatever way you can, if people are willing to give you money, it's good enough, it'll work, make it happen, ship it. And one of the essays was all about the zero harm policy or something. It's an argument that's used against using genetically modified foods that says, if you can't prove that it's harmless, you shouldn't do it. At the time, I had this reaction, Oh, come on, we would never do anything, because you can never prove that it's harmless. It's very interesting to look at that with my current perspective, because I'm not sure I totally disagree with me for myself. But I get it a lot more.

JONATHAN: I think one context I think about, which is really deviating from privacy, but self driving cars, depending on your definition, it will either be here very soon or never, because it really does come down to, how do you define harm? And what are reasonable levels of harm? I think it's an interesting, we will have to write a lot about this one thing, but I do think it could also be applicable elsewhere. What is the gating threshold? The other way of looking at it is by default, there is some harm that's involved. We are accepting the status quo of, let's take genetically modified food, you could talk about a golden rice, how many people are unable to get access to rice, and by having this genetically modified option, your choices are either less food overall, or this risky food. And so I think in specific contexts, it might be worth also piecing apart, are we already implicitly saying, the status quo is the acceptable thing?

In the Starlink example, I think you could maybe put it as, it's easy for us to say that Oh, yes, if we do this thing, we're introducing space junk and whatever. The other question is, what about the people who don't have access to the internet, but want it? What are we implicitly saying about, because you weren't born in the right area, you don't deserve access to all of this wealth of free knowledge that is just there.

KELSEY: I've seen a similar argument used around nuclear power, where we have this idea that you can't use nuclear unless you have no waste, where you can store it for literally forever. Versus, how much radiation is emitted by uranium unharvested? Can we at least get to that level? Now you have a reasonable-seeming line, where before you just had this, let's just never use it.

GREG: Have y'all seen the feminist data Manifest-no? manifestno.com. In my field, I'm known as the leftmost edge, but reading this, I was like, Oh, fuck, they are far to my left, and they seem correct. Where they're just like, they're basically like, we refuse to operate under the assumption that risk and harm associated with data practices can be bounded to mean the same thing for everyone everywhere at the same time. That's just how it starts. And it gets harder from there. And I'm reading through this, and I'm like, you're right. And at the end of it, I don't know what I'm left with, as someone who wants to reduce harm in these fields. What they're basically making the case for is refusal and rejection, and it seems solid to me, which is worrisome, because where does that leave me as someone who's trying to do ethical work? I've made my peace with it to date, but looking ahead to what I expect will be a very bad situation next month, and what I expect will become much worse in January, these fields are going to want to continue pretending everything's basically fine, and politics is so weird, indefinitely. And I think that, I'm personally approaching a point in my professional work where I'm going to have to start taking this manifest-no more seriously under an explicitly fascist administration. Hospitals shouldn't be collecting people's personal information and sharing it with Child and Family Services. It shouldn't happen, at all!

KELSEY: I think you're right, Greg. I think this thing is totally right. I don't think it means you can't do anything with data, I think it means you have to do things with data that specifically are really participatory.

GREG: Right. And, I don't know, the field that I'm in, I was already struggling just to be like, let's create a group of people who aren't just users, but who are setting priorities for this entire system and evaluating outcomes. And maybe that's a little bit what this is. Maybe that's a version of this. But yeah, I am concerned that the forces of power are inherently going to have the upper hand when it comes to complex information systems, no matter what kind of participatory action research you throw at them. That's my concern. And in a situation where we've gone from passive white supremacy in this country to active white supremacy, I think that leaves those of us who who believed in innovation in a very in a very difficult spot.

KELSEY: I'm wondering in our last eight minutes if there's more ground we want to cover because I feel like we've thoroughly talked through and depressed ourselves about this.

GREG: No, give me some bright spots! What's working? Give me appreciative inquiry: have you seen it gone well? Who's doing it right? Other than Buurtzorg.

KELSEY: Buurtzorg is good. I like what we're doing with the Environmental Enforcement Watch, we're trying to get different people involved with EPA enforcement data.

GREG: I mean, Design Justice as a text, and the many networks of thought and activism that Sasha points to there was really helpful for me to bring into some spaces. And they specifically linked to Our Data Bodies, which I find to be a really good start as far as a report that brings some of these ideas into a digestible framework.

KELSEY: Max Liboiron's lab in Canada does a lot of data justice and true academia moving the needle on who's allowed to be participatory in what spaces.

GREG: How can I talk to people about the risks of deanonymization?

KELSEY: I mean, honestly, you came in and you were like, I feel like I've been the first person in a lot of these spaces to think about harms that aren't just legal harms. And I feel very similarly, including in leading a decentralized web meetup here in Seattle. There's definitely a cycle going on of, you can become a technologist, because you had access to the materials and the community. And so some of the people doing it just haven't really thought that far beyond how cool it is that you can do the thing. I've had a lot of conversations here; somebody said to me, it'd be so cool if every car had a camera on it, and you could monetize, just by driving around, showing wait times at restaurants. And I was like, I don't want anything about that. Can I break that down for you? That's kind of a source of hope, what you're doing, integrating into different communities. And saying, hey, have you been asked this question, have you thought about this? That's what I want Data Together to be too, for people, a space to really think about this stuff.

GREG: I feel like we need grief counseling for technologists. Our peers are going through it. I've seen many people going through it, and it's clearly painful. But more people need to go through the process of mourning for the internet that we grew up believing in, and for the notions that inspired us to do this work. We have to grieve. We have to let it go, and see what comes up through that process. What are we left with?

KELSEY: I think I asked someone on the last Data Together call how do we implement this idea on a broader scale? How do we implement trust on a larger scale? And I can't remember who it was who just basically clapped back at me and said, you don't. You grow it from the ground every time.

KELSEY: And that's not wrong.

GREG: Yeah, yeah. It's what I was worried about.

KELSEY: We can build better infrastructure, too. I've got some ideas for how the EPA can have better hygiene on their enforcement data practices.

Chat log

00:15:28 Greg Bloom: hello! 00:51:16 Jonathan Victor: I really miss the complacency :( 01:06:35 Kelsey Breseman: have to go find a charger, I’m still on audio 01:17:10 Kelsey Breseman: some of my friends have a 10-year bet going re whether then the date arrives there will be some US city where non automatically driven cars are illegal 01:18:43 Kelsey Breseman: https://www.manifestno.com/