It feels like we have a new privacy threat that’s emerged in the past few years, and this year especially.
I kind of think of the privacy threats over the past few decades as happening in waves of:
First we were concerned about governments spying on us. The way we fought back (and continue to fight back) was through encrypted and secure protocols.
Then we were concerned about corporations (Big Tech) taking our data and selling it to advertisers to target us with ads, or otherwise manipulate us. This is still a hard battle being fought, but we’re fighting it mostly by avoiding Big Tech (“De-Googling”, switching from social media to communities, etc.).
Now we’re in a new wave. Big Tech is now building massive GPTs (ChatGPT, Google Bard, etc.) and it’s all trained on our data. Our reddit posts and Stack Overflow posts and maybe even our Mastodon or Lemmy posts! Unlike with #2, avoiding Big Tech doesn’t help, since they can access our posts no matter where we post them.
So for that third one…what do we do?
Anything that’s online is fair game to be used to train the new crop of GPTs.
Is this a battle that you personally care a lot about, or are you okay with GPTs being trained on stuff you’ve provided?
If you do care, do you think there’s any reasonable way we can fight back?
Can we poison their training data somehow?
I have a similar kind of idea.
I think if it had been a free/open source/community project that made the headlines I would have been all like “this is so awesome”.
I guess what I don’t like is the economic system that makes that impractical.
In order to build one of those giant GPTs, you need tonnes of hardware (capital), so the community projects are always going to be playing catchup, and I think quite serious catchup in this arena.
So the economic system requires that instead of our posts going to a “collective hive mind” that aid human knowledge, they go to some walled garden owned by OpenAI, which filters and controls it for us, and gives us little bits of access to our own data, as long as we used it only in approved ways (i.e., ways that benefit them).
Most of the data used in training GPT4 has been gathered through open initiatives like Wikipedia and CommonCrawl. Both are freely accessible by anyone. As for building datasets and models, there are many non-profits like LAION and EleutherAI involved that release their models for free for others to iterate on.
While actually running the larger models at a reasonable scale will always require expensive computational resources, you really only need to do the expensive base model training once. So the cost is not nearly as expensive as one might first think.
Any headstart OpenAI may have gotten is quickly diminishing, and it’s not like they actually have any super secret sauce behind the scenes. The situation is nowhere as bleak as you make it sound.
Fighting against the use of publicly accessible data is ultimately as self-sabotaging ludditism as fighting against encryption.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !privacyguides@lemmy.one
In the digital age, protecting your personal information might seem like an impossible task. We’re here to help.
This is a community for sharing news about privacy, posting information about cool privacy tools and services, and getting advice about your privacy journey.
You can subscribe to this community from any Kbin or Lemmy instance:
Check out our website at privacyguides.org before asking your questions here. We’ve tried answering the common questions and recommendations there!
Want to get involved? The website is open-source on GitHub, and your help would be appreciated!
This community is the “official” Privacy Guides community on Lemmy, which can be verified here. Other “Privacy Guides” communities on other Lemmy servers are not moderated by this team or associated with the website.
Moderation Rules:
We prefer posting about open-source software whenever possible.
No soliciting engagement: Don’t ask for upvotes, follows, etc.
Surveys, Fundraising, and Petitions must be pre-approved by the mod team.
Be civil, no violence, hate speech. Assume people here are posting in good faith.
Don’t repost topics which have already been covered here.
News posts must be related to privacy and security, and your post title must match the article headline exactly. Do not editorialize titles, you can post your opinions in the post body or a comment.
Memes/images/video posts that could be summarized as text explanations should not be posted. Infographics and conference talks from reputable sources are acceptable.
No help vampires: This is not a tech support subreddit, don’t abuse our community’s willingness to help. Questions related to privacy, security or privacy/security related software and their configurations are acceptable.
No misinformation: Extraordinary claims must be matched with evidence.
Do not post about VPNs or cryptocurrencies which are not listed on privacyguides.org. See Rule 2 for info on adding new recommendations to the website.
General guides or software lists are not permitted. Original sources and research about specific topics are allowed as long as they are high quality and factual. We are not providing a platform for poorly-vetted, out-of-date or conflicting recommendations.
deleted by creator
I agree. I’ve always thought that AI would be our successor species, humanity’s child. I like that I might be some small part of its heritage.
I have a similar kind of idea. I think if it had been a free/open source/community project that made the headlines I would have been all like “this is so awesome”.
I guess what I don’t like is the economic system that makes that impractical. In order to build one of those giant GPTs, you need tonnes of hardware (capital), so the community projects are always going to be playing catchup, and I think quite serious catchup in this arena. So the economic system requires that instead of our posts going to a “collective hive mind” that aid human knowledge, they go to some walled garden owned by OpenAI, which filters and controls it for us, and gives us little bits of access to our own data, as long as we used it only in approved ways (i.e., ways that benefit them).
Most of the data used in training GPT4 has been gathered through open initiatives like Wikipedia and CommonCrawl. Both are freely accessible by anyone. As for building datasets and models, there are many non-profits like LAION and EleutherAI involved that release their models for free for others to iterate on.
While actually running the larger models at a reasonable scale will always require expensive computational resources, you really only need to do the expensive base model training once. So the cost is not nearly as expensive as one might first think.
Any headstart OpenAI may have gotten is quickly diminishing, and it’s not like they actually have any super secret sauce behind the scenes. The situation is nowhere as bleak as you make it sound.
Fighting against the use of publicly accessible data is ultimately as self-sabotaging ludditism as fighting against encryption.
deleted by creator