-
-
Notifications
You must be signed in to change notification settings - Fork 75
fix: Add Netlify Edge function to block China from accessing feed.xml #796
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
✅ Deploy Preview for fr-eslint ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Hi @Copilot!, thanks for the Pull Request The pull request title isn't properly formatted. We ask that you update the pull request title to match this format, as we use it to generate changelogs and automate releases.
To Fix: You can fix this problem by clicking 'Edit' next to the pull request title at the top of this page. Read more about contributing to ESLint here |
✅ Deploy Preview for hi-eslint ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for new-eslint ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for zh-hans-eslint ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for de-eslint ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for ja-eslint ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>
|
Hi @Copilot!, thanks for the Pull Request The pull request title isn't properly formatted. We ask that you update the pull request title to match this format, as we use it to generate changelogs and automate releases.
To Fix: You can fix this problem by clicking 'Edit' next to the pull request title at the top of this page. Read more about contributing to ESLint here |
1 similar comment
|
Hi @Copilot!, thanks for the Pull Request The pull request title isn't properly formatted. We ask that you update the pull request title to match this format, as we use it to generate changelogs and automate releases.
To Fix: You can fix this problem by clicking 'Edit' next to the pull request title at the top of this page. Read more about contributing to ESLint here |
✅ Deploy Preview for pt-br-eslint ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for es-eslint ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>
|
Hi @Copilot!, thanks for the Pull Request The pull request title isn't properly formatted. We ask that you update the pull request title to match this format, as we use it to generate changelogs and automate releases.
To Fix: You can fix this problem by clicking 'Edit' next to the pull request title at the top of this page. Read more about contributing to ESLint here |
2 similar comments
|
Hi @Copilot!, thanks for the Pull Request The pull request title isn't properly formatted. We ask that you update the pull request title to match this format, as we use it to generate changelogs and automate releases.
To Fix: You can fix this problem by clicking 'Edit' next to the pull request title at the top of this page. Read more about contributing to ESLint here |
|
Hi @Copilot!, thanks for the Pull Request The pull request title isn't properly formatted. We ask that you update the pull request title to match this format, as we use it to generate changelogs and automate releases.
To Fix: You can fix this problem by clicking 'Edit' next to the pull request title at the top of this page. Read more about contributing to ESLint here |
Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>
|
Hi @Copilot!, thanks for the Pull Request The pull request title isn't properly formatted. We ask that you update the pull request title to match this format, as we use it to generate changelogs and automate releases.
To Fix: You can fix this problem by clicking 'Edit' next to the pull request title at the top of this page. Read more about contributing to ESLint here |
2 similar comments
|
Hi @Copilot!, thanks for the Pull Request The pull request title isn't properly formatted. We ask that you update the pull request title to match this format, as we use it to generate changelogs and automate releases.
To Fix: You can fix this problem by clicking 'Edit' next to the pull request title at the top of this page. Read more about contributing to ESLint here |
|
Hi @Copilot!, thanks for the Pull Request The pull request title isn't properly formatted. We ask that you update the pull request title to match this format, as we use it to generate changelogs and automate releases.
To Fix: You can fix this problem by clicking 'Edit' next to the pull request title at the top of this page. Read more about contributing to ESLint here |
|
Hi @Copilot!, thanks for the Pull Request The pull request title isn't properly formatted. We ask that you update the pull request title to match this format, as we use it to generate changelogs and automate releases.
To Fix: You can fix this problem by clicking 'Edit' next to the pull request title at the top of this page. Read more about contributing to ESLint here |
netlify.toml
Outdated
| function = "og" | ||
| path = "/og" | ||
|
|
||
| [[edge_functions]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can replace this with defining the path in the edge function file, e.g.
export const config = {
path: "/feed.xml"
}or, typed:
import type { Config } from "@netlify/edge-functions";
...
export const config: Config = {
path: "/feed.xml"
}but idk if you want to add the needed dependency for that one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot revert the changes in this file and insert this into the edge function file:
export const config = {
path: "/feed.xml"
}|
@copilot revert the changes netlify.toml and add this code to export const config = {
path: "/feed.xml"
} |
Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>
JoshuaKGoldberg
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we use Netlify's rate limiting instead, so that all of China doesn't stay blocked until a more comprehensive solution is figured out?
| @@ -0,0 +1,21 @@ | |||
| export const config = { | |||
| path: "/feed.xml" | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem with blocking all requests from any one region is that it also blocks that region's legitimate traffic. I imagine there are quite a few real users who would be blocked from subscribing to the feed by this. To start, we could instead rate limit the requests to make this more dynamic to spiked times & regions:
| path: "/feed.xml" | |
| path: "/feed.xml", | |
| rateLimit: { | |
| aggregateBy: ["ip", "domain"], | |
| windowSize: 60, | |
| windowLimit: 100, | |
| } |
https://docs.netlify.com/manage/security/secure-access-to-sites/rate-limiting
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't seen the specific requests / their aggregations causing issues. If they're split across domain/IP and the only commonality is just "China" then this wouldn't work. At that point I'd suggest using an edge function after all, with a basic quota set up per-country. It could use Netlify Blobs as a basic key-value store.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is available on only enterprise accounts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As of yesterday this is available to all! was done as part of the AI launch...
What you don't get without enterprise is advanced rules that can apply that for China only, etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gotcha thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JoshuaKGoldberg I can provide more stats from our systems. We would have better observability for all customers soon enough, but meanwhile I'm glad to assist.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh that'd be great, yes please (and thank you for engaging+offering!). I'm mostly curious about the mentioned characteristics of the mass-requesting users: what commonalities exist between them? Do they only match on geo code / country? Is there something more granular, such as domain or IP, that we can match on?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've seen various IP ranges coming from data centers in China, and I don't have at the moment other "stable" markers to isolate that traffic - there's a specific user-agent string, but this can change over time as the bot in question gets updates, etc.
If I have any more tools or information at my disposal, I'll update you on this!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Separately: I think an ideal solution would be to cache the responses to any static endpoint/page, including the feed, at the CDN level. As in, putting the CDN in front of Netlify. I've used Cloudflare in the past to have zero-cost caching for similar situations. But if this is urgently costing the project money then ACK that it's too much work for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'd then have to move all of our DNS to Cloudflare, which introduces another provider we need to keep track of and manage access to. I just don't think this is important enough for that extra overhead.
eladroz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added another suggested optimization, which doesn't block the release.
functions/feed.ts
Outdated
| } | ||
|
|
||
| // For all other countries, serve the feed.xml content | ||
| return await context.next(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can actually remove this line -
If you just end the function (or do an early return somewhere), this also signals to Netlify to proceed with the actual content serving. Edge functions only terminate response processing if they return a Response object (like in the code above when denying access).
The explicit await context.next() is great for when you actually want to capture the response and transform it (the response headers, the body, whatever), but here it's not needed - and it carries some additional latency because when you await the content, control goes back to the edge function before returning to the client. With a simple return, this doesn't need to happen.
The impact of this added latency in this case is a non-issue, but one less line is cleaner :-) I guess we need Netlify eslint rules! 🙂
Also, I realized this behavior is not really super clear in our docs, so would fix that as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah nice!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot remove this line.
|
@copilot revert changes to package.json |
Co-authored-by: nzakas <38546+nzakas@users.noreply.github.com>
|
If anyone notices this and wants to know why, you can check out https://x.com/geteslint/status/1973732635676684540 |
feed.tsinfunctions/directory that:Original prompt
💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.