403 with food network

[n8henrie]

Failing website: foodnetwork.com

Checking if valid metadata are present: Yes

Yes, I check the source code of the website and found metadata (the recipe schema)

Cookbook version: 0.11.2,

Problem description (if applicable):

For the last few weeks, I've been getting 403s with all FoodNetwork recipes that I've tried.

For example, first one I tried today: https://www.foodnetwork.com/recipes/food-network-kitchen/air-fryer-fried-shrimp-8059268

I also suspect scraper protection, as a plain curl fails with 403 (even if I spoof user agent):

$ curl -I 'https://www.foodnetwork.com/recipes/food-network-kitchen/air-fryer-fried-shrimp-8059268'
HTTP/2 403
mime-version: 1.0
content-type: text/html
content-length: 464
cache-control: max-age=0
expires: Thu, 26 Dec 2024 15:57:36 GMT
date: Thu, 26 Dec 2024 15:57:36 GMT
set-cookie: layout=DESKTOP; path=/
set-cookie: __ds_loc_country=US; path=/; domain=.foodnetwork.com
set-cookie: __ds_loc_state=NM; path=/; domain=.foodnetwork.com
vary: User-Agent
server-timing: ak_p; desc="1735228656043_399338645_418606152_25_8642_38_56_15";dur=1
$
$ curl --user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:62.0) Gecko/20100101 Firefox/62.0" -I https://www.foodnetwork.com/re
cipes/food-network-kitchen/air-fryer-fried-shrimp-8059268
HTTP/2 403
mime-version: 1.0
content-type: text/html
content-length: 464
cache-control: max-age=0
expires: Thu, 26 Dec 2024 15:58:14 GMT
date: Thu, 26 Dec 2024 15:58:14 GMT
set-cookie: layout=DESKTOP; path=/
set-cookie: __ds_loc_country=US; path=/; domain=.foodnetwork.com
set-cookie: __ds_loc_state=NM; path=/; domain=.foodnetwork.com
vary: User-Agent
server-timing: ak_p; desc="1735228694042_399338635_2258563962_22_8673_41_44_15";dur=1

But it loads fine from the same machine in Firefox.

Interestingly, it looks like it works if I include the --compressed flag (as well as spoof user agent):

$ curl 'https://www.foodnetwork.com/recipes/food-network-kitchen/air-fryer-fried-shrimp-8059268' \
    --compressed \
    -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:133.0) Gecko/20100101 Firefox/133.0'
HTTP/2 200
content-type: text/html;charset=utf-8
strict-transport-security: max-age=63072000; includeSubDomains
last-modified: Thu, 26 Dec 2024 15:56:13 GMT
accept-ranges: bytes
content-encoding: gzip
access-control-allow-origin: *
access-control-allow-method: GET
content-security-policy: frame-ancestors 'self';
content-length: 88727
cache-control: max-age=3329
expires: Thu, 26 Dec 2024 16:57:11 GMT
date: Thu, 26 Dec 2024 16:01:42 GMT
vary: Accept-Encoding
set-cookie: layout=DESKTOP; path=/
set-cookie: __ds_loc_country=US; path=/; domain=.foodnetwork.com
set-cookie: __ds_loc_state=NM; path=/; domain=.foodnetwork.com
vary: User-Agent
server-timing: ak_p; desc="1735228902481_399338574_35083294_1060_8198_44_42_15";dur=1

Some overlap with #115, #1840

Food Network seems like it would be a really nice website to support for the URL import feature.