Forgotten parts of RFC2616
2023-01-17
by Cal Paterson
How come when you visit
https://csvbase.com/meripaterson/stock-exchanges
in a browser you get a webpage –:
but when you curl the same url you get a csv file?:
The url is the same – so how come?
The answer is HTTP’s built-in “content negotiation”.
How content negotiation works
When an HTTP client sends any request, it sends “headers” with that request.
Here are the headers that Google Chrome sends:
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9 accept-encoding: gzip, deflate, br accept-language: en-GB,en-US;q=0.9,en;q=0.8 cache-control: no-cache pragma: no-cache sec-ch-ua: "Google Chrome";v="105", "Not)A;Brand";v="8", "Chromium";v="105" sec-ch-ua-mobile: ?0 sec-ch-ua-platform: "Linux" sec-fetch-dest: document sec-fetch-mode: navigate sec-fetch-site: none sec-fetch-user: ?1 upgrade-insecure-requests: 1 user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36
That’s a lot. Only the first of these is relevant: the accept header:
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
The accept header is an unordered list of preferences for what media type
(aka “Content Type”, or file format) the server should send.
Chrome is saying:
-
Ideally, give me:
text/html(ie: HTML)application/xhtml
