> There was even a proposal to reduce this surface area, but it wasn't adopted:
>> Instead of sending a full list of the users' preferred languages from browsers and letting sites figure out which language to use, we propose a language negotiation process in the browser, which means in addition to the Content-Language header, the site also needs to respond with a header indicating all languages it supports
Who thought that made sense? Show me the website that (1) is available in multiple languages, and also (2) can't display a list of languages to the user for manual selection.
What language do you put that list in? Would you still want to show it to every visitor when you know most of them speak a particular language?
I use to do some work in this area. The first question is difficult and the second is no. We had the best results when we used various methods to detect the preferred language and then put up a language selector with a welcome message in that language. After they made a selection, it would stick on return visits.
> What language do you put that list in? Would you still want to show it to every visitor when you know most of them speak a particular language?
Judging by... a large number of websites, you make the list available in a topbar, and each language is named in itself. You don't apply one language to the entire list.
Here's the first page that popped into my head as one that would probably offer multiple languages (and it does!):
They've got the list in a page footer instead of a header, but otherwise it's an absolutely standard language selector. It does technically identify countries rather than languages. The options range from Azərbaycan to Україна. They are -- of course -- displayed to every visitor.
Why would you want to force someone to consume your website in the wrong language?
And why would the list be in a single language, again?
You’re looking at it with the perspective of someone who understands the language the site defaults to. Most non-native speakers have a hard time finding the link and they leave.
No, I'm looking at it from the perspective of someone who has needed to use that language selector in the past. Understanding the language the site defaults to wouldn't help, because the selector doesn't use that language anyway.
> Most non-native speakers have a hard time finding the link
You might notice the colorful flag right next to it.
Flags are a terrible way to indicate language. At best, they are unclear. At worst, they can be offensive.
Assuming you are a US company catering to non-English speakers in the US, which flag would you use for Spanish? Which flags would you use to differentiate between Mandarin and Cantonese? What would you do in Canada where they speak English and French? Show a French flag?
Except they're recognizable across languages. Faced with a UI in a language I don't know, going to settings -> languages -> my preferred language is a total guessing game. Meanwhile, if I'm confronted by a UI that has a tiny flag icon in the top, I know I can click on that and get to something familiar. Yes, someone looking to get offended can nitpick your flag choice, but a Spanish flag vs a Mexican flag for Spanish will at least let the user get to something closer to what they know, even though there's quite a bit of difference on the ground between Spanish in Spain and Spanish in Mexico. If your internationalization team is well funded enough to offer both, then show both flags. Same for UK English and American English, Chinese Simplify, Traditional, and Cantonese. And yes, Quebecoise French and French in France. Offer as many flags as you actually have translations for. If you can have a Chinese flag and a Hong Kong flag, users will appreciate it. Having a two level menu is also an option. Click on the Canada flag, which then offers Francaise and English is also an option.
Well, one of us has done research and work in this area. I don’t know what you’ve been doing. All of your suggestions perform poorly in the real world.
You can determine user's language from IP address location. Of course, there are users with VPNs, but they probably are used to seeing foreign content. For example, Youtube shows me advertisement in a language I don't understand despite my language header saying I only understand "en-US" and "en" languages. So this header is unnecessary, even Youtube ignores it.
Also, when using VPN, Google typically uses a language based on IP address, not my language header. I assume the header is only useful for fingerprinting today.
> You can determine user's language from IP address location.
There are reasons why it might not work (VPN is only one of them; there are others such as places with multiple languages, people traveling to foreign countries, and others), although it is also a bad idea for other reasons as well.
If the user specifies the language then you should use that one. I think it would probably be better to use the following order of figuring out which language you should want:
1. If the URL specifies the language to use, then use the language specified by the URL.
2. If the language is not specified by the URL, use the language specified by any cookies that are set for the purpose of selecting the language.
3. If the language is not specified by URL or cookies, but the user is logged in and the user account has a language setting, use the language specified by the user account. (If TLS client authentication is being used, then you might consider adding an extension into the client's X.509 certificate to select the language.)
4. If the language is not specified by URL or cookies or the user's account, or the user is not logged in, use the Accept-Language header.
5. If the language is not specified by URL or cookies or the user's account, or the user is not logged in, or the Accept-Language header is not present or cannot be parsed or does not specify any language that the request file is available in, then use the default, such as the language that it was originally written in.
> You can determine user's language from IP address location.
I live in Hyderabad, Telangana, India. I do not yet speak enough Telugu or Hindi or Urdu to be useful, and cannot read Hindi or Urdu at all; but I’m a foreigner who grew up on English only, rather rare around here, so let’s consider native Indians instead. Many can speak these languages but not read them in their native scripts, only romanised (in which case they can probably speak English tolerably). And many (many) come from other parts of India (or even Nepal) and can’t speak Telugu. Or are Muslim and at least prefer to deal in Hindi, often not having very good Telugu. And so on. It’s messy.
Some IP geolocation doesn’t even get the city right—I’ve seen Noida suggested, which is up north in Hindi territory.
More and more international audiences websites literally do this themselves, putting a language (sometimes even currency) select box option on top when they detect your settings don’t match best at first the page you are on.
Why not have this negotiation implemented at the browser level?
Because that prevents all of your users from selecting the language they want. It's a terrible idea with no upside and not-high-but-still-not-no downside.
It has a remarkably inconspicuous language selector, also using the names of countries rather than languages, located in the page footer. Compared to Dyson, Apple's list of country names is much more willing to use English in preference to whatever someone from that country would call it. This isn't consistent; many countries are rendered in their own language (日本 / Ελλάδα) and many aren't (Georgia / Kazakhstan).
The page defaults to the locale that you request in the URL. https://www.apple.com/ shows up in English, regardless of your country;† https://www.apple.com/bg/ shows up in Bulgarian. Switching your preferred location simply takes you to the page for that location. (Dyson does the same thing.) Some locations support more than one language; there's https://www.apple.com/lae/ for Latin America (English) and https://www.apple.com/la/ for Latin America (Spanish). If you're on the page for a location like this, a language selector (with language names) displays next to the location selector. In the case of Latin America, only two languages are supported, and the language selector automatically displays "Español" if you're on the English site and "English" if you're on the Spanish site, which makes sense but won't generalize.
Apple's selector is inconspicuous because it refuses to display flags, which I would guess is due to much higher political exposure than Dyson. So it's lower-quality in two ways, but fundamentally the same approach. The user asks for a language, and the site honors that.
Given that I presented Dyson as an example of doing language selection correctly, I'm confused about what you wanted me to see on apple.com. They're trying to do the right thing, but less effectively.
† I tested this by accessing the site(s) from Mongolia, Vietnam, and Morocco using ExpressVPN.
That was my point. Not comparing Apple/Dyson/whatever, but showing that website do have this need.
If this was designed and implemented as a standard at the browser level, we would get something better in the end, rather than re-implementations on each and every website.
>> Instead of sending a full list of the users' preferred languages from browsers and letting sites figure out which language to use, we propose a language negotiation process in the browser, which means in addition to the Content-Language header, the site also needs to respond with a header indicating all languages it supports
Who thought that made sense? Show me the website that (1) is available in multiple languages, and also (2) can't display a list of languages to the user for manual selection.