Subject to telemetry results from bug 1648464, replace the Text Encoding menu with a single menu item called Override Text Encoding that performs the action currently performed by the item Automatic in the Text Encoding menu.

Benefits:

• User doesn’t need to know which item to choose.
• Removal of one level of submenus, which might allow the new single item to remain in the hamburger menu.
• Harder to get the user to self-XSS, because the attack needs to fool the detector instead of stating a particular encoding to the user.
• Allows for even more conditions where the item can be disabled (since there will be more cases where we know the item won’t change anything), which saves users’ time by not allowing the feature to be activated when useless.

Additional non-user-facing benefit:

• The back end code for supporting the non-Automatic menu items got in the way of implementing bug 673087. Based on that experience, chances are that fixing bug 1701828 would be much easier if we first implemented the change proposed here.

Attachment #9217362 –
Attachment description: Bug 1687635 part 1 – Add new localizable strings. → Bug 1687635 part 1 – Add new localizable strings for Override Text Encoding.

Assignee: nobody → hsivonen

Status: NEW → ASSIGNED

Attachment #9217363 –
Attachment description: Bug 1687635 part 2 – Change UI code for Override Text Encoding. → Bug 1687635 part 1 – Replace Text Encoding submenu with Override Text Encoding item.

Attachment #9217373 –
Attachment description: Bug 1687635 part 3 – Disable Override Text Encoding when known not to have effect. → Bug 1687635 part 2 – Disable Override Text Encoding when known not to have effect.

Flags: needinfo?(mwalkington)

Hi Henri, can you help me understand where this label appears (perhaps via screenshot and trigger instructions)?

Changes to menus added via the Customize pane were not in scope for MR1 and Flod advised not to make any changes to this menu.

Flags: needinfo?(mwalkington) → needinfo?(hsivonen)

Hi Henri, can you help me understand where this label appears (perhaps via screenshot and trigger instructions)?

The string “Override Text Encoding” appears in two places:

• In the menubar View menu as a menu item in the place currently occupied by the Text Encoding submenu. (I.e. the submenu is replaced with a single item called Override Text Encoding whose function is the same as the current item Automatic in the current submenu.)
• As the label of the toolbar button, which is currently labeled Text Encoding in toolbar customization or in the overflow menu. (Instead of the button opening a menu, with this patch clicking the button performs the operation that is currently performed by the item Automatic in the menu currently opened by the button.)

The string “Guess text encoding from page content” appears as the tooltip for the toolbar button.

The accelerator key, c, for the Override Text Encoding menu item in the View menu intentionally remains the same as the accelerator key for the current submenu.

I generated try builds for experimentation:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=cd2f329ffee9e3cf540c45bc0fa419e7125ef133

Note that the menu item and button are disabled on pages where they’d do nothing. Here’s an example of a page where they are enabled:
https://hsivonen.com/test/charset/unlabeled-utf8/ja.htm

I’m attaching an image of screenshots with the current situation on the left and the situation with the patch on the right.

In the View menu, the item Automatic of the Text Encoding submenu becomes an item Override Text Encoding in the View menu directly and the submenu and the other submenu items go away.

The item Automatic in the menu opened by the toolbar button becomes the action of the toolbar button itself so that that the toolbar button no longer opens a menu. When the action is not available, the toolbar button itself is disabled instead of the current state where the toolbar button is always enabled but its menu items are disabled.

Flags: needinfo?(hsivonen) → needinfo?(mwalkington)

Thanks, Henri. I also spoke with Flod to get a better understanding of text encoding in general.

What is the feature overriding text encoding with? i.e., the page is not displaying correctly. I select “Override Text Encoding” to fix the issue, which overrides the current text (or characters?) with _____.

Flags: needinfo?(mwalkington) → needinfo?(hsivonen)

What the user wants to do is to override the text encoding that they visually believe to be wrong with the correct one. What the menu item does is it overrides the text encoding with a guess made from the page content.

Whether the verb should be what the user is trying to do (Correct, Fix) or less promising of success (Override, Guess) depends on whether we want to claim success for mechanism that is successful with a high probability (extremely high for the main uses cases) but that doesn’t guarantee success.

I’m shy to use a verb that over-promises, because:

• There are going to be failures but very rarely for the main use case of applying the feature to non-Latin-script content. (I still believe that this change will be a net usability improvement; users aren’t really that good at choosing right on the first try from the long manual list.)
• Overpromising invites the question “If you know how to correct it, why don’t you just do it by default?” (But see previous point.)
• Experience indicates some users who use this feature believe that they know encoding stuff better than Firefox does and are upset if Firefox takes away their options when Firefox knows that none of the options that were available in an older version would work. However, in this case (see first point), there are going to be (rare) cases where this patch takes away options and the user actually could know better than Firefox. I don’t want to flame users in that case.

Flags: needinfo?(hsivonen) → needinfo?(mwalkington)

Flags: needinfo?(mwalkington) → needinfo?(hsivonen)

(In reply to Meridel [:meridel] from comment #17)

It sounds like “Override text encoding” is technically accurate and safe. It would nice if the label could communicate the benefit of the feature without over-promising. Did you consider something like: “Troubleshoot text encoding”?

I hadn’t considered “Troubleshoot”. Now that I consider it, my first thought is that the UI items that I’ve seen before that have said “Troubleshoot” are more ceremonious than this one: They either launch a separate more or run something automated that takes for long enough to have a progress bar. In this case, the menu item / toolbar button just does its thing instantaneously without any further UI.

If “Troubleshoot” sets the expectation of some further UI appearing and then no further UI appears, might that be confusing?

Flags: needinfo?(hsivonen) → needinfo?(mwalkington)

I am less concerned about “Troubleshooting” setting a false expectations of a separate or elongated process, and more that users would expect something called ‘troubleshoot’ to diagnose a problem. Would it be accurate to say this feature diagnoses a problem? By correcting the text encoding do they then understand the source of the issue?

Documenting label options:

1. Override text encoding: Technically accurate but does not communicate the benefit of the feature
2. Troubleshoot text encoding: Gets at the benefit of the feature but could set expectations of diagnosis?
3. Fix text encoding: Communicates benefit but could over-promise. Our goal in product copy isn’t to be 100% technically accurate, though, so this label may be the best option for most contexts.
4. Fix garbled text: Communicates benefit but could over-promise. Plainer language than “text encoding” but is it accurate enough?
5. Fix default text encoding: More specific than option 3 and 4, but longer

FYI, if a request like this is time-sensitive and I become a blocker for shipping, please don’t hesitate to reach out to me on Slack (@Meridel). Betsy and I make up the content design team and are spread thin across projects so these kind of string requests can get lost in the daily shuffle!

Flags: needinfo?(mwalkington) → needinfo?(hsivonen)

(In reply to Meridel [:meridel] from comment #19)

I am less concerned about “Troubleshooting” setting a false expectations of a separate or elongated process, and more that users would expect something called ‘troubleshoot’ to diagnose a problem.

To get impressions from more people on various options, I asked about this in the DOM Core team meeting. The result was that “Troubleshoot” suggested a bigger thing than other options.

Would it be accurate to say this feature diagnoses a problem?

I guess it’s accurate enough that the feature makes a partial diagnosis as part of its internal operation, but what’s exposed to the user is the page became readable (likely) or the page remained unreadable (unlikely).

By correcting the text encoding do they then understand the source of the issue?

I expect users who use this feature to be aware of the conceptual source of the problem before invoking the UI even if they couldn’t identify the exact pair of wrong and right encoding in a particular case.

Documenting label options:

1. Override text encoding: Technically accurate but does not communicate the benefit of the feature
2. Troubleshoot text encoding: Gets at the benefit of the feature but could set expectations of diagnosis?

“Troubleshoot” also could set expectations of a bigger operation than what’s going to happen.

3. Fix text encoding: Communicates benefit but could over-promise. Our goal in product copy isn’t to be 100% technically accurate, though, so this label may be the best option for most contexts.

Apart from potentially over-promising, this raises the question “If you know how to fix it, why didn’t you just do it already?” (This concern was raised in the DOM Core meeting.)

4. Fix garbled text: Communicates benefit but could over-promise. Plainer language than “text encoding” but is it accurate enough?

I’d like to keep “Text Encoding” in the string in order to keep the term that people who’ve previously used the submenu have already seen and to enable the same accelerator key letter to be found in the string.

The technically accurate term would be “Character Encoding”, and we used to use that. However, we previously changed it from “Character Encoding” to “Text Encoding” in order to have both the more colloquial “Text” and the technical “Encoding” there and to align with Apple’s terminology. Safari’s corresponding submenu is labeled “Text Encoding”.

5. Fix default text encoding: More specific than option 3 and 4, but longer

I think “default” suggests technically a somewhat wrong thing considering how there default is understood to be a now-obsolete browser setting and not “whatever was initially determined for this page this time”.

However, if we’re OK with a longer string, others suggested “Force Automatic Text Encoding” and observed that text editors have a “Reopen with Encoding” feature. In the browser context and with the automation here, the closest to the latter would be “Reload with Automatic Text Encoding”.

It’s unclear if the nuance contemplated here reaches the users of this feature, since if we assume that users in the regions where the feature is used the most use localized versions of Firefox, users will see the Japanese, Traditional Chinese, or Simplified Chinese translation (and, to lesser extent, other non-Latin-script translations).

Flags: needinfo?(hsivonen) → needinfo?(mwalkington)

I expect users who use this feature to be aware of the conceptual source of the problem before invoking the UI even if they couldn’t identify the exact pair of wrong and right encoding in a particular case.

That is, for users of the feature, I expect the exact verb not to be the critical factor and I expect them to be looking for the “Text Encoding” part.

Thanks, Henri. I appreciate you asking the DOM team and sharing context. Using “automatic” is a bit confusing because I assume what the user sees before trying to fix the issue is ‘automatically’ there.

New and remaining options:

1. Force text encoding (leave it at this?)
2. Manual text encoding (implies that whatever showed up by default or automatically is not correct and a one-off, manual fix is needed)
3. Repair text encoding (this is pretty pedantic but “repair” might not promise as much as “fix” since “repair” is the act of repairing something — the outcome is not promised — while “fix” is the action itself)
4. Unscramble text encoding (in articles on this topic I am seeing people use the word ‘unscramble’ to describe fixing text encoding)
5. Override text encoding

Flags: needinfo?(mwalkington) → needinfo?(hsivonen)

(In reply to Meridel [:meridel] from comment #22)

1. Force text encoding (leave it at this?)

I think this is less informative than Override. This one also doesn’t explain why and this seems less suggestive of changing it than Override.

2. Manual text encoding (implies that whatever showed up by default or automatically is not correct and a one-off, manual fix is needed)

I think this is somewhat weird in the sense that what this patch is doing is taking away the possibility of manually choosing which one and leaving only the option to trigger the automated guessing.

3. Repair text encoding (this is pretty pedantic but “repair” might not promise as much as “fix” since “repair” is the act of repairing something — the outcome is not promised — while “fix” is the action itself)

I agree that in English this seems better than Fix. The nuance between Repair and Fix might be lost in some translations.

4. Unscramble text encoding (in articles on this topic I am seeing people use the word ‘unscramble’ to describe fixing text encoding)

I think this could work, although what’s being unscrambled is, pedantically, the text and not the text encoding.

5. Override text encoding

How about I change the menu item and the button label to “Repair Text Encoding”, keep the button tooltip as “Guess text encoding from page content”, and land?

Flags: needinfo?(hsivonen) → needinfo?(mwalkington)

Yes, I am happy with that solution, with a couple minor tweaks:

Label: Repair text encoding (can we make this sentence case? Or are you treating this as a proper feature name? For Proton/MR1, we are moving to sentence case in all core UI)

Tooltlip: Guess correct text encoding from page content (added “correct” to be more specific and based on your comment above that ‘What the user wants to do is to override the text encoding that they visually believe to be wrong with the correct one. ‘

If you are good with these tweaks, we should then get Flod review.

Flags: needinfo?(mwalkington) → needi