Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set goals for editing investigation #1

Closed
jgraham opened this issue Mar 1, 2022 · 40 comments
Closed

Set goals for editing investigation #1

jgraham opened this issue Mar 1, 2022 · 40 comments

Comments

@jgraham
Copy link
Contributor

jgraham commented Mar 1, 2022

We need a set of goals that will represent substantial progress toward enabling interoperable editing on the web.

As I understand it, the current situation is:

So a good overall goal here might be to have a genuine shared understanding of how editing on the web should work, and have that reflected in the test suite. One possible route toward that would be to audit the existing test failures and figure out which existing behaviour we want to / are able to follow. A testsuite that people believe in seems like a prerequisite to future implementation work that actually aligns the behaviour.

@jgraham
Copy link
Contributor Author

jgraham commented Mar 29, 2022

Proposed participants, to be updated once people confirm or we get different volunteers:

Gecko

@jgraham
@masayuki-nakano

Blink

@BoCupp-Microsoft
@dlibby- ?

WebKit

@megangardner
@whsieh

Others

?

Once we have a list of participants, I propose scheduling a meeting to get things moving and set some goals.

@jensimmons
Copy link

Wenson Hsieh @whsieh and Megan Gardner @megangardner for Apple.

@jgraham
Copy link
Contributor Author

jgraham commented Apr 22, 2022

I'd really like to schedule an initial meeting for this in the next week or two, so we can set some clear goals/outcomes for this effort. Unfortunately I don't have everyone's email addresses, so it's hard to send direct email. Let's see who's reading their GH notifications, and if that fails I'll try to track down more email addresses :)

Looking at the people here, it seems we have a pretty global mix of people, but I might be the only European, and everyone else might be on the West Coast of NA or Japan? In that case I'd suggest we should probably aim for a meeting time that works for Asia/USA, and not for Europe. I think getting all three at the same time is ~impossible.

So I suggest we aim for something in the range 16:00-18:00 PST / 08:00-10:00 JST? Could people reply with days in the next two weeks that would work for them in that time range and we can see where we have some overlap? Alternatively we could discuss this semi-synchronously on matrix at https://matrix.to/#/#Interop-2022-Editing:matrix.org

@BoCupp-Microsoft
Copy link

I'm reading my GitHub notifications. I'm just not good at responding to them. ;-)

We discussed this issue briefly in the context of this issue raised by @whsieh at our last Web Editing WG meeting.

I'm happy to meet or discuss in issues. I know @johanneswilm would probably also like to attend and is CET. I'm Pacific Time. At the time you suggested I think I can make any day work. My work email is pcupp@microsoft.com.

@BoCupp-Microsoft
Copy link

@foolip I see there is an editing agenda item for May 12th (today). Could you share details on the meeting time / place?

@foolip
Copy link
Member

foolip commented May 12, 2022

@BoCupp-Microsoft web-platform-tests/interop#77 is the general Interop 2022 meeting. @jgraham has sent out an email suggesting a time for the editing investigation meeting.

@marcoscaceres
Copy link

@jgraham wrote:

So a good overall goal here might be to have a genuine shared understanding of how editing on the web should work, and have that reflected in the test suite.

I think we all want a spec too. My preference would be that we (re)specify execCommand and contentEditable from scratch (maybe directly in the HTML spec?). The existing docs have been effectively unmaintained for years, are not well specified, and have only received a few tweaks @saschanaz in the last year. And, as @jgraham eluded to, the spec prose doesn't meet the requirement that there be at least two interoperable engines actually following the spec.

(They are also not even in scope of the charter for the Web Editing WG... so they don't even have a real home or a venue to be published!)

One possible route toward that would be to audit the existing test failures and figure out which existing behaviour we want to / are able to follow. A testsuite that people believe in seems like a prerequisite to future implementation work that actually aligns the behaviour.

As above... it would be good to do both a "modern" spec and tests simultaneously. Or, better yet, we fix and land fixes across engines as we go. I'm sympathethic that it's going to be challenging because of all the libraries depending on quirky behavior, but I guess we will need to deal with that too... cost of getting interop, I guess.

If this is high priority across all user agents, and is (or appears to be) the most lagging part of the 2022 compat effort, then it might be good to gather some folks who know how this part of the platform works to get a sense of what we need to specify and test.

I've personally never looked at how this area of the platform works, but it feels like there is sufficient developer pain (and implementer frustration with low quality specs/tests) here that we (collectively) might want to throw some editorial resources at this. I'm not volunteering at this point - but I'd personally like to get a sense of how much work there is and how this works.

@johanneswilm
Copy link

johanneswilm commented May 13, 2022

It seems like we are going a bit in circles on execCommand. If the browser engines are all willing to put the resources in over time to create a version of execCommand that is working the exact same everywhere and can be extended in JavaScript so that it can be used by JavaScript projects to build their custom editors, then I would be all for it. But this discussion has been going on for the 8 years I have been around and it sounds like browser makers are not willing to put in the amount of work this would require. Focus has instead been on eliminating the last few remaining usecases of execCommand - for example to turn off table controls in Firefox. Other technologies, such as Input Events and EditContext, can be used as more flexible replacements for creating text editors.

It was discussed at the April and May calls of the Editing Working Group. I proposed in April to possibly work on standardizing caret movement in contenteditable, but there was little interest in that among browser makers. There was also no interest working on execCommand. Instead, the priorities that were found were EditContext, Clipboard APIs and Input Events.

@masayuki-nakano
Copy link

In my understanding, execCommand is an API to do same things if user can do the operation (except whether beforeinput event is dispatched or not). So standardizing:

  • forwarddelete (corresponding to Delete)
  • delete (corresponding to Backspace)
  • insertLineBreak (corresponding to Shift+Enter)
  • insertParagraph (corresponding to Enter)
  • insertText (corresponding to type some text)

Would make the users' operation result consistent between browsers. Especially, the behavior around/with white-spaces, in preformatted content, and immediately before a block boundary causes web-compat issues in the wild a lot. So I'd like those behaviors to be at least covered by WPT.

Additionally, and if it's possible, the following commands should be standardized:

  • insertHTML (may be corresponding to pasting same fragment)
  • bold (Gecko does not support Ctrl (Command) + B yet, but can add it soon)
  • italic (Gecko does not support Ctrl (Command) + I yet, but can add it soon)

@johanneswilm
Copy link

johanneswilm commented May 16, 2022

From the point of JavaScript editors, the points @masayuki-nakano mentions make sense to have standardized even when the execcommand itself is not used.

@johanneswilm
Copy link

johanneswilm commented May 16, 2022

Note though also @masayuki-nakano that there are is an almost infinite amount of circumstances and combinations of those commands. For example, enter could be pressed while in something like this:

<div>Some text<span class="outer">12<span class="inner">34<b>bo|ld</b>56</span>78</span></div>
| = caret

or the key combination for bold could be pressed while the selection looks something like this:

<p>Some [text<i>italic<b>bold</b></i></p>
<p>Second ]paragraph</p>
[] = selection

And you'd have to describe in detail every one of those cases if you really wanted all browsers to react the same to each command. If you are just covering a few basic cases or just make sure that the browsers support the same key combinations, then that will be easier to do, but it will also mean that JS editors will need to continue having their own logic for these commands.

Having Firefox support CTRL+b and CTRL+i would be useful by itself.

@BoCupp-Microsoft
Copy link

@masayuki-nakano, to make sure I'm understanding your suggestion, are you saying that we just standardize, for example, that execCommand("forwarddelete") does the same thing as the user pressing delete, but that we wouldn't specify the transformation of the markup?

@jgraham
Copy link
Contributor Author

jgraham commented May 16, 2022

I doubt that kind of high-level specification would get meaningful interop wins (although @masayuki-nakano may of course have a different opnion!). If you look at https://bugzilla.mozilla.org/buglist.cgi?quicksearch=execCommand quite a lot of the bugs seem to be from authors expecting quite specific behaviour.

For contentEditable, the situation is similar, see the "depends on" list in https://bugzilla.mozilla.org/show_bug.cgi?id=1341152 for the kinds of bugs that authors are running into in the wild.

@masayuki-nakano
Copy link

@masayuki-nakano, to make sure I'm understanding your suggestion, are you saying that we just standardize, for example, that execCommand("forwarddelete") does the same thing as the user pressing delete, but that we wouldn't specify the transformation of the markup?

I hope that the behavior should be standardized as far as possible, but I understand it's an ideal goal and it may be impossible. However, I think that standardizing even only some simple cases may improve the compatibility in usual web apps.

I doubt that kind of high-level specification would get meaningful interop wins (although @masayuki-nakano may of course have a different opnion!).

Sure, but I hope it should be a goal of the long term.

If you look at https://bugzilla.mozilla.org/buglist.cgi?quicksearch=execCommand quite a lot of the bugs seem to be from authors expecting quite specific behaviour.

Yes, but most of them are not related to the basic commands which I listed above. My proposal is what it'd be really nicer if the basic behavior of editor (not only via execCommand) were standardized.

One of the difficult web-compat issues which Gecko needs to change its behavior is, the other browser engines behave odd but web apps in the wild depend on the behavior. So, standardizing the basic behavior would help for solving such issue. Anyway, I don't persist this. I just wanted you to see my idea.

@marcoscaceres
Copy link

marcoscaceres commented May 17, 2022

@masayuki-nakano wrote:

However, I think that standardizing even only some simple cases may improve the compatibility in usual web apps.

Worth a shot. On MDN, there is already a huge red warning that this area of the platform is deprecated. So, if we can at least standardize a minimal set of behaviors it may indeed help.

What might also help is documenting where we don't actual expect to get any interop and explaining why (platform differences!) and suggesting alternatives.... which, if I understand correctly, is basically what @masayuki-nakano is proposing here👇

My proposal is what it'd be really nicer if the basic behavior of editor (not only via execCommand) were standardized.

Agree. At least, we should direct people to the newer APIs or provide notes to the alternative/modern ways of doing things.

One of the difficult web-compat issues which Gecko needs to change its behavior is, the other browser engines behave odd but web apps in the wild depend on the behavior. So, standardizing the basic behavior would help for solving such issue. Anyway, I don't persist this. I just wanted you to see my idea.

Again, I think this is worth a shot. The contention here is: who's behavior should we follow? And how much willingness is there on each browser to change (within reason, for minimal viable interop). And we take list at #1 (comment) as a basis.

@jgraham, @foolip, others, thoughts?

@johanneswilm
Copy link

johanneswilm commented May 17, 2022

we should direct people to the newer APIs or provide notes to the alternative/modern ways of doing things.

Yes, is it possible to get the note updated to point to Input Events? Once EditContext has shipped in two browsers, it would be good to also link to it. We put a link to Input Events in the execcommand document [1] previously. Something similar on MDN would be a good idea.

For the majority of web developers, it is best to use a JavaScript library to create an editor rather than try create one oneself. And that is not always obvious. There are many out there who have lost a year to execcommand, only to find out one needs to use a library in the end. Without promoting any single one of the existing open source JavaScript libraries, would it be possible to put a warning on there saying that using a library for editing will make most sense in many circumstances?

who's behavior should we follow?

Possibly not for all editing behavior, but in the case of shortcuts for bold and italic, it is Firefox that is the odd one out and because it lacks those, when creating an editor, it is not enough to listen for input events as one needs to also listen to key events as well to catch ctrl+i/ctrl+b.

CC: @Comandeer

[1] https://w3c.github.io/editing/docs/execCommand/

@jgraham
Copy link
Contributor Author

jgraham commented May 17, 2022

As a general note about EditContext and other things that aren't yet widely supported: hoping that a new technology displaces an existing one isn't a good compatibility strategy until we're at the point where implementations are looking to unship the existing feature. For ContentEditable and execCommand we're still a long way from that point. Historically there aren't too many examples of web technologies being totally displaced to the point they could be unshipped; AppCache is the main example that comes to mind, but that was relatively safe to drop since doing so was unlikely to actually break sites, but might make them unusable offline.

I totally agree that editing is mostly done by libraries, but even with prominent libraries interop problems are leading to site compatibility issues. For example draft-js has had a number of problems in gecko that have impacted very popular sites (e.g. https://bugzilla.mozilla.org/show_bug.cgi?id=1739489). So "just use libraries that work around all the problems" isn't a good enough answer. This is, if you like, just the priority of contituencies in action: although library authors are only a small subset of web authors, there are still more libraries than there are rendering engines so we should favour solutions that improve interop at the engine level for library authors to depend on rather than solutions which require libraries to continue to implement browser-specific codepaths.

who's behavior should we follow?

It seems pretty clear that for anything where Blink and WebKit agree we should align on that behaviour. Where WebKit and Blink differ it's not so clearcut, but certainly aligning based on compatibility with existing expectations makes sense. That would be much easier to do in general if we had a good description of what the model leading to that bewhaviour is, and tests covering it. Getting those things seems like it should be part of the goal here.

@jgraham
Copy link
Contributor Author

jgraham commented May 17, 2022

I've scheduled a meeting for 15:00 UTC on Thursday 19th. If you didn't get an invite and would like one please let me know. If that time is going to be a problem for a lot of people we could push it into next week at a more agreeable time.

@johanneswilm
Copy link

As a general note about EditContext and other things that aren't yet widely supported: hoping that a new technology displaces an existing one isn't a good compatibility strategy until we're at the point where implementations are looking to unship the existing feature. For ContentEditable and execCommand we're still a long way from that point.

Input Events are being shipped everywhere and have been shipped for several years in Webkit and Blink.

As for replacement of execcommand: We have not used the term "deprecated" nor said that it is being "unshipped" in the execcommand document itself. Using those terms has been the decision of someone at MDN.

For those who may not be aware of this: We have been running the editing task force since 2014. It has been the view of the vast majority of participants, both from the browser makers and JavaScript editor creators since the start that execCommand is not a viable option for creating editors, nor can it ever be even if all the bugs in it are fixed unless it is made to be extendible by JavaScript so that editors can have their various desired custom behaviors.

Nevertheless, we are all aware that there are some groups out there who have written usable software programs using execCommand in some way or other, and there is a knowledgeable minority that has a different view who would prefer for us to forget everything developed after 2014 and instead go back to try to fix execCommand. I think we should all acknowledge that it is always possible that some time in the future this becomes the majority view and for that reason we are not using the word "deprecated". However, as fact as I am aware, that is not the case, and the point of the editing taskforce all these years has been to create a viable alternative that does not have the shortcomings of execCommand.

I think if the call is to have another discussion on whether or not to go back and try to fix execCommand instead of working on the alternatives Input Events and EditContext, we probably need to plan the meeting enough time in advance so that all participants of what is now the Editing Working Group have the opportunity to participate.

@johanneswilm
Copy link

So "just use libraries that work around all the problems" isn't a good enough answer.

Part of the point of Input Events and EditContext is that they allow to get around a lot of the issues and simplify the code of these libraries. We have worked together with many of the editor libraries to make sure these specs really answer their needs. However, it's still not simple enough to someone to just put it on their personal website and create an editor in half a day. That is the impression developers could get of execCommand as it looks like it is so easy to do initially - you create two buttons, add event listeners and in 15-20 lines of code you have a simple richtext editor. The problem is then that it breaks under such and such circumstances, you need to do a lot of your own programming to "make bold" in some custom way, etc. . I myself spent more than a year trying to create a JS editor using execCommand and pur contenteditable and I've heard the story from quite a few others: It would be really nice if someone had warned me at the beginning that this is not as easy as it might seem at first.

although library authors are only a small subset of web authors, there are still more libraries than there are rendering engines so we should favour solutions that improve interop at the engine level for library authors to depend on rather than solutions which require libraries to continue to implement browser-specific codepaths.

I agree. The question though is what we need to do similarly. If editors cancel input events and/or use diffing mechanisms and their own model of the document, then changes to how the markup is being changed will not make much of a difference, whereas some other things will (such as supporting the same keyboard shortcuts and similar caret movement).

@Comandeer
Copy link

Comandeer commented May 17, 2022

I agree with @johanneswilm – trying to fix execCommand would require a lot of work and its current state makes it nearly unusable or worse than modern alternatives in most cases:

  • cut and copy commands – they work pretty well  (I have a pleasure working with it also in IE – it's the only environment where they don't work well…) but it's better to use Async Clipboard API,
  • paste command – the current model of deciding if paste should be allowed is quite cumbersome and due to that it does not work more often than it works; Async Clipboard API is better (and it's async!),
  • undo and redo commands – the fact that the undo stack is global in most browsers makes these commands de facto useless (as pressing undo in one editor could undo the content in another one); unfortunately, there is no native, modern alternative so every RTE needs to reinvent the wheel here,
  • styling commands – the variety of possible output is really wide: b, strong, span[style="font-weight: bold"]… 

Everything possible in execCommand is possible in some other, nearly always a better way. I agree that "just use libraries" should not be the answer but I'd risk saying that until there aren't really good low-level editing APIs it is the answer.

At the same time, I agree that execCommand should not be seen as deprecated. It's rather frozen as is – without the guarantee that it would work the same way in all browsers. I saw some tutorials in which authors expressed their confusion about it being deprecated according to MDN but working "well" everywhere.

Personally, I (or rather – we at CKSource) would like to see better interop in other areas:

  • IME – the bane of (nearly) every RTE's existence. Input Events help here but Edit Context API looks even more promising so it will be really nice if all browsers support it.
  • sensible selection handling inside Shadow DOM – that's probably number 1 on my list. Shadow DOM allows real style isolation – something that requires an iframe now. There were several attempts to this (e.g. shadowRoot#getSelection() in Chrome which was used back in 2018 in my experimental implementation in CKEditor 4), there is even a proposal but not much happened for nearly 6 years.
  • feature parity between old (paste event with DataTransfer) and new (navigator.clipboard#read()) Clipboard APIs  – it is important for RTEs to be able to get some more "exotic" MIME types from the clipboard as e.g. LibreOffice or Word put some of the data only inside the RTF format (especially images). This format is not available inside the new API. The same goes for e.g. pasting files copied from the filesystem. It's possible via the paste event and impossible via navigator.clipboard#read(). Due to that, the new API can't replace the old one. Additionally, there are some differences between browsers in how they implement this new API (e.g. around sanitization).
  • handling images from LO or Word – this part of the pain-point above has also another possible solution: reuse Safari's attempt in other browsers.
  • some other areas of selection – e.g. behavior around [contenteditable=false] elements inside editable elements, allowing/disallowing multi-range selection, etc.

@jgraham
Copy link
Contributor Author

jgraham commented May 17, 2022

So taking a step back here: the goal of the Interop 2022 effort in general, and this item in particular, is to improve the user and developer experience of the web by reducing unexpected differences between engines.

Editing is part of the overall effort because it's an area where we see a lot of user frustration from sites, including big, well resourced sites, not working as expected. But when looking at how to include editing it wasn't even clear what would need to be done to set a metric for improving the cross-browser compatibility / user experience. So that combination of importance and uncertainty is why we've got an "investigate" item in Interop 2022 for editing. The hope is that in 2023 we'd be able to set some concrete goals for implementors in terms of testcases they need to pass to improve the developer and user experience for web-based editing.

I personally don't really have a horse in this race; I understand that a lot of the current editing primitives are too complex and hard to use. But I also have some experience of working on web standards and it's relatively common to hear "we don't need to bother fixing X because it will soon be displaced by Y". Typically that displacement either never happens at all or if it does it's over a period of decades. A slightly relevant example: you never hear about jQuery anymore, and the development is basically in maintainance mode, but it's apparently still used on over 70% of "top" websites. So I'm quite skeptical of the idea that it will be possible to fix the current user/developer pain points by rapidly migrating the editing ecosystem over to a new set of technologies.

That said it's not impossible that this is a special case and that the outcome of the "investigate" could be e.g. a set of tests for the new APIs that, if they were all to pass in major browsers, would give library developers the confidence to abandon contentEditable and execCommand APIs entirely. I think there would need to be some pretty substantial supporting evidence that would actually happen and not be held up by e.g. supporting older browsers.

But at the very least I think the view that execCommand is working well enough with libraries and therefore it's OK to just abandon it in the "every browser is different" state isn't supported by the kinds of compat bug reports we get. @masayuki-nakano already identified some of the areas that seem to be low hanging fruit, and I'd hope that we can turn that into a set of achieveable improvements, backed with agreed-upon tests, which would allow us to better align browsers without waiting for every editing library to be rewritten.

@johanneswilm
Copy link

But I also have some experience of working on web standards and it's relatively common to hear "we don't need to bother fixing X because it will soon be displaced by Y". Typically that displacement either never happens at all or if it does it's over a period of decades.

Absolutely. But in this case we have at least one spec that is implemented and shipped everywhere (Input Events) that is also seen as having a future by all relevant parties, but where there are differences in those implementations (mainly concerning event order). Also, we have had the question of what to do in terms of interop on the agenda of two of our monthly Editing Working Group meetings and the conclusion from those were to work in the areas mentioned above and specifically not execCommand.

Firefox is the one browser that is not always represented in these calls and I think there are some things we likely did not consider sufficiently because of that, such as the lack of keyboard shortcuts for italic and bold in Firefox.

But at the very least I think the view that execCommand is working well enough with libraries.

The libraries, one of which @Comandeer is working for, are not heavy users of execCommand. Some examples:

Draft.js: In the case of draft.js that was referred to above, in the entire source I find one instance [1] and that is only used in case of Internet Explorer where it uses the IE-specific AutoUrlDetect command.

ProseMirror: I can find two instances of execCommand [2]: enableObjectResizing and enableInlineTableEditing. These were Firefox specific commands prior to Firefox 64 that one needed to run once in order to make Firefox behave more like other browsers.

CKEditor: Somewhat more difficult to find as they are using the term "execCommand" for an internal command, but no obvious uses I could make out [3].

So it is not that we say that execCommand is used but broken and can be fixed by using some libraries that use execCommand in the background - it is that these libraries have largely stopped using execCommand and therefore it does not help them much either.

[1] https://github.com/facebook/draft-js/search?q=execCommand

[2] https://github.com/search?q=org%3AProseMirror+execCommand&type=code

[3] https://github.com/search?q=org%3Acksource+execCommand&type=code

@Comandeer
Copy link

In fact, CKEditor has preferred not to use native execComannd for many years (if to believe our GH repo's history – at least 13 years now). Even back then execCommand was quite messy and not really interoperable.

no obvious uses I could make out

The one use case I can think of are cut & copy commands in CKEditor 4 used for the cut/copy toolbar buttons – mainly due to the fact that it was created way before the new async Clipboard API. But the code for these commands also shows well what is wrong with execCommand – there are two flows for IE and other browsers, some hacky way to detect if the cut/copy succeeded and some fix for IE because selection becomes broken after cutting content. In CKEditor 5 we decided not to include toolbar buttons for cut and copy (one of the reasons was the fact that cut and copy buttons without the paste button look weird; the paste one was omitted as native pasting back then did not work well outside IE) and there is no need to use native execCommand anymore.

@marcoscaceres
Copy link

@jgraham wrote:

I've scheduled a meeting for 15:00 UTC on Thursday 19th. If you didn't get an invite and would like one please let me know. If that time is going to be a problem for a lot of people we could push it into next week at a more agreeable time.

It's 1am for me, unfortunately. I'm based in Melbourne, Australia. However, if you have critical mass already for tomorrow's meeting, I'm happy to read the minutes.

@johanneswilm wrote:

I think if the call is to have another discussion on whether or not to go back and try to fix execCommand instead of working on the alternatives Input Events and EditContext, we probably need to plan the meeting enough time in advance so that all participants of what is now the Editing Working Group have the opportunity to participate.

It would be ideal if the Web Editing WG continues to focus on its chartered deliverables. As you said, and as @Comandeer pointed out, coming up with the APIs to eventually supplant execCommand and contentEditable, such as Edit Context API, is equally as important - so I kindly ask that you please continue to focus the Editing WG energy on that.

As @jgraham stated, folks here can undertake the investigation and come up with an interop plan in the meantime.

@johanneswilm
Copy link

Hey @marcoscaceres ,
execCommand is not a spec that is going to be delivered by the Editing WG, for the various reasons given above, but both contentEditable and execCommand fall within the scope and under the mission of the Editing Working Group.
Over the years, while we were still the Editing Taskforce, we have discussed a number of questions related to execCommand in cases where a developer from one of the browsers was about to change the behavior to make the browsers behave more alike.
And to be clear - contentEditable without execCommand is still being used by a lot of JavaScript-based editors. So it is clearly an area where standardization makes sense. The question though is of which part because a lot of the dom manipulation is being overriden by the editors' internal logic and it doesn't sound like the browsers will want to spend the time to really specify all the various dom changes that are happen in each possible situation to the extend that JavaScript editors can start removing some or all of their own code. But there are other areas where standardization of contentEditable will have a big impact immediately: For example, I mentioned selection/caret movement in the Editing Working Group March meeting and @Comandeer mentioned more specifically selection around contenteditable=false and inside Shadow DOM above.

@masayuki-nakano
Copy link

My point is, some commands of execCommand may be handled in same paths to handle user input in the browser engines. If this is true (I believe so), standardizing such commands' behavior means that standardizing basic editing behavior of contenteditable too. Then, it makes web apps handling contenteditable (including the libraries which are maintained) stabler.

And the reason of I'm too busy in several years is, I'm working on making Gecko align the basic editing behavior to Blink's behavior or WebKit's if an incompatible behavior is reported and it causes inconvenience of Firefox users. In most cases, the issues can be tested with adding new tests under editing/data to check and fix the compatibility between browsers. As you know, the WPT path uses execCommand to check the browsers' behavior. So, incompatibility issues of execCommand (only with the basic commands) are not the past issues, they are still alive. And currently, WPT is the only place to post a reasonable behavior. I think that this is not a good thing because I think that nobody checks the WPT result unless existing ones fail when changing their behavior. I.e., WPT does not give a motivation to fix existing bugs of browser developers. That's a really bad thing.

On the other hand, I agree with the standardizing all paths even of the basic editing behavior is impossible due to too many situations. So, using and reorganizing WPT may be reasonable to standardize the behavior indirectly. If this idea would be taken, there should be a protocol to discuss about reasonable test result for the tests which do not pass in some browser engines.

@johanneswilm
Copy link

Thanks for the clarification @masayuki-nakano .
I noticed you wrote lots of WPT tests that use execCommand to check certain behavior. I take it that means that it is guaranteed that executing an execCommand will work the exact same as if the user had made the input by means of keyboard/mouse, etc. ? Is that also the case in Blink, @BoCupp-Microsoft ? If yes, then I can see how it can make sense for testing user input.

It is also possible to write a lot of tests without execCommand by using Selenium's built-in methods like send_keys. See for example this input events test [1]. There are some problems with that as one will test user input, which really the input events spec does not specify. So, for example, a browser in the Spanish speaking part of the world may decide to use CTRL+N instead of CTRL+B for bold and still be spec compliant, but the test will fail because it just sends the CTRL+B key combination.

On the other hand, it seems like all the existing browsers actually agree on a lot of the user inputs that will cause certain input events. The fact that Firefox doesn't support CTRL+B/I is not that one has decided against it, but as I understand it, it was rather that it was an oversight and we would probably have caught it earlier had we had a test that used send_keys rather than just execCommand.

[1] https://github.com/web-platform-tests/wpt/blob/master/input-events/input-events-typing.html

@jgraham
Copy link
Contributor Author

jgraham commented May 18, 2022

In terms of testing I agree that we should prefer using testdriver (which is how WebDriver/Selenium is exposed to web-platform-tests) with injected key input events for basic text editing and cursor movement. However there's no guarantee in the WebDriver spec that inputting application-level keyboard commands like ctrl+b work, and indeed it's considered something of an antipattern to support that because tests which rely on it tend to be quite fragile.

If we think that a problem is that we're missing testing primitives then documenting those and working out how to get them added one be one possible goal for the investigation. I'm probably the right person to help you with that ;)

More generally it sounds like we're starting to make progress on indetifying some goals. As something to argue against, how does the following sound:

  • Identify the existing tests in editing/data (or elsewhere0 that correspond to problematic compat / interop issues with contenteditable and the basic edting commands.
  • Update those tests to avoid execCommand where possible.
  • Look for areas where we have known interop / compat issues around the basic editing behaviour in contenteditable but don't have test coverage. Add tests for those areas or, where we are blocked on testing infrastructure, specify new testing APIs that will allow those areas to get proper test coverage.
  • Examine those tests in detail and agree on the desired behaviour in each case. Document that agreement in some way (e.g. as changes to the existing spec document or in some other form).

Then at the end of the year we'd have a set of tests that everyone could agree were testing the desired behaviour and where fixing them would improve the lives of library developers and end users. That could form the basis of a Interop 2023 proposal which would measure browsers against that set of tests.

One obvious thing that's missing from the above is editing-related events. These have come up several times as a source of issues. Should we also be looking to identify existing tests and improve coverage in those areas?

@masayuki-nakano
Copy link

@jgraham Sounds really reasonable except there are too many tests under editing/data 😥

@marcoscaceres
Copy link

@jgraham, that sounds like a great plan. I'm tentatively putting my hand up to help with spec editing when the time is appropriate. In the meantime, I'll try to educate myself a bit more in this area. I can help with tests also if need be, or anything else. Just let me know.

@johanneswilm
Copy link

johanneswilm commented May 19, 2022

One obvious thing that's missing from the above is editing-related events. These have come up several times as a source of issues. Should we also be looking to identify existing tests and improve coverage in those areas?

Yes, specifically, the order of IME and editing related input events should be tested. However, AFAIK there are some limitations as to what can be tested currently [1].

Look for areas where we have known interop / compat issues around the basic editing behaviour in contenteditable but don't have test coverage.

You are referring to DOM changes that follow from user input? It is difficult to limit to what is "basic". If one takes the list of execCommands listed by @masayuki-nakano as a starting point, there is still an almost unlimited amount of combinations of tags/situations. So maybe one needs to limit the tags down to a similarly short list (for example <p>, <br>, <b>, <i>) to avoid the "pandora's box" effect of this becoming an all-consuming and never-ending task.

How about other basic editing behavior, that is not directly related to manipulation of the DOM, such as formatting shortcuts that are available by default, caret/selection movement, behavior around contenteditable=false, SVGs or IMGs inside of contenteditable containers? Those are items requested by editing libraries and they could have a direct impact on a lot of end users.

[1] w3c/editing#357

@jgraham
Copy link
Contributor Author

jgraham commented May 19, 2022

there are too many tests under editing/data

I think we need to come up with a way to select the tests that we think are most likely to have user impact. On the gecko side maybe we can relate some of the tests to known interop pain points e.g. open bugs under the contenteditable metabug, or tests we've added where we don't want to align with Blink/WebKit because we don't properly understand their behaviour or think it's user hostile.

On the library side, identifying areas where there are lots of bug reports from users, or browser-specific workarounds might give us a list of areas in which we should prioritise writing tests.

Yes, specifically, the order of IME and editing related input events should be tested. However, AFAIK there are some limitations as to what can be tested currently [1].

Ah, yes, @masayuki-nakano had a proposal for this. Sounds like maybe we should set part of the goal for this project to make that usable for writing web-platform-tests by adding it to WebDriver and writing some implementations. I can contribute to that pretty directly!

Those are items requested by editing libraries and they could have a direct impact on a lot of end users/

I think UI-level stuff like "which shortcuts are available by default" is going to be hard to write tests for, but informally it sounds good for us to understand where there are fixable painpoints there. All the other stuff you mentioned seems like it culd be in scope if that's where libraries are struggling; the idea is really to make things better for users and developers after all.

@BoCupp-Microsoft
Copy link

I take it that means that it is guaranteed that executing an execCommand will work the exact same as if the user had made the input by means of keyboard/mouse, etc. ? Is that also the case in Blink, @BoCupp-Microsoft ? If yes, then I can see how it can make sense for testing user input.

From memory, yes, that's how it works in Blink.

@BoCupp-Microsoft
Copy link

@johanneswilm wrote:

execCommand is not a spec that is going to be delivered by the Editing WG, for the various reasons given above, but both contentEditable and execCommand fall within the scope and under the mission of the Editing Working Group.

Agreed that a spec for execCommand is not in our charter, but the topic is within the scope of the Editing WG. If someone undertakes producing a spec I prefer that activity and efforts on consensus building etc. happen as part of the current Editing WG.

@jgraham
Copy link
Contributor Author

jgraham commented May 19, 2022

Thanks to everyone who was able to attend the meeting today. The minutes are below.

We identified four important areas which seem high impact:

  • Selection (specifically focusing on where the caret can be positioned)
  • Input-related events including order.
  • IME events including a mechanism to test IME input.
  • Top web-compat issues in the contenteditable editing model

I'll file an issue in this repo for each identified area and some next steps.

Minutes

May 19, 2022 Interop 2022 Editing Investigation

Attendees: @foolip @jgraham @johanneswilm @masayuki-nakano @BoCupp-Microsoft

James chairing, Philip taking notes. Notes will be posted to GitHub.

Introductions!

Goal of the meeting is to make progress on the investigation effort.

James: Editing is an area where we see a lot of compatibility problems, affecting users as well.

James: Interop 2022 is a public metric for high value areas for developers and users, where browsers aren’t interoperable enough. For each area we’ve selected tests that would show we’ve improved interop. Editing is an area with a lot of problems for Gecko. But here there wasn’t a good spec and a good set of tests. So we created “investigate” as a category, where we figure out what needs to be done to make interop better. One possible outcome is that for Interop 2023 we do have a set of test cases that we agree are the high impact things to work on. 10% of the over Interop 2022 is split between 3 investigation efforts, scored as “did you make the progress you set out to?” So we should set meaningful and achievable goals and get there by the end of the year.

Bo: Focus here is mainly execCommand? There’s also clipboard and input event work in the editing working group. These are the inputs to editing, where execCommand is the default behaviors that many are overriding.

Johannes: When I first got involved I tried to get browsers to fix execCommand to be more similar, and browsers said they’ll never do that because it’s a never ending story. Nevertheless, I spend a lot of time on changing some small behavior in execCommand. If browsers have the resources to standardize this, that would be great. But if not, there’s lower hanging fruit. Some editors use contenteditable and built-in selection, and there are problems there, that would be nice to focus on.

Philip: In https://wpt.fyi/interop-2022 the name of the area is “Editing, contenteditable, and execCommand” so it’s as broad as would be useful. My goal would be whatever most improves things for web developers and users.

James: We’re interested in the outcomes. In Gecko we have a pile of bugs around contenteditable differences more than execCommand I think. Masayuki?

Masayuki: Gecko is the only engine with a different root. Now rewriting behavior that matched IE to instead match WebKit. Sometimes finding that Blink’s behavior is buggy. Sometimes behavior can be tested using execCommand even if it’s not really about execCommand. For example hitting enter might map to a specific command.

Bo: If there’s someone that could drive this forward and suggest a model, the right place to discuss it would be in the Editing WG. We’d be happy to host that discussion and consensus building.

Johannes: There’s been someone hired by Bloomberg who fixed things in Chromium.

Bo: Masayuki, the bugs you’re working on, are they around user input, and goal isn’t to improve execCommand, but execCommand (at least in Blink) is the same code, so it’s a proxy?

Masayuki: Yeah. Gecko is also designed like this. I don’t think we need to rewrite tests using execCommand to use WebDriver.

Johannes: Libraries work around differences in execCommand with DOM diffing, just replacing whatever the result was with what they expected.

James: The reason we care is that in practice we get bug reports about this stuff breaking editors. Probably not as much as there are differences, but still a lot. Maybe we should identify a specific subset of contenteditable or execCommand that is the most problematic. I agree this shouldn’t be the only goal. Other areas selection (including Shadow DOM), input events, and IMEs.

Johannes: Testing of IMEs is still difficult, and thus difficult to fix.

James: Masayuki had a proposal for integrating IME input with WebDriver. Defining that could be part of our work. I know how to make that happen at least :)

Bo: I like that if we’re talking about low hanging fruit! IME test infra is definitely a gap. We had an intern who worked on IME test infra based on some external requests. We updated CDP and Playwright, but not WebDriver itself, but would be interested in collaborating on that, a worthwhile area!

James: We have something for internal testing too. It integrates with the WebDriver Actions API, so a bit different. Should turn that into a public proposal.

Bo: Are there other easy wins? I’m wary about Selection because browsers all have normalized selection internally, a bit hairy because of arbitrarily complex markup. Maybe there’s a subset we can focus on?

Johannes: The main issue I understand is that selection can’t go certain places. There can be bits between non-editable “islands” where the bit in between cannot be selected, so libraries draw their own carets. We once made a list of places that the caret needs to be able to go.

Bo: On positioning of selection, we have a tendency in editors to “hug to the left” to continue the formatting at the cursor. Libraries sometimes put the selection right after a bold tag, hoping to influence what commands do. We do things to avoid dealing with both arbitrary selection and arbitrary markup, to simplify.

Johannes: Some editors chose to have two different positions for inside and outside the bold, same position but a bold or thinner caret. That’s not what browsers would do though. By now libraries have their own model of what’s going to happen that doesn’t depend on this though.

James: It sounds like there are a list of situations that you or editing library maintainers consider problematic. Browser makers should go through that list and turn that into test cases. I guess it’s possible to test?

Bo: There are lots of existing tests written by Aryeh Gregor testing what the impact of a command on markup and selection is. It’s challenging. I’d like to steer us away from “let’s spec everything about editing commands”. Pick just once, like inserting a paragraph, and proposing a model for how to spec and test that. Try that for one case. Scope it as narrowly as possible. Is that workable?

Bo: My proposal would be going after event order. And clipboard, Blink and WebKit don’t even agree on what number of clipboard entries there should be.

Johannes: I know Koji took a list of issues and fixed a bunch of them. Two of the items on that list showed up again now. (Where?) Even with something like “insert paragraph” you can have an unlimited number of things to test with enough markup combinations. Limit to a certain set of elements? Start with something as simple as splitting a paragraph.

James: For DOM structures interaction with contenteditable and execCommand, it seems clear we need to go look for a narrow set of things to focus on that would help the most. We’ll propose a set of things, and later hopefully get agreement on a set of test cases that cover those.

Bo: One thing that’s frustrating in this area is that I don’t feel we’re converging towards a shared behavior, more band-aid fixes where you fix one thing and another one breaks. A model on which we can build would be very helpful. As Johannes said, limit the document model, leave things beyond that undefined. Or agree how we produce innerText and define how to insert a paragraph in innerText. Not just point to some tests and say “make them pass”.

Masayuki: nods

James: Agree we need to scope things.

James: To summarize we want:

  • List of 15 high-impact selection bugs.
  • Clipboard (should it be out of scope, or maybe Interop 2023?)
  • Event order, we want a set of tests backed by a spec
  • Gecko top issues (web compat bugs)

Johannes: https://github.com/w3c/contentEditable/blob/ba10fb177f68567ff742d0acb6caab337224af2d/index.html#L145-L187

Bo: Next step is evidence gathering. Demonstrating what the most impactful areas are.

James: Let’s keep meeting. Once a month in a similar time slot?

Bo: Make it part of the Editing WG? I’d like to at least have the same members showing up.

James: It’s separate in a way and I don’t want to dominate WG time with this stuff, it’s also important for them to make progress on future looking things.

Bo: Maybe just more advertising for this meeting then.
<,/details>

@jgraham
Copy link
Contributor Author

jgraham commented May 19, 2022

I've filed #2 #3 #4 and #5 for the areas we identified. This isn't set in stone, so if you have concerns with the scope please bring them forward.

I've also extended the meeting to be monthly. I know this time doesn't work for everyone. Maybe we should consider also having a second time that works better for people in Asia / Australia?

@johanneswilm
Copy link

We briefly discussed this priority list during today's Editing Taskforce call.

I should add that selection issues are high on the priority list of JavaScript libraries that make use of the native selection. Certainly much higher than the dom changes that happen due to the execution of execCommand commands. So if the point is to work on some of the code that has been along for a long time, then that is to be preferred.

However, the efforts browser makers are putting into newer specs such as Input Events, Clipboard API, EditContext, VirtualKeyboard API are also all of importance as they will over time lead to improvements for JS editor developers, so if browser makers prioritize to put their limited resources into these areas, that would also be seen positively.

The hope from the perspective of JS editor developers seems to be that the effort does not end with prioritizing standardizing dom changes due to user input over everything else.

@gsnedders
Copy link
Member

gsnedders commented Sep 11, 2022

Is there any reason for this to still be open? It sounds like there was, eventually, broad consensus about what to do—am I right in thinking what's in the README is essentially the conclusion of this?

@jgraham
Copy link
Contributor Author

jgraham commented Sep 12, 2022

I think the other open issues are indeed a well agreed plan at this point.

@jgraham jgraham closed this as completed Sep 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants