Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extensibility: How should we best support accessible Custom widgets and interactivity with ARIA? #553

Closed
richschwer opened this issue Aug 23, 2016 · 12 comments

Comments

@richschwer
Copy link

  1. Authors may want to create an entirely new custom control for which there is no role in ARIA that fits. Should we follow the UIA model and define scripted control patterns, accompanied by a localized role name for the widget?
  2. If control patterns are used does that mean that the browser may perform device independent actions on them? This would mean a JavaScript interface.
  3. If such patterns were created with the browser manufacturers agree to expose them through standard interfaces on the platform so that assistive technology could interact with them?
  4. Would there be a preferred other mechanism for extensibility?
@domenic
Copy link
Collaborator

domenic commented Aug 23, 2016

Authors may want to create an entirely new custom control for which there is no role in ARIA that fits. Should we follow the UIA model and define scripted control patterns, accompanied by a localized role name for the widget?

I'm not sure what the UIA model is, but I've been impressed with https://github.com/a11y-api/a11y-api/ as a way of solving this problem.

On the latter question, maybe you could clear something up for me. How does accessibility technology deal with novel widget names? My understanding from reading https://w3c.github.io/aria/html-aam/html-aam.html#html-element-role-mappings is that there is a predefined set of roles that accessibility technologies support. For example, ROLE_SYSTEM_GROUPING / IA2_ROLE_SECTION / Group / ATK_ROLE_BLOCK_QUOTE / AXGroup. So I don't understand how it's possible to create custom widget names. Do these technologies have some sort of escape hatch?

If control patterns are used does that mean that the browser may perform device independent actions on them? This would mean a JavaScript interface.

By this do you mean things like https://github.com/a11y-api/a11y-api/'s accessibility input events?

Not sure if I can help answer your latter two questions.

@richschwer
Copy link
Author

This is the first time I have seen this API work. There was a rumor that Mozilla and Google were working on one.
That is an excellent question!

Each platform accessibility API has a mapping for a custom role description. Look at the mapping for aria-roledescription: http://rawgit.com/w3c/aria/master/core-aam/core-aam.html

However, that is a localized role name that is difficult to tee off of for an AT in and by itself.

Microsoft's UI Automation has a notion of control patterns. (that include interfaces that can be called as well as properties that are exposed) that can be used for accessibility as well as automated testing. The combination of these control patterns forms a signature for which you can expose a standard recognized role and/or a custom role. An AT would look at the combination of control patterns to assess how to work with the new customer control. This is why we have been looking at the UIA model for custom controls.

So, what we really need is a vehicle for discovery. UIA does do that by producing an "signature": tabular, scrollable, clickable, selectable, etc. These can be applied to objects like any other accessibility api. I brought the API you mentioned up on the ARIA call today and Microsoft said they were no longer involved with it. Whatever we do we need everyone playing in the same boat.

Now, to understand Microsoft's perspective they have a common API strategy designed to work across all their platforms: XBox, Windows Desktop, Tablets, Phones. It is all UIA based.

So, circling back to your question, combining common control patterns with existing role mappings and/or custom, localized roles is a way forward. Another possibility that was not adopted by browser manufactures was to have the role value be a name-spaced (I know the browser community hates those) value that could be converted to say a JSON object reference that allowed and AT to discover about the new custom role and how to interact with it. That would include information like inheritance. Originally, when we created ARIA we had role values that could be prefixed and converted to a semantic web RDF/OWL representation of that information. The browser vendors told us to get rid of it or they would not accept it and we lost the functionality. We certainly could do something like but find a way to ditch the name spaces.

You can see remnants of this in the Chrome and Firefox MSAA/IA2 implementation where you have the xml-roles object attribute that exposes the content role value. That used to be name-spaced prefixed.

Here is a reference information on UIA Control Patterns: https://msdn.microsoft.com/en-us/library/ms752362(v=vs.110).aspx

@richschwer
Copy link
Author

i just spoke with Joanie Diggs and discovered she was not really involved with the AAPI either. So, my recommendation is that this work be integrated into the Web Community Group where we had expected. The graph of committers shows this effort to largely by Google. People's names have been used without really fully having committed to the effort. The attributions are very misleading.

@cookiecrook
Copy link

cookiecrook commented Aug 30, 2016

@richschwer wrote:

People's names have been used without really fully having committed to the effort.

At the face-to-face meeting, all the attendees agreed to list their names as editors. Cynthia @cyns was not in attendance, but had contributed enough to the lead-in API discussions and WAPA that Bogdan @boggydigital requested she be left on the list. It may be more accurate to refer to the list as "authors" rather than "editors" based on their active and prior contributions. If anyone wants their names removed, we're happy to do so. Everyone listed has editor privileges, so they're welcome to make additional changes, too.

The attributions are very misleading.

I believe you're mistaken. In any case, your suggestion that this is a Google-only proposal isn't true. Here's a little more history to fill in the gaps.

The idea started as an incubation effort between browser implementors, and we've had active discussions since around January of this year. Dominic Mazzoni (@minorninth) noticed that several prior efforts (Intentional Events, IndieUI, WAPA, some RTE features, Mozilla's AAPI proposal) had been attempted within the normal W3C process and had failed due to a number of reasons. Among them, implementation problems too late in the process, scope bloat, vendor-specific API style, etc. Nevertheless, Dominic M and @alice Boxhall spotted that we were all trying to solve the same problems and suggested we work amongst ourselves to bring a shared proposal to the table. @cyns is the one who originally suggested the WICG.

In the Spring, we had active email discussions of the pros and cons of the different proposals, and started focusing the effort for a minimum viable product that we could all live with. The API discussions included participation by every major browser vendor prior to anything being committed to a spec draft.

We had a face-to-face meeting in May. The active contributors in that face-to-face were:

  • Alex Surkov, Mozilla
  • Bogdan Brinza, Microsoft
  • Alice Boxhall, Google
  • Dominic Mazzoni, Google
  • James Craig (me), Apple

The Accessibility Object Model (AOM) spec is the culmination of the effort from that meeting. Each of the people listed contributed to the draft, regardless of whether they explicitly made edits in the Github repo. A number of changes were made at Bogdan's request because he thought the initial ideas may not be implementable in Edge. A number of ideas were based on Alex's draft of AAPI, or my contributions to IndieUI, or Cynthia's work on WAPA.

Since that time, @minorninth put the technical draft to paper, and I primarily contributed the process sections: Introduction and Inclusion/Exclusion/Objection criteria. The group discussion slowed this Summer due to company-focused conferences (I/O, WWDC, etc) and beta cycles for upcoming releases.

At this point, there are proof-of-concept patches for both Chrome and Firefox, and we have implementation plans in WebKit. We put out the call for comment to other engineers and standards contributors within each company, which is how Domenic D heard about it. Within Google, Domenic D @domenic and Alex Russell @slightlyoff submitted comments. Within Mozilla, Marcos @marcoscaceres has made a number of comments and Alex Surkov @asurkov has contributed heavily. Within Apple, about half a dozen members of the Standards, WebKit, and Accessibility Engineering teams have reviewed the drafts, and I've been submitting changes or comments on their behalf.

Hopefully this history alleviates your concern that the Accessibility Object Model (AOM) proposal was not a joint effort. We just hadn't formally announced it because it wasn't yet ready. It's almost ready now, so I added AOM as a topic proposal for Wednesday's technical plenary at TPAC in Lisbon, and was planning to propose it as an agenda item for the ARIA F2F, too. I'm happy to give a summary in one of the upcoming ARIA meetings, too.

@cookiecrook
Copy link

The group discussion slowed this Summer…

It slowed for everyone, but I looked back and confirmed that @cyns and @boggydigital have not contributed actively since the face-to-face meeting in May. As of sometime last week, @cyns was looking into whether they'd want to remain on the authors/editors list, or be moved to the Acknowledgements section.

@richschwer
Copy link
Author

When I spoke with Cynthia she was very surprised she was listed as an editor.

@richschwer
Copy link
Author

I am reading your reasons for working on AOM seprately ("several prior efforts (Intentional Events, IndieUI, WAPA, some RTE features, Mozilla's AAPI proposal) had been attempted within the normal W3C process and had failed due to a number of reasons. Among them, implementation problems too late in the process, scope bloat, vendor-specific API style, etc. ") and yet I see little or no discussion on the WICG list to support any of this analysis or requirements gathering even took place. ... Just saying. I did not see Mozilla's AAPI being discussed in W3C. Do you have a link?

@cyns
Copy link

cyns commented Aug 30, 2016

I am working on solidifying Microsoft's position on this work. I hope to have an answer next week.

@cookiecrook
Copy link

cookiecrook commented Aug 30, 2016

I see little or no discussion on the WICG list to support any of this analysis or requirements gathering even took place.

The discussion did not happen on the WICG list. The AOM work is planned for contribution to WICG but has not been contributed yet.

I did not see Mozilla's AAPI being discussed in W3C. Do you have a link?

https://wicg.github.io/a11yapi/

@richschwer
Copy link
Author

that is the link I had for Mozilla's early work

@asurkov
Copy link
Member

asurkov commented Sep 7, 2016

yes, that's it. I stopped contributions to the 'Accessibility API' draft,
shifting focus to AOM, considering it a continuation of the original draft
ideas.

On Wed, Aug 31, 2016 at 3:58 PM, Richard Schwerdtfeger <
notifications@github.com> wrote:

that is the link I had for Mozilla's early work


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#553 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABdnqTuEpi5Mmat9t0l1AqS2c7x8ZfTTks5qldzrgaJpZM4Jrafr
.

@annevk
Copy link
Collaborator

annevk commented Feb 17, 2018

Closing this given the efforts noted in #729. Please do comment if there's something left to address here.

@annevk annevk closed this as completed Feb 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants