Admin accessibility

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Admin accessibility

Tom Carrick
Sorry, long post incoming.

The admin currently has some accessibility issues. Not a huge amount, actually, but they should be fixed regardless. I hope I don't need to convince anyone here of the importance of accessibility, but I'll try to make the case as briefly as possible for the benefit of the wider community.

There is a trend towards stronger accessibility laws - there is a good roundup of them here: https://www.w3.org/WAI/policies/ - and I'll come back to this later. Most of these cover the public sector and small segments of the private sector, but there's also an overview of some laws used against private entities more generally here: https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines#Legal_obligations

I should make it clear that I'm not a lawyer and have no idea if any of this would apply to the admin, since it's not intended for general consumption, so I think I'd rather make this point: Developers and other people using the admin - "staff users" of some kind - can have disabilities too, and we should be catering for them, especially since the remedies are not at all difficult. It's also worth pointing out that accessibility improvements almost always improve the general experience. For example, making sure everything is easily accessible for keyboard only users is also beneficial to power users. Better contrast and larger fonts are more legible for everyone, not just for those with low vision, etc.

So I think the place to start here is to decide on a guideline to aim for, and I'm pretty convinced at this point that WCAG 2.1 at Level AA is the way to go: https://www.w3.org/TR/WCAG21/

Why not AAA? Well, it's really hard / time-consuming. For example, users have to be able to select their own foreground / background colours, if a user's session expires, they need to be able to login in again and pick off where they left off (forms filled, etc.), and more. Moreover, every law I've looked at seems to use WCAG 2.0 (2.1 is just a couple years old and is backwards compatible) at Level AA, so this seems like the target to aim for. I don't see a reason to not make specific improvements to AAA where they're relatively simple, however, but my point is that we should make AA a minimum requirement.

So if that is accepted, we need a few things:

1. Document it and update the contributing docs.
2. Audit the admin properly.
3. Fix any issues.
4. Make sure reviewers have this in mind for admin changes (I'm not sure if there's any CI solution for this, but it would be nice to have).

I haven't audited the entire admin, but I have run a checker through some pages. The main issues are with contrast and small font sizes, and the changelist also seems quite painful. For example, there are no labels on the checkboxes to select rows, which is going to be pretty hard to understand and use if you're blind. A quick aria-label can fix this without affecting it for sighted users.

Another issue here is harder to solve, it requires two tabs to move to another row of the change list in the default state (one for the checkbox, one for the link to the change form page). If you have editable fields in the change list, it's another tab for each. It would be nice to be able to use vim keys to move up and down rows, perhaps be able to hit * to select a row for an action. In general, keyboard shortcuts would be handy elsewhere too. It would be nice to be able to hit a key to open the nav sidebar which also sets tab focus on the first link, for example.

But these details aren't the point of the post. The point of the post is to decide if this is worth it (clearly I think it is), and how to move forwards on it. Any thoughts?

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMaQhWCtH%2BRZrp8JuXOyPFghAk7GVsJQPD15YHE9DUzQyw%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Adam Johnson-2
Hi Tom

It's a laudable goal to make the admin accessible. Thank you for doing the research on currently laws and guidelines.

I haven't audited the entire admin, but I have run a checker through some pages.

Which checker did you use? I see Google's Lighthouse checks for many accessibility issues: https://web.dev/lighthouse-accessibility/

I think fixing any "low hanging fruit" would be acceptable PR's right now.

Setting a target and keeping to it would be harder though. Documentation is easy to write, but it can easily be forgotten/ignored, so I'd prefer a CI solution. One potential CI solution is running admin tests through the Lighthouse CLI.

There will certainly add an ongoing maintenance cost. Would you be able to work on this on an ongoing basis? If you know others who are interested, perhaps we could even form a Django a11y team, just like we have an i18n team.

On Tue, 19 May 2020 at 12:11, Tom Carrick <[hidden email]> wrote:
Sorry, long post incoming.

The admin currently has some accessibility issues. Not a huge amount, actually, but they should be fixed regardless. I hope I don't need to convince anyone here of the importance of accessibility, but I'll try to make the case as briefly as possible for the benefit of the wider community.

There is a trend towards stronger accessibility laws - there is a good roundup of them here: https://www.w3.org/WAI/policies/ - and I'll come back to this later. Most of these cover the public sector and small segments of the private sector, but there's also an overview of some laws used against private entities more generally here: https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines#Legal_obligations

I should make it clear that I'm not a lawyer and have no idea if any of this would apply to the admin, since it's not intended for general consumption, so I think I'd rather make this point: Developers and other people using the admin - "staff users" of some kind - can have disabilities too, and we should be catering for them, especially since the remedies are not at all difficult. It's also worth pointing out that accessibility improvements almost always improve the general experience. For example, making sure everything is easily accessible for keyboard only users is also beneficial to power users. Better contrast and larger fonts are more legible for everyone, not just for those with low vision, etc.

So I think the place to start here is to decide on a guideline to aim for, and I'm pretty convinced at this point that WCAG 2.1 at Level AA is the way to go: https://www.w3.org/TR/WCAG21/

Why not AAA? Well, it's really hard / time-consuming. For example, users have to be able to select their own foreground / background colours, if a user's session expires, they need to be able to login in again and pick off where they left off (forms filled, etc.), and more. Moreover, every law I've looked at seems to use WCAG 2.0 (2.1 is just a couple years old and is backwards compatible) at Level AA, so this seems like the target to aim for. I don't see a reason to not make specific improvements to AAA where they're relatively simple, however, but my point is that we should make AA a minimum requirement.

So if that is accepted, we need a few things:

1. Document it and update the contributing docs.
2. Audit the admin properly.
3. Fix any issues.
4. Make sure reviewers have this in mind for admin changes (I'm not sure if there's any CI solution for this, but it would be nice to have).

I haven't audited the entire admin, but I have run a checker through some pages. The main issues are with contrast and small font sizes, and the changelist also seems quite painful. For example, there are no labels on the checkboxes to select rows, which is going to be pretty hard to understand and use if you're blind. A quick aria-label can fix this without affecting it for sighted users.

Another issue here is harder to solve, it requires two tabs to move to another row of the change list in the default state (one for the checkbox, one for the link to the change form page). If you have editable fields in the change list, it's another tab for each. It would be nice to be able to use vim keys to move up and down rows, perhaps be able to hit * to select a row for an action. In general, keyboard shortcuts would be handy elsewhere too. It would be nice to be able to hit a key to open the nav sidebar which also sets tab focus on the first link, for example.

But these details aren't the point of the post. The point of the post is to decide if this is worth it (clearly I think it is), and how to move forwards on it. Any thoughts?

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMaQhWCtH%2BRZrp8JuXOyPFghAk7GVsJQPD15YHE9DUzQyw%40mail.gmail.com.


--
Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM3BFwKmXDw7rR46QswdA%2BKbGWM7euz1Zujt%2BtVwhgHKjw%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Tom Carrick
Hi Adam,

> Which checker did you use?

I had a brief check with a few. Firefox's devtools, Lighthouse, and WebAIM's WAVE browser extension. WAVE is quite nice in that it shows the problems directly on the page, but they all give more or less the same information. If Lighthouse can be integrated into the CI it seems the most logical option. Lighthouse also gives a 0 - 100 score metric for each page, which would be nice as part of a coverage-style CI process, i.e. the score can remain the same or increase, but a decrease will fail the test.

> Would you be able to work on this on an ongoing basis? If you know others who are interested, perhaps we could even form a Django a11y team, just like we have an i18n team.

I'm happy to work on this on an ongoing basis. Thinking over the idea of an a11y team, at first I thought it would have too narrow a scrope as it only really concerns the admin, but I realised it also concerns the docs, the djangoproject website, and other sites under the django org (django-people and so on), so perhaps it's worth doing. That said, I don't have any physical disabilities to speak of. My eyes are pretty bad, small text can be a problem, but that's about it. It would be good to get some feedback from people that this directly affects, but I don't know of many. It'd also be hugely beneficial of course if any a11y team includes people with physical disabilities or at least experience with it. I can only immediately think of one person in the Django community that might fit, so I will reach out to them and ask if they'd be interested in giving any input here, but more voices would be useful.

For the low-hanging fruit, I will run Lighthouse on some more pages and write some tickets up when I get some time, hopefully over the weekend.

Cheers,
Tom

On Thu, 21 May 2020 at 14:04, Adam Johnson <[hidden email]> wrote:
Hi Tom

It's a laudable goal to make the admin accessible. Thank you for doing the research on currently laws and guidelines.

I haven't audited the entire admin, but I have run a checker through some pages.

Which checker did you use? I see Google's Lighthouse checks for many accessibility issues: https://web.dev/lighthouse-accessibility/

I think fixing any "low hanging fruit" would be acceptable PR's right now.

Setting a target and keeping to it would be harder though. Documentation is easy to write, but it can easily be forgotten/ignored, so I'd prefer a CI solution. One potential CI solution is running admin tests through the Lighthouse CLI.

There will certainly add an ongoing maintenance cost. Would you be able to work on this on an ongoing basis? If you know others who are interested, perhaps we could even form a Django a11y team, just like we have an i18n team.

On Tue, 19 May 2020 at 12:11, Tom Carrick <[hidden email]> wrote:
Sorry, long post incoming.

The admin currently has some accessibility issues. Not a huge amount, actually, but they should be fixed regardless. I hope I don't need to convince anyone here of the importance of accessibility, but I'll try to make the case as briefly as possible for the benefit of the wider community.

There is a trend towards stronger accessibility laws - there is a good roundup of them here: https://www.w3.org/WAI/policies/ - and I'll come back to this later. Most of these cover the public sector and small segments of the private sector, but there's also an overview of some laws used against private entities more generally here: https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines#Legal_obligations

I should make it clear that I'm not a lawyer and have no idea if any of this would apply to the admin, since it's not intended for general consumption, so I think I'd rather make this point: Developers and other people using the admin - "staff users" of some kind - can have disabilities too, and we should be catering for them, especially since the remedies are not at all difficult. It's also worth pointing out that accessibility improvements almost always improve the general experience. For example, making sure everything is easily accessible for keyboard only users is also beneficial to power users. Better contrast and larger fonts are more legible for everyone, not just for those with low vision, etc.

So I think the place to start here is to decide on a guideline to aim for, and I'm pretty convinced at this point that WCAG 2.1 at Level AA is the way to go: https://www.w3.org/TR/WCAG21/

Why not AAA? Well, it's really hard / time-consuming. For example, users have to be able to select their own foreground / background colours, if a user's session expires, they need to be able to login in again and pick off where they left off (forms filled, etc.), and more. Moreover, every law I've looked at seems to use WCAG 2.0 (2.1 is just a couple years old and is backwards compatible) at Level AA, so this seems like the target to aim for. I don't see a reason to not make specific improvements to AAA where they're relatively simple, however, but my point is that we should make AA a minimum requirement.

So if that is accepted, we need a few things:

1. Document it and update the contributing docs.
2. Audit the admin properly.
3. Fix any issues.
4. Make sure reviewers have this in mind for admin changes (I'm not sure if there's any CI solution for this, but it would be nice to have).

I haven't audited the entire admin, but I have run a checker through some pages. The main issues are with contrast and small font sizes, and the changelist also seems quite painful. For example, there are no labels on the checkboxes to select rows, which is going to be pretty hard to understand and use if you're blind. A quick aria-label can fix this without affecting it for sighted users.

Another issue here is harder to solve, it requires two tabs to move to another row of the change list in the default state (one for the checkbox, one for the link to the change form page). If you have editable fields in the change list, it's another tab for each. It would be nice to be able to use vim keys to move up and down rows, perhaps be able to hit * to select a row for an action. In general, keyboard shortcuts would be handy elsewhere too. It would be nice to be able to hit a key to open the nav sidebar which also sets tab focus on the first link, for example.

But these details aren't the point of the post. The point of the post is to decide if this is worth it (clearly I think it is), and how to move forwards on it. Any thoughts?

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMaQhWCtH%2BRZrp8JuXOyPFghAk7GVsJQPD15YHE9DUzQyw%40mail.gmail.com.


--
Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM3BFwKmXDw7rR46QswdA%2BKbGWM7euz1Zujt%2BtVwhgHKjw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMajRhzFMPKHXYQuOSouk4jej%2BwQ%2B%2B7JrtWQv02P4ZTQVQ%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Kye Russell
Hi all,

I am legally blind. Accessibility discussions tend to largely revolve around contrast and screen readers (which I do not use) as it’s easy to measure for (albeit poorly without actually trying to use a screen reader yourself, which nobody tends to do).

I am unsure if I have any valuable / unique perspective to share, and I probably don’t, but I am always happy to help. I have wanted an excuse to get involved in the framework's development and this would be as good a time as any.

The one thing I will say (and this doesn’t come from any lived experience, rather what I’ve heard) is not to rely to heavily on automated accessibility metrics, especially when it comes to screen reader friendliness. An interface can tick all the automated boxes but still be terrible to use. Just as with other user interfaces, automation can only get us so far and additional manual checking would be worthwhile.

Kye




On 21 May 2020 at 9:33:51 pm, Tom Carrick ([hidden email]) wrote:

Hi Adam,

> Which checker did you use?

I had a brief check with a few. Firefox's devtools, Lighthouse, and WebAIM's WAVE browser extension. WAVE is quite nice in that it shows the problems directly on the page, but they all give more or less the same information. If Lighthouse can be integrated into the CI it seems the most logical option. Lighthouse also gives a 0 - 100 score metric for each page, which would be nice as part of a coverage-style CI process, i.e. the score can remain the same or increase, but a decrease will fail the test.

> Would you be able to work on this on an ongoing basis? If you know others who are interested, perhaps we could even form a Django a11y team, just like we have an i18n team.

I'm happy to work on this on an ongoing basis. Thinking over the idea of an a11y team, at first I thought it would have too narrow a scrope as it only really concerns the admin, but I realised it also concerns the docs, the djangoproject website, and other sites under the django org (django-people and so on), so perhaps it's worth doing. That said, I don't have any physical disabilities to speak of. My eyes are pretty bad, small text can be a problem, but that's about it. It would be good to get some feedback from people that this directly affects, but I don't know of many. It'd also be hugely beneficial of course if any a11y team includes people with physical disabilities or at least experience with it. I can only immediately think of one person in the Django community that might fit, so I will reach out to them and ask if they'd be interested in giving any input here, but more voices would be useful.

For the low-hanging fruit, I will run Lighthouse on some more pages and write some tickets up when I get some time, hopefully over the weekend.

Cheers,
Tom

On Thu, 21 May 2020 at 14:04, Adam Johnson <[hidden email]> wrote:
Hi Tom

It's a laudable goal to make the admin accessible. Thank you for doing the research on currently laws and guidelines.

I haven't audited the entire admin, but I have run a checker through some pages.

Which checker did you use? I see Google's Lighthouse checks for many accessibility issues: https://web.dev/lighthouse-accessibility/

I think fixing any "low hanging fruit" would be acceptable PR's right now.

Setting a target and keeping to it would be harder though. Documentation is easy to write, but it can easily be forgotten/ignored, so I'd prefer a CI solution. One potential CI solution is running admin tests through the Lighthouse CLI.

There will certainly add an ongoing maintenance cost. Would you be able to work on this on an ongoing basis? If you know others who are interested, perhaps we could even form a Django a11y team, just like we have an i18n team.

On Tue, 19 May 2020 at 12:11, Tom Carrick <[hidden email]> wrote:
Sorry, long post incoming.

The admin currently has some accessibility issues. Not a huge amount, actually, but they should be fixed regardless. I hope I don't need to convince anyone here of the importance of accessibility, but I'll try to make the case as briefly as possible for the benefit of the wider community.

There is a trend towards stronger accessibility laws - there is a good roundup of them here: https://www.w3.org/WAI/policies/ - and I'll come back to this later. Most of these cover the public sector and small segments of the private sector, but there's also an overview of some laws used against private entities more generally here: https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines#Legal_obligations

I should make it clear that I'm not a lawyer and have no idea if any of this would apply to the admin, since it's not intended for general consumption, so I think I'd rather make this point: Developers and other people using the admin - "staff users" of some kind - can have disabilities too, and we should be catering for them, especially since the remedies are not at all difficult. It's also worth pointing out that accessibility improvements almost always improve the general experience. For example, making sure everything is easily accessible for keyboard only users is also beneficial to power users. Better contrast and larger fonts are more legible for everyone, not just for those with low vision, etc.

So I think the place to start here is to decide on a guideline to aim for, and I'm pretty convinced at this point that WCAG 2.1 at Level AA is the way to go: https://www.w3.org/TR/WCAG21/

Why not AAA? Well, it's really hard / time-consuming. For example, users have to be able to select their own foreground / background colours, if a user's session expires, they need to be able to login in again and pick off where they left off (forms filled, etc.), and more. Moreover, every law I've looked at seems to use WCAG 2.0 (2.1 is just a couple years old and is backwards compatible) at Level AA, so this seems like the target to aim for. I don't see a reason to not make specific improvements to AAA where they're relatively simple, however, but my point is that we should make AA a minimum requirement.

So if that is accepted, we need a few things:

1. Document it and update the contributing docs.
2. Audit the admin properly.
3. Fix any issues.
4. Make sure reviewers have this in mind for admin changes (I'm not sure if there's any CI solution for this, but it would be nice to have).

I haven't audited the entire admin, but I have run a checker through some pages. The main issues are with contrast and small font sizes, and the changelist also seems quite painful. For example, there are no labels on the checkboxes to select rows, which is going to be pretty hard to understand and use if you're blind. A quick aria-label can fix this without affecting it for sighted users.

Another issue here is harder to solve, it requires two tabs to move to another row of the change list in the default state (one for the checkbox, one for the link to the change form page). If you have editable fields in the change list, it's another tab for each. It would be nice to be able to use vim keys to move up and down rows, perhaps be able to hit * to select a row for an action. In general, keyboard shortcuts would be handy elsewhere too. It would be nice to be able to hit a key to open the nav sidebar which also sets tab focus on the first link, for example.

But these details aren't the point of the post. The point of the post is to decide if this is worth it (clearly I think it is), and how to move forwards on it. Any thoughts?

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMaQhWCtH%2BRZrp8JuXOyPFghAk7GVsJQPD15YHE9DUzQyw%40mail.gmail.com.


--
Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM3BFwKmXDw7rR46QswdA%2BKbGWM7euz1Zujt%2BtVwhgHKjw%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMajRhzFMPKHXYQuOSouk4jej%2BwQ%2B%2B7JrtWQv02P4ZTQVQ%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CANK-ykkkuBOGGBBFOhP7BViN8zk5_HCBnw-TUVWZHoUwduTwhA%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Andreu Vallbona
In reply to this post by Tom Carrick

Hi there,

 

I think it's a great initiative, even if as a person with physical disabilities, I've never been issues to use any kind of software. 

 

I would be happy to contribute but I'm not aware of any tool that can be integrated in any continuous integration system. Whenever I've been working in projects that implied accessibility requirements, I've always used "manual" processes to verify the web accessibility.

 

Let me know if a team is formed for this purpose. I think I could contribute a few hours of work.

 

Personally, I think that, as Adam says, it would be not difficult to amend the “low hanging fruits“, such as:

  • alt text for images

  • semantic use of headings (h1, h2, …)

  • avoid use of style tags inside the html code

  • lack of contrast in texts

 

Web accessibility tools I'm aware of:

Kind regards




El dimarts, 19 maig de 2020 13:12:17 UTC+2, Tom Carrick va escriure:
Sorry, long post incoming.

The admin currently has some accessibility issues. Not a huge amount, actually, but they should be fixed regardless. I hope I don't need to convince anyone here of the importance of accessibility, but I'll try to make the case as briefly as possible for the benefit of the wider community.

There is a trend towards stronger accessibility laws - there is a good roundup of them here: <a href="https://www.w3.org/WAI/policies/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2Fpolicies%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHuXyCEAwXzhQM1FYyrOjwonnSQOQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2Fpolicies%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHuXyCEAwXzhQM1FYyrOjwonnSQOQ&#39;;return true;">https://www.w3.org/WAI/policies/ - and I'll come back to this later. Most of these cover the public sector and small segments of the private sector, but there's also an overview of some laws used against private entities more generally here: <a href="https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines#Legal_obligations" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fen.wikipedia.org%2Fwiki%2FWeb_Content_Accessibility_Guidelines%23Legal_obligations\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNH143LeV65kZTx5tZZN8hFO0VOUNg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fen.wikipedia.org%2Fwiki%2FWeb_Content_Accessibility_Guidelines%23Legal_obligations\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNH143LeV65kZTx5tZZN8hFO0VOUNg&#39;;return true;">https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines#Legal_obligations

I should make it clear that I'm not a lawyer and have no idea if any of this would apply to the admin, since it's not intended for general consumption, so I think I'd rather make this point: Developers and other people using the admin - "staff users" of some kind - can have disabilities too, and we should be catering for them, especially since the remedies are not at all difficult. It's also worth pointing out that accessibility improvements almost always improve the general experience. For example, making sure everything is easily accessible for keyboard only users is also beneficial to power users. Better contrast and larger fonts are more legible for everyone, not just for those with low vision, etc.

So I think the place to start here is to decide on a guideline to aim for, and I'm pretty convinced at this point that WCAG 2.1 at Level AA is the way to go: <a href="https://www.w3.org/TR/WCAG21/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FTR%2FWCAG21%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGMeLOIbdr8pp-cOtOUVVbuTTilfw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FTR%2FWCAG21%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGMeLOIbdr8pp-cOtOUVVbuTTilfw&#39;;return true;">https://www.w3.org/TR/WCAG21/

Why not AAA? Well, it's really hard / time-consuming. For example, users have to be able to select their own foreground / background colours, if a user's session expires, they need to be able to login in again and pick off where they left off (forms filled, etc.), and more. Moreover, every law I've looked at seems to use WCAG 2.0 (2.1 is just a couple years old and is backwards compatible) at Level AA, so this seems like the target to aim for. I don't see a reason to not make specific improvements to AAA where they're relatively simple, however, but my point is that we should make AA a minimum requirement.

So if that is accepted, we need a few things:

1. Document it and update the contributing docs.
2. Audit the admin properly.
3. Fix any issues.
4. Make sure reviewers have this in mind for admin changes (I'm not sure if there's any CI solution for this, but it would be nice to have).

I haven't audited the entire admin, but I have run a checker through some pages. The main issues are with contrast and small font sizes, and the changelist also seems quite painful. For example, there are no labels on the checkboxes to select rows, which is going to be pretty hard to understand and use if you're blind. A quick aria-label can fix this without affecting it for sighted users.

Another issue here is harder to solve, it requires two tabs to move to another row of the change list in the default state (one for the checkbox, one for the link to the change form page). If you have editable fields in the change list, it's another tab for each. It would be nice to be able to use vim keys to move up and down rows, perhaps be able to hit * to select a row for an action. In general, keyboard shortcuts would be handy elsewhere too. It would be nice to be able to hit a key to open the nav sidebar which also sets tab focus on the first link, for example.

But these details aren't the point of the post. The point of the post is to decide if this is worth it (clearly I think it is), and how to move forwards on it. Any thoughts?

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/25ae3ff2-9a9b-4af0-97d2-3cf654ccbed1%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Gustavo Sinovsky
In reply to this post by Tom Carrick
Hi everybody,

I'm very interested in joining this conversation since I believe accessibility is an important issue and something often overlooked in applications such as the Django admin site. I come in representation of a software company called Talpor, located in Santiago de Chile, and we are very much interested in helping any way we can to make Django a better library. We just posted a ticket with some ideas on how to initially tackle this in order to keep the momentum going on this issue, I invite you to continue this conversation over there. Also, we actually plan to invest some time ourselves in solving some of the issues outlined in the ticket, so any specific suggestions are more than welcomed

On Tuesday, May 19, 2020 at 7:12:17 AM UTC-4, Tom Carrick wrote:
Sorry, long post incoming.

The admin currently has some accessibility issues. Not a huge amount, actually, but they should be fixed regardless. I hope I don't need to convince anyone here of the importance of accessibility, but I'll try to make the case as briefly as possible for the benefit of the wider community.

There is a trend towards stronger accessibility laws - there is a good roundup of them here: <a href="https://www.w3.org/WAI/policies/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2Fpolicies%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHuXyCEAwXzhQM1FYyrOjwonnSQOQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2Fpolicies%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHuXyCEAwXzhQM1FYyrOjwonnSQOQ&#39;;return true;">https://www.w3.org/WAI/policies/ - and I'll come back to this later. Most of these cover the public sector and small segments of the private sector, but there's also an overview of some laws used against private entities more generally here: <a href="https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines#Legal_obligations" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fen.wikipedia.org%2Fwiki%2FWeb_Content_Accessibility_Guidelines%23Legal_obligations\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNH143LeV65kZTx5tZZN8hFO0VOUNg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fen.wikipedia.org%2Fwiki%2FWeb_Content_Accessibility_Guidelines%23Legal_obligations\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNH143LeV65kZTx5tZZN8hFO0VOUNg&#39;;return true;">https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines#Legal_obligations

I should make it clear that I'm not a lawyer and have no idea if any of this would apply to the admin, since it's not intended for general consumption, so I think I'd rather make this point: Developers and other people using the admin - "staff users" of some kind - can have disabilities too, and we should be catering for them, especially since the remedies are not at all difficult. It's also worth pointing out that accessibility improvements almost always improve the general experience. For example, making sure everything is easily accessible for keyboard only users is also beneficial to power users. Better contrast and larger fonts are more legible for everyone, not just for those with low vision, etc.

So I think the place to start here is to decide on a guideline to aim for, and I'm pretty convinced at this point that WCAG 2.1 at Level AA is the way to go: <a href="https://www.w3.org/TR/WCAG21/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FTR%2FWCAG21%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGMeLOIbdr8pp-cOtOUVVbuTTilfw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FTR%2FWCAG21%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGMeLOIbdr8pp-cOtOUVVbuTTilfw&#39;;return true;">https://www.w3.org/TR/WCAG21/

Why not AAA? Well, it's really hard / time-consuming. For example, users have to be able to select their own foreground / background colours, if a user's session expires, they need to be able to login in again and pick off where they left off (forms filled, etc.), and more. Moreover, every law I've looked at seems to use WCAG 2.0 (2.1 is just a couple years old and is backwards compatible) at Level AA, so this seems like the target to aim for. I don't see a reason to not make specific improvements to AAA where they're relatively simple, however, but my point is that we should make AA a minimum requirement.

So if that is accepted, we need a few things:

1. Document it and update the contributing docs.
2. Audit the admin properly.
3. Fix any issues.
4. Make sure reviewers have this in mind for admin changes (I'm not sure if there's any CI solution for this, but it would be nice to have).

I haven't audited the entire admin, but I have run a checker through some pages. The main issues are with contrast and small font sizes, and the changelist also seems quite painful. For example, there are no labels on the checkboxes to select rows, which is going to be pretty hard to understand and use if you're blind. A quick aria-label can fix this without affecting it for sighted users.

Another issue here is harder to solve, it requires two tabs to move to another row of the change list in the default state (one for the checkbox, one for the link to the change form page). If you have editable fields in the change list, it's another tab for each. It would be nice to be able to use vim keys to move up and down rows, perhaps be able to hit * to select a row for an action. In general, keyboard shortcuts would be handy elsewhere too. It would be nice to be able to hit a key to open the nav sidebar which also sets tab focus on the first link, for example.

But these details aren't the point of the post. The point of the post is to decide if this is worth it (clearly I think it is), and how to move forwards on it. Any thoughts?

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/396d34bd-c435-423c-9d83-8b60625d7791%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Thibaud Colas
In reply to this post by Tom Carrick
Hi all,

I’m also interested in helping with this, although I’ve never been involved with Django development before. I’ve recently worked on a similar accessibility effort for Wagtail, a CMS built upon Django, which largely replaces the Django admin interface on projects where it’s used – and thus has similar use cases and audiences. We discussed which standards to aim for, why, which tools to use when, what to expect of contributors, etc, which I believe would be similarly relevant to Django. If Django had done this before we’d definitely have borrowed from its approach :)

I would recommend Pa11y as a tool to consider adding in CI. It uses HTML CodeSniffer and Axe under the hood (similarly to Lighthouse), which both score relatively well in how many issues they find. As Kye mentions this only goes so far (30-40% of issues at most), but still helpful to have automated checks if possible.

Cheers,

Thibaud



On Tuesday, 19 May 2020 12:12:17 UTC+1, Tom Carrick wrote:
Sorry, long post incoming.

The admin currently has some accessibility issues. Not a huge amount, actually, but they should be fixed regardless. I hope I don't need to convince anyone here of the importance of accessibility, but I'll try to make the case as briefly as possible for the benefit of the wider community.

There is a trend towards stronger accessibility laws - there is a good roundup of them here: <a href="https://www.w3.org/WAI/policies/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2Fpolicies%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHuXyCEAwXzhQM1FYyrOjwonnSQOQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2Fpolicies%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHuXyCEAwXzhQM1FYyrOjwonnSQOQ&#39;;return true;">https://www.w3.org/WAI/policies/ - and I'll come back to this later. Most of these cover the public sector and small segments of the private sector, but there's also an overview of some laws used against private entities more generally here: <a href="https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines#Legal_obligations" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fen.wikipedia.org%2Fwiki%2FWeb_Content_Accessibility_Guidelines%23Legal_obligations\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNH143LeV65kZTx5tZZN8hFO0VOUNg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fen.wikipedia.org%2Fwiki%2FWeb_Content_Accessibility_Guidelines%23Legal_obligations\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNH143LeV65kZTx5tZZN8hFO0VOUNg&#39;;return true;">https://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines#Legal_obligations

I should make it clear that I'm not a lawyer and have no idea if any of this would apply to the admin, since it's not intended for general consumption, so I think I'd rather make this point: Developers and other people using the admin - "staff users" of some kind - can have disabilities too, and we should be catering for them, especially since the remedies are not at all difficult. It's also worth pointing out that accessibility improvements almost always improve the general experience. For example, making sure everything is easily accessible for keyboard only users is also beneficial to power users. Better contrast and larger fonts are more legible for everyone, not just for those with low vision, etc.

So I think the place to start here is to decide on a guideline to aim for, and I'm pretty convinced at this point that WCAG 2.1 at Level AA is the way to go: <a href="https://www.w3.org/TR/WCAG21/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FTR%2FWCAG21%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGMeLOIbdr8pp-cOtOUVVbuTTilfw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FTR%2FWCAG21%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGMeLOIbdr8pp-cOtOUVVbuTTilfw&#39;;return true;">https://www.w3.org/TR/WCAG21/

Why not AAA? Well, it's really hard / time-consuming. For example, users have to be able to select their own foreground / background colours, if a user's session expires, they need to be able to login in again and pick off where they left off (forms filled, etc.), and more. Moreover, every law I've looked at seems to use WCAG 2.0 (2.1 is just a couple years old and is backwards compatible) at Level AA, so this seems like the target to aim for. I don't see a reason to not make specific improvements to AAA where they're relatively simple, however, but my point is that we should make AA a minimum requirement.

So if that is accepted, we need a few things:

1. Document it and update the contributing docs.
2. Audit the admin properly.
3. Fix any issues.
4. Make sure reviewers have this in mind for admin changes (I'm not sure if there's any CI solution for this, but it would be nice to have).

I haven't audited the entire admin, but I have run a checker through some pages. The main issues are with contrast and small font sizes, and the changelist also seems quite painful. For example, there are no labels on the checkboxes to select rows, which is going to be pretty hard to understand and use if you're blind. A quick aria-label can fix this without affecting it for sighted users.

Another issue here is harder to solve, it requires two tabs to move to another row of the change list in the default state (one for the checkbox, one for the link to the change form page). If you have editable fields in the change list, it's another tab for each. It would be nice to be able to use vim keys to move up and down rows, perhaps be able to hit * to select a row for an action. In general, keyboard shortcuts would be handy elsewhere too. It would be nice to be able to hit a key to open the nav sidebar which also sets tab focus on the first link, for example.

But these details aren't the point of the post. The point of the post is to decide if this is worth it (clearly I think it is), and how to move forwards on it. Any thoughts?

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1ff1afc8-20e4-47c9-9f58-db42ff7285d8%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Tobias Bengfort
Hi,

felixxm proposed in https://code.djangoproject.com/ticket/31617 to add a
DEP for this. The more I think about it, the less I agree.

There is no DEP for security, usability, developer ergonomics or
anything like that, so why should there be one for accessibility?

Like with any of the other categories, there are laws and regulations
that define a lower bound, but more is always better. And like with the
other categories, there are ways to find issues (think fuzzers) but no
way to verify the absence of issues.

So I think accessibility issues should be handled much like security
issues: When we find one, we fix it as quickly as possible. If there is
a conflict, say between security and accessibility, we decide on a
case-by-case basis.

Concerning which standard we should target: Tom made the case to target
WCAG 2.1 AA. I have not noticed any disagreement in this thread and
completely agree personally. Also note that many national regulations
are based on WCAG. So this question seems to be answered.

tobias

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/1a86d2d0-b3a2-0c56-8e13-cc64a32bead4%40posteo.de.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Mariusz Felisiak
In reply to this post by Tom Carrick
Hi Tobias,

  I'm not a WCAG expert, and it's not clear to me what steps we would like to take. In the ticket we have only steps describes as "very basics which should include", so I can imagine that's not all we need to do to be WCAG 2.1 compliant on AAA, AA or A level. As far as I'm concerned WCAG is quite massive. Moreover if we want to make changes in CI we should discussed available tools, etc. We can change colors but what next? We will be not able to guarantee that the contrast of all elements remains appropriate, we cannot do this manually.

Best,
Mariusz

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Tom Carrick
Hi,

Thanks for the feedback. I had thought about a DEP when I was writing up the original post actually, I just wasn't sure what it should contain. Here are my thoughts, based on the feedback so far:

- Defining a standard to target.
- Forming an a11y team that covers the django admin and all sites in the django github organization, and defining its role, membership, etc.
- Deciding on a CI process
- An outline of current issues and how to solve them.

If anyone can think of anything else, please let me know. If/when there's a consensus I'll start writing a draft.

Mariusz, I mentioned this in the original post, but https://www.w3.org/WAI/policies/ has a good overview of laws and EU directives in this area. From my browsing through, it seems that they all either, have their own national standard, or are using WCAG 2.0/2.1 AA. Since we probably don't want to define our own standard, I think AA is the way to go. This seems to also be the recommendation I hear from people in the accessibility field working on regular websites. AAA seems to be more suited for very specialist sites that explicitly target disabled people.

Most of AAA is completely feasible for us, but there are some reasons it would be difficult:

- All language needs to be at a lower secondary education level, or have an alternative that is. This doesn't seem feasible, for e.g. the documentation.
- Users that are logged out need to be able to resume their session where they left off after logging in, with all form fields filled, etc.

There are others that are difficult, but I don't think any apply to us currently.

Cheers,
Tom


On Mon, 25 May 2020 at 11:09, Mariusz Felisiak <[hidden email]> wrote:
Hi Tobias,

  I'm not a WCAG expert, and it's not clear to me what steps we would like to take. In the ticket we have only steps describes as "very basics which should include", so I can imagine that's not all we need to do to be WCAG 2.1 compliant on AAA, AA or A level. As far as I'm concerned WCAG is quite massive. Moreover if we want to make changes in CI we should discussed available tools, etc. We can change colors but what next? We will be not able to guarantee that the contrast of all elements remains appropriate, we cannot do this manually.

Best,
Mariusz

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMaG5C7hHJ%3DJZaLZ8DNsevZzr8hmJ%3Dd4S%3D1mmTsXkiyPZw%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Thibaud Colas
Hi Tom,

It’s exciting to see this getting started! To me a DEP would be highly beneficial, because there is a lot to this that goes beyond finding and fixing individual issues – it’s more about detailing the a process for parts of Django to stay accessible over time. Here are things I’d suggest to cover, in addition to your list:

- Going further than the targeted standard and tests in CI –  outline clear steps developers should take when testing their work: using developer tools, screen readers, etc. It’s not practical to just state the standard as WCAG is quite big, and open to interpretation. And CI testing will never be enough to reach any degree of compliance.
  - I’m not familiar with Django’s development process but generally I would recommend to use a combination of linting, browser extensions, manual testing instructions – and CI tests.
- If the accessibility team was to cover more than the Django admin, I think it would be worthwhile to also include accessibility of sites built with Django. Django is largely unopinionated about markup, but I remember its forms markup not being particularly good for screen readers, and the HTML code snippets in the docs would also be worth reviewing.
- Same goes for admin docs – not too familiar with them myself so I’m not sure whether they are considered to be part of the admin, or separate.
- I wouldn’t find it surprising if an audit of the Django admin turned up a lot of issues. IMHO part of the DEP should cover how to manage this – whether individual issues warrant individual tickets or not, and how to prioritize the issues if relevant. Which kinds of issues are likely to need design input. If the DEP is done first, also talk about what kind of auditing would be valuable, and how to make it happen.

To help with the initial audit and prioritisation of issues, I would suggest to first start by creating a map of all of the parts of the Django admin that are to be audited – list all of the individual pages, and all of the states the pages can be in (success, error, loading – empty content / some content / paginating, etc). I also find it worthwhile to add a succinct definition of how each of those pages is likely to be used – for example things that are only configured once for a site’s lifetime are presumably not as worth improving as a screen a user would see on a daily basis. Here is an example in practice: https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit. This can also be used to track the auditing effort, and form the basis of an automated test suite for CI (or simply repeat testing).

Finally, consider ATAG2.0 (Authoring Tool Accessibility Guidelines) compliance additionally to WCAG. At a high level, ATAG is made of two parts:

- Part A: Make the authoring tool user interface accessible. That sounds like what we’re discussing here.
- Part B: Support the production of accessible content. That’s a whole other topic but feels relevant too.

ATAG is nowhere near as well known / established / easy to test for, but it feels very relevant to the Django admin in particular, and I’m sure there will at least be some useful bits to consider in its success criteria

Hope this helps!

Thibaud


On Monday, May 25, 2020 at 11:56:39 AM UTC+1, Tom Carrick wrote:
Hi,

Thanks for the feedback. I had thought about a DEP when I was writing up the original post actually, I just wasn't sure what it should contain. Here are my thoughts, based on the feedback so far:

- Defining a standard to target.
- Forming an a11y team that covers the django admin and all sites in the django github organization, and defining its role, membership, etc.
- Deciding on a CI process
- An outline of current issues and how to solve them.

If anyone can think of anything else, please let me know. If/when there's a consensus I'll start writing a draft.

Mariusz, I mentioned this in the original post, but <a href="https://www.w3.org/WAI/policies/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2Fpolicies%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHuXyCEAwXzhQM1FYyrOjwonnSQOQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2Fpolicies%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHuXyCEAwXzhQM1FYyrOjwonnSQOQ&#39;;return true;">https://www.w3.org/WAI/policies/ has a good overview of laws and EU directives in this area. From my browsing through, it seems that they all either, have their own national standard, or are using WCAG 2.0/2.1 AA. Since we probably don't want to define our own standard, I think AA is the way to go. This seems to also be the recommendation I hear from people in the accessibility field working on regular websites. AAA seems to be more suited for very specialist sites that explicitly target disabled people.

Most of AAA is completely feasible for us, but there are some reasons it would be difficult:

- All language needs to be at a lower secondary education level, or have an alternative that is. This doesn't seem feasible, for e.g. the documentation.
- Users that are logged out need to be able to resume their session where they left off after logging in, with all form fields filled, etc.

There are others that are difficult, but I don't think any apply to us currently.

Cheers,
Tom


On Mon, 25 May 2020 at 11:09, Mariusz Felisiak <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="70rLM5tWAQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">felisia...@...> wrote:
Hi Tobias,

  I'm not a WCAG expert, and it's not clear to me what steps we would like to take. In the ticket we have only steps describes as "very basics which should include", so I can imagine that's not all we need to do to be WCAG 2.1 compliant on AAA, AA or A level. As far as I'm concerned WCAG is quite massive. Moreover if we want to make changes in CI we should discussed available tools, etc. We can change colors but what next? We will be not able to guarantee that the contrast of all elements remains appropriate, we cannot do this manually.

Best,
Mariusz

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="70rLM5tWAQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">django-d...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Tom Carrick
Some further thoughts...

Mariusz, I don't think I agree that WCAG is massive. It can look a little large but I think that's mostly because each guideline is split into smaller pieces and it's quite wordy to avoid ambiguity. There are in 2.1 50 success criteria for AA, many of which can be checked automatically, and many of those that can't don't apply to us as we don't use audio / video. There's also a nice quick reference that can be filtered to remove AAA and anything else unwanted. Once you drill down it can again get quite wordy, but it covers a lot. I think it's worth remembering here is that the point is not to strictly adhere to a standard for the sake of it, just to improve the overall accessibility, and WCAG is a useful and measurable tool to get us some of the way there.

Thibaud, I more or less agree with everything there. I'm not sure developers should have to do full accessibility checks for their changes in the admin. Using a screen reader for example can be very frustrating and confusing if you're not used to it, and it can take an inordinate amount of time, and I don't really wish that burden on (for example) first time committers who already have a lot of stuff to do, I think this will only discourage contributions. I think it should be the responsibility of the accessibility team to provide feedback, suggestions, etc. on relevant PRs or fix things when they're noticed after manual audits.

ATAG looks handy, but at first glance we couldn't even get to A, it requires auto-saving. It also generally seems like something that would cover a rich text editor plugged into the admin more than the admin itself, though I think it's laudable as a future target.

The more I think about it, I think the DEP should be a process DEP rather than a feature DEP. As others have mentioned, we don't need a DEP to fix accessibility issues - just fix them. But forming a team seems useful, and I think a DEP is required for that. I think the DEP should be limited to:

1. The accessibility team, how membership is decided, who it's accountable to, it's role/responsibilities, etc.
2. Perhaps the initial standard(s) to target, although this could also be decided by the a11y team post-assembly and agreed with the technical board and/or core devs.

As I said, I'm happy to write up the DEP, but since it'll be a process DEP, I think I'd need the support of a core dev, someone on the technical board, or just someone who has more knowledge of Django's "bureaucracy", for lack of a better word.

Tom

On Tue, 26 May 2020 at 01:20, Thibaud Colas <[hidden email]> wrote:
Hi Tom,

It’s exciting to see this getting started! To me a DEP would be highly beneficial, because there is a lot to this that goes beyond finding and fixing individual issues – it’s more about detailing the a process for parts of Django to stay accessible over time. Here are things I’d suggest to cover, in addition to your list:

- Going further than the targeted standard and tests in CI –  outline clear steps developers should take when testing their work: using developer tools, screen readers, etc. It’s not practical to just state the standard as WCAG is quite big, and open to interpretation. And CI testing will never be enough to reach any degree of compliance.
  - I’m not familiar with Django’s development process but generally I would recommend to use a combination of linting, browser extensions, manual testing instructions – and CI tests.
- If the accessibility team was to cover more than the Django admin, I think it would be worthwhile to also include accessibility of sites built with Django. Django is largely unopinionated about markup, but I remember its forms markup not being particularly good for screen readers, and the HTML code snippets in the docs would also be worth reviewing.
- Same goes for admin docs – not too familiar with them myself so I’m not sure whether they are considered to be part of the admin, or separate.
- I wouldn’t find it surprising if an audit of the Django admin turned up a lot of issues. IMHO part of the DEP should cover how to manage this – whether individual issues warrant individual tickets or not, and how to prioritize the issues if relevant. Which kinds of issues are likely to need design input. If the DEP is done first, also talk about what kind of auditing would be valuable, and how to make it happen.

To help with the initial audit and prioritisation of issues, I would suggest to first start by creating a map of all of the parts of the Django admin that are to be audited – list all of the individual pages, and all of the states the pages can be in (success, error, loading – empty content / some content / paginating, etc). I also find it worthwhile to add a succinct definition of how each of those pages is likely to be used – for example things that are only configured once for a site’s lifetime are presumably not as worth improving as a screen a user would see on a daily basis. Here is an example in practice: https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit. This can also be used to track the auditing effort, and form the basis of an automated test suite for CI (or simply repeat testing).

Finally, consider ATAG2.0 (Authoring Tool Accessibility Guidelines) compliance additionally to WCAG. At a high level, ATAG is made of two parts:

- Part A: Make the authoring tool user interface accessible. That sounds like what we’re discussing here.
- Part B: Support the production of accessible content. That’s a whole other topic but feels relevant too.

ATAG is nowhere near as well known / established / easy to test for, but it feels very relevant to the Django admin in particular, and I’m sure there will at least be some useful bits to consider in its success criteria

Hope this helps!

Thibaud


On Monday, May 25, 2020 at 11:56:39 AM UTC+1, Tom Carrick wrote:
Hi,

Thanks for the feedback. I had thought about a DEP when I was writing up the original post actually, I just wasn't sure what it should contain. Here are my thoughts, based on the feedback so far:

- Defining a standard to target.
- Forming an a11y team that covers the django admin and all sites in the django github organization, and defining its role, membership, etc.
- Deciding on a CI process
- An outline of current issues and how to solve them.

If anyone can think of anything else, please let me know. If/when there's a consensus I'll start writing a draft.

Mariusz, I mentioned this in the original post, but https://www.w3.org/WAI/policies/ has a good overview of laws and EU directives in this area. From my browsing through, it seems that they all either, have their own national standard, or are using WCAG 2.0/2.1 AA. Since we probably don't want to define our own standard, I think AA is the way to go. This seems to also be the recommendation I hear from people in the accessibility field working on regular websites. AAA seems to be more suited for very specialist sites that explicitly target disabled people.

Most of AAA is completely feasible for us, but there are some reasons it would be difficult:

- All language needs to be at a lower secondary education level, or have an alternative that is. This doesn't seem feasible, for e.g. the documentation.
- Users that are logged out need to be able to resume their session where they left off after logging in, with all form fields filled, etc.

There are others that are difficult, but I don't think any apply to us currently.

Cheers,
Tom


On Mon, 25 May 2020 at 11:09, Mariusz Felisiak <[hidden email]> wrote:
Hi Tobias,

  I'm not a WCAG expert, and it's not clear to me what steps we would like to take. In the ticket we have only steps describes as "very basics which should include", so I can imagine that's not all we need to do to be WCAG 2.1 compliant on AAA, AA or A level. As far as I'm concerned WCAG is quite massive. Moreover if we want to make changes in CI we should discussed available tools, etc. We can change colors but what next? We will be not able to guarantee that the contrast of all elements remains appropriate, we cannot do this manually.

Best,
Mariusz

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMavsZmszUrhnQY5L%3DTFBD8KJS4smCTYwj5M0k0PsoJhJw%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Thibaud Colas
Hey Tom,

I wanted to check if there is anything we/I could do to help in the meantime? Whether that’s by starting to map or audit the Django admin, or setting up a sample CI pipeline with accessibility tests, or something else altogether. It’s a bit daunting to get started with this but I think I could realistically do one of:

1. Create a draft map of the Django admin UI for later manual auditing purposes.
2. Audi a specific part of Django and creating a ticket with concrete things to fix.
3. Set up static analysis for the CSS or HTML to look for common accessibility gotchas.
4. Sett up automated in-browser accessibility checks on some parts of Django to show what that might look like in a CI pipeline.

… and document the steps along the way, and report back here or as a ticket.

Cheers,

Thibaud

On Tuesday, May 26, 2020 at 10:22:26 AM UTC+1, Tom Carrick wrote:
Some further thoughts...

Mariusz, I don't think I agree that WCAG is massive. It can look a little large but I think that's mostly because each guideline is split into smaller pieces and it's quite wordy to avoid ambiguity. There are in 2.1 50 success criteria for AA, many of which can be checked automatically, and many of those that can't don't apply to us as we don't use audio / video. There's also a nice <a href="https://www.w3.org/WAI/WCAG21/quickref/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2FWCAG21%2Fquickref%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHe7AsEzIzpK1MA5iGZ403GlhWZGw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2FWCAG21%2Fquickref%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHe7AsEzIzpK1MA5iGZ403GlhWZGw&#39;;return true;">quick reference that can be filtered to remove AAA and anything else unwanted. Once you drill down it can again get quite wordy, but it covers a lot. I think it's worth remembering here is that the point is not to strictly adhere to a standard for the sake of it, just to improve the overall accessibility, and WCAG is a useful and measurable tool to get us some of the way there.

Thibaud, I more or less agree with everything there. I'm not sure developers should have to do full accessibility checks for their changes in the admin. Using a screen reader for example can be very frustrating and confusing if you're not used to it, and it can take an inordinate amount of time, and I don't really wish that burden on (for example) first time committers who already have a lot of stuff to do, I think this will only discourage contributions. I think it should be the responsibility of the accessibility team to provide feedback, suggestions, etc. on relevant PRs or fix things when they're noticed after manual audits.

ATAG looks handy, but at first glance we couldn't even get to A, it requires auto-saving. It also generally seems like something that would cover a rich text editor plugged into the admin more than the admin itself, though I think it's laudable as a future target.

The more I think about it, I think the DEP should be a process DEP rather than a feature DEP. As others have mentioned, we don't need a DEP to fix accessibility issues - just fix them. But forming a team seems useful, and I think a DEP is required for that. I think the DEP should be limited to:

1. The accessibility team, how membership is decided, who it's accountable to, it's role/responsibilities, etc.
2. Perhaps the initial standard(s) to target, although this could also be decided by the a11y team post-assembly and agreed with the technical board and/or core devs.

As I said, I'm happy to write up the DEP, but since it'll be a process DEP, I think I'd need the support of a core dev, someone on the technical board, or just someone who has more knowledge of Django's "bureaucracy", for lack of a better word.

Tom

On Tue, 26 May 2020 at 01:20, Thibaud Colas <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="AU4UhgugAQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">thibau...@...> wrote:
Hi Tom,

It’s exciting to see this getting started! To me a DEP would be highly beneficial, because there is a lot to this that goes beyond finding and fixing individual issues – it’s more about detailing the a process for parts of Django to stay accessible over time. Here are things I’d suggest to cover, in addition to your list:

- Going further than the targeted standard and tests in CI –  outline clear steps developers should take when testing their work: using developer tools, screen readers, etc. It’s not practical to just state the standard as WCAG is quite big, and open to interpretation. And CI testing will never be enough to reach any degree of compliance.
  - I’m not familiar with Django’s development process but generally I would recommend to use a combination of linting, browser extensions, manual testing instructions – and CI tests.
- If the accessibility team was to cover more than the Django admin, I think it would be worthwhile to also include accessibility of sites built with Django. Django is largely unopinionated about markup, but I remember its forms markup not being particularly good for screen readers, and the HTML code snippets in the docs would also be worth reviewing.
- Same goes for admin docs – not too familiar with them myself so I’m not sure whether they are considered to be part of the admin, or separate.
- I wouldn’t find it surprising if an audit of the Django admin turned up a lot of issues. IMHO part of the DEP should cover how to manage this – whether individual issues warrant individual tickets or not, and how to prioritize the issues if relevant. Which kinds of issues are likely to need design input. If the DEP is done first, also talk about what kind of auditing would be valuable, and how to make it happen.

To help with the initial audit and prioritisation of issues, I would suggest to first start by creating a map of all of the parts of the Django admin that are to be audited – list all of the individual pages, and all of the states the pages can be in (success, error, loading – empty content / some content / paginating, etc). I also find it worthwhile to add a succinct definition of how each of those pages is likely to be used – for example things that are only configured once for a site’s lifetime are presumably not as worth improving as a screen a user would see on a daily basis. Here is an example in practice: <a href="https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit#gid=0" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit#gid\x3d0&#39;;return true;" onclick="this.href=&#39;https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit#gid\x3d0&#39;;return true;">https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit. This can also be used to track the auditing effort, and form the basis of an automated test suite for CI (or simply repeat testing).

Finally, consider <a href="https://www.w3.org/TR/ATAG20/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FTR%2FATAG20%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHBSk1uWS0fY-b9tH1h5P64cd5MrA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FTR%2FATAG20%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHBSk1uWS0fY-b9tH1h5P64cd5MrA&#39;;return true;">ATAG2.0 (Authoring Tool Accessibility Guidelines) compliance additionally to WCAG. At a high level, ATAG is made of two parts:

- Part A: Make the authoring tool user interface accessible. That sounds like what we’re discussing here.
- Part B: Support the production of accessible content. That’s a whole other topic but feels relevant too.

ATAG is nowhere near as well known / established / easy to test for, but it feels very relevant to the Django admin in particular, and I’m sure there will at least be some useful bits to consider in its success criteria

Hope this helps!

Thibaud


On Monday, May 25, 2020 at 11:56:39 AM UTC+1, Tom Carrick wrote:
Hi,

Thanks for the feedback. I had thought about a DEP when I was writing up the original post actually, I just wasn't sure what it should contain. Here are my thoughts, based on the feedback so far:

- Defining a standard to target.
- Forming an a11y team that covers the django admin and all sites in the django github organization, and defining its role, membership, etc.
- Deciding on a CI process
- An outline of current issues and how to solve them.

If anyone can think of anything else, please let me know. If/when there's a consensus I'll start writing a draft.

Mariusz, I mentioned this in the original post, but <a href="https://www.w3.org/WAI/policies/" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2Fpolicies%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHuXyCEAwXzhQM1FYyrOjwonnSQOQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.w3.org%2FWAI%2Fpolicies%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHuXyCEAwXzhQM1FYyrOjwonnSQOQ&#39;;return true;">https://www.w3.org/WAI/policies/ has a good overview of laws and EU directives in this area. From my browsing through, it seems that they all either, have their own national standard, or are using WCAG 2.0/2.1 AA. Since we probably don't want to define our own standard, I think AA is the way to go. This seems to also be the recommendation I hear from people in the accessibility field working on regular websites. AAA seems to be more suited for very specialist sites that explicitly target disabled people.

Most of AAA is completely feasible for us, but there are some reasons it would be difficult:

- All language needs to be at a lower secondary education level, or have an alternative that is. This doesn't seem feasible, for e.g. the documentation.
- Users that are logged out need to be able to resume their session where they left off after logging in, with all form fields filled, etc.

There are others that are difficult, but I don't think any apply to us currently.

Cheers,
Tom


On Mon, 25 May 2020 at 11:09, Mariusz Felisiak <[hidden email]> wrote:
Hi Tobias,

  I'm not a WCAG expert, and it's not clear to me what steps we would like to take. In the ticket we have only steps describes as "very basics which should include", so I can imagine that's not all we need to do to be WCAG 2.1 compliant on AAA, AA or A level. As far as I'm concerned WCAG is quite massive. Moreover if we want to make changes in CI we should discussed available tools, etc. We can change colors but what next? We will be not able to guarantee that the contrast of all elements remains appropriate, we cannot do this manually.

Best,
Mariusz

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="AU4UhgugAQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">django-d...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4baf0cff-e82a-4c7d-b8c0-81f16919ad92o%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Tom Carrick
A quick update first. I'm pretty busy with a combination of day job and personal projects, but I should have time to start writing the draft DEP in the next week or two.

Thibaud, I think the absolute most important thing is a way to measure progress, even if - as others have mentioned - that measure is imperfect. I think getting a proof of concept of Lighthouse running against a few admin pages would be extremely helpful. It's also probably one of the more difficult things. If that seems like too much, I think catching what the CI would miss is the next most important thing, so I think your #1 suggestion would work well.

On Tue, 23 Jun 2020 at 22:34, Thibaud Colas <[hidden email]> wrote:
Hey Tom,

I wanted to check if there is anything we/I could do to help in the meantime? Whether that’s by starting to map or audit the Django admin, or setting up a sample CI pipeline with accessibility tests, or something else altogether. It’s a bit daunting to get started with this but I think I could realistically do one of:

1. Create a draft map of the Django admin UI for later manual auditing purposes.
2. Audi a specific part of Django and creating a ticket with concrete things to fix.
3. Set up static analysis for the CSS or HTML to look for common accessibility gotchas.
4. Sett up automated in-browser accessibility checks on some parts of Django to show what that might look like in a CI pipeline.

… and document the steps along the way, and report back here or as a ticket.

Cheers,

Thibaud

On Tuesday, May 26, 2020 at 10:22:26 AM UTC+1, Tom Carrick wrote:
Some further thoughts...

Mariusz, I don't think I agree that WCAG is massive. It can look a little large but I think that's mostly because each guideline is split into smaller pieces and it's quite wordy to avoid ambiguity. There are in 2.1 50 success criteria for AA, many of which can be checked automatically, and many of those that can't don't apply to us as we don't use audio / video. There's also a nice quick reference that can be filtered to remove AAA and anything else unwanted. Once you drill down it can again get quite wordy, but it covers a lot. I think it's worth remembering here is that the point is not to strictly adhere to a standard for the sake of it, just to improve the overall accessibility, and WCAG is a useful and measurable tool to get us some of the way there.

Thibaud, I more or less agree with everything there. I'm not sure developers should have to do full accessibility checks for their changes in the admin. Using a screen reader for example can be very frustrating and confusing if you're not used to it, and it can take an inordinate amount of time, and I don't really wish that burden on (for example) first time committers who already have a lot of stuff to do, I think this will only discourage contributions. I think it should be the responsibility of the accessibility team to provide feedback, suggestions, etc. on relevant PRs or fix things when they're noticed after manual audits.

ATAG looks handy, but at first glance we couldn't even get to A, it requires auto-saving. It also generally seems like something that would cover a rich text editor plugged into the admin more than the admin itself, though I think it's laudable as a future target.

The more I think about it, I think the DEP should be a process DEP rather than a feature DEP. As others have mentioned, we don't need a DEP to fix accessibility issues - just fix them. But forming a team seems useful, and I think a DEP is required for that. I think the DEP should be limited to:

1. The accessibility team, how membership is decided, who it's accountable to, it's role/responsibilities, etc.
2. Perhaps the initial standard(s) to target, although this could also be decided by the a11y team post-assembly and agreed with the technical board and/or core devs.

As I said, I'm happy to write up the DEP, but since it'll be a process DEP, I think I'd need the support of a core dev, someone on the technical board, or just someone who has more knowledge of Django's "bureaucracy", for lack of a better word.

Tom

On Tue, 26 May 2020 at 01:20, Thibaud Colas <[hidden email]> wrote:
Hi Tom,

It’s exciting to see this getting started! To me a DEP would be highly beneficial, because there is a lot to this that goes beyond finding and fixing individual issues – it’s more about detailing the a process for parts of Django to stay accessible over time. Here are things I’d suggest to cover, in addition to your list:

- Going further than the targeted standard and tests in CI –  outline clear steps developers should take when testing their work: using developer tools, screen readers, etc. It’s not practical to just state the standard as WCAG is quite big, and open to interpretation. And CI testing will never be enough to reach any degree of compliance.
  - I’m not familiar with Django’s development process but generally I would recommend to use a combination of linting, browser extensions, manual testing instructions – and CI tests.
- If the accessibility team was to cover more than the Django admin, I think it would be worthwhile to also include accessibility of sites built with Django. Django is largely unopinionated about markup, but I remember its forms markup not being particularly good for screen readers, and the HTML code snippets in the docs would also be worth reviewing.
- Same goes for admin docs – not too familiar with them myself so I’m not sure whether they are considered to be part of the admin, or separate.
- I wouldn’t find it surprising if an audit of the Django admin turned up a lot of issues. IMHO part of the DEP should cover how to manage this – whether individual issues warrant individual tickets or not, and how to prioritize the issues if relevant. Which kinds of issues are likely to need design input. If the DEP is done first, also talk about what kind of auditing would be valuable, and how to make it happen.

To help with the initial audit and prioritisation of issues, I would suggest to first start by creating a map of all of the parts of the Django admin that are to be audited – list all of the individual pages, and all of the states the pages can be in (success, error, loading – empty content / some content / paginating, etc). I also find it worthwhile to add a succinct definition of how each of those pages is likely to be used – for example things that are only configured once for a site’s lifetime are presumably not as worth improving as a screen a user would see on a daily basis. Here is an example in practice: https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit. This can also be used to track the auditing effort, and form the basis of an automated test suite for CI (or simply repeat testing).

Finally, consider ATAG2.0 (Authoring Tool Accessibility Guidelines) compliance additionally to WCAG. At a high level, ATAG is made of two parts:

- Part A: Make the authoring tool user interface accessible. That sounds like what we’re discussing here.
- Part B: Support the production of accessible content. That’s a whole other topic but feels relevant too.

ATAG is nowhere near as well known / established / easy to test for, but it feels very relevant to the Django admin in particular, and I’m sure there will at least be some useful bits to consider in its success criteria

Hope this helps!

Thibaud


On Monday, May 25, 2020 at 11:56:39 AM UTC+1, Tom Carrick wrote:
Hi,

Thanks for the feedback. I had thought about a DEP when I was writing up the original post actually, I just wasn't sure what it should contain. Here are my thoughts, based on the feedback so far:

- Defining a standard to target.
- Forming an a11y team that covers the django admin and all sites in the django github organization, and defining its role, membership, etc.
- Deciding on a CI process
- An outline of current issues and how to solve them.

If anyone can think of anything else, please let me know. If/when there's a consensus I'll start writing a draft.

Mariusz, I mentioned this in the original post, but https://www.w3.org/WAI/policies/ has a good overview of laws and EU directives in this area. From my browsing through, it seems that they all either, have their own national standard, or are using WCAG 2.0/2.1 AA. Since we probably don't want to define our own standard, I think AA is the way to go. This seems to also be the recommendation I hear from people in the accessibility field working on regular websites. AAA seems to be more suited for very specialist sites that explicitly target disabled people.

Most of AAA is completely feasible for us, but there are some reasons it would be difficult:

- All language needs to be at a lower secondary education level, or have an alternative that is. This doesn't seem feasible, for e.g. the documentation.
- Users that are logged out need to be able to resume their session where they left off after logging in, with all form fields filled, etc.

There are others that are difficult, but I don't think any apply to us currently.

Cheers,
Tom


On Mon, 25 May 2020 at 11:09, Mariusz Felisiak <[hidden email]> wrote:
Hi Tobias,

  I'm not a WCAG expert, and it's not clear to me what steps we would like to take. In the ticket we have only steps describes as "very basics which should include", so I can imagine that's not all we need to do to be WCAG 2.1 compliant on AAA, AA or A level. As far as I'm concerned WCAG is quite massive. Moreover if we want to make changes in CI we should discussed available tools, etc. We can change colors but what next? We will be not able to guarantee that the contrast of all elements remains appropriate, we cannot do this manually.

Best,
Mariusz

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4baf0cff-e82a-4c7d-b8c0-81f16919ad92o%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMYdYUqLu9k1wng7zusW6Qns673E54Z5B28CR1mitGgQNw%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Thibaud Colas
Hi Tom,

Great to hear – no rush if you’re busy with other things. Here’s a quick proof of concept, with 20 different pages/scenarios, tested with Axe, HTML Code Sniffer, and Lighthouse: https://github.com/thibaudcolas/django_admin_tests.

I’m not a big fan of Lighthouse personally, and I already had the rest of the tools set up, hence why I went with this. I spent a bit of time making a report out of the test results, which you can see an example of at https://thibaudcolas.github.io/django_admin_tests/. This is generated in Travis, currently set up to run those tests & regenerate the report from Django’s `master` branch once per week.

I added instructions on the README to run this locally if anyone wants to try it out, and a few words about what to make of the report / issues. I didn’t look into what the issues were though.

Let me know what you think, and I should be able to spend more time on this next week if it helps.

On Wednesday, 24 June 2020 at 09:11:52 UTC+1 [hidden email] wrote:
A quick update first. I'm pretty busy with a combination of day job and personal projects, but I should have time to start writing the draft DEP in the next week or two.

Thibaud, I think the absolute most important thing is a way to measure progress, even if - as others have mentioned - that measure is imperfect. I think getting a proof of concept of Lighthouse running against a few admin pages would be extremely helpful. It's also probably one of the more difficult things. If that seems like too much, I think catching what the CI would miss is the next most important thing, so I think your #1 suggestion would work well.

On Tue, 23 Jun 2020 at 22:34, Thibaud Colas <[hidden email]> wrote:
Hey Tom,

I wanted to check if there is anything we/I could do to help in the meantime? Whether that’s by starting to map or audit the Django admin, or setting up a sample CI pipeline with accessibility tests, or something else altogether. It’s a bit daunting to get started with this but I think I could realistically do one of:

1. Create a draft map of the Django admin UI for later manual auditing purposes.
2. Audi a specific part of Django and creating a ticket with concrete things to fix.
3. Set up static analysis for the CSS or HTML to look for common accessibility gotchas.
4. Sett up automated in-browser accessibility checks on some parts of Django to show what that might look like in a CI pipeline.

… and document the steps along the way, and report back here or as a ticket.

Cheers,

Thibaud

On Tuesday, May 26, 2020 at 10:22:26 AM UTC+1, Tom Carrick wrote:
Some further thoughts...

Mariusz, I don't think I agree that WCAG is massive. It can look a little large but I think that's mostly because each guideline is split into smaller pieces and it's quite wordy to avoid ambiguity. There are in 2.1 50 success criteria for AA, many of which can be checked automatically, and many of those that can't don't apply to us as we don't use audio / video. There's also a nice quick reference that can be filtered to remove AAA and anything else unwanted. Once you drill down it can again get quite wordy, but it covers a lot. I think it's worth remembering here is that the point is not to strictly adhere to a standard for the sake of it, just to improve the overall accessibility, and WCAG is a useful and measurable tool to get us some of the way there.

Thibaud, I more or less agree with everything there. I'm not sure developers should have to do full accessibility checks for their changes in the admin. Using a screen reader for example can be very frustrating and confusing if you're not used to it, and it can take an inordinate amount of time, and I don't really wish that burden on (for example) first time committers who already have a lot of stuff to do, I think this will only discourage contributions. I think it should be the responsibility of the accessibility team to provide feedback, suggestions, etc. on relevant PRs or fix things when they're noticed after manual audits.

ATAG looks handy, but at first glance we couldn't even get to A, it requires auto-saving. It also generally seems like something that would cover a rich text editor plugged into the admin more than the admin itself, though I think it's laudable as a future target.

The more I think about it, I think the DEP should be a process DEP rather than a feature DEP. As others have mentioned, we don't need a DEP to fix accessibility issues - just fix them. But forming a team seems useful, and I think a DEP is required for that. I think the DEP should be limited to:

1. The accessibility team, how membership is decided, who it's accountable to, it's role/responsibilities, etc.
2. Perhaps the initial standard(s) to target, although this could also be decided by the a11y team post-assembly and agreed with the technical board and/or core devs.

As I said, I'm happy to write up the DEP, but since it'll be a process DEP, I think I'd need the support of a core dev, someone on the technical board, or just someone who has more knowledge of Django's "bureaucracy", for lack of a better word.

Tom

On Tue, 26 May 2020 at 01:20, Thibaud Colas <[hidden email]> wrote:
Hi Tom,

It’s exciting to see this getting started! To me a DEP would be highly beneficial, because there is a lot to this that goes beyond finding and fixing individual issues – it’s more about detailing the a process for parts of Django to stay accessible over time. Here are things I’d suggest to cover, in addition to your list:

- Going further than the targeted standard and tests in CI –  outline clear steps developers should take when testing their work: using developer tools, screen readers, etc. It’s not practical to just state the standard as WCAG is quite big, and open to interpretation. And CI testing will never be enough to reach any degree of compliance.
  - I’m not familiar with Django’s development process but generally I would recommend to use a combination of linting, browser extensions, manual testing instructions – and CI tests.
- If the accessibility team was to cover more than the Django admin, I think it would be worthwhile to also include accessibility of sites built with Django. Django is largely unopinionated about markup, but I remember its forms markup not being particularly good for screen readers, and the HTML code snippets in the docs would also be worth reviewing.
- Same goes for admin docs – not too familiar with them myself so I’m not sure whether they are considered to be part of the admin, or separate.
- I wouldn’t find it surprising if an audit of the Django admin turned up a lot of issues. IMHO part of the DEP should cover how to manage this – whether individual issues warrant individual tickets or not, and how to prioritize the issues if relevant. Which kinds of issues are likely to need design input. If the DEP is done first, also talk about what kind of auditing would be valuable, and how to make it happen.

To help with the initial audit and prioritisation of issues, I would suggest to first start by creating a map of all of the parts of the Django admin that are to be audited – list all of the individual pages, and all of the states the pages can be in (success, error, loading – empty content / some content / paginating, etc). I also find it worthwhile to add a succinct definition of how each of those pages is likely to be used – for example things that are only configured once for a site’s lifetime are presumably not as worth improving as a screen a user would see on a daily basis. Here is an example in practice: https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit. This can also be used to track the auditing effort, and form the basis of an automated test suite for CI (or simply repeat testing).

Finally, consider ATAG2.0 (Authoring Tool Accessibility Guidelines) compliance additionally to WCAG. At a high level, ATAG is made of two parts:

- Part A: Make the authoring tool user interface accessible. That sounds like what we’re discussing here.
- Part B: Support the production of accessible content. That’s a whole other topic but feels relevant too.

ATAG is nowhere near as well known / established / easy to test for, but it feels very relevant to the Django admin in particular, and I’m sure there will at least be some useful bits to consider in its success criteria

Hope this helps!

Thibaud


On Monday, May 25, 2020 at 11:56:39 AM UTC+1, Tom Carrick wrote:
Hi,

Thanks for the feedback. I had thought about a DEP when I was writing up the original post actually, I just wasn't sure what it should contain. Here are my thoughts, based on the feedback so far:

- Defining a standard to target.
- Forming an a11y team that covers the django admin and all sites in the django github organization, and defining its role, membership, etc.
- Deciding on a CI process
- An outline of current issues and how to solve them.

If anyone can think of anything else, please let me know. If/when there's a consensus I'll start writing a draft.

Mariusz, I mentioned this in the original post, but https://www.w3.org/WAI/policies/ has a good overview of laws and EU directives in this area. From my browsing through, it seems that they all either, have their own national standard, or are using WCAG 2.0/2.1 AA. Since we probably don't want to define our own standard, I think AA is the way to go. This seems to also be the recommendation I hear from people in the accessibility field working on regular websites. AAA seems to be more suited for very specialist sites that explicitly target disabled people.

Most of AAA is completely feasible for us, but there are some reasons it would be difficult:

- All language needs to be at a lower secondary education level, or have an alternative that is. This doesn't seem feasible, for e.g. the documentation.
- Users that are logged out need to be able to resume their session where they left off after logging in, with all form fields filled, etc.

There are others that are difficult, but I don't think any apply to us currently.

Cheers,
Tom


On Mon, 25 May 2020 at 11:09, Mariusz Felisiak <[hidden email]> wrote:
Hi Tobias,

  I'm not a WCAG expert, and it's not clear to me what steps we would like to take. In the ticket we have only steps describes as "very basics which should include", so I can imagine that's not all we need to do to be WCAG 2.1 compliant on AAA, AA or A level. As far as I'm concerned WCAG is quite massive. Moreover if we want to make changes in CI we should discussed available tools, etc. We can change colors but what next? We will be not able to guarantee that the contrast of all elements remains appropriate, we cannot do this manually.

Best,
Mariusz

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4baf0cff-e82a-4c7d-b8c0-81f16919ad92o%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/125fc1db-6d9f-404c-86aa-2f2eeb75a192n%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Tom Carrick
I've managed to have a quick look, it looks really promising. Having a total number of issues to compare against would be good to avoid regressions or introducing new issues. I do have one concern. To be viable, it needs to live in the django repo somewhere. And we'd need a full admin site to test against, with every admin feature (stacked / tabular inlines, filtering, search, date filters, list editable, autocomplete, ...) to be fully representative and catch everything. This seems like a really big amount of stuff to add into the repo, that will need to be maintained, and possibly an easy thing to forget about. Also the maintenance is tricky, we should ideally not update this site too often or the total number of issues won't be as easily comparable. I wonder if there's another way to handle it, but I can't think of one.

I also imagine this would be pretty intensive to run, it would probably be best to not run it by default and have a buildbot command to run this manually for PRs where there are changes to the admin.

On Thu, 25 Jun 2020 at 02:15, Thibaud Colas <[hidden email]> wrote:
Hi Tom,

Great to hear – no rush if you’re busy with other things. Here’s a quick proof of concept, with 20 different pages/scenarios, tested with Axe, HTML Code Sniffer, and Lighthouse: https://github.com/thibaudcolas/django_admin_tests.

I’m not a big fan of Lighthouse personally, and I already had the rest of the tools set up, hence why I went with this. I spent a bit of time making a report out of the test results, which you can see an example of at https://thibaudcolas.github.io/django_admin_tests/. This is generated in Travis, currently set up to run those tests & regenerate the report from Django’s `master` branch once per week.

I added instructions on the README to run this locally if anyone wants to try it out, and a few words about what to make of the report / issues. I didn’t look into what the issues were though.

Let me know what you think, and I should be able to spend more time on this next week if it helps.

On Wednesday, 24 June 2020 at 09:11:52 UTC+1 [hidden email] wrote:
A quick update first. I'm pretty busy with a combination of day job and personal projects, but I should have time to start writing the draft DEP in the next week or two.

Thibaud, I think the absolute most important thing is a way to measure progress, even if - as others have mentioned - that measure is imperfect. I think getting a proof of concept of Lighthouse running against a few admin pages would be extremely helpful. It's also probably one of the more difficult things. If that seems like too much, I think catching what the CI would miss is the next most important thing, so I think your #1 suggestion would work well.

On Tue, 23 Jun 2020 at 22:34, Thibaud Colas <[hidden email]> wrote:
Hey Tom,

I wanted to check if there is anything we/I could do to help in the meantime? Whether that’s by starting to map or audit the Django admin, or setting up a sample CI pipeline with accessibility tests, or something else altogether. It’s a bit daunting to get started with this but I think I could realistically do one of:

1. Create a draft map of the Django admin UI for later manual auditing purposes.
2. Audi a specific part of Django and creating a ticket with concrete things to fix.
3. Set up static analysis for the CSS or HTML to look for common accessibility gotchas.
4. Sett up automated in-browser accessibility checks on some parts of Django to show what that might look like in a CI pipeline.

… and document the steps along the way, and report back here or as a ticket.

Cheers,

Thibaud

On Tuesday, May 26, 2020 at 10:22:26 AM UTC+1, Tom Carrick wrote:
Some further thoughts...

Mariusz, I don't think I agree that WCAG is massive. It can look a little large but I think that's mostly because each guideline is split into smaller pieces and it's quite wordy to avoid ambiguity. There are in 2.1 50 success criteria for AA, many of which can be checked automatically, and many of those that can't don't apply to us as we don't use audio / video. There's also a nice quick reference that can be filtered to remove AAA and anything else unwanted. Once you drill down it can again get quite wordy, but it covers a lot. I think it's worth remembering here is that the point is not to strictly adhere to a standard for the sake of it, just to improve the overall accessibility, and WCAG is a useful and measurable tool to get us some of the way there.

Thibaud, I more or less agree with everything there. I'm not sure developers should have to do full accessibility checks for their changes in the admin. Using a screen reader for example can be very frustrating and confusing if you're not used to it, and it can take an inordinate amount of time, and I don't really wish that burden on (for example) first time committers who already have a lot of stuff to do, I think this will only discourage contributions. I think it should be the responsibility of the accessibility team to provide feedback, suggestions, etc. on relevant PRs or fix things when they're noticed after manual audits.

ATAG looks handy, but at first glance we couldn't even get to A, it requires auto-saving. It also generally seems like something that would cover a rich text editor plugged into the admin more than the admin itself, though I think it's laudable as a future target.

The more I think about it, I think the DEP should be a process DEP rather than a feature DEP. As others have mentioned, we don't need a DEP to fix accessibility issues - just fix them. But forming a team seems useful, and I think a DEP is required for that. I think the DEP should be limited to:

1. The accessibility team, how membership is decided, who it's accountable to, it's role/responsibilities, etc.
2. Perhaps the initial standard(s) to target, although this could also be decided by the a11y team post-assembly and agreed with the technical board and/or core devs.

As I said, I'm happy to write up the DEP, but since it'll be a process DEP, I think I'd need the support of a core dev, someone on the technical board, or just someone who has more knowledge of Django's "bureaucracy", for lack of a better word.

Tom

On Tue, 26 May 2020 at 01:20, Thibaud Colas <[hidden email]> wrote:
Hi Tom,

It’s exciting to see this getting started! To me a DEP would be highly beneficial, because there is a lot to this that goes beyond finding and fixing individual issues – it’s more about detailing the a process for parts of Django to stay accessible over time. Here are things I’d suggest to cover, in addition to your list:

- Going further than the targeted standard and tests in CI –  outline clear steps developers should take when testing their work: using developer tools, screen readers, etc. It’s not practical to just state the standard as WCAG is quite big, and open to interpretation. And CI testing will never be enough to reach any degree of compliance.
  - I’m not familiar with Django’s development process but generally I would recommend to use a combination of linting, browser extensions, manual testing instructions – and CI tests.
- If the accessibility team was to cover more than the Django admin, I think it would be worthwhile to also include accessibility of sites built with Django. Django is largely unopinionated about markup, but I remember its forms markup not being particularly good for screen readers, and the HTML code snippets in the docs would also be worth reviewing.
- Same goes for admin docs – not too familiar with them myself so I’m not sure whether they are considered to be part of the admin, or separate.
- I wouldn’t find it surprising if an audit of the Django admin turned up a lot of issues. IMHO part of the DEP should cover how to manage this – whether individual issues warrant individual tickets or not, and how to prioritize the issues if relevant. Which kinds of issues are likely to need design input. If the DEP is done first, also talk about what kind of auditing would be valuable, and how to make it happen.

To help with the initial audit and prioritisation of issues, I would suggest to first start by creating a map of all of the parts of the Django admin that are to be audited – list all of the individual pages, and all of the states the pages can be in (success, error, loading – empty content / some content / paginating, etc). I also find it worthwhile to add a succinct definition of how each of those pages is likely to be used – for example things that are only configured once for a site’s lifetime are presumably not as worth improving as a screen a user would see on a daily basis. Here is an example in practice: https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit. This can also be used to track the auditing effort, and form the basis of an automated test suite for CI (or simply repeat testing).

Finally, consider ATAG2.0 (Authoring Tool Accessibility Guidelines) compliance additionally to WCAG. At a high level, ATAG is made of two parts:

- Part A: Make the authoring tool user interface accessible. That sounds like what we’re discussing here.
- Part B: Support the production of accessible content. That’s a whole other topic but feels relevant too.

ATAG is nowhere near as well known / established / easy to test for, but it feels very relevant to the Django admin in particular, and I’m sure there will at least be some useful bits to consider in its success criteria

Hope this helps!

Thibaud


On Monday, May 25, 2020 at 11:56:39 AM UTC+1, Tom Carrick wrote:
Hi,

Thanks for the feedback. I had thought about a DEP when I was writing up the original post actually, I just wasn't sure what it should contain. Here are my thoughts, based on the feedback so far:

- Defining a standard to target.
- Forming an a11y team that covers the django admin and all sites in the django github organization, and defining its role, membership, etc.
- Deciding on a CI process
- An outline of current issues and how to solve them.

If anyone can think of anything else, please let me know. If/when there's a consensus I'll start writing a draft.

Mariusz, I mentioned this in the original post, but https://www.w3.org/WAI/policies/ has a good overview of laws and EU directives in this area. From my browsing through, it seems that they all either, have their own national standard, or are using WCAG 2.0/2.1 AA. Since we probably don't want to define our own standard, I think AA is the way to go. This seems to also be the recommendation I hear from people in the accessibility field working on regular websites. AAA seems to be more suited for very specialist sites that explicitly target disabled people.

Most of AAA is completely feasible for us, but there are some reasons it would be difficult:

- All language needs to be at a lower secondary education level, or have an alternative that is. This doesn't seem feasible, for e.g. the documentation.
- Users that are logged out need to be able to resume their session where they left off after logging in, with all form fields filled, etc.

There are others that are difficult, but I don't think any apply to us currently.

Cheers,
Tom


On Mon, 25 May 2020 at 11:09, Mariusz Felisiak <[hidden email]> wrote:
Hi Tobias,

  I'm not a WCAG expert, and it's not clear to me what steps we would like to take. In the ticket we have only steps describes as "very basics which should include", so I can imagine that's not all we need to do to be WCAG 2.1 compliant on AAA, AA or A level. As far as I'm concerned WCAG is quite massive. Moreover if we want to make changes in CI we should discussed available tools, etc. We can change colors but what next? We will be not able to guarantee that the contrast of all elements remains appropriate, we cannot do this manually.

Best,
Mariusz

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4baf0cff-e82a-4c7d-b8c0-81f16919ad92o%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/125fc1db-6d9f-404c-86aa-2f2eeb75a192n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMba%3DxZt%2Bc%2B-6QjJ0G%3D3HrjVDdG6Jp7utq0KW-PL02D7kA%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Tom Carrick
I've had some time to work on this, I've written up a draft... draft DEP today, and I'd like to request feedback. I've written it quite hastily over the last few hours with the expectation that it will need to be substantially rewritten. I've added it as a local PR on my fork of DEPS to keep the discussion in one place. I'm happy to take broad feedback here, but try to keep specific feedback on the PR itself to keep the noise off this thread. PR: https://github.com/knyghty/deps/pull/1

I'm also requesting someone to shepherd this, and ideally also a co-author, particularly someone on the technical board, a core dev, or someone else with experience of Django's governance, as I think this is the first process DEP written by a non-core dev.

Cheers!
Tom

On Thu, 25 Jun 2020 at 19:31, Tom Carrick <[hidden email]> wrote:
I've managed to have a quick look, it looks really promising. Having a total number of issues to compare against would be good to avoid regressions or introducing new issues. I do have one concern. To be viable, it needs to live in the django repo somewhere. And we'd need a full admin site to test against, with every admin feature (stacked / tabular inlines, filtering, search, date filters, list editable, autocomplete, ...) to be fully representative and catch everything. This seems like a really big amount of stuff to add into the repo, that will need to be maintained, and possibly an easy thing to forget about. Also the maintenance is tricky, we should ideally not update this site too often or the total number of issues won't be as easily comparable. I wonder if there's another way to handle it, but I can't think of one.

I also imagine this would be pretty intensive to run, it would probably be best to not run it by default and have a buildbot command to run this manually for PRs where there are changes to the admin.

On Thu, 25 Jun 2020 at 02:15, Thibaud Colas <[hidden email]> wrote:
Hi Tom,

Great to hear – no rush if you’re busy with other things. Here’s a quick proof of concept, with 20 different pages/scenarios, tested with Axe, HTML Code Sniffer, and Lighthouse: https://github.com/thibaudcolas/django_admin_tests.

I’m not a big fan of Lighthouse personally, and I already had the rest of the tools set up, hence why I went with this. I spent a bit of time making a report out of the test results, which you can see an example of at https://thibaudcolas.github.io/django_admin_tests/. This is generated in Travis, currently set up to run those tests & regenerate the report from Django’s `master` branch once per week.

I added instructions on the README to run this locally if anyone wants to try it out, and a few words about what to make of the report / issues. I didn’t look into what the issues were though.

Let me know what you think, and I should be able to spend more time on this next week if it helps.

On Wednesday, 24 June 2020 at 09:11:52 UTC+1 [hidden email] wrote:
A quick update first. I'm pretty busy with a combination of day job and personal projects, but I should have time to start writing the draft DEP in the next week or two.

Thibaud, I think the absolute most important thing is a way to measure progress, even if - as others have mentioned - that measure is imperfect. I think getting a proof of concept of Lighthouse running against a few admin pages would be extremely helpful. It's also probably one of the more difficult things. If that seems like too much, I think catching what the CI would miss is the next most important thing, so I think your #1 suggestion would work well.

On Tue, 23 Jun 2020 at 22:34, Thibaud Colas <[hidden email]> wrote:
Hey Tom,

I wanted to check if there is anything we/I could do to help in the meantime? Whether that’s by starting to map or audit the Django admin, or setting up a sample CI pipeline with accessibility tests, or something else altogether. It’s a bit daunting to get started with this but I think I could realistically do one of:

1. Create a draft map of the Django admin UI for later manual auditing purposes.
2. Audi a specific part of Django and creating a ticket with concrete things to fix.
3. Set up static analysis for the CSS or HTML to look for common accessibility gotchas.
4. Sett up automated in-browser accessibility checks on some parts of Django to show what that might look like in a CI pipeline.

… and document the steps along the way, and report back here or as a ticket.

Cheers,

Thibaud

On Tuesday, May 26, 2020 at 10:22:26 AM UTC+1, Tom Carrick wrote:
Some further thoughts...

Mariusz, I don't think I agree that WCAG is massive. It can look a little large but I think that's mostly because each guideline is split into smaller pieces and it's quite wordy to avoid ambiguity. There are in 2.1 50 success criteria for AA, many of which can be checked automatically, and many of those that can't don't apply to us as we don't use audio / video. There's also a nice quick reference that can be filtered to remove AAA and anything else unwanted. Once you drill down it can again get quite wordy, but it covers a lot. I think it's worth remembering here is that the point is not to strictly adhere to a standard for the sake of it, just to improve the overall accessibility, and WCAG is a useful and measurable tool to get us some of the way there.

Thibaud, I more or less agree with everything there. I'm not sure developers should have to do full accessibility checks for their changes in the admin. Using a screen reader for example can be very frustrating and confusing if you're not used to it, and it can take an inordinate amount of time, and I don't really wish that burden on (for example) first time committers who already have a lot of stuff to do, I think this will only discourage contributions. I think it should be the responsibility of the accessibility team to provide feedback, suggestions, etc. on relevant PRs or fix things when they're noticed after manual audits.

ATAG looks handy, but at first glance we couldn't even get to A, it requires auto-saving. It also generally seems like something that would cover a rich text editor plugged into the admin more than the admin itself, though I think it's laudable as a future target.

The more I think about it, I think the DEP should be a process DEP rather than a feature DEP. As others have mentioned, we don't need a DEP to fix accessibility issues - just fix them. But forming a team seems useful, and I think a DEP is required for that. I think the DEP should be limited to:

1. The accessibility team, how membership is decided, who it's accountable to, it's role/responsibilities, etc.
2. Perhaps the initial standard(s) to target, although this could also be decided by the a11y team post-assembly and agreed with the technical board and/or core devs.

As I said, I'm happy to write up the DEP, but since it'll be a process DEP, I think I'd need the support of a core dev, someone on the technical board, or just someone who has more knowledge of Django's "bureaucracy", for lack of a better word.

Tom

On Tue, 26 May 2020 at 01:20, Thibaud Colas <[hidden email]> wrote:
Hi Tom,

It’s exciting to see this getting started! To me a DEP would be highly beneficial, because there is a lot to this that goes beyond finding and fixing individual issues – it’s more about detailing the a process for parts of Django to stay accessible over time. Here are things I’d suggest to cover, in addition to your list:

- Going further than the targeted standard and tests in CI –  outline clear steps developers should take when testing their work: using developer tools, screen readers, etc. It’s not practical to just state the standard as WCAG is quite big, and open to interpretation. And CI testing will never be enough to reach any degree of compliance.
  - I’m not familiar with Django’s development process but generally I would recommend to use a combination of linting, browser extensions, manual testing instructions – and CI tests.
- If the accessibility team was to cover more than the Django admin, I think it would be worthwhile to also include accessibility of sites built with Django. Django is largely unopinionated about markup, but I remember its forms markup not being particularly good for screen readers, and the HTML code snippets in the docs would also be worth reviewing.
- Same goes for admin docs – not too familiar with them myself so I’m not sure whether they are considered to be part of the admin, or separate.
- I wouldn’t find it surprising if an audit of the Django admin turned up a lot of issues. IMHO part of the DEP should cover how to manage this – whether individual issues warrant individual tickets or not, and how to prioritize the issues if relevant. Which kinds of issues are likely to need design input. If the DEP is done first, also talk about what kind of auditing would be valuable, and how to make it happen.

To help with the initial audit and prioritisation of issues, I would suggest to first start by creating a map of all of the parts of the Django admin that are to be audited – list all of the individual pages, and all of the states the pages can be in (success, error, loading – empty content / some content / paginating, etc). I also find it worthwhile to add a succinct definition of how each of those pages is likely to be used – for example things that are only configured once for a site’s lifetime are presumably not as worth improving as a screen a user would see on a daily basis. Here is an example in practice: https://docs.google.com/spreadsheets/d/1rTMilzKhcb43Y3fZKGJJeoKWLCR00bfgwVB9f6OaC2U/edit. This can also be used to track the auditing effort, and form the basis of an automated test suite for CI (or simply repeat testing).

Finally, consider ATAG2.0 (Authoring Tool Accessibility Guidelines) compliance additionally to WCAG. At a high level, ATAG is made of two parts:

- Part A: Make the authoring tool user interface accessible. That sounds like what we’re discussing here.
- Part B: Support the production of accessible content. That’s a whole other topic but feels relevant too.

ATAG is nowhere near as well known / established / easy to test for, but it feels very relevant to the Django admin in particular, and I’m sure there will at least be some useful bits to consider in its success criteria

Hope this helps!

Thibaud


On Monday, May 25, 2020 at 11:56:39 AM UTC+1, Tom Carrick wrote:
Hi,

Thanks for the feedback. I had thought about a DEP when I was writing up the original post actually, I just wasn't sure what it should contain. Here are my thoughts, based on the feedback so far:

- Defining a standard to target.
- Forming an a11y team that covers the django admin and all sites in the django github organization, and defining its role, membership, etc.
- Deciding on a CI process
- An outline of current issues and how to solve them.

If anyone can think of anything else, please let me know. If/when there's a consensus I'll start writing a draft.

Mariusz, I mentioned this in the original post, but https://www.w3.org/WAI/policies/ has a good overview of laws and EU directives in this area. From my browsing through, it seems that they all either, have their own national standard, or are using WCAG 2.0/2.1 AA. Since we probably don't want to define our own standard, I think AA is the way to go. This seems to also be the recommendation I hear from people in the accessibility field working on regular websites. AAA seems to be more suited for very specialist sites that explicitly target disabled people.

Most of AAA is completely feasible for us, but there are some reasons it would be difficult:

- All language needs to be at a lower secondary education level, or have an alternative that is. This doesn't seem feasible, for e.g. the documentation.
- Users that are logged out need to be able to resume their session where they left off after logging in, with all form fields filled, etc.

There are others that are difficult, but I don't think any apply to us currently.

Cheers,
Tom


On Mon, 25 May 2020 at 11:09, Mariusz Felisiak <[hidden email]> wrote:
Hi Tobias,

  I'm not a WCAG expert, and it's not clear to me what steps we would like to take. In the ticket we have only steps describes as "very basics which should include", so I can imagine that's not all we need to do to be WCAG 2.1 compliant on AAA, AA or A level. As far as I'm concerned WCAG is quite massive. Moreover if we want to make changes in CI we should discussed available tools, etc. We can change colors but what next? We will be not able to guarantee that the contrast of all elements remains appropriate, we cannot do this manually.

Best,
Mariusz

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/be97c4e7-f961-4d46-998f-693ca6076f09%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/148d658c-2062-4973-ac6c-9f1abd5e95e3%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4baf0cff-e82a-4c7d-b8c0-81f16919ad92o%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/125fc1db-6d9f-404c-86aa-2f2eeb75a192n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMbUD6w4Dj8tGDbBR2o4Xv2-L6m7GT15%3DZ3vsz54Yw0AMg%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Tobias Bengfort
Nice writeup! I found it easy to read (I did not catch myself skipping
paragraphs which is always a good sign). Contentwise, I would have no
issue if this was accepted as is. Maybe I am forgetting about some
important details though.

I had just some ideas that might be good additions:

- mention ATAG somewhere

- It would be nice to have a real commitment to accessibility. Something
along the lines of "accessibility bugs must be treated with the same
priority as any other bugs".

- The step from "leaving accessibility in the hands of
individual contributors" to "you have to commit for 9 months" is a tad
big. I keep wondering if we can do something to improve the options in
between those. One idea would be to formalize an "a11y" keyword so
interested contributors can easily find related tickets.

tobias

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/9b194f07-8d00-0540-109a-190ee950d92a%40posteo.de.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Thibaud Colas
Update for the proof of concept CI tests from my side – thank you Tom for the feedback. Here are the latest additions to the test suite & reports, still living at https://thibaudcolas.github.io/django_admin_tests/,

- Added as much as I know about in the admin, and now also outside of it a bit (startproject welcome page, error pages)
- Separated the issues between Axe and HTML_CS so the numbers are easier to understand
- Added anchor links everywhere for easier navigation
- I’ve also started a draft list of "things to (potentially) audit", over at https://github.com/thibaudcolas/django_admin_tests#scope-for-future-audits

I think the next two big steps are what you mention:

- Having a way to track the number of issues over time. Currently the report overwrites itself every week (well, every build). If you have a suggestion on ways to demo this that would be useful please let me know. Currently I’m thinking sparklines for each test case could be nice as a proof of concept, and a sparkline for the total number of issues. Or see whether I should get out of my comfort zone a bit and find a dashboard/graphing tool to send the metrics to and graph there.
- Testing more features of modeladmin. I don’t use it too frequently myself so don’t really know what’s “easy” to enable – if you know of an existing test suite I could repurpose, or of an example of using a lot of functionality – I’d be keen to invest time to add it to my test site.

Alternatively something else I could do is to file a ticket for accessibility issues with the welcome page – I’ve tested it with screen readers, there are a few issues, but nothing that should be too hard to fix, and it might be a good demo of what reporting accessibility issues in general could look like?

Cheers,

Thibaud


On Tuesday, 30 June 2020 at 18:59:53 UTC+1 Tobias Bengfort wrote:
Nice writeup! I found it easy to read (I did not catch myself skipping
paragraphs which is always a good sign). Contentwise, I would have no
issue if this was accepted as is. Maybe I am forgetting about some
important details though.

I had just some ideas that might be good additions:

- mention ATAG somewhere

- It would be nice to have a real commitment to accessibility. Something
along the lines of "accessibility bugs must be treated with the same
priority as any other bugs".

- The step from "leaving accessibility in the hands of
individual contributors" to "you have to commit for 9 months" is a tad
big. I keep wondering if we can do something to improve the options in
between those. One idea would be to formalize an "a11y" keyword so
interested contributors can easily find related tickets.

tobias

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e65e3879-d74c-4401-9232-29eb0a73385cn%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Admin accessibility

Tom Carrick
I've added a PR to the DEPs repo to hopefully get some eyes on it: https://github.com/django/deps/pull/69

Thibaud, I think whatever you have the time and motivation for sounds good, all of those things are useful. If you're not sure about all the admin features, I think that's pretty normal. One project I've had on my mind for a while now is to build a simple django site that is essentially just there to use every feature of the admin, so I might bump that up the priority list, though it's somewhat daunting.

Cheers,
Tom

On Mon, 13 Jul 2020 at 01:15, Thibaud Colas <[hidden email]> wrote:
Update for the proof of concept CI tests from my side – thank you Tom for the feedback. Here are the latest additions to the test suite & reports, still living at https://thibaudcolas.github.io/django_admin_tests/,

- Added as much as I know about in the admin, and now also outside of it a bit (startproject welcome page, error pages)
- Separated the issues between Axe and HTML_CS so the numbers are easier to understand
- Added anchor links everywhere for easier navigation
- I’ve also started a draft list of "things to (potentially) audit", over at https://github.com/thibaudcolas/django_admin_tests#scope-for-future-audits

I think the next two big steps are what you mention:

- Having a way to track the number of issues over time. Currently the report overwrites itself every week (well, every build). If you have a suggestion on ways to demo this that would be useful please let me know. Currently I’m thinking sparklines for each test case could be nice as a proof of concept, and a sparkline for the total number of issues. Or see whether I should get out of my comfort zone a bit and find a dashboard/graphing tool to send the metrics to and graph there.
- Testing more features of modeladmin. I don’t use it too frequently myself so don’t really know what’s “easy” to enable – if you know of an existing test suite I could repurpose, or of an example of using a lot of functionality – I’d be keen to invest time to add it to my test site.

Alternatively something else I could do is to file a ticket for accessibility issues with the welcome page – I’ve tested it with screen readers, there are a few issues, but nothing that should be too hard to fix, and it might be a good demo of what reporting accessibility issues in general could look like?

Cheers,

Thibaud


On Tuesday, 30 June 2020 at 18:59:53 UTC+1 Tobias Bengfort wrote:
Nice writeup! I found it easy to read (I did not catch myself skipping
paragraphs which is always a good sign). Contentwise, I would have no
issue if this was accepted as is. Maybe I am forgetting about some
important details though.

I had just some ideas that might be good additions:

- mention ATAG somewhere

- It would be nice to have a real commitment to accessibility. Something
along the lines of "accessibility bugs must be treated with the same
priority as any other bugs".

- The step from "leaving accessibility in the hands of
individual contributors" to "you have to commit for 9 months" is a tad
big. I keep wondering if we can do something to improve the options in
between those. One idea would be to formalize an "a11y" keyword so
interested contributors can easily find related tickets.

tobias

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/e65e3879-d74c-4401-9232-29eb0a73385cn%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHoz%3DMZ_%2BR%2BMWxeJtsQt1%3DW-9hT-wS7Nk7ALUKnozDaRKzQFbw%40mail.gmail.com.
12