100 Days of A11y
Day 81: Manual vs. Automated A11y Testing Tools
Published on
Today I went into my study time with the intent to list out pros and cons of automated versus manual accessibility testing. Instead I walked away with a comparison of what each had to offer, and understanding that both are valuable when used cooperatively during website and web app development.
Things I accomplished
Permalink for "Things I accomplished"Submitted my request to take the Web Accessibility Specialist certification exam in early April via private proctor.
Read:
- Testing Guidance for Developers
- What is accessibility testing?
- Manual and Automated Web Accessibility Testing
- The Importance of Manual Accessibility Testing (so much packed into one article!)
Created a comparison table to jot down ideas about manual and automated testing (see under What I learned today).
What I learned today
Permalink for "What I learned today"| Manual Testing | Automated Testing |
|---|---|
| Slower process | Faster process |
| Mostly accurate | Sometimes accurate |
| Easier to miss a link | Guaranteed check of all links |
| Identifies proper state of elements | Automated user input can miss state |
| Page by Page | Site-wide |
| Assurance of conformance | Misleading in assurance of conformance |
| Guidance for alternative solutions | Yes/No (boolean) checks and solutions |
| Human and software | Software |
| Context | Patterns |
| Finds actual problems | Lists potential problems |
| Appropriate HTML semantics | HTML validation |
| Accurate alt text | Existence of alt attribute |
| Heading hierarchy | Headings exist |
| Follows intention of usability | Follows WCAG success criteria |
| Test is/isn't readable | Programmatic color contrast |
| Exploratory | Automated |
| Part of the testing process | Part of the testing process |
| Appropriate use of ARIA | Presence and validity of ARIA |
| In real life | Hypothetical |
| Identifies granular challenges of usability | Quickly identifies low-hanging fruit and repeated offenders |
In conclusion
Permalink for "In conclusion"Deciding on testing methods and tools shouldn't be an either-or mandate. Each has their strengths and weaknesses. Using both methods should be a part of every testing process. Why not strengthen your product's usability by incorporating tools from each methodology into your process?