Day 81: Manual vs. Automated A11y Testing Tools

Today I went into my study time with the intent to list out pros and cons of automated versus manual accessibility testing. Instead I walked away with a comparison of what each had to offer, and understanding that both are valuable when used cooperatively during website and web app development.

Things I accomplished

Submitted my request to take the Web Accessibility Specialist certification exam in early April via private proctor.

Read:

Created a comparison table to jot down ideas about manual and automated testing (see under What I learned today).

What I learned today

Manual Testing Automated Testing
Slower process Faster process
Mostly accurate Sometimes accurate
Easier to miss a link Guaranteed check of all links
Identifies proper state of elements Automated user input can miss state
Page by Page Site-wide
Assurance of conformance Misleading in assurance of conformance
Guidance for alternative solutions Yes/No (boolean) checks and solutions
Human and software Software
Context Patterns
Finds actual problems Lists potential problems
Appropriate HTML semantics HTML validation
Accurate alt text Existence of alt attribute
Heading hierarchy Headings exist
Follows intention of usability Follows WCAG success criteria
Test is/isn’t readable Programmatic color contrast
Exploratory Automated
Part of the testing process Part of the testing process
Appropriate use of ARIA Presence and validity of ARIA
In real life Hypothetical
Identifies granular challenges of usability Quickly identifies low-hanging fruit and repeated offenders

In conclusion

Deciding on testing methods and tools shouldn’t be an either-or mandate. Each has their strengths and weaknesses. Using both methods should be a part of every testing process. Why not strengthen your product’s usability by incorporating tools from each methodology into your process?