AUTOMATED SEMANTIC ALIGNMENT ASSESSMENT FOR WEB ACCESSIBILITY USING LARGE LANGUAGE MODELS

Abstract

Automated verification of web content accessibility remains an acute problem, as traditional tools are capable of fully testing only approximately 4 out of 50 Web Content Accessibility Guidelines (WCAG) criteria. One of the criteria that is difficult to verify is WCAG 2.5.3, which requires semantic correspondence between the visible text of an element and its accessible name for users of assistive technologies. The objective of this study is to verify the feasibility of using large language models for automated assessment of semantic correspondence according to WCAG 2.5.3 criterion and to determine optimal models by price-quality ratio. A comparative analysis of 17 large language models across different price categories was conducted on a specially created dataset in English and Ukrainian languages. To measure model quality relative to the consensus of leading models, a statistical framework based on consensus assessment was employed with metrics including bias, deviation variance, and coefficient of determination The leading models demonstrated a high level of consistency in semantic similarity assessments (R² = 0.85-0.91). Mid-range price segment models showed the best quality-to-cost ratio, notably gemini-2.5-flash-preview achieved the highest consistency (R² = 0.91) with minimal noise. The absence of direct correlation between syntactic correctness of responses and the quality of semantic analysis was established. Large language models can be effectively utilized for assessment of WCAG 2.5.3 semantic correspondence. Optimal models for practical application have been identified and directions for further research have been outlined, including knowledge distillation into smaller specialized models to reduce computational costs.