Improving Content Validity Evaluation of Assessment Instruments Through Formal Content Validity Analysis

Content validity is defined as the degree to which elements of an assessment instrument are relevant to and representative of the target construct. The available methods for content validity evaluation typically focus on the extent to which a set of items are relevant to the target construct, but do not afford precise evaluation of items’ behavior, nor their exhaustiveness with respect to the elements of the target construct. Formal content validity analysis (FCVA) is a new procedure combining methods and techniques from various areas of psychological assessment, such as (a) constructing Boolean classification matrices to formalize relationships among an assessment instrument’s items and target construct elements, and (b) computing interrater agreement indices. We discuss how FCVA can be extended through the implementation of a Bayesian procedure to improve the interrater agreement indices’ accuracy (Bayesian formal content validity analysis [B-FCVA]). With respect to extant methods, FCVA and B-FCVA can provide a great amount of information about content validity while not demanding much more work for authors and experts.