Discuss the different types of reliability and validity that can be measured in survey research, specifically: convergent validity, discriminant validity, construct validity, test-retest reliability, and internal consistency reliability. What can researchers do to improve upon these different types of reliability and validity in a survey?
Inter-Rater or Inter-Observer Reliability is used to assess the degree to which different raters/observers give consistent estimates of the same phenomenon. Test-Retest Reliability is used to assess the consistency of a measure from one time to another. Parallel-Forms Reliability is used to assess the consistency of the results of two tests constructed in the same way from the same content domain. Internal Consistency Reliability is used to assess the consistency of results across items within a test.
Construct ...view middle of the document...
Discriminant validity occurs where constructs that are expected not to relate do not, such that it is possible to discriminate between these constructs.
Convergence and discrimination are often demonstrated by correlation of the measures used within constructs. Convergent validity and Discriminant validity together demonstrate construct validity.
Content validity occurs when the experiment provides adequate coverage of the subject being studied. This includes measuring the right things as well as having an adequate sample. Content validity is related very closely to good experimental design. A high content validity question covers more of what is sought.
Internal validity occurs when it can be concluded that there is a causal relationship between the variables being studied. A danger is that changes might be caused by other factors.
Conclusion validity occurs when you can conclude that there is a relationship of some kind between the two variables being examined.
External validity occurs when the causal relationship discovered can be generalized to other people, times and contexts.
This examines the ability of the measure to predict a variable that is designated as a criterion. A criterion may well be an externally-defined 'gold standard'. Achieving this level of validity thus makes results more credible. Criterion-related validity is related to external validity.
This measures the extent to which a future level of a variable can be predicted from a current measurement. This includes correlation with measurements made with different instruments.
This measures the relationship between measures made with existing tests. The existing tests is thus the criterion.
Face validity occurs where something appears to be valid. This of course depends very much on the judgment of the observer. In any case, it is never sufficient and requires more solid validity to enable acceptable conclusions to be drawn.