Y học - Chapter 15: Measurement in research

Chapter 15 Measurement in ResearchEssential Characteristics of DataCollected data must have certain characteristics in order for them to be considered acceptable evidence upon which to base conclusionsObjectivityReliabilityValidityEstablish before collecting research dataObjectivityThe degree to which multiple scorers agree on the values of collected measures or scoresa.k.a rater reliabilityImportant because data should not be dependent on who collected itReliabilityConsistency of test scoresScores should reflect a true indication of one’s ability and not change over a short period of timeReliability of test scores is a necessary condition for validityValidityThe degree to which interpretations of test scores or measures derived from a measuring instrument lead to correct interpretationsMost important characteristic of dataEnables researcher to have confidence that conclusions are based on correct informationEstablishing Data CharacteristicsBefore collecting research dataDocumented by other researchersPilot study – situation specific documentationUniqueness of setting and participation characteristicsSample size does not need to be largeTechnique generally involves calculating a correlation coefficientDetermining ObjectivityTwo or more scorers or raters should independently score a group of people on the testIntraclass correlation coefficient (R) is calculated as an estimate of the objectivity coefficientANOVA techniqueSelect appropriate computational formula for RDetermining ReliabilityTo determine reliability, each person must have at least two repeated scores, such as. . .Scores on multiple trials of a test within a dayScores from multiple administrations on different daysScores from multiple items on a testScores on two different forms of a testTypes of Relative ReliabilityInternal consistency reliabilityConsistency of scores within a dayStability reliabilityConsistency of scores across daysStatistical TechniquesIntraclass correlation coefficient (R) ANOVA techniqueSelect appropriate computational formula for RCronbach’s AlphaKappa coefficientTechnique of determining reliability of dichotomously scored testsDetermining ValiditySeveral types of validity evidenceLogical evidenceCriterion evidenceConstruct evidenceOther evidenceLogical EvidenceBased on the knowledge that a test measures an attribute and yields scores that can be validly interpretedInsufficient as the only type of validity evidenceCombine with other forms of evidenceSubjective judgmentJury of expertsCriterion EvidenceBased on a correlation coefficient between scores on a test and scores on a criterion measure or standardTraditional type of validity evidenceSelection of suitable criterion measure often challengingConstruct EvidenceA construct is something known to exist although it may not be precisely defined or measuredExperts maintain that all validity evidence is actually construct validity evidenceEvidence that scores are interpreted correctlyGroup differences approachOther Measurement IssuesEconomy of measuresEase of administrationPreparation of participantsScoring directionsCharacteristics of participantsExperience of testerOther