5
reasons, direct observation by an independent observer of children's behavior and their interactions with relevant individuals in their environment is considered to provide the most objective description of target behaviors, such as noncompliance to parental commands and the effectiveness of parents' responses (McMahon & Forehand, 1988).
Although behavioral observations were once seen as inherently objective and valid, it is now agreed that there are measurement issues related to aspects of observational methodology that can lead to confounds in the data collected, thereby generating invalid results (Hops, Davis, & Longoria, 1995). A basic requirement of an observational system is that it be both reliable and valid. Furthermore, a system's utility is enhanced if it provides normative data and is clinically practical. Reliability
Reliability traditionally refers to consistency in measurement, or the extent to
which a person's score, using the same assessment device, remains constant under varying conditions (Anastasi, 1988). From this perspective, agreement between two observers, identically trained using the same coding system and observing the same situation, can be viewed as a measure of consistency.
Numerous methods of assessing interobserver agreement have been developed over the past 50 years. Only a subset of methods, however, dominate the behavioral literature (Hops, Davis, & Longoria, 1995). One of the most easily calculated indices involves computing the percent of agreements out of the total number of agreements and disagreements. Unfortunately, because this method does not control for chance levels of agreement between observers, it is regarded by some as the least desirable of the reliability