Summary
Complete summary Psychodiagnostics, final course of the bachelor
- Course
- Institution
This is a complete English summary of all literature and lectures for the course psychodiagnostics
[Show more]
Seller
Followveracreemers
Reviews received
Content preview
SummaryPsychodiagnostics 2021
1: Test your memory
Learning goals:
1. What is reliability?
a. What are ways to measure this?
2. What are the different theories mentioned in the course manual?
3. What makes a test important? When is it adequate?
4. What is the reliability in the case of individual diagnostics?
What is reliability?
The primary source for this learning goal is the chapter in Gregory, additional information can come from the
articles.
Reliability refers to the attribute of consistency in measurement and can best be viewed as a continuum
ranging from minimal consistency of measurement to a near-perfect repeatability of results. To
understand the concept of reliability a basic understanding of Classical Test Theory is required, however,
this will be further explained under the relevant learning goal. Errors in measurement reduce the
reliability or repeatability of psychological test results. According to Classical Test Theory (CTT) test scores
result form the influence of 2 factors:
1. Factors that contribute to consistency
2. Factors that contribute to inconsistency
X (the obtained score)= T (the true score) + e ( errors of measurement), e = X-T. test developers strive to
minimize nuisance through careful attention to the sources of measurement error because a true score is
never really known.
Sources of measurement error
Item selection: the instrument itself is a source of measurement error because the developer
needs to settle on a finite nr of items from a potentially infinite pool of questions.
o The chosen set of questions might not be equally fair to all persons
Test administration: the circumstances of administration can result in measurement errors;
uncomfortable temperature, lighting, noise, as well as personal aspects like anxiety, motivation,
attention and fatigue
Test scoring: any test that has a format other than machine-scored multiple-choice items requires
some form of judgment
unsystematic: their effects are unpredictable and inconsistent
Systematic measurement error: arises when a test consistently measures something other than
the trait for which it was intended
,Taking into account these sources our equation looks like this: X= T= es (systematic errors) +eu
(unsystematic errors). Systematic errors constitute a significant problem in the development of
psychological tests, but by using proper test development procedures this can be minimized.
Measurement error and reliability
Reliability bears a precise statistical relationship to measurement error and CTT makes a few assumptions:
1. Unsystematic measurement errors are random, and thus their mean is 0
a. Errors are equally likely to be positive or negative, averaging them out to 0 across a
group of subjects
2. True scores and errors are uncorrelated rTe=0
a. If not they would be systematic
3. Errors on different tests are uncorrelated, r12=0
These assumptions have important implications for reliability and measurement and are based on the
optimistic assumption that systematic errors are minimal.
CTT adds value because it permits to partition the variance of obtained scores into 2 separate sources; the
variance of true scores and the variance of errors of measurement:
σ 2X =σ 2T + σ 2e
The reliability coefficient
Reliability expresses the relative influence of true and error scores on obtained test scores, the reliability
coefficient (rXX) is the ratio of true score variance to the total variance of test scores:
2 2
σT σT
r XX = 2 or r XX = 2 2
σX σT σ e
If the measurement error is very small reliability coefficient approaches a value of 1
If the measurement error is very large reliability coefficient approaches a value of 0
In the literal sense, rXX indicates the proportion of variance in obtained test scores that is accounted for by
the variability in true scores. There is also an additional interpretation, rXX is an index of the potential or
actual consistency of obtained scores.
What are ways to measure reliability?
The correlation coefficient
A correlation coefficient, r, expresses the degree of linear relationship between 2 sets of
scores obtained from the same person. R can range from -1 to +1 and is independent of
the mean.
The correlation coefficient as a reliability coefficient
We can use the correlation coefficient to gauge the consistency of psychological test
scores; if results are highly consistent, the scores of persons taking the test on 2 separate occasions
should be strongly correlated. In this context, the correlation coefficient is a reliability coefficient, because
it reflects the proportion of variance in obtained tests cores accounted for by the variability in true scores.
This is called the test-retest approach/ reliability and entails administering an instrument twice to the
same group of persons and computing the correlation between the 2.
,Different methods of measuring reliability can be divided into 2 groups; temporal stability and internal
consistency. Test-retest reliability is a temporal stability method.
Reliability as temporal stability
Test-retest: the most straightforward method for determining reliability
o We can expect subjects to score somewhat higher the 2nd time they do a test because of
practice, maturation, schooling or other effects and this is especially so for ability and
achievement tests
Alternate-forms reliability: test developers can produce 2 forms of the same test which are
independently constructed to meet the same specifications
o Estimates of this type of reliability are derived from administering both forms to the
same group and correlation the 2 sets of scores
o Intervening effects can also occur here, reducing the reliability estimate somewhat
A fundamental difference with test-retest is that the alternate-forms introduces item-sampling
differences as an additional source of error variance. Some test takers might be better or worse on one
form because of the particular items. It is also more expensive.
Reliability as internal consistency
Split-half reliability: by correlating the pairs of scores obtained from equivalent halves of a test
that are administered only once to a sample, we obtain an estimate of the split-half reliability.
o Supplementary to the Golden Standard of test-Retest, but tends to yield higher estimates
o Reasons for using it include; logistical problems or costs that render it impractical to
obtain a 2nd set of scores, test-rest can be misleading in certain cases, when there are
large and variable practice effects the rank order of scores from a 2nd administration will
at best sustain only a modest association to the rank order of some scores from the 1st
administration, if the trait being measured is known to fluctuate rapidly test-retest can
yield misleadingly low estimates
o A major challenge is dividing the test into 2 nearly equivalent halves, but the most
common method is comparing scores on the odd items versus those on the even items
o A split-half reliability needs to adjusted using the Spearman-Brown formula because of
the 2 tests
Coefficient Alpha: Cronbach’s coefficient alpha may be thought of as the mean of all possible
split-half coefficients corrected by the Spearman-Brown formula
∑σj 2
o rα= ( N
N−1)(1− 2 )
σ
o Rα is the coefficient alpha, N the nr of items, σ 2j the variance of 1 item and ∑ σ 2j is the
sum of the variances of all items, σ 2 is the variance of the total test scores
o Coefficient alpha can vary between 0 and 1 and is an index of the internal consistency of
items their tendency to correlate positively with one another.
o Used to be thought of as an index of unidimensionality, but it rather is an index of
interrelatedness and that is not synonymous with unidimensionality. It is possible for a
scale to measure 2 or more distinct factors and still possess a very strong coefficient
alpha
The Kuder-Richardson estimate of reliability: coefficient alpha is the general application of a more
specific formula by Kuder and Richardson. It is known as the Kuder-Richardson 20 (KR-20) and is
relevant to the special case in which each test item is dichotomous (0 or 1)
, ∑ pq )
o
( N
N−1 )
(1−
σ
2
P is the proportion of examinees getting each item correct and q is the proportion of
o
examinees getting each item wrong
o Coefficient alpha extends KR-20 to tests with items that are not dichotomous
Interscorer reliability: this involves a sample of tests being independent scored by 2 or more
examiners where scores for pairs of examiners are correlated
o It supplements but does not replace other reliability estimates
Spearman-Brown Formula:
The split-half reliability gives us an estimate of reliability for an instrument half as long as the full test and
shorter tests are generally less reliable than longer tests. The Pearson r between 2 halves of a test will
therefore usually underestimate the reliability of the full instrument Spearman-Brown formulae offers
the appropriate adjustment for this:
2 r hh
r SB=
1+ r hh
RSB is the estimated reliability of the full test and rhh is the half-test reliability. Instead of giving a single
coefficient for the test, the procedure gives different coefficients depending on which items are grouped
when the test is split. Coefficient alpha proposes a more typical value by being a mean of the split-half
coefficients.
Which reliability is appropriate?
For tests designed to be administered to individuals more than once, a test-retest reliability is reasonable.
Tests that purport to factorial purity can benefit from using coefficient alpha, but measures like those of
general intelligence, that have factorial complexity, do not. split-half methods work well for instruments
that have items carefully ordered according to difficulty level and interscorer reliability is appropriate for
any test that involves subjectivity of scoring.
method Sources of error variance
Test-retest Changes over time
Alternate-forms (immediate) Item sampling
Alternate-forms (delayed) Item sampling
Changes over time
Split-half Item sampling
Nature of split
Coefficient alpha Item sampling
Test heterogeneity
interscorer Scorer difference
Special circumstances in the estimation of reliability
Unstable characteristics: if test-retest method is used they should be nearly instantaneously in
order to provide an accurate index of reliability
Speed and power tests: a speed test contains items of uniform and generally simple levels of
difficulty and has restrictive time limit that guarantees few subjects complete the entire test, thus
the score largely represents speed of performance. A power test allows enough time to attempt
all items but is constructed so that no test taker is able to obtain a perfect score
o Split-half reliability yields spuriously high reliability coefficients for speed tests should
be based on test-retest or split-half from 2 separately timed half tests
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller veracreemers. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $10.73. You're not tied to anything after your purchase.
Can Stuvia be trusted?
4.6 stars on Google & Trustpilot (+1000 reviews)
77333 documents were sold in the last 30 days
Founded in 2010, the go-to place to buy study notes for 14 years now