Educational effectiveness
Examen: 12/20 punten (25 juni)
Open boek examen (3-5 open vragen) → 5-10 examenvragen worden gegeven in de les
o Vraag 1: je krijgt de resultaten van een studie en je moet je opinie hierover schrijven
o Vraag 2: schrijf een advies voor de minister van onderwijs
Paper: 8/20 punten (15 juni): schrijf een korte academische review over een EER onderwerp
Onderwerpen: Effectiveness criteria, Scientific properties of school effects, Process characteristics,
Methodological issues
o Lijst op Toledo/vrij keuze → Voorbeeld papers op Toledo
3-5 pagina’s lang
Web of Science!
Inhoud: verbondenheid met EER, probleemstelling/onderzoeksvraag, samenvatten van resultaten
(antwoord op onderzoeksvraag), implicaties voor praktijk, beleid, onderzoek, kritische reflectie
General research-questions
What makes a “good” school?
Are there differences between schools in student outcomes?
How large are the differences between schools?
Why do students have more learning gains in school A than in school B? = explaining differences
How can we measure the effectiveness of schools? Effectiveness of teachers? → fair and a valid way?
o Standardized test → do we want to compare teachers?
Is an effective school effective for all subjects? For all students?
Inhoud
1. Introduction to Educational Effectiveness Research (EER)..............................................................................................3
1.1. History of EER..........................................................................................................................................................3
1.2. Educational effectiveness (scheerens, 2004)...........................................................................................................4
2. Five factor model (Edmonds’ Big Five)............................................................................................................................4
3. Model of school learning (Carroll)..................................................................................................................................4
4. Comprehensive model of educational effectiveness......................................................................................................5
5. Integrated multilevel model of education......................................................................................................................5
5.1. Methodology in EER................................................................................................................................................5
5.2. Scientific properties of school effects......................................................................................................................6
6. School effects: Value added models...............................................................................................................................8
6.1. Multilevel structures (= the basic concepts)............................................................................................................8
6.2. Multilevel analysis: Variance decomposition, empty models and value-added models..........................................8
6.3. From an empty multilevel model to a value-added multilevel model.....................................................................9
7. Empty multilevel model (Model 0): model without covariates.....................................................................................10
8. Value added model(s) (model1, model 2): model with covariates...............................................................................11
8.1. Four types of value-added models........................................................................................................................13
8.2. Interpreting the results of a multilevel analysis: an example.................................................................................14
9. International and comparative studies: part 1..............................................................................................................17
9.1. Today’s objectives..................................................................................................................................................17
9.2. Which is the most effective educational system/country?....................................................................................17
9.3. Criticism.................................................................................................................................................................17
9.4. Why international studies......................................................................................................................................22
,10. International and comparative studies: part 2............................................................................................................22
10.1. Text reader: Immigrant children’s educational achievement in Western countries: Origin, destination, and
community effects on mathematical performance. (2008)..........................................................................................22
10.2. Text reader: Educational quasi-markets, school effectiveness and social inequalities.........................................26
11. Absolute versus relative school effects (rankings)......................................................................................................27
12. Online practicum multilevel analyses: exercises.........................................................................................................27
12.1. Exercise 1: variance composition.........................................................................................................................27
12.2. Exercise 2: PIRLS repeat 2018: variance composition and interpreting a multilevel outcome.............................28
12.3. Exercise 3: System level.......................................................................................................................................29
13. Evidence-based education..........................................................................................................................................30
13.1. Evidence based education...................................................................................................................................30
13.2. Why evidence based education?.........................................................................................................................30
13.3. Example 1: successful for all................................................................................................................................30
13.4. Example 2: Grade retention.................................................................................................................................31
13.5. Side-effects of interventions................................................................................................................................31
13.6. Evidence-based education impossible?...............................................................................................................31
13.7. Problems with experimental research.................................................................................................................32
13.8. Evidence-based practice and policy.....................................................................................................................32
13.9. What is essential in education?...........................................................................................................................32
13.10. Monitoring/ data use.........................................................................................................................................33
14. Criticism on EER + Q&A...............................................................................................................................................33
14.1. Methodological criticisms....................................................................................................................................33
14.2. Theoretical criticism............................................................................................................................................33
14.3. Political criticism..................................................................................................................................................34
,1. Introduction to Educational Effectiveness Research (EER)
Tekst uit reader: Educational effectiveness research (EER): a state-of-the-art review. = important
NOT: What makes a “good” school/teacher/system? → effective and less effective schools instead!
o NOT: Are we doing the right things? → teaching right things/ teach the right content
o WEL: Are we doing the things right? → to develop certain competences
Educational quality: EER is 1 perspective to look at this
Educational effectiveness: how process characteristics seen in student outcomes
o We need 2 things for this:
(1) Effectiveness criterion = always student outcomes! Ex. cognitive (average score of math)
Easy to measure and easy to compare
A way to say something about a school/teacher → more/less effective
(2) Effective enhancing conditions
Why there are differences between teachers/schools → Interest in processes
NOT input characteristics: School can’t change: low SES, girls score better on reading
WEL: Process characteristics: elements they can change: instruction methods
1.1. History of EER
Response to Coleman et al. (1966) and Jencks et al. (1972)
Before: School matters! → important wich school you go to
’60-’70
They say: “Schools cannot compensate for society”
o Chances are dependent on the family you are born in
Multilevel models / scientific properties / differential effects
Caused a lot of reaction → EER was born as a reaction
‘80 They want to show that SCHOOLS MATTERS
Differential effects: if a school is effective: is it effective for
every student are specific for some students?
Open the black box of education → related to OV → entering
the right variables
Input/output characteristics → input/process/output
‘90 o Focus on school process: evaluation, policy, …
Importance of class/teacher level
o Focus on school before, here focus on teacher
Large-scale longitudinal studies (LOSO, SiBO, … )
o New types of studies: not 1 measurement anymore
o You can see what the schools adds to the student
> ‘00
International comparative research (PISA, TIMSS, PIRLS, …)
Methodological advances (multivariate ML, SEM,…)
Dynamic nature of education
Examples of an input on the exam!
o Regression coefficients → effect sizes → does the school matter here? → NO
o First model 1: you see student characteristics: all correlated to results of students
Girls score .20 better than boys (references group: here males are not written down)
Matters in which family where you are born (high or low SES) → determined your chances
R²: 20% of differences in achievement scores are explained by students characteristics
o Then model 2: add also school characteristics
Average school SES had a big effect
Student and school characteristics together: not much more explained
o But they are looking at the wrong school characteristics → those are not the effective processes
The teaching matters! → but not inserted in the study
, They did single-level-analysis: they did not take into account that students not all students are
withing the same schools → schools can differ from each others!
o ALL THE ESTIMATES ARE WRONG IN THIS STUDY
1.2. Educational effectiveness (scheerens, 2004)
The most important model
Association between:
o (1) Effectiveness enhancing conditions of schooling
Process characteristics
Important: these characteristics schools can change
Input characteristics (= not manageable): gender, home language, age student/teacher, SES
But important: they influence the process and the outcomes (direct an indirectly)
They explain also differences between students and schools
They need to be taken into account: control for input characteristics
Then you can investigate the true effective!
Context: lager characteristics that have an influence: school ranking, rural settings, …
o (2) effectiveness criterion
Always the students: their outcomes
→ CIPO Model: break open the black box
Effectiveness criteria Theoretical models
1. School results / test results Edmonds (1979)
2. School career (grade repeating, drop-out, …) o Five factor model
3. Metacognitive skills / self-regulated learning Carroll (1963)
4. Student well-being o Model of school learning
5. Motivation / engagement Scheerens (1990)
6. Selfesteem / academic self-concept o CIPO-model
7. Values / attitudes Creemers (1994)
8. Social skills o Comprehensive model of educational
9. Behaviour (truancy, …) effectiveness
10. Long term results (succes in labour market, …) Scheerens (2015):
o Multilevel model of education
11. Learning ga ins
Creemers & Kyriakides (2006):
12. Equity = differences between different types of
o Dynamic model of educational effectiveness
students → high when the gap is small
= the right side of the CIPO model
2. Five factor model (Edmonds’ Big Five)
5 characteristics that are necessary to make you school more effective (= this is confirmed)
o 1. Strong educational leadership
o 2. Emphasis on basic skills acquisition
o 3. Safe and orderly learning climate
o 4. High expectations of pupils’ achievement
o 5. Frequent monitoring of pupils’ progress
3. Model of school learning (Carroll)
Main concept is time (time spend and time need for learning
Achievement is function of 2 things (active and required time)
o Required time: some students need more/less time
Should be as low as possible
Function of 3 things:
Attitude: how good in a course
General ability
Quality of instruction = interest in EER
o High = required time to learn is lower than
Les avantages d'acheter des résumés chez Stuvia:
Qualité garantie par les avis des clients
Les clients de Stuvia ont évalués plus de 700 000 résumés. C'est comme ça que vous savez que vous achetez les meilleurs documents.
L’achat facile et rapide
Vous pouvez payer rapidement avec iDeal, carte de crédit ou Stuvia-crédit pour les résumés. Il n'y a pas d'adhésion nécessaire.
Focus sur l’essentiel
Vos camarades écrivent eux-mêmes les notes d’étude, c’est pourquoi les documents sont toujours fiables et à jour. Cela garantit que vous arrivez rapidement au coeur du matériel.
Foire aux questions
Qu'est-ce que j'obtiens en achetant ce document ?
Vous obtenez un PDF, disponible immédiatement après votre achat. Le document acheté est accessible à tout moment, n'importe où et indéfiniment via votre profil.
Garantie de remboursement : comment ça marche ?
Notre garantie de satisfaction garantit que vous trouverez toujours un document d'étude qui vous convient. Vous remplissez un formulaire et notre équipe du service client s'occupe du reste.
Auprès de qui est-ce que j'achète ce résumé ?
Stuvia est une place de marché. Alors, vous n'achetez donc pas ce document chez nous, mais auprès du vendeur EmmaP. Stuvia facilite les paiements au vendeur.
Est-ce que j'aurai un abonnement?
Non, vous n'achetez ce résumé que pour €4,69. Vous n'êtes lié à rien après votre achat.