Methods of Communication Research
Table of contents
1. Basics of communication research ...................................................................................1
1.1 The discipline of communication research ..................................................................... 1
1.1.1 Communication research as interdisciplinary research .............................................. 2
1.1.2 Quantitative vs. Qualitative? ..................................................................................... 3
1.2 Research needs strategic choices ................................................................................. 3
2. Literature review in Communication Science ....................................................................4
2.1 Why do we need literature reviews? ............................................................................... 4
2.2 The types of literature reviews ....................................................................................... 5
2.2.1 Narrative review ....................................................................................................... 5
2.2.2 Systematic review..................................................................................................... 5
2.2.3 Meta-analysis ........................................................................................................... 9
2.3 Summary: Literature Reviews ...................................................................................... 11
3. Experiments in Communication Science ......................................................................... 12
3.1 Types of experiments in communication science ......................................................... 12
3.2 Terminology ............................................................................................................... 13
3.2.1 Independent Variable ............................................................................................. 13
3.2.2 Dependent Variable ................................................................................................ 13
3.3 Experimental Designs ................................................................................................. 14
3.3.1 Classic experimental design / Pure experimental design........................................... 14
3.3.2 Quasi-experimental design ..................................................................................... 14
3.4 Important design terms ............................................................................................... 15
3.4.1 Distinction between ‘Factors’ & ‘Levels’ .................................................................. 15
3.4.2 Interaction effects .................................................................................................. 16
3.4.3 Manipulation check ................................................................................................ 16
3.5 Experiment: step-by-step ............................................................................................ 17
3.6 Important statistical terms .......................................................................................... 17
3.6.1 Statistical power..................................................................................................... 19
, 3.7 Three obligatory articles on experimental designs ........................................................ 19
3.7.1 Experiment on goal priming ..................................................................................... 19
3.7.2 Experiment on virtual reality .................................................................................... 20
3.7.3 Experiment on framing & priming ............................................................................. 22
3.8 Experiments: conclusion............................................................................................. 23
4. Survey Research in Communication Science................................................................... 24
4.1 Method of data collection ........................................................................................... 26
4.2 Risks of survey research.............................................................................................. 27
4.2.1 Self-Reports: How the questions shape the answers ................................................ 27
4.2.2 Surveys do not (always) correlate with (more) objective measures ............................ 28
4.2.3 Types of questions .................................................................................................. 30
4.3 The obligatory articles on survey research.................................................................... 30
4.3.1 Survey research on app-addoption .......................................................................... 31
4.3.2 Survey research on social media ............................................................................. 33
4.3.3 Survey research about misinformation .................................................................... 34
4.3.4 Conclusion of the three articles............................................................................... 35
4.4 Tips for online survey research .................................................................................... 35
4.4.1 Randomization of concepts & items ........................................................................ 35
4.4.2 Take the participants on a ‘questionnaire-journey’ ................................................... 36
4.4.3 Try to avoid: ............................................................................................................ 36
5. Quantitative Content Analysis in Communication Science .............................................. 37
5.1 Content...................................................................................................................... 37
5.1.1 Content: two key areas ........................................................................................... 37
5.1.2 Content: three key structures .................................................................................. 37
5.1.3 Content: how to decide on what content? ................................................................ 38
5.1.4 Content: how to get a dataset? ................................................................................ 38
5.2 Analysis ..................................................................................................................... 39
5.2.1 Analysis: how to get results from content analysis data? .......................................... 39
5.3 The obligatory article on content analysis .................................................................... 39
5.4 Guest lecture on quantitative content analysis............................................................. 41
,6. Introduction to Qualitative Communication Research into the Nexus of Media, Culture, and
Society ................................................................................................................................. 45
6.1 Doing Research into the Nexus of Media, Culture, and Society ...................................... 46
6.1.1 Society ................................................................................................................... 46
6.1.2 Media..................................................................................................................... 46
6.1.3 Culture................................................................................................................... 48
6.1.4 Qualitative communication research as a response to the media effects-model ....... 51
6.2 The Foundations of Doing Qualitative Communication Research .................................. 52
6.2.1 The ONTOLOGY of qualitative communication research ........................................... 54
6.2.2 The EPISTEMOLOGY of qualitative communication research .................................... 54
6.2.3 SUMMARY: Foundations of Qualitative Communication Research ............................ 56
6.3 Formatting your Research Field in Qualitative Communication Research ...................... 57
6.3.1 What, why, how ...................................................................................................... 58
6.3.2 Unit of analysis & sampling ..................................................................................... 58
7. Researching Mediated Content and Fictions ................................................................... 62
7.1 Objectivist forms of content analysis ........................................................................... 62
7.1.1 Quantitative content analysis .................................................................................. 64
7.1.2 Qualitative content analysis .................................................................................... 65
7.2 Constructivist forms of content analysis in media and cultural studies ......................... 68
7.2.1 Fact and fiction....................................................................................................... 68
7.2.2 Representation ....................................................................................................... 69
7.2.3 Why constructivist analysis of mediated fictions? .................................................... 71
7.2.4 Genre..................................................................................................................... 71
7.2.5 Textual analysis in media and cultural studies.......................................................... 72
8. (Critical) Discourse Analysis ........................................................................................... 78
8.1 What is (critical) discourse analysis? ........................................................................... 79
8.1.1 Unpacking the term of discourse analysis ................................................................ 79
8.1.2 Discourse............................................................................................................... 80
8.1.3 Discourse analysis has specific areas of interest: .................................................... 82
8.2 Critical discourse analysis (CDA)................................................................................. 82
8.3 Key concepts in CDA................................................................................................... 85
, 8.3.1 Key concepts in CDA: POWER ................................................................................. 85
8.3.2 Key concepts in CDA: IDEOLOGY ............................................................................ 88
8.3.3 Key concepts in CDA: ARTICULATION ...................................................................... 88
8.4 Doing CDA step-by-step.............................................................................................. 90
Step 1: Choice of the research problem ................................................................................ 90
Step 2: Formulating research questions ................................................................................ 90
Step 3: Data & sampling ....................................................................................................... 90
Step 4: Transcription of the data ........................................................................................... 91
Step 5: Analysis of the data .................................................................................................. 91
Step 6: Data & sampling ....................................................................................................... 92
9. Researching people’s engagements with media, technology, and cultures ...................... 93
9.1 Approaches to studying media engagement ................................................................. 94
9.2 Audiences as Active Agents......................................................................................... 95
9.2.1 Uses & Gratifications .............................................................................................. 96
9.2.2 Interpreting & decoding (mass) media texts ............................................................. 97
9.2.3 Contexts of Reception & Use ..................................................................................100
10. Techniques in automated content analysis: Topics, entities and sentiment ............... 104
10.1 Automated content analysis.......................................................................................104
10.2 Approaches to automated content analysis ................................................................105
10.3 The possibilities demonstrated trough a hypothetical use case....................................108
10.3.1 Manual approach ..............................................................................................108
10.3.2 Automatic approach ........................................................................................108
10.4 Example of research .................................................................................................113
10.5 Takeaways ................................................................................................................114
,1. Basics of communication research
“A certain look comes over the faces of some of my students when they hear the word research. Their eyes glaze over, and their
faces take on a pained expression as if they had a migraine or a bad stomach-ache. They see the required course on research as
some kind of an ordeal they must survive before being allowed to take the courses they want and go on to live a normal life.”
-> methods courses aren’t usually the favourite courses of students
-> but it doesn’t have to be like that!
-> research is about unleashing the inner child in you, let yourself be driven by curiosity
-> what do you want to know about society? About the role of media in society? …
-> good researchers have 1 thing in common: curiosity
➢ Is everyone a researcher?
-> Research (ENG) /chercher (Fr) → to search, to seek
=> Research means looking for information about something
-> You research everyday (without realising)
o What phone do I buy?
o What is healthy to eat?
o Is this Netflix series for me?
o What is this conflict about?
➢ BUT scientific research ≠ everyday research
o More systematic -> there are procedures
o Based more on facts than feelings
o More cautious -> need to be careful when claiming certainty about a certain fact
o More focus on accuracy & veracity (waarachtigheid)
o Are data, numbers & statistics always part of scientific research? → Not necessarily
(historical research within literary studies <-> econometrics (which is only about statistics & numbers))
1.1 The discipline of communication research
- = a young discipline -> started +- 50 years ago
- = an interdisciplinary field
-> we borrow concepts, theories, methods,… from other fields in the social sciences, but also from
psychology, the humanities (because media = part of society) & even the natural sciences
(eg. use of psychophysiological metrics in the social lab -> tools that measure eg. biological
responses to eg. media content)
- The true start of the discipline: a response to the increasing important role of print and broadcast mass
communication media in everyday life
(cf. the impact of newspapers on society -> they are deeply ingrained in the changing of societies & their
structures) (cf. the enormous impact of propaganda)
- With the development of digital media, the field is more central than ever to the political, economic and
cultural developments in societies and the world
1
,1.1.1 Communication research as interdisciplinary research
→ Two major currents within communication science research (see Deuze, 2021)
1. Social science oriented study of media and communication (1e deel gegeven door prof de Bruijn)
-> Link with disciplines that analyse, understand & predict human behavior such as sociology,
psychology, political science ...
2. Humanities-oriented study of media & communication (2e deel gegeven door prof De Ridder)
-> Link with disciplines that analyse & understand products of human culture such as language, art
history literature, history, philosophy, culture, etc.
No strict separation, lots of interdisciplinary research
1.1.1.1 Social science-oriented study of media and communication
- Does a lot of media impact research, media facts driven research
-> connects to the field of strategic communication (eg. test if a campaign will actually work)
- Examines influence & effects of media and communication on behavior of people & society
(positive and/or negative)
- Usually bases their analyses on the concepts, ideas & theories of social scientists
& (experimental) psychology
➔ Often quantitative research
Eg. research questions: What are the negative effects of media?
How does social media affect how we communicate with each other?
1.1.1.2 Humanities-oriented study of media and communication
- Connected to fields such as media studies, cultural studies & the study of popular culture
- Examines everything from art house cinema and fiction to comics, television, movies, music, digital
media & everyday life (eg. Instagram content,…)
- Usually bases their analyses on the concepts, ideas & theories of philosophers, psychologists, social
scientists, linguists & others
➔ Often qualitative research
Eg. research questions: How is cultural diversity represented in Harry Potter movies?
(focusses on the texts of media products themselves & connects that RQ to broader questions about power in society)
The 2nd part of this course, on qualitative methods will be
about 3 interrelated concepts (media, culture & society)
→ how can we do research on them?
-> the relation between media & culture (= studied in cultural media studies)
-> the relationship between media & society (= studied in media sociology)
=> the broad link between these 2 = the school of media studies
-> we will talk less about the relationship between culture & society (because that is another field) (= studied in cultural studies)
2
,1.1.2 Quantitative vs. Qualitative?
➢ “Quantitative research” in media & communications → we think of numbers, effect & measurement
→ Quantitative researchers = sometimes accused of being too limited, basing their research on what
they can count, measure & observe -> neglecting other things
➢ “Qualitative research” → we think of media texts, interpretations & ethnographic research
→ Qualitative researchers = often accused of "reading" all sorts of things in texts / making
interpretations of how people consume media that are not there
-> Interpretations often seem very personal (not driven by objective numbers)
& too much evaluative/value-driven
! we need to think beyond
previous stereotypes
1.2 Research needs strategic choices
→ Different research methods (qualitative / quantitative / combinations)
≠ inherently "better" / "truer"
→ they lend themselves to different research questions & observations
-> if you want to choose a method, you always start from a RQ
→ Determining which research method = part of a strategic negotiation
-> you need to think about what you precisely want to know & what kind of method do I need for that?
-> eg. What method do you use when you are interested in mass media representations & how the
mass media shape social ideas about sexual identity based on these representations?
→ probably a longitudinal content / qualitative textual analysis
eg. Which method do you use when you are interested in the effectiveness of communication
campaigns to promote healthy behavior?
→ probably some kind of experiment (because it is an effect-driven question)
→ How do you read a research article?
-> Questions of interest in evaluating these strategic choices made by researchers
1. What methodology was used in the research, what are the strategic choices & reasoning?
2. How important is the topic & does it constitute a challenging scientific investigation?
3. What conclusions were drawn?
4. Are the conclusions supported by the data? Are the conclusions credible?
5. Can one generalize from the research?
6. Can these findings lead to practical recommendations?
3
,Part 1: Quantitative Methods
➢ Deals with methods with which you can quantify
-> Quantify: putting things into numbers
➢ Assumption is that you can put all things in numbers
o How often something occurs (e.g. what is the amount of misinformation on Facebook)
o How things are related
▪ Correlation: more misinformation leads to less public trust
▪ Percentage: 80% chance that a fear appeal will make public smoke less
o What is more effective?
▪ Means & standard deviations: Do social robots at home lead to less loneliness?
➢ Four distinct methods in Part 1
o Literature review
o Experiments
o Survey
o Quantitative content analysis
2. Literature review in Communication Science
2.1 Why do we need literature reviews?
➢ They generate an overview of current knowledge regarding a communication science topic for…
• …Knowledge: what is known about a topic at this moment?
• …Discussion: how to proceed with future research on the topic?
• …Policy: e.g. How to deal with misinformation? How do we provide guidelines for patients? Etc
➢ It is important to know that the method exists & what you can do with it
-> Very likely you will encounter a literature review during your time at UA
➢ You summarize a lot of studies & pull them together => a lot of participants
➢ For any topic, you can probably find a meta-analysis
Example:
- How much behavior change (in %) occurs after seeing a mass-mediated campaign?
-> 168.362 participants
=> Effects of campaign exposure on
Alcohol: r = .09 -> 09*.09 = 0.008% of behavior change for alcohol can be explained by campaign exposure
Smoking: r = .05 -> .05*.05 = 0.003% “…” smoking can be explained by campaign exposure
Sexual health: r = .04 -> .04*.04 = 0.002% “…” sexual health can be explained by campaign exposure
-> by multiplying a correlation (r) by itself, you get the amount of explained variance
→ in this lecture, we will look at the way we can get to such a conclusion
4
,2.2 The types of literature reviews
2.2.1 Narrative review
➢ A non-systematic approach to literature reviews regarding
• Search strategies (what databases / search terms / … are used)
and/or
• The selection process (which criteria were used to include / exclude a study)
-> you normally start from a lot of articles and then systematically narrow the selection<
<-> narrative reviews just randomly select
Different researchers will conclude different things despite reviewing the same topic
➢ Results
• Very little quantitative information -> unclear
~> abacus at most (you just count, the most simple version of quantitative analysis)
➢ There are a lot of downsides of narrative reviews
• Sensitive to the biases of the researcher (biased towards the opinion of the researcher)
• Little to no transparency about the review process
Not replicable by other researchers
Example:
2.2.2 Systematic review
Systematic reviews follow a systematic approach
=> if person A does the research in a systematic manner, person B should find exactly the same things
- The search process = systematic
o Databases (PubMed,…) use specific terms -> they have key-words that everybody has to use
-> Bv PubMed: so-called MESH headings
(a comprehensive controlled vocabulary for the purpose of indexing journal articles)
-> terms like for example Exercise
(<-> itt het vb-artikel v narrative reviews: zij gebruiken de zoekterm Fitbit, maar wanneer je deze
term in de database v PubMed zoekt, vind je niets → unknown term => no systematic search)
5
, - Reporting = systematic (eg. IMRAD / PRISMA structure)
-> there is a standardized way of reporting
(all systematic reviews ‘look the same’)
-> researchers need to check a checklist before publishing article
-> you need to adhere to the ‘rules’
(eg. is there a risk of bias? Is there additional analysis?
Study characteristics?...)
- Criteria for selection = systematic
-> Inclusion vs exclusion
-> authors are very specific about which articles they found
& which articles they included/excluded AND WHY
2.2.2.1 Example of a systematic review
➢ About CDSS = Clinical Decision Support System
-> A computer system that supports / alerts clinicians when they have to take a clinical decision
(For instance changing medications or treatment options)
➢ From previous research we know that clinicians disregard 50-95% of CDSS alerts
➢ Research question of the review:
What are the barriers and facilitators of CDSS- acceptance, as reported by clinicians?
→ This RQ was the starting point from the systematic review
➢ Systematic search in collaboration with ‘information specialist’
= person whose full-time job it is to make sure researchers can have the most adequate access
to clinical knowledge from research studies
-> knows which standard search terms are used in the databases (e.g. MESH headings, such as
“e-prescription”), knows relevant free search terms & provides recommendations for search terms
(so you do not end up with too much, but also not too few hits)
➢ Registration of review protocol in Prospero → enhances transparency
• Preregistration of entire process! (what are we going to do?)
• You need to be open & transparent about what you are going to do, before you are doing it
➢ Determine inclusion & exclusion criteria
-> Before you start the review, you say ‘these studies will be included, these will be excluded’
-> What are the requirements studies should have to be included in the review?
e.g. Study population: patients; young adults; elderly; etc…; Study methodology: objective vs
subjective measures; research designs (experiments only?); etc
6
