Worked example: screen-time & focus — a mixed-methods study, start to finish

This page is a single, fully worked research project that threads together every method in the course: framing a testable question, operationalizing fuzzy constructs, drawing a representative sample, writing a clean questionnaire, designing a quasi-experiment, and analyzing a mock dataset with real formulas — then auditing it for bias, confounding and ethics. Treat it as a template for the group field study (Sessions 26–29) and a model of what "doing it properly" looks like.

The scenario. A study team at a university wants to know whether heavy smartphone use is associated with worse sustained attention in first-year students, and whether a simple "notifications-off" intervention measurably improves focus. They run a correlational survey phase and a small quasi-experiment, then combine the qualitative and quantitative findings.

Sessions exercised

Each phase below maps to specific course sessions. The tags throughout the page link back to the relevant session and to the matching interactive demo.

1 · Research question, constructs & hypotheses

A good study starts from one sharp question and a handful of falsifiable predictions — not a vague topic. Compare the textbook progression from a "topic" to a testable hypothesis.

Hypotheses

2 · Design: sampling, instrument & experiment

2.1 Sampling plan

The target population is first-year undergraduates at one university (≈ 1,200 students). A census is infeasible, so we sample. We use stratified random sampling to guarantee the sample mirrors the population on a variable likely tied to the outcome (degree area).

• Frame: the registrar's first-year enrolment list (closest available list to the true population).
• Strata: degree area (STEM / business / humanities), since baseline study habits may differ.
• Within strata: simple random sampling, allocation proportional to stratum size.
• Target size: $n=180$ (see margin-of-error calc below); over-recruit to $n=210$ for ~15% attrition.

2.2 The questionnaire

A short instrument with mixed item types. Note the deliberate use of neutral wording, a reverse-scored item to catch straight-lining, and a behavioral anchor (the OS screenshot) so we are not relying on self-report alone.

2.3 The (quasi-)experimental design

Q2 needs a manipulation. Ideally we randomly assign participants to condition, making it a true experiment; if assignment must respect existing tutorial groups, it becomes a quasi-experiment with intact groups (weaker causal claim). We use a pretest–posttest two-group design.

• Independent variable: notification condition (off vs. unchanged) — manipulated.
• Dependent variable: change in attention score $\Delta = O_2-O_1$.
• Control of confounds: randomization (true experiment) breaks the link between condition and lurking variables; single-blind scoring; identical task instructions; same time-of-day testing.
• Why pretest–posttest: using $\Delta$ removes stable individual differences in baseline attention.

3 · Data collection plan & mock analysis

3.1 Descriptive statistics (correlational wave, n = 180)

Always describe before you infer. Means, standard deviations and the shape of each variable.

3.2 Correlation: screen-time vs. attention

# 3.2 — correlation + significance (Python / SciPy)
import numpy as np
from scipy import stats

r, p_two = stats.pearsonr(screen_time, attention)
p_one    = p_two / 2            # one-sided: predicted negative
print(f"r = {r:.2f},  r^2 = {r**2:.2f},  one-sided p = {p_one:.4f}")
# r = -0.34,  r^2 = 0.12,  one-sided p = 0.0000

3.3 Two-group test: did the intervention work?

We compare the attention change $\Delta=O_2-O_1$ between treatment and control with an independent-samples (Welch) $t$-test, and report a standardized effect size.

# 3.3 — Welch t-test + Cohen's d
t, p_two = stats.ttest_ind(delta_T, delta_C, equal_var=False)
sp = np.sqrt((delta_T.var(ddof=1) + delta_C.var(ddof=1)) / 2)
d  = (delta_T.mean() - delta_C.mean()) / sp
print(f"t = {t:.2f},  one-sided p = {p_two/2:.4f},  d = {d:.2f}")
# t = 3.08,  one-sided p = 0.0014,  d = 0.65

3.4 Results summary

4 · Threats to validity, bias & ethics

A finding is only as trustworthy as the design that produced it. Here we audit our own study against the standard threats — the kind of critique the final exam asks you to perform.

5 · Mapping to learning outcomes

How each part of this project demonstrates a course learning objective.

6 · Extensions & variations

• Add a factor: turn Q2 into a $2\times2$ factorial — notifications (off/on) × study environment (silent/social) — and look for an interaction (Sessions 12–14).
• Go within-subjects: a crossover design where each participant does both conditions in random order increases power and controls for individual differences.
• Deepen the qualitative arm: run focus groups (Sessions 24–25) on Q8 responses and code themes, then triangulate against the quantitative results.
• Interrupted time series: track attention daily across a phone-free week for a single cohort (quasi-experimental, Session 20) to see the trajectory, not just pre/post.
• Robustness: replace Pearson $r$ with Spearman $\rho$ if attention scores are skewed; bootstrap the confidence interval for $d$.

7 · References & further reading

1. Privitera, G. J. Research Methods for the Behavioral Sciences. SAGE. — Ch. 5 (Sampling), Ch. 8 (Correlational Designs), Ch. 10 (Between-Subjects), Ch. 11 (Quasi-Experimental), Ch. 13 (Descriptive Statistics).
2. American Psychological Association. Publication Manual of the APA (7th ed.). — report structure & statistics reporting.
3. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). — effect-size conventions ($d$, $r$).
4. Field, A. Discovering Statistics. SAGE. — Pearson $r$, $t$-tests, partial correlation, Cronbach's $\alpha$.
5. Course syllabus — Learning to Observe, Experiment and Survey (PDF).

All numbers on this page are illustrative mock data created to demonstrate the analysis workflow — not results from a real study.