EDP 591/COE 501:
Introduction to Research & Evaluation in Education
Sarah
K Brem
sarah.brem@asu.edu
Payne
308L
fax:
480-965-0300
voice
msg: 480-965-7191
Objectives
--Become
familiar with the most widely used and discussed methods of educational
research and evaluation.
--Learn
to critique published research/evaluations using these methods, and to design
studies using these methods.
--Begin
reflecting on the role that research and evaluation might play in your future!
Materials
There
is a book available at the ASU bookstore for this class. I will not be
assigning readings from it, or lecturing directly from it, but you may find it
useful if you are the sort of learner who likes to have a textbook for
reference and/or a different perspective. It is very concise and relatively
inexpensive, so if you think you might benefit from having a reference text, I
recommend you go ahead and buy it.
All
required materials are available online; follow the links below. If you have
any problems accessing the materials, let me know IMMEDIATELY! “I couldn’t
download the readings” is not a valid excuse for not being ready for class,
unless I announce to the entire class that there is a problem that cannot be resolved
in a reasonable amount of time (e.g., the servers go down for several days). If
this does happen, I will provide an alternate plan ASAP.
Assignments
For
each section of the course (see calendar), you need to write 1 or 2 papers of
approximately 3 pages each. You’ll create a research question that is
appropriate to the method (how to do this is something we’ll work on!), and
provide the following:
1. The question the research or
evaluation addresses, in one sentence.
NOTE: This is a strict limit! The
reasons will become clear (I hope!) in time. In the beginning, just keep in
mind that (a) the study must be kept small and tight if you’re going to
describe it fully in a short paper, (b) that even larger studies are made up of
a set of tight, well-formulated questions, and (c) boiling the question down to
one sentence requires you to consider each element very carefully.
2. A thorough description of the
study itself, including all of the elements needed for that type of study.
NOTE:
This will vary depending on the methodology, and we’ll talk about
what’s needed for each sort of study as we go. I can’t provide a master list,
but it will be clear when we get there.
3. A critique of the study from the
perspective of a harsh critic.
NOTE:
The problems raised by your critic should not be the result of a poor
design, but should address inherent limitations of the methodology, or the
resources available for the study. For example, if your experiment lacks proper
controls, this is a poor design; if your correlational study fails to show
causality, this is an inherent limitation of the methodology. If the problem is
one of design, you should fix it!
4. Your response to the critique: an
acknowledgement of the limitation, and a justification as to why the study is
nevertheless valuable.
The
exact length of each section (except #1) isn’t set; it will depend on the
question and the methodology. When 2 papers are required, two different
methodologies must be used; the questions and studies can be closely related,
but they will inevitably be different, since different methodologies involve
different questions and designs. Likewise, you can address related questions in
the course of the term.
Scoring
The
assignments are scored on the following scale:
0.5 or
1: A response that has serious problems or errors that undermine the research
or evaluation; the study cannot answer the question posed (because there is a
problem with the question, a problem with the study, or both), or cannot be
adequately justified to critics.
1.5 or
2.0: A good response; some significant problems or errors. The study would have
to be revised, but there are no issues that would require completely rethinking
the study.
2.5 or
3.0: An excellent response with, at most, a few trivial errors (e.g., misspellings).
The study is worthwhile, the question is clear and succinct, and the study is
designed to answer that particular question.
The
questions are entirely up to you—they can be about any aspect of educational
research and/or evaluation. They can be extensions of existing studies or
entirely new; if they are related to other coursework, your job, or other
aspects of your everyday life—great! The only real way to determine whether a
question is appropriate is to: (a) ask yourself whether it’s about “education”
(loosely defined to include job training, traditional classrooms, museum-going,
watching Sesame Street, learning to hangglide on your days off, etc.), and (b)
determine whether you can write about it in the way described above.
In-class Review
The
week before the papers are due, we will have in-class peer review. You do not
need to have a complete draft of each paper, but you should have at least an
outline of each paper. You do not need to handout your outlines or drafts; be
prepared to give a succinct description (about 10 minutes/paper). The class
will break up into small groups, and each group member will present their two
ideas for papers to their group, and receive feedback.
I
never take attendance, but you should make every effort to attend all of the
review sessions, not only because it will be of benefit to you, but also as a
service to others in the class.
Calendar
|
18-Jan-05 |
Introduction |
NONE |
|
|
|
25-Jan-05 |
Principles
of Measurement |
|
||
|
1-Feb-05 |
Experimental
Approaches |
Example
1: Emotion
& Reasoning Example
2: Emotion
& Logic |
|
|
|
8-Feb-05 |
Experimental
Approaches No new slides |
Example
1: Stereotype
Threat Example
2: Working
in Groups |
|
|
|
15-Feb-05 |
Design
& Quasi Experiments |
Note added 2-13-05: Don’t worry
if you can’t access the Example 1 article. The journal website is apparently
down. Prepare the others for class as usual. Example
1: Transfer Example
2: Career
Counseling Example
3: Share
the Parenting |
|
|
|
22-Feb-05 |
In
class peer review |
|
||
|
1-Mar-05 |
Survey
Studies |
Example
1: Effects
of 9-11 Example
2: Attitudes
about Animal Research |
Papers 1 and 2 due |
|
|
8-Mar-05 |
Using
Existing Databases |
Example
1: Ability
and Expertise Example
2: Evaluating
the Effects of NCLB |
|
|
|
15-Mar-05 |
SPRING
BREAK |
|
||
|
22-Mar-05 |
NO
CLASS; Use this time for online peer review |
|
||
|
29-Mar-05 |
Qualitative:
Grounded Theory |
Example
1: Urban
Education Example
2: Consequence
of High-Stakes Testing |
Paper 3 due—extended to 5 April due to server crash |
|
|
5-Apr-05 |
Qualitative:
Cultural Analysis & Standpoint Methodologies |
Example
1: Themes
of Control & Domination Example
2: Postcolonial
Hawai’i |
|
|
|
12-Apr-05 |
NO
CLASS |
|
||
|
18-Apr-05 |
Qualitative:
Activity Theory |
Definitions of
Activity Theory Example
1: Studying
Diversity Example
2: Improving
Educational Outcomes for Impoverished Youth |
|
|
|
25-Apr-05 |
Qualitative:
Arts-Based Research |
Example
1: Educational
Leadership |
|
|
|
2-May-05 |
In
class peer review |
|
||
|
9-May-05 |
|
Papers 4 and 5 due |
||