The E-learning Curve at Edublogs

A e-learning blog focusing on user performance, enhancing skills, knowledge, and organizational development using digitally-mediated learning.

Creating a Research Design to evaluate the effectiveness of e-learning projects

March 31st, 2008 · No Comments
assessment · case study · e-learning · evaluate learning · learning outcomes · non-formal learning · program logic model · research methodologies · research methods · training evaluation model · triangulation

In a previous blog entry, I mentioned the importance of data triangulation, particularly if you use a case study approach when evaluating an e-learning initiative. Another strategy I recommend is to collect data over a number of phases, using a number of techniques.

Case studies involve in-depth, descriptive data collection and analysis of a case, or a number of related instances of the same type within the case. In particular, the case study design is useful when answering “how” and “why” questions, and in understanding the particulars, and diversity of the case (Yin, 1994). As Yin asserted in his 1993 paper Applications of Case Study Research

the case study is the method of choice when the phenomenon under study is not readily distinguishable from its context.
(p.3)

The case study design (see Table 1) suited the conditions of this research project very well as it allowed naturalistic methods of enquiry such as individual interviews to be carried out, side-by-side with quantitative data gathering from surveys and archival records.

Table 1 Features of a case study (Russ-Eft & Preskill, 2001, pp.173-174)

Advantages

Disadvantages

Provides descriptive data

Results may not lead to scientific generalizability

Does not require control of participants or setting

Researcher bias may interfere with validity of the findings

Reports include verbatim quotes

May take too long to conduct

Leads to a greater understanding of the context of the evaluand

May produce more data than can be analysed in an effective manner

Gather data using multiple methods

Provides data that are rich in examples

Captures what is important to the participants

Portrays the multiplicity of causes that are associated with various outcomes

Embraces diversity of perspectives and experiences of participants

Allows the researcher to collect information on outcomes not known prior to the learning and performance initiative

As Robson points out, the flexibility of the case study approach allows the design to “emerge” (2002, p.89) during data collection and analysis. Similarly, the case study facilitates the application of other research designs within the case study design framework, enabling the learning professional to capture data during different phases of a project.

References:

Robson, C. (2002) Real World Research. 2nd ed. Oxford: Blackwell Publishing.

Russ-Eft, D. & Preskill H. (2001) Evaluation in Organizations: A Systematic Approach to Enhancing Learning, Performance and Change. New York, NY. Perseus Books.

Yin, R. K. (1993). Applications of Case Study Research. Newbury Park, CA. Sage.

Yin, R. K. (1994). Discovering the future of the case study method in evaluation research. Evaluation Practice [Internet] 15. Available from: http://aje.sagepub.com/cgi/reprint/15/3/283 [Accessed 15th January 2008 subscription required]

Create a free edublog to get your own comment avatar (and more!)

0 responses so far ↓

  • There are no comments yet...Kick things off by filling out the form below.

Leave a Comment

*
To prove you're a person (not a spam script), type the security word shown in the picture.
Anti-Spam Image