The E-learning Curve at Edublogs

In my last posting on this topic, we discussed Kirkpatrick’s guidelines for evaluating training programs (see Table 1).

Table 1 Guidelines for evaluating learning

In this post, I will outline the difficulties encountered in applying these guidelines to non-formal learning initiatives.

Now read on…

Kirkpatick’s approach is incompatible with the assessment of non-formal learning in three ways: firstly, the singular nature of each non-formal learning event means that no control group can exist to measure a difference between a control group and the “experimental group” (p.43). Similarly; a significant number of the learners that access this kind of content do so asynchronously, via a number of different learning channels and over a broad time span in a relatively ad hoc manner. This “just enough, just in time” aspect of non-formal learning is one of its strengths, but it is unrealistic to attempt to measure learning using an experimental method in such an environment.

Secondly, the lack of summative assessment in non-formal learning precludes both pencil-and-paper and performance testing to measure learning. Finally, the distributed nature of access to these events, over both time and location makes achieving a 100 per cent learner response rate (expressed by the value R) impossible. As the content (C) is always available (D) the potential (+) always exists for the current number of learners (N₁) to increment upwards (N₂). A complete learner response rate will always be a hostage to N₂ in this context.

R = R+ N₂

where C (is a known value) and N₁ (is a known value) and D (is a known value) + N₂

It follows that you can never achieve a one hundred per cent response rate. Contrast this with formal training courses. They are, by their nature, finite: each course has a specified duration. Content is delivered to learners in an instructor-led environment in a sequence of modules over the duration of the course. Once the course instruction is complete, members of any given class are required to take a comprehensive summative assessment on the learning objectives outlined in the course, usually within a set time after they complete a course to achieve certification.

Thus R = 100

where C = 10 and N₁ = 12 and D = 10

As a result, a 100 per cent response rate to knowledge and skills testing is achieved.

However, I believe that it is possible to capture and use data with the retrospective pre-test method, and I’ll be discussing an approach to undertaking this activity next time.

References:

Kirkpatrick, D. & Kirkpatrick, P. (2006) Evaluating Training Programs. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers, Inc.

–