E-learning Curve Blog at Edublogs » Rosenberg

ADDIE, and the Systems Approach: a Critical Analysis

Michael Hanley — Tue, 10 Jun 2008 15:47:00 +0000

In yesterday’s post I looked at a systems approach to instructional design and development, and suggested that this approach was an emergent property of the process whereby instructor-led courseware developers in the 1990’s attempted to apply previously-effective training development techniques to the new domain of e-learning.

As an example, I looked at the ADDIE design appraoch and suggested that it could be characterized as a set of heuristics (or conceptual framework) to enable learning professionals to output content commensurate with the systems approach.

Now read on…

In this context, we can say that ADDIE will only deliver results consistent with the expertise of the content designer, as it relies on their (the designers’) skills and experience in transferring information and knowledge to “deliver” learning to the recipient. In E-Learning Strategies: how to get implementation and delivery right first time, Don Morrison remarks that

…like many ISD models it fails to give clear guidance about what makes good learning content. You can follow ADDIE to the letter and still deliver mediocre content.

(2004, p.255)

It can be said that ADDIE (and similar systems’ approaches to learning content development) are closer in philosophy and practice to a project management methodology than a pedagogy. In this respect they can be a useful approach when developing “one-shot” training content in environments where repetitive worker activities are the norm.

Take this scenario as an example:

Production workers have just started using the new WidgetMaker 300-0 tool: it can create 3o widgets per hour.

As it’s operating principles are very similar to the the older WM 299-8 (20 widgets per hour) model, the organization decides that workers only need a 2-day familiarization course before beginning production with the new tool.

A new production target (30 WPH) is set for workers to reflect the increased efficiency of the tool.

After three months, production is still 5WPH short of target. It turns out that workers are using the old production process on the new machine, and with the expectation of attaining 30WPH are actually working harder than ever, but are still 50% short of the difference between the new and old hourly production rate. Much disgruntlement all round.

It seems that there’s an extra gadget on the handle of the WM300-0 that nobody noticed because of the cursory nature of the 2-day familiarization course.

Time for some training…

Table 1. Formative evaluation is present in each stage of the ADDIE process.

2. Summative evaluation consists of tests designed for criterion-related referenced items and providing opportunities for feedback from the users.

Once they have undergone formative and summative evaluation, learners are encouraged to review and revise the courseware as necessary, until they have successfully passed the proscribed tests

The transactional nature of the systems approach assumes that the very act of communicating information to the supplicant (sorry, learner) results in the output of “learning.”

Adhering to this content development methodology, instructors consider that the channels used to “deliver” knowledge are cognitively neutral and merely replicate and modalities of the classroom – voice-over narration equating to instructor explanation, the screen being equivalent to the overhead projector or blackboard, computer-mediated interaction being the essentially the same as teacher-student interaction, and so on.

By balancing the cognitive load across the learning modalities, it is supposed that knowledge can be effectively delivered, and the student will “learn.” A corollory of this is that the learning delivery channels (visual, audial, text-based, and so forth) themselves have no affect upon the learner’s interpretation of the content, so to all intents and purposes, the “sage on the stage” transforms into the “guide by the side.”

_____________________

References:

Brown, J. S. Collins, C. & Duguid, P. (1989) Situated Cognition and the Culture of Learning [Internet] Educational Researcher 18(1), pp. 32-42, Jan-Feb 1989. Available from: http://tiger.coe.missouri.edu/%7Ejonassen/courses/CLE/index.html [Accessed January 12th 2007]

Dick, W. & Carey, L. (1996). The Systematic Design of Instruction (4th Ed.). New York: Harper Collins College Publishers.

Dillon, P. & Hallett, C. (2001, October). Powering the leap to maturity: The eLearning ecosystem. Cisco Systems white paper.

Exclusive Interview with Professor David Jonassen (2001) IN: elearningpost [Internet] Available from: http://www.elearningpost.com/articles/archives/
exclusive_interview_with_professor_david_jonassen [Accessed 12th January 2007]

Molenda, M. (2003). In Search of the Elusive ADDIE Model. [Internet] Available from: http://www.indiana.edu/~molpage/
In%20Search%20of%20Elusive%20ADDIE.pdf Accessed 12 May 2008

Rosenberg, M.J. (2001) What Lies Beyond E-Learning? learningcircuits.org e-zine [Internet] Available from: http://www.learningcircuits.org/2006/March/rosenberg.htm Accessed 14th April 2007

–

E-learning ecosystems and the failure of ADDIE

Michael Hanley — Mon, 09 Jun 2008 16:35:00 +0000

Onwards with our investigation of the e-learning ecosystem and learning & development in organizations. Today’s post looks at the similarly-sounding but fundamentally different Systems Approach to instructional design and learning, and why the doubtful provenance of ADDIE combined with a misunderstanding of the role of content delivery channels have failed a generation of e-learners.

Now read on.

I have suggested that a growing number of researchers and learning professionals (Dillon & Hallett, Jonassen, Rosenberg etc) are recognizing the negative affect of what Brown, Collins, & Duguid (1989) call the “breach between learning and use” (p.1) of training interventions in organizations. We have seen that both instructor-led and ‘traditional’ self-paced learning courseware is relatively limited in its application, and is most effective in a restricted range of circumstances. Brown, Collins, & Duguid elaborate:

Many teaching practices implicitly assume that conceptual knowledge can be abstracted from the situations in which it is learned and used. [The authors] argue that this assumption inevitably limits the effectiveness of such practices. Drawing on recent research into cognition as it is manifest in everyday activity, the authors argue that knowledge is situated, being in part a product of the activity, context, and culture in which it is developed and used.

(p.1)

To meet these conditions, we can say that it is appropriate to adapt both pedagogical approaches and the technology resources now available to enhance learners performance.

Marc J. Rosenberg concurs:

Although e-learning began as a new way to deliver training, it cannot remain that way because it is no longer able to adequately support all the learning needs of individuals and organizations by itself – if it every was. E-learning has moved in a new, somewhat unanticipated direction that is not always reminiscent of an instructional framework. To be more influential, e-learning must be reinvented. While continuing to provide a viable instructional option in a formal learning setting, it must also move toward informational and collaborative solutions that focus more prominently on the specific jobs people do. It must move beyond courseware and classrooms and into work. To reinvent e-learning is, in many ways, to reinvent learning itself.

In essence, this means transforming workplace learning, so that learning activities and resources are situated around the learner, their work environment, and their tasks, enabling learners to construct their own knowledge in the context of what resources they need to carry out their work effectively. As David Jonassen says:

In constructivist learning environments, technologies are used to situate learning tasks in a variety of contexts. With video, very rich and engaging contexts can be created.

He asserts that in the traditional organizational approach that

[u]nfortunately, most e-learning replicates the worst features of face-to-face instruction. So, it may be cheaper to “deliver” knowledge over the Internet, but it will not be more effective.

This is commonly known as the systems-based approach to instructional design & development (ISD). A system is a set of elements or components that must integrate to perform a specific function. Every job in an organization is used by the organizational ecosystem to produce a product or output. The product or output is the means by which a organization generates its assets and remains self-supporting.

There are four inputs necessary in every system to produce a product or output:

People: The workers making up a group and linked by a common activity.
Material: The raw products which go into the system.
Technology: The technique for achieving a practical purpose or goal.
Time: The measured period during which an action or process begins and ends.

In learning and development, this systems-based approach is epitomized by the ADDIE conceptual framework (see Figure 1), most notably refined by Dick & Carey in The Systematic Design of Instruction (1996).

Figure 1. The ADDIE ISD Heuristic

The ADDIE approach has been one of the core tenets of instructional design for the best part of two decades, but curiously, it may not exist! In his article In Search of the Elusive ADDIE Model (2003), Michael Molenda of Indiana University, the author undertook a Livingstonian attempt to discover the source for the original reference to the ADDIE model. Molenda’s research uncovered no original reference for the ADDIE model. This lack of an original reference led Molenda to write,

I am satisfied at this point to conclude that the ADDIE Model is merely a colloquial term used to describe a systematic approach to instructional development, virtually synonymous with instructional systems development (ISD). The label seems not to have a single author, but rather to have evolved informally through oral tradition. There is no original, fully elaborated model, just an umbrella term that refers to a family of models that share a common underlying structure.

(p.1)

Hence my suggestion above that ADDIE is more proerly labeled a conceptual framework; I would go so far as to say that it could more properly be called a set of heuristics or “rules of thumb” to develop learning content by adhering to the precepts of the systems approach (see Table 1).

Table 1. Formative evaluation is present in each stage of the ADDIE process.

2. Summative evaluation consists of tests designed for criterion-related referenced items and providing opportunities for feedback from the users.

Once they have undergone formative and summative evaluation, learners are encouraged to review and revise the courseware as necessary, until they have successfully passed the proscribed tests

The transactional nature of the systems approach assumes that the very act of communicating information to the supplicant (sorry, learner) results in the output of “learning.”

_____________________

References:

Dick, W. & Carey, L. (1996). The Systematic Design of Instruction (4th Ed.). New York: Harper Collins College Publishers.

Dillon, P. & Hallett, C. (2001, October). Powering the leap to maturity: The eLearning ecosystem. Cisco Systems white paper.

Molenda, M. (2003). In Search of the Elusive ADDIE Model. [Internet] Available from: http://www.indiana.edu/~molpage/
In%20Search%20of%20Elusive%20ADDIE.pdf Accessed 12 May 2008

Rosenberg, M.J. (2001) What Lies Beyond E-Learning? learningcircuits.org e-zine [Interent] Available from: http://www.learningcircuits.org/2006/March/rosenberg.htm Accessed 14th April 2007

–

E-Learning along the curve: adapting to knowledge workers learning needs in organizations

Michael Hanley — Tue, 03 Jun 2008 11:44:00 +0000

As I discussed in the previous blog entry, formal, structured approaches to learning (such as to both instructor-led and traditional CBT-type training interventions), which by their nature are long in duration, relatively generalized in terms of subject matter, and are best deployed to novices in the relevant discipline or skill area. As such, they represent the best value for organizations when to knowledge worker is not expected to be a full contributor in their role. In this context, I would suggest that organizations can justify using these learning solutions for situations such as

entry-level or new hire orientation and competency building
internal transfers to a new discipline (i.e. the worker moves from Production to QA)
retraining on a new production system (i.e. a new type of widget-making tool or a new software system)
employee career advancement (i.e. from individual contributor to manager)

Psychology tells us that the learning curve obeys what is called a power law (Ritter & Scholler, 2002).

As such they are often said to conform to “the power law of practice”. Cognitive psychology has shown that the power law of practice is ubiquitous, and cognitive modeling has explained both the general speedup and variability in performance.

(The Learning Curve, p.2)

So, as a worker learns a task, skill, or process (”progresses along the learning curve”), their competency improves and productivity increases with some variations but broadly within generally accepted parameters.

As workers advance from their neophyte status, they begin to attain what Marc J. Rosenberg (2006) calls “performer” status; the are transitioning from being Novice to Competent, along a path that will enable them to become Experienced, until they achieve Expert (or Master) status.

We can also say that once a worker reaches a certain level of competence, their learning needs are met less by generic courses and curricula, and more by specific, even personalized, learning interventions such as task/skill practise and coaching, access to knowledge and performance resources, and collaboration and problem solving (see Figure 1).

Figure 1. Levels of mastery and appropriate learning strategies (after Marc J. Rosenberg, 2006)

As an extension of this (and effectively demonstrated by Rosenberg), organizations failing to move beyond the classroom or traditional CBT-type courseware for their ongoing learning and development needs, are probably impeding the development of their workers, as well as negatively affecting their (the organization’s) own potential (see Figure 2).

Figure 2. Advantages of workflow learning (after Marc J. Rosenberg, 2006)

In order to provide effective learning and performance support to workers after they become competent, organizations must strive to develop their workers’ skills as employees undertake their regular workplace activities. It is my view that this level of performance support can only be provided through access to networked knowledge assets.

And that is what I’ll be discussing tomorrow.
______________________

References:

Dillon, P. & Hallett, C. (2001, October). Powering the leap to maturity: The eLearning ecosystem. Cisco Systems white paper.

Ritter, F. E., & Schooler, L. J. (2002). The learning curve. In International encyclopedia of the social and behavioral sciences. 8602-8605. Amsterdam: Pergamon.
[Internet] Available from: http://www.iesbs.com/ Accessed 27 May 2008.

Rosenberg, M. J. (2006) Beyond e-Learning. San Francisco, CA: John Wiley & Sons, Inc.

–

E-Learning along the curve: adapting to knowledge workers learning needs in organizations

Michael Hanley — Tue, 03 Jun 2008 11:44:00 +0000

entry-level or new hire orientation and competency building
internal transfers to a new discipline (i.e. the worker moves from Production to QA)
retraining on a new production system (i.e. a new type of widget-making tool or a new software system)
employee career advancement (i.e. from individual contributor to manager)

Psychology tells us that the learning curve obeys what is called a power law (Ritter & Scholler, 2002).

As such they are often said to conform to “the power law of practice”. Cognitive psychology has shown that the power law of practice is ubiquitous, and cognitive modeling has explained both the general speedup and variability in performance.

(The Learning Curve, p.2)

Figure 1. Levels of mastery and appropriate learning strategies (after Marc J. Rosenberg, 2006)

Figure 2. Advantages of workflow learning (after Marc J. Rosenberg, 2006)

And that is what I’ll be discussing tomorrow.
______________________

References:

Dillon, P. & Hallett, C. (2001, October). Powering the leap to maturity: The eLearning ecosystem. Cisco Systems white paper.

Rosenberg, M. J. (2006) Beyond e-Learning. San Francisco, CA: John Wiley & Sons, Inc.

–

Learning Evaluation and Strategy: Using an e-learning readiness survey

Michael Hanley — Mon, 14 Apr 2008 20:07:00 +0000

During this series of posts on evaluating non-formal learning programs, I have mentioned carrying out an e-learning readiness survey without characterizing or discussing how to implement such a research instrument.

This was deliberate; in my view e-learning readiness surveys represent an alpha and an omega of evaluation: on one level they are the starting point for any evaluation of an organization’s learning initiative, on another level they define an organization’s ability to implement an effective e-learning strategy. As such, they are a bridge between the theory and the practice of implementing a learning program.

Readiness surveys enable the learning practitioner to understand and measure both the effectiveness of organizational learning and to identify the critical factors for success when developing learning programs.

I have found Marc J. Rosenberg’s E-Learning Readiness Survey to be a very effective instrument to evaluate both the effectiveness of an organization’s learning strategy, and one of the foundations of any serious evaluation of the effectiveness of learning programs.

The questions are grouped into seven areas of understanding:

business readiness
the changing nature of learning and e-learning
value of instructional and information design
change management
reinventing the training organization
the e-learning industry
your personal commitment

The questions provided in this survey represent some of the most important strategic issues organizations face when transitioning to e learning. Certainly there are additional questions and issues that deserve attention; add your own, organization-specific items as required.

Downloads:

Marc J. Rosenberg’s E-learning Readiness Survey

References:

Rosenberg, M. J. (2006) Beyond e-Learning. San Francisco, CA: John Wiley & Sons, Inc.
–

Learning Evaluation and Strategy: Using an e-learning readiness survey

Michael Hanley — Mon, 14 Apr 2008 20:07:00 +0000

The questions are grouped into seven areas of understanding:

business readiness
the changing nature of learning and e-learning
value of instructional and information design
change management
reinventing the training organization
the e-learning industry
your personal commitment

Downloads:

Marc J. Rosenberg’s E-learning Readiness Survey

References:

Rosenberg, M. J. (2006) Beyond e-Learning. San Francisco, CA: John Wiley & Sons, Inc.
–

Measuring the learning effect of non-formal learning programs

Michael Hanley — Fri, 22 Feb 2008 11:02:00 +0000

I have suggested in a previous blog entry that non-formal learning (NFL) is located on a continuum between so-called informal and formal learning (see Figure 1)

Figure 1 the Learning Continuum

and that learning is situated in a larger socio-cultural context – a cognitive web – that provides the knowledge-centric framework within which learners operate (see Figure 1.2).

Figure 1.2 Knowledge-centric View of Knowledge (after Marc J. Rosenberg)

Similarly, NFL shares characteristics with both informal and formal learning, but is differentiated enough from both of these approaches to merit it’s own category (see Table 1)

Table 1 Definition of learning types

Learning Type	Description
Formal Learning	Learning typically provided by an education or training institution, structured (in terms of learning objectives, learning time or learning support) and leading to certification. Formal learning is intentional from the learner’s perspective [my italics].
Non-formal Learning	Learning that is not provided by an education or training institution and typically does not lead to formalised certification. It is, however, structured (in terms of learning objectives, learning time or learning support). Non-formal learning is intentional from the learner’s perspective [my italics].
Informal Learning	Learning resulting from daily life activities related to work, family or leisure. It is not structured (in terms of learning objectives, learning time or learning support) and typically does not lead to certification. Informal learning may be intentional but in most cases it is non-intentional (or “incidental”/ random) [my italics].

Given the non-formal nature of the learning (and by extension a paucity of the type of assessment metrics available when evaluating the efficacy of formal training programmes) I am going to explore the challenge of assessing non-formal learning in the context of Kirkpatrick’s 4-level model, particularly between Level 1 (Reaction) and Level 3 (Behaviour); as Kirkpatrick & Kirkpatrick (2006, p.22) assert “learning has taken place …when one or more of the following occurs: Attitudes (sic) are changed. Knowledge is increased. Skill is improved. One or more of these changes must take place if a change in behaviour is to occur.”

More tomorrow…

References:

Kirkpatrick, J. & Kirkpatrick, P. (2006) Evaluating Training Programs. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers, Inc.

Rogers, A. (2004) Looking again at non-formal and informal education – towards a new paradigm [Internet] Available from: http://www.infed.org/biblio/non_formal_paradigm.htm [Accessed 30th January 2008]

Rosenberg, M. J. (2001) e-Learning: Strategies for Delivering Knowledge in the Digital Age London: McGraw-Hill.