E-learning Curve Blog at Edublogs

E-learning Curve Blog is Michael Hanley's elearning blog about skills, knowledge, and organizational development using web-based training and technology in education

Entries Tagged as 'College Education'

Approaches to evaluating learning

August 15, 2008 by Michael Hanley · 1 Comment · College Education, formal learning, Knowledge Economy, knowledge worker, leaving cert, the apprentice

In yesterday’s post I put forward some thoughts on terminal second-level examinations and the effect that faring poorly in these could have for young adults’ future lives.

I received a comment on the topic from a correspondent who asserted:

…does success in exams equal education? Exams are an outmoded assessment tool – success in them means nothing more than a certificate and maybe entry into higher education – where you sit more exams until you finally move into the real world…where you realise your so called education did you a grave disservice.
Can you think? Can you create? Can you problem solve? Does an exam system fit into 21st century learning?

Now read on…

Do we have practical solutions (as opposed to high aspirations) about the best way to serve students in the 21st century?

Hard to say.

I would suggest that the best way to prepare people for the workplace is to evaluate learners as if they were in the workplace. So let’s look briefly at the role of certification in this context.

Certification is the ability to prove through testing if an individual has achieved a mastery of skills, knowledge and attitude. Certification can also prove the ability of an individual to apply those skills and knowledge in specified areas and job functions.

(Certification: Corporate America’s Secret Weapon. Hilbink, P. 2004 p.2).

In 1959, Donald L. Kirkpatrick first published his four-level training evaluation model (see Table 1) in a series of articles for the US Training and Development Journal. “The reason for evaluating is to determine the effectiveness of a training program” (Evaluating Training Programs, 2006, p3). The reason for the four-level model then “was to clarify the elusive term evaluation” (2006, xv). In articulating evaluation through each of the four levels – reaction, learning, behaviour and results – the model aspires to

inspire us to look beyond our traditional classroom content delivery model and opens windows to the many way we can improve the performance of our organisations.

(2006, p.xi)

Table 1 Kirkpatrick’s Four-level Model

kirkpatricks4levelmodel

In the context of Kirkpatrick’s Four Level Model, second-level certification is typically interpreted at Levels Two. We can say that the most effective approach to understanding how well learners have acquired new knowledge, skills, and expertise is to ask them to demonstrate what they have learned: they sit a test. In the context of second-level education, certification assesses the learner at Level 2: Transfer of Learning. It is important to measure learning transfer because no change in behaviour can be expected unless one or more of these learning objectives have been accomplished. Measuring learning means determining one or more of the following metrics:

  • What knowledge was learned?
  • What skills were developed or improved?
  • What attitudes were changed?

The benefits to conducting Level Two tests are that the learner must demonstrate that the learning transfer has occurred, and that the assessment provides verifiable and conclusive evidence that an improvement has occurred in knowledge, skills, or attitudes.

Assessment tests are a powerful tool for organisations, institutions and society-at-large, as they combine a hierarchical observation and a normalising judgement. Testing makes individuals visible (who has attained the qualification? who has not attained certification?) and enables them to be categorised (how well did they do?). Exams also normalize people by assessing them according to the same metric, and subsequently measures them in relation to one common standard.

Having established the value of Level Two assessments to understand how well knowledge has been transferred, what next? Does the education system as it currently exists meet the needs of students as they prepare for life in the Information Age, or is there a lag between national educational policies & strategies, and how students’ skill need to be shaped?

Substantial resources (time, well-paid teacher, continuous training for educators etc) are required to undertake these types of evaluations, and the ministries and agencies with responsibility for managing these tasks don’t seem to have the influencing powers to ensure these assets are in place.

A cynic might say that it’s because such an initiative requires long-term planning. The results of such an approach mightn’t be seen for five to ten years, and a politician who advocated such a spend on education might not be in a position, a decade or so later, to reap the rewards of such an innovation in national education policy.

_____________________

References:

Hilbink, P. (2004) Certification: Corporate America’s Secret Weapon [Internet] Available from: <http://www.digital-latitudes.com/docs/Cert_White_Paper.pdf> [Accessed 7 July, 2008]

Kirkpatrick, D. & Kirkpatrick, J. (2006) Evaluating Training Programs. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers, Inc.

[Read more →]

Approaches to evaluating learning

August 15, 2008 by Michael Hanley · 1 Comment · College Education, formal learning, Knowledge Economy, knowledge worker, leaving cert, the apprentice

In yesterday’s post I put forward some thoughts on terminal second-level examinations and the effect that faring poorly in these could have for young adults’ future lives.

I received a comment on the topic from a correspondent who asserted:

…does success in exams equal education? Exams are an outmoded assessment tool – success in them means nothing more than a certificate and maybe entry into higher education – where you sit more exams until you finally move into the real world…where you realise your so called education did you a grave disservice.
Can you think? Can you create? Can you problem solve? Does an exam system fit into 21st century learning?

Now read on…

Do we have practical solutions (as opposed to high aspirations) about the best way to serve students in the 21st century?

Hard to say.

I would suggest that the best way to prepare people for the workplace is to evaluate learners as if they were in the workplace. So let’s look briefly at the role of certification in this context.

Certification is the ability to prove through testing if an individual has achieved a mastery of skills, knowledge and attitude. Certification can also prove the ability of an individual to apply those skills and knowledge in specified areas and job functions.

(Certification: Corporate America’s Secret Weapon. Hilbink, P. 2004 p.2).

In 1959, Donald L. Kirkpatrick first published his four-level training evaluation model (see Table 1) in a series of articles for the US Training and Development Journal. “The reason for evaluating is to determine the effectiveness of a training program” (Evaluating Training Programs, 2006, p3). The reason for the four-level model then “was to clarify the elusive term evaluation” (2006, xv). In articulating evaluation through each of the four levels – reaction, learning, behaviour and results – the model aspires to

inspire us to look beyond our traditional classroom content delivery model and opens windows to the many way we can improve the performance of our organisations.

(2006, p.xi)

Table 1 Kirkpatrick’s Four-level Model

kirkpatricks4levelmodel

In the context of Kirkpatrick’s Four Level Model, second-level certification is typically interpreted at Levels Two. We can say that the most effective approach to understanding how well learners have acquired new knowledge, skills, and expertise is to ask them to demonstrate what they have learned: they sit a test. In the context of second-level education, certification assesses the learner at Level 2: Transfer of Learning. It is important to measure learning transfer because no change in behaviour can be expected unless one or more of these learning objectives have been accomplished. Measuring learning means determining one or more of the following metrics:

  • What knowledge was learned?
  • What skills were developed or improved?
  • What attitudes were changed?

The benefits to conducting Level Two tests are that the learner must demonstrate that the learning transfer has occurred, and that the assessment provides verifiable and conclusive evidence that an improvement has occurred in knowledge, skills, or attitudes.

Assessment tests are a powerful tool for organisations, institutions and society-at-large, as they combine a hierarchical observation and a normalising judgement. Testing makes individuals visible (who has attained the qualification? who has not attained certification?) and enables them to be categorised (how well did they do?). Exams also normalize people by assessing them according to the same metric, and subsequently measures them in relation to one common standard.

Having established the value of Level Two assessments to understand how well knowledge has been transferred, what next? Does the education system as it currently exists meet the needs of students as they prepare for life in the Information Age, or is there a lag between national educational policies & strategies, and how students’ skill need to be shaped

Substantial resources (time, well-paid teacher, continuous training for educators etc) are required to undertake these types of evaluations, and the ministries and agencies with responsibility for managing these tasks don’t seem to have the influencing powers to ensure these assets are in place.

A cynic might say that it’s because such an initiative requires long-term planning. The results of such an approach mightn’t be seen for five to ten years, and a politician who advocated such a spend on education might not be in a position, a decade or so later, to reap the rewards of such an innovation in national education policy.

_____________________

References:

Hilbink, P. (2004) Certification: Corporate America’s Secret Weapon [Internet] Available from: <http://www.digital-latitudes.com/docs/Cert_White_Paper.pdf> [Accessed 7 July, 2008]

Kirkpatrick, D. & Kirkpatrick, J. (2006) Evaluating Training Programs. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers, Inc.

[Read more →]

It’s not the end of the world, you know

August 14, 2008 by Michael Hanley · 2 Comments · College Education, formal learning, Knowledge Economy, knowledge worker, leaving cert, the apprentice

Today’s blog post is not so much about e-learning or learning and development but rather more about education in general. Whether it’s the Leaving lc_testCertificate (the final course in the Irish secondary school system), SATs in the US, O- and A-Levels in the UK, or le bac in France, I’m sure you are familiar with the day of reckoning just-finished secondary-level students are facing about now as their results are published in advance of matriculation for third-level and university places, as well as those former students who intend to go straight into employment, or take a gap year to figure out what they want to do with their lives.

Now read on…

In Ireland, the examination results were published yesterday, and today they’re released in the UK; tuning in to Irish and British radio and TV broadcasts over the last twenty-four hours you would be forgiven for thinking that we’re on the edge of a natural catastrophe (well, apart from the monsoon-like rain we’ve been experiencing for the last week, but that’s another story).

You may be familiar with the format:

  • Images / sounds of students opening their results envelopes and shouting for joy / groaning with disappointment
  • Vox-pop of said students as they discuss their plans now that they’ve achieved / failed to achieve the marks they needed
  • Outro from reporter which goes along the lines of “… you might not have got what you wanted, but it’s not the end of the world, you know” before an short excerpt of some successful local business person or politician describing how they left school at 15 with no qualifications and worked their way up from tea-boy to head of a transnational organisation (before giving out the number of the helpline worried parents can call for advice).

While I completely understand the need to sympathize with devastated young adults who see their hopes, dreams, and career options evaporating before their eyes, I believe that to glibly state that “it’s not the end of everything” simultaneously devalues the emotional and psychological impact of performing poorly in such an important life event, and provides falselc_2 hope that somehow it will be all right.

Sadly, the reality is that in the 21st Century knowledge economy, a less-than-average result in these examinations seriously affects most young peoples’ ability to move forward with their lives – particularly in these increasingly straitened times. As the world transitions to an Information Age where the primary asset an individual possesses is their expertise, a misstep on this lowest rung of the ladder has the potential to damage an otherwise bright, intelligent individual’s potential to both contribute to, and make their way in their society.

For every business leader who “did it the hard way” or “learned from the university of life,” there are ten frustrated employees working in the wrong career, and maybe ten times that number pumping gas, or just about making enough to hang in there, not really living, just existing. Unless you’re in a position where you have the financial resources, the family or social contacts, or just the pure luck to break into your chosen path, you have to pretty much generate your own career based upon your abilities and talents – hopefully enhanced by what you learned in school.

I never cease to be amazed by (for example) politicians who state that through hard work and perseverance they finally got elected and rose to the position they’re in now… while omitting that their father held the seat before them and they’re based in a traditional constituency where the electorate has voted for the ‘name they know’ for generations, or the business-person who was a millionaire by thirty, through their ceaseless efforts and dedication …and the fact that they come from a wealthy family with the resources to set them up.

I heard one such person on the radio yesterday who asserted that “all you need to be successful in your chosen career is to be focused on your goal and to be passionate about what you want to do.” Try this experiment – say for example you want to be a project manager – send your résumé to as many organisations as you like, outlining your passion, enthusiasm and lack of formal qualifications in the field. Then, wait for the employment offers to roll in.

But don’t hold your breath.

Passion is great, but in many cases it’s the last resort of the incompetent (look at all those candidates on The Apprentice who, when about to be fired protest that they’re “passionate about what they do”). Enthusiasm is an admirable quality, but certainly no substitute for expertise, ability, and experience: organisations understand this, though professional competence is boring and makes for poor reality TV.

lc_apprentice

Which one of these would YOU hire?

One of the primary reasons I take such joy in my career (and it’s not even a logical reason) is that it puts me in a position to see people reach their potential. Aristotle (Nicomachean Ethics) believed that education was fundamental to the human condition – the fulfilled person was an educated person.

Perhaps this week, more than at any other time of the year his assertion that

the roots of education are bitter, but the fruit is sweet

is at it’s most evocative and apposite.

_____________

References:

Aristotle. Nicomachean Ethics

_____________

Images’ source:

Radio Telefís Éireann image library

BBC. The Apprentice. [Internet] Available from: http://www.bbc.co.uk/apprentice/ Accessed 14 August 2008

[Read more →]

It’s not the end of the world, you know

August 14, 2008 by Michael Hanley · 2 Comments · College Education, formal learning, Knowledge Economy, knowledge worker, leaving cert, the apprentice

Today’s blog post is not so much about e-learning or learning and development but rather more about education in general. Whether it’s the Leaving lc_testCertificate (the final course in the Irish secondary school system), SATs in the US, O- and A-Levels in the UK, or le bac in France, I’m sure you are familiar with the day of reckoning just-finished secondary-level students are facing about now as their results are published in advance of matriculation for third-level and university places, as well as those former students who intend to go straight into employment, or take a gap year to figure out what they want to do with their lives.

Now read on…

In Ireland, the examination results were published yesterday, and today they’re released in the UK; tuning in to Irish and British radio and TV broadcasts over the last twenty-four hours you would be forgiven for thinking that we’re on the edge of a natural catastrophe (well, apart from the monsoon-like rain we’ve been experiencing for the last week, but that’s another story).

You may be familiar with the format:

  • Images / sounds of students opening their results envelopes and shouting for joy / groaning with disappointmentlc_2
  • Vox-pop of said students as they discuss their plans now that they’ve achieved / failed to achieve the marks they needed
  • Outro from reporter which goes along the lines of “… you might not have got what you wanted, but it’s not the end of the world, you know” before an short excerpt of some successful local business person or politician describing how they left school at 15 with no qualifications and worked their way up from tea-boy to head of a transnational organisation (before giving out the number of the helpline worried parents can call for advice).

While I completely understand the need to sympathize with devastated young adults who see their hopes, dreams, and career options evaporating before their eyes, I believe that to glibly state that “it’s not the end of everything” simultaneously devalues the emotional and psychological impact of performing poorly in such an important life event, and provides false hope that somehow it will be all right.

Sadly, the reality is that in the 21st Century knowledge economy, a less-than-average result in these examinations seriously affects most young peoples’ ability to move forward with their lives – particularly in these increasingly straitened times. As the world transitions to an Information Age where the primary asset an individual possesses is their expertise, a misstep on this lowest rung of the ladder has the potential to damage an otherwise bright, intelligent individual’s potential to both contribute to, and make their way in their society.

For every business leader who “did it the hard way” or “learned from the university of life,” there are ten frustrated employees working in the wrong career, and maybe ten times that number pumping gas, or just about making enough to hang in there, not really living, just existing. Unless you’re in a position where you have the financial resources, the family or social contacts, or just the pure luck to break into your chosen path, you have to pretty much generate your own career based upon your abilities and talents – hopefully enhanced by what you learned in school.

I never cease to be amazed by (for example) politicians who state that through hard work and perseverance they finally got elected and rose to the position they’re in now… while omitting that their father held the seat before them and they’re based in a traditional constituency where the electorate has voted for the ‘name they know’ for generations, or the business-person who was a millionaire by thirty, through their ceaseless efforts and dedication …and the fact that they come from a wealthy family with the resources to set them up.

I heard one such person on the radio yesterday who asserted that “all you need to be successful in your chosen career is to be focused on your goal and to be passionate about what you want to do.” Try this experiment – say for example you want to be a project manager – send your résumé to as many organisations as you like, outlining your passion, enthusiasm and lack of formal qualifications in the field. Then, wait for the employment offers to roll in.

But don’t hold your breath.

Passion is great, but in many cases it’s the last resort of the incompetent (look at all those candidates on The Apprentice who, when about to be fired protest that they’re “passionate about what they do”). Enthusiasm is an admirable quality, but certainly no substitute for expertise, ability, and experience: organisations understand this, though professional competence is boring and makes for poor reality TV.

lc_apprentice

Which one of these would YOU hire?

One of the primary reasons I take such joy in my career (and it’s not even a logical reason) is that it puts me in a position to see people reach their potential. Aristotle (Nicomachean Ethics) believed that education was fundamental to the human condition – the fulfilled person was an educated person.

Perhaps this week, more than at any other time of the year his assertion that

the roots of education are bitter, but the fruit is sweet

is at it’s most evocative and apposite.

_____________

References:

Aristotle. Nicomachean Ethics

_____________

Images’ source:

Radio Telefís Éireann image library

BBC. The Apprentice. [Internet] Available from: http://www.bbc.co.uk/apprentice/ Accessed 14 August 2008

[Read more →]

100 Helpful Web Tools for Every Kind of Learner

June 13, 2008 by Michael Hanley · Comments Off · collaboration tools, College Education, Features, learning styles, mind tools, Productivity, VAK

As you’ll know if you viewed my post on Jane Hart’s Centre for Learning & Performance Technologies (C4PT) I feel that the more exposure learning professionals have to the range of tools at our disposal, the more effectively we can develop effective learning solutions for our customers.

In a similar vein, I’ve just received a nice e-mail from Fiona King at College@Home about a similar list of tools.

I’ll let herself explain:

I realize this is out of the blue, but we just posted an article, “100 Helpful Web Tools for Every Kind of Learner.” I thought I’d bring it to your attention in case you think your readers would find it interesting.

If you follow the link to the web page, you’ll see that Fiona has arranged the tools according to VAK learning style and by tool type:

  • Visual Learners
    • Mind Mapping
    • Charting and Diagrams
    • Videos and Photos
  • Auditory Learners
    • Podcasts
    • Presentation Tools
    • Audio Tools
    • Text Readers
    • Audio Books
  • Kinaesthetic Learners
    • Note Taking Tools
    • Bookmarking
    • Interaction
    • Collaboration

I have to state at this point that in my view I would question the effectiveness of the VAK-type learning style approach – see Coffield’s article Learning styles and pedagogy in post-16 learning (link below) evidence for further information, but as a set of sensory categories for Fiona’s purpose, they work just fine.

I think that even for those among us who would be familiar with a lot of the tools here (many of the “usual suspects” are on the list), there will be one or two new tools, and even useful new information about old favourites; did you know that Adobe Acrobat 8 has a text-to-speech reader, for example?

Me neither.

But I do now.

_____________

Links:

100 Helpful Web Tools for Every Kind of Learner

_____________

References:

Coffield, F., Moseley, D., Hall, E., Ecclestone, K. (2004). Learning styles and pedagogy in post-16 learning. A systematic and critical review [Internet] Available from: http://www.lsda.org.uk/files/PDF/1543.pdf Accessed 12 June 2008

[Read more →]

100 Helpful Web Tools for Every Kind of Learner

June 13, 2008 by Michael Hanley · Comments Off · collaboration tools, College Education, Features, learning styles, mind tools, Productivity, VAK

As you’ll know if you viewed my post on Jane Hart’s Centre for Learning & Performance Technologies (C4PT) I feel that the more exposure learning professionals have to the range of tools at our disposal, the more effectively we can develop effective learning solutions for our customers.

In a similar vein, I’ve just received a nice e-mail from Fiona King at College@Home about a similar list of tools.

I’ll let herself explain:

I realize this is out of the blue, but we just posted an article, “100 Helpful Web Tools for Every Kind of Learner.” I thought I’d bring it to your attention in case you think your readers would find it interesting.

If you follow the link to the web page, you’ll see that Fiona has arranged the tools according to VAK learning style and by tool type:

  • Visual Learners
    • Mind Mapping
    • Charting and Diagrams
    • Videos and Photos
  • Auditory Learners
    • Podcasts
    • Presentation Tools
    • Audio Tools
    • Text Readers
    • Audio Books
  • Kinaesthetic Learners
    • Note Taking Tools
    • Bookmarking
    • Interaction
    • Collaboration

I have to state at this point that in my view I would question the effectiveness of the VAK-type learning style approach – see Coffield’s article Learning styles and pedagogy in post-16 learning (link below) evidence for further information, but as a set of sensory categories for Fiona’s purpose, they work just fine.

I think that even for those among us who would be familiar with a lot of the tools here (many of the “usual suspects” are on the list), there will be one or two new tools, and even useful new information about old favourites; did you know that Adobe Acrobat 8 has a text-to-speech reader, for example?

Me neither.

But I do now.

_____________

Links:

100 Helpful Web Tools for Every Kind of Learner

_____________

References:

Coffield, F., Moseley, D., Hall, E., Ecclestone, K. (2004). Learning styles and pedagogy in post-16 learning. A systematic and critical review [Internet] Available from: http://www.lsda.org.uk/files/PDF/1543.pdf Accessed 12 June 2008

[Read more →]