For the last twenty five years, I have insisted that our clients allow us to build a very early prototype, or sample of the learning strategy that we intend to use. Then we simply "try it out" with no more than 4 to 6 actual learners. We have had a lot of pushback over the years:  "it will take too much time; we won’t be able to get access to actual learners, etc." The reason I have insisted on this small investment of time is that Formative Evaluation can become a quality guarantee. What better way to measure the results of learning than with actual learners? The time required for a Learner Try-out is minimal. We need four to six people for about two to four hours on average. Sure, there is some preparation time, so we use up a couple of days. What we are doing is testing the validity of our instructional strategy, or in plain English, we find out if people can actually learn something from the strategy we have selected. It seems a small price to pay to find out how our plan is going to work before we have spent all our money and completed the project.   Let me cite a recent example. Our design team was working with a large client that was willing to take a risk to totally re-design a core program to employ a task-based strategy rather than one that presented content organized by topic areas. The guideline on how much time to spend on Formative Evaluation is: "the more risk there is in the project, the more time and money we are willing to spend on Formative Evaluation." In this case the new approach was quite different from the traditional approach. We wanted to be sure our assumptions, which were based on careful analysis, were correct and that our instructional strategy worked, before we completed the entire program. The pay-off from this session exceeded our expectations. Check out the next blog post to read about the results. 2/8 of series

Let’s go back to our example from the previous blog. Since we were running sophisticated simulations in a classroom environment, our Learner Tryout lasted an entire day and actually trained eight new learners in the process. The team had a debate about two different strategies for how to implement a particular simulation. Without the tryout, we would have slugged it out until someone picked a direction. Both strategies were tested, resulting in a clear consensus from learners, the instructor, and the designers. Why guess when you can measure? Many other decisions and enhancements were made as a result of this type of measurement. The team estimates they saved potential rework by about 40% of what we projected. This type of saving translates to several weeks of work saved as a result of the learner tryout. But this wasn’t the only pay off.  An operations manager from the client area came to do her standard introduction for the class. This normally takes her thirty minutes, and then she leaves. She was so fascinated by what was going on that she stayed all day, and has now become our biggest internal advocate. Learners and clients love to be asked for their input into how their learning is going to be developed. The benefits of this kind of internal PR are hard to quantify, but are worth their weight in gold. With a pay-off like this, why would anyone not want to invest a small amount of time and money to measure instead of guessing? 3/8 of series

I recently presented this workshop to two different teams at TIAA-CREF.  Read below to find out what the managers at TIAA-CREF had to say about their experience. "Dick showed our consultants how to remain professional in their approach, acknowledge the client concerns, and help the client arrive at a decision on their own that is focused on results and not actions.  Kudos to Dick and Handshaw, Inc for providing such a valuable program!  I hope others take advantage of the expertise and value that Dick can bring to a session on performance consulting."   Kathleen Skelly, Learning Manager at TIAA-CREF "Dick Handshaw facilitated a reframing workshop for our team of performance consultants during a time when we were trying to shift from order-taker to trusted advisor.  It was one of the best development opportunities that TIAA-CREF could have invested in for our team.  Our consultants have been able to successfully apply these skills immediately following the workshop in many different interactions with their clients.  I would highly recommend this workshop to anyone who wants to transform their relationships with their clients!" Heather Fausnaugh, Learning Director at TIAA-CREF Want to experience this workshop firsthand?  Join me for the half-day session in November. Date:  Friday, November 5thth Time:  8:30 am - 12:00 noon Cost:  $95 Location:  Interact Studios, Charlotte, NC To register or learn more visit:  http://dickhandshawworkshop.eventbrite.com/ ** Only 10 seats available

I was working with a client who was implementing a new software system for front-line personnel. The system was fairly user-friendly, but it did require some practice in the form of e-learning in order to master its new functionality for that all important first day in front of customers. As is our practice, we created a couple of 20-minute prototype modules that were representative of our overall instructional strategy. One of the modules was rich in simulations with plenty of opportunities for extra tries with nice instructional feedback. Everything was proceeding nicely until we had a design meeting with our client to discuss something as innocent as a color scheme for the simulation screens. Our client wanted the colors of our instruction, scenario and feedback boxes to match the color scheme of the system. Our designers advised that this would cause confusion for the learners, because they would not know which messages were part of the learning and which might be actual messages from the system. We simply could not convince our client to use contrasting colors for the learning messages. The project team was stumped, worried, and a little bit frantic because they were sure the Learner Try-out would fail. Now here’s the thing, a prototype may fail, but a Try-out that yields data that leads to improvement of the strategy or any part of the solution is a success.   I sat next to the insistent and well-intending client representative during the Learner Try-out. After the third person turned to us and said, "I can’t tell when I’m looking at a message from the real system and when I’m reading a message from the e-learning," my client turned to me and said, "Why don’t you use the colors you originally recommended?" It didn’t take us long to go back and make those changes to the one 20 minute prototype. As a result of our investment in a four hour Learner Tryout, we did not have to make those changes to all 12 modules in the course. That was one improvement we expected to find, but just one of many other improvements we find every time we do a Learner Try-out the right way.   4/8 of series

Whenever I hear that a team from our company has returned from a client site to conduct a Learner Try-out, I always ask, "How did it go?" The answer I usually get is, "Oh, it went great." Now this always concerns me. You can conduct a Learner Tryout in such a manner that you verify that every single design decision you made was spot on. It can also be done methodically and objectively so that it yields useful results for future design decisions and revisions.  The real value of a Learner Try-out depends on how you collect the data. We have three phases of data collection during a Learner Try-out. The first phase is qualitative. We interview the client using a standard set of questions, which we always customize, before the Try-out begins. The next phase is more quantitative. We observe the learners one-on-one as they proceed through whatever prototype they are testing. We ask them to "think out loud" for us as they proceed. We watch body language and frequently stop them to ask questions. We must avoid coaching them. The idea is to let the prototype fail if it isn’t working. We will coach our learners if necessary to make sure they don’t fail, but failures are our best opportunity to collect useful data. Being an observer during a Learner Try-out is not for the timid. The third and final data collection phase is qualitative again. We interview with a list of open-ended questions to get personal opinions and impressions about the learning experience. What surprises us is that even with a small sampling of six learners, clear trends always develop. For complete procedures on how to select sample learners, how to prepare for, and how to conduct Learner Try-outs, and how to collect and interpret data, click on the Resources tab and select Formative Evaluation and Formative Evaluation Procedures.   5/8 of series

Chris Adams and I will be presenting a breakout session at the 10th Annual U.S. Coast Guard Human Performance Technology Workshop on Friday, September 17th.  Our session is titled "For Training Purposes Only" and focuses on the practical and creative use of content management systems. The title is significant to Coast Guard personnel who are used to seeing documents and other materials stamped ‘FOR TRAINING PURPOSES ONLY’ because materials are often out of date and can’t be used as performance support on the job. The session is based on a project that Handshaw, Inc. is doing with partner Vector, CSP for the Coast Guard, in which we will deliver maintenance procedures on hand held devices. The devices are capable of instant updates through our hosted content management system, Lumenix. To learn more about the workshop visit www.uscghpt.org. Check back on Monday, 9/20 for highlights from our session and the HPT Workshop.

Stay tuned for details as they become available.

Stay tuned for additional details as they become available.

I am pleased to announce my new partnership with Interact Studio in Charlotte, NC, where I’ll begin teaching courses this fall.  Read more below to learn about my course offering and register for the first workshop.    The Results-Based Learning Series: It doesn’t cost any more to design learning that gets results - it just works better.  "Training Request?  Ask Questions First" - Friday, November 5th"Learning Analysis?  You Can’t Afford Not To" - Friday, January 14th  "Learning Design:  Using a Practical Systems Approach" - Friday, March 11th"Learning Measurement:  Making it Practical and Useful" - Friday, May 13th To register visit: http://dickhandshawworkshop.eventbrite.com  Courses are $95 each or $300 for the series.  Use the discount code "resultsbased" for the series discount. Course Descriptions: Training Request? Ask Questions First Many performance improvement and training professionals cringe when a client suggests, "We need to do some good old-fashioned, back to basics training."  This re-framing workshop, led by Dick Handshaw, will equip participants to handle this request in a way that yields better results for the participant and the client.  Participants will learn an effective process for partnering with clients to define the performance needs that will lead to business results.  Participants will develop skills using a re-framing exercise that allows them to become both consultant and client as they role-play the skill of turning a training request into a performance consulting opportunity.  Because participants use real projects and people for the basis of the role-play, results are overwhelmingly positive with a high degree of mastery.  This workshop allows participants to discover their own abilities as a consultant and equips them to develop the confidence to use them. Performance Objectives: Engage the client in an open discussion about business needs and the performance required to support them. Practice and improve eight skills that will facilitate the re-framing discussion. Identify opportunities to conduct further analysis of the performance needs in order to identify learning needs. Friday, November 5th :: 8:30 am - 12:00 noon  Learning Analysis? You Can’t Afford Not To "Analysis? We don’t have time for that. It’s too expensive. Truth is, I’m not really sure how to do it." Sound familiar? Sure it does. In this session, Dick Handshaw will show participants how analysis has saved him more money and earned him more clients than anything else he has done in the past twenty-five years in his business. Completing a task analysis may be tedious, but it is neither difficult nor expensive. Participants will complete a task analysis during the session and will learn how analysis benefits them in course design and development. Participants will also learn what to do with an audience analysis and a learning culture analysis. Performance Objectives: Determine what type of analysis is needed given the situation, risk and the business case. Conduct a task analysis that is useful in course design and doesn’t take too long or cost too much. Conduct an appropriate audience and learning culture analysis that serves a useful purpose in learning design and development. Friday, January 14th :: 8:30 am - 12:00 noon Learning Design—Using a Practical, Systems Approach Many opinions have been shared about the value of Instructional design over the past twenty years. Some say it is obsolete and out-dated. Some say you can’t even think about designing learning solutions without it. Dick Handshaw says both schools of thought are right. Dick says, "It’s not really instructional design that’s outdated, it’s the way people are using it that’s outdated." In this session, participants will learn how to apply ISD principles to real-life learning design opportunities in a way that focuses on results, and also saves time and money. Participants will know how to write performance objectives and be able to write them correctly and easily by the end of the session. Dick will help participants understand why they should design the measurement strategy before they design the learning and participants will begin using a systematic way to select delivery systems for blended learning solutions. Performance Objectives: Write performance objectives that actually define something to measure against. Use design of measurement instruments as a way to improve learning design. Make intelligent decisions about delivery systems that take into account learning culture and business issues Friday, March 11th :: 8:30 am - 12:00 noon Learning Measurement—Making it Practical and Useful Measurement is undoubtedly the least understood aspect of learning design. Most people write test questions the way their teachers from grade school wrote test questions. A lot of people don’t write them at all. And when they do, writing tests is something done as an afterthought. Many trainers look through their written material and make up test questions. In this session, Dick Handshaw will help participants learn why it is important to design, or at least think about the testing instruments, before designing the learning itself. There are only two major mistakes made when designing tests. One is selecting the wrong type of testing instrument. The other is writing test items incorrectly. Through Dick’s proven techniques, participants will be able to avoid both of these mistakes. Participants will leave with the ability to use test data to improve learning design and to evaluate the mastery level of their learners. Performance Objectives: Select the correct type of testing instrument for the learning solution. Develop testing instruments that are valid and reliable. Use testing data to improve the learning and to evaluate the mastery level of learners. Friday, May 13th :: 8:30 am - 12:00 noon About Interact InteractAuthentically.com focuses on Authentic Communication in Business, with programs in speaking, writing, listening, e-learning, productivity, instructional design, social media and leadership communication.  Interact Studio is located in Charlotte, NC. Directions to the studio:  http://www.interactauthentically.com/wp-content/uploads/2010/08/Interact-Studio-Directions.pdf

Stay tuned for additional details as they become available.