“All models are wrong, some are useful.”

Image of George P Box

The statistician George Box (1976) coined the phrase “All models are wrong, but some are useful.” I have used this quote many times… in workshops, as content in lessons for clients on modeling and simulation and at cocktail hour. Okay not at cocktail hour!!

If you are one of the seven avid followers of this blog, you have seen the continual efforts to make the Performance Improvement Process Model, (PIPM) now in it’s tenth iteration, useful. Every time I post an update or use it in an article, I get more feedback that moves me and the model closer to that goal.

Recently I received a comment from my friend and colleague Dr. Jim Hill, the CEO of Organizational Performance Systems, regarding the PIPM on LinkedIn. Jim reminded me that part of the value consultants bring to their clients is the adaptation of “our” models (there are soooo many) to fit their processes and language. So very true!

Another good friend and colleague of mine, Lieutenant Commander Janice Kirk, is doing exactly what Jim described, using the PIPM and a host of other models to inform the creation of a process model for the Canadian Armed Forces (CAF) Materiel Division (the folks that buy the big equipment aka tanks, planes and submarines) to clearly explain to project stakeholders what the differences are between Needs Assessment/Analysis and Training Needs Analysis. With her permission, I am sharing it below.

Janice’s NA to TNA process

I know… hard to read in this form! If you right click the image and select “open in a new tab” you can see a larger image that’s much easier to read. (You’re welcome!) If you are a “keener” and compare this model to the PIPM, you will see that the major steps are the same… it’s the bits in between that:

1. Drill down more,
2. Address the ADM Mat specific requirements, and
3. Link to the military guidance (Canadian Forces Individual Training and Education System (CFITES) policies and processes.

It’s a great example to underscore Dr. Hill’s advice that we need to work within and adjust to the client’s processes and just as importantly, speak their language. If you did open up Janice’s diagram in a new tab, you have seen it is rife with “CAF” TLA’s (Three Letter Acronyms) and more! Just for fun, I have included them below under TLA’s in case you are REALLY interested 🙂

Box, G.P. (1976). Science and statistics. Journal of the American Statistical Association 71(356) pages 791-799.


ICT: Initial Cadre Training – training which enables CAF members to perform the tasks associated with new equipment, systems or directives upon their fielding, delivery or initiation. The responsibility for this training rests with the Project Management Office/Contractor for new equipment and systems, unless specified otherwise in the statement of requirements.

JBS: Job Based Specification – the document that describes the tasks, medical requirements etc for specific jobs.

MES: Military Employment Structure – The arrangement of CF [Canadian Forces] jobs into structural elements, consisting of military career fields, occupations and sub-occupations that collectively provide the necessary management framework for the personnel life cycle of activities across all components of the CF and throughout the spectrum of conflict.

MOS: Military Occupation Structure- The arrangement of Canadian Armed Forces (CAF) jobs into structural elements, consisting of military career fields, occupations and sub-occupations that collectively provide the necessary management framework for the personnel life cycle of activities across all components of the CAF and throughout the spectrum of conflict. It is a structural arrangement of the work performed by members of the CAF.

MOS ID: Military Occupation Specification Identification Code – Equivalent to a National Occupation Code.

QS: Qualification Standard – Describes how well the job should be done using Performance Objectives (POs).

QSP: A combined QS and TP. Far more efficient.

RQ: Rank Qualification – A qualification, obtained via formal training, that enables a member to perform one or more entry level occupational jobs, where required to attain a new substantive rank (RQs replace former occupation qualification levels, such as QL3, QL5, etc.)

SST: Steady State Training – training identified in the QS/RQ for normal career progression.

TP: Training Plan – describes the instructional programme which will enable the learner to achieve, at optimum cost, the performance objectives from the QS.

Just gimme training…

Author’s note: The names have been changed, faces blurred and voices altered to protect the innocent in this post. 

It’s a story I have heard many times. The boss wants training, the Instructional designer – performance technologist – analyst KNOWS that training isn’t going to fix the problem but is told to “gimme the training.”


A client told me awhile back that they needed a course. That’s always the first red flag right!?

I asked “Why? What’s the problem?” Turned out that my client was told that a group in another part of the organization wasn’t performing as needed when requesting and using my client’s services to get their job done. Their boss wanted my client to train his people. Classic example of picking the solution before understanding the problem.


To make the story easier to understand, let’s call the group that is experiencing the performance issue the “Jocks.” We’ll call my client – the one told to fix the problem, the “Wizards.” The Wizards are responsible for producing some pretty cool magic that helps the Jocks prepare for their games. The problem is that the Jocks don’t understand exactly what type of magic the Wizards can provide, how to ask for or use the magic once they do get it and as you can imagine – Jocks and Wizards don’t exactly speak the same language, so they don’t even know how to interact with each other at this stage.

As a good performance technologist or CPT, I asked for more information from the Wizards and the Jocks – starting at the end – the outcome – the desired level of performance. In this case, that would be the Jocks being ready for the game. Two other key pieces of the puzzle were the Jock’s “playbook” and the Wizard’s “spells book.” Much to my surprise, the playbook used by the jocks to get ready for games had much the same process as used by the Wizards.  It may look familiar:

  1. Define the objective
  2. Plan
  3. Prepare
  4. Execute
  5. Analyse/Evaluate results

Yes… at a deeper level of fidelity, there are some differences, but overall, the Jocks and the Wizards are using the same process, with the same goal. So after carefully reviewing the plays and the spells, it was pretty apparent to me that if we added the information that the jocks had to give to the Wizards at each step above, training really wasn’t necessary. All we had to do was update the playbook!


A meeting was held! After showing the Jocks and the Wizards that what they are doing is similar enough that the time and effort required to build a course might be better spent on improving the playbook, I was not at all surprised to be told, “just gimme the training.”

It might be because the direction given to my client was for a training solution. It might be because the Jocks don’t know what they don’t know at this point and they want to have the training first, work through the process a couple time first and THEN update the playbook. It might be because the Wizards don’t want to pay me to update the Jock’s playbook. I’m not sure at this point. I noted in a previous post that you may just end up getting a training solution when you really need a performance solution. It’s killing me that we are going to spend a lot of time and energy developing this course, knowing that the solution is an updated process.

In 2007 I attended the International Society of Performance Improvement’s (ISPI) “Principles and Practices of Performance Technology Workshop.” Geary Rummler and Roger Addison were my teachers. To this day I remember our discussion about this very situation and Roger’s advice that no matter how much training you develop – always leave them a job-aid.


In 2012, my good friend and mentor Dr. Roger Kaufman – a legend in the field of performance technology (far right), introduced me to another legend, Dr. Joe Harless (middle). I had read many of Joe’s articles which all hold true today. As Guy Wallace explains at the linked article about Joe, you never say no to training. Harless (1985) is also credited with coining the term “Inside every fat course there’s a thin Job Aid crying to get out.”

So I am bashing ahead with designing and developing the course. I’m going to add in a “Student Manual” that will contain one page that explains the whole process. Thanks Geary, Roger A., Roger K. and Joe.


Harless, J. (1985). Performance technology and other popular myths. Performance & Instruction Journal, July 1985.