What is a Smile Sheet?

Even if you have never heard of a “smile sheet”, it’s almost certain that you’ve seen one or had to complete one.

The smile sheet goes by many names: evaluation form, feedback sheet, level 1 form and so on.

The main driver for the adoption of the smile sheet as a near-standard practice, comes from the popularity of the Kirkpatrick evaluation framework. We’ve talked about Kirkpatrick’s framework a bit more deeply in other blog articles, but suffice it to say that it’s an effective and powerful model to evaluate the actual impact of your training efforts. Especially when it is combined with modern digital evaluation tools and systems.

Ground Floor

Smile sheets are the primary tool that we use to gather information for the first level of evaluation in the Kirkpatrick model. This first level is all about the immediate reaction trainees have to the session they just completed. The rationale behind this first level is that for the training to have any chance at being effective it needs to be well-received at a visceral, emotional level.

This means that people should enjoy the session, feel comfortable, welcome and stimulated. When they reflect on their training we want it to be a positive memory. This means that any actual learning that occurs is associated with that positive memory. It means that trainees are more likely to volunteer or attend future training sessions.

So smile sheets are used to get an idea of how trainees experienced the training session. They have to be administered directly after training so that recollections don’t become distorted by time. Despite the fact that smile sheets have become a sort of throwaway ritual to many trainers, it’s actually a precious and rare opportunity to gather information that can transform the quality of your training. All you need to do is ask the right questions.

Asking the Wrong Questions

The truth however, is that most trainers tend to ask the wrong questions when putting together a smile sheet.

When you compile any sort of questionnaire it’s vitally important that the way you formulate the items is done in line with your measurement needs. In other words, you need to make sure that the answers your items are likely to receive, are actually the answer that you want and need. In general, smile sheets are not put together with a great deal of care. It would be surprising if many organisations bother to peer-review or otherwise test something as taken-for-granted as a smile sheet. The truth is that the small window of time we have when smile sheets are usually handed out is an incredibly valuable and rare opportunity.

We should therefore be taking extra-special measure to make sure that our smile sheets generate maximum value for everyone involved. By re-thinking the questions we put on our smile sheets we can efficiently and effectively transform the quality and kind of information we gather for much more important insights.

Asking the Right Questions

Smile sheets have become diluted and aimless as they have gone from a novel and exciting training evaluation tool to an almost ritualized chore for most trainers. So the first thing we must do in order to  revitalize our smile sheets is to undo that ritualization. We must “un-traditionalize” them and check our own assumptions about how they are constructed.

Jim Kirkpatrick, Don Kirkpatrick’s son, notes that we tend to ask questions of trainees that relate to us more than it does to them. The first level of the Kirkpatrick evaluation model is meant to look at the trainee’s’ immediate reaction to the training. Instead we tend to ask our trainees things that relate to us as trainers and what we do instead. For example, we may ask people if the food was good or the seats were comfortable. We are asking them to evaluate us as facilitators, course designers and so on. Dr. Kirkpatrick junior basically sees two broad categories of evaluation item. The ones we tend to use are trainer-centered and the ones we should be making more use of are learner-centered.

The key difference is that learner-centered questions focus on the learner’s experience. It may not matter that the facilitator spoke a bit too quietly, that the air conditioner was  a bit too cold or anything else like that. As long as they reacted well to the training in terms of the learning experience. Often this is simply a case of phrasing, but importantly, poor item formulation can seriously bias the information you gather. So we should formulate questions that make the trainee reflect on their own experience rather than impersonal opinions on the training as a whole.

Ask the trainee to relate their experience of the the training. Ask them whether they learned something new. Don’t ask them whether they think you were a good facilitator. After all, we want their reactions to the training itself, not to the trainers or the quality of the biscuits.  

For the most part, Jim Kirkpatrick suggests that we use “I” questions when presenting smile sheet to trainees. For example, you wouldn’t ask if “the” questions were easy to understand. Rather you would ask for agreement with a statement like “I found the questions easy to understand”. 

It’s a subtle difference, but as anyone with a background in questionnaire design will tell you, if you are sloppy with the formulation of your items, even a seemingly subtle difference can give you wildly different results.

Don’t believe me? Try this: create a new smile sheet that contains the same items, but with different formulations. Randomly hand the old and new sheets to half of your trainees. Afterwards you can do a comparison between the results and you should see a significant difference between the average ratings of the two groups. Even though they attended the same session. 

When a person responds to a questionnaire item they themselves are not aware of the subconscious priming and framing that’s taking place, but it’s always there.

Leveling Up

While we think of smile sheets as being a tool to gather information for Kirkpatrick’s first level, there is no reason why some questions can’t also give us a heads up on higher levels in the model. It’s fine to also ask questions about trainees thoughts on application of the knowledge in their jobs. You can ask what sort of help they think they’ll need in order apply things in practical terms. Asking reflection on learning questions (that belong more to level 2) is also fine. You don’t have to compromise on level 1 data in order to get a head start on level 2 and 3 data.

Opportunity Knocks

As we said earlier, this is one of the best and only time you will have direct access to the trainees so the smile sheet is the frontline weapon in your training evaluation arsenal. If you spend a little time testing and refining it, you may find that your insights into your training practice becomes substantially deeper, which will show as improvements on all four levels of the Kirkpatrick model.