An experimental or preliminary trial or test of a solution on a limited scale. In general, there is a way to pilot all or some part of every solution or new design that you want to implement.

Why use a pilot?

Lower risk
  • Lower risk of failure by limiting resource usage
  • Assess true performance in controlled-but-live experiments
Learn
  • Confirm or disprove expected results and relationships
  • Test and validate the benefit of the proposed solutions before roll out
  • Validate your measurement system
  • Understand expected variation in the process and its possible impact to the customer
  • Improve the ability to better predict monetary savings from a proposed solution
Improve solution
  • Identify additional improvements in either the solution or implementation launch itself
  • Improve future projections of benefits of full roll out
Stakeholder relations
  • Increase opportunities for feedback
  • Increase stakeholder buy-in
  • Quickly deliver a version of the solution to a targeted segment of the client population

What is a pilot?

An experimental or preliminary trial or test of your solution on a limited scale.

When to pilot

In general, there is a way to pilot all or some part of every solution or new design you want to implement. It is almost always worth the extra effort to pilot. Consider doing a pilot when:

  • The scope of the design is large
  • The new product/service could cause far-reaching, unintended consequences
  • Implementing the design and/or solutions will be a costly process
  • The design and/or solutions would be difficult to reverse

Traits of a pilot

  • Small scale
  • Reversible
  • Designed to incrementally increase employee involvement
  • Utilise simulations — process improvement may be simulated using mathematical or computer-based modelling

How do we do it?

Design the pilot

  • What are you trying to do?
  • What do you hope to achieve?
  • What is the definition of success?
  • How are you going to get there?
  • Who is impacted?
  • From whom do you need help?

Pilot plan

Plan Pilot Purpose and Goals
  • Define what needs to be piloted/implemented
  • Reference the Project Charter, especially the Project Benefits and Objectives. Will your pilot be able to demonstrate that achieving these is possible?
Pilot Scope
  • Prioritisation matrix can be used to rationally narrow down the focus of the team before detailed implementation planning can happen
  • What is the size or boundaries to what will be measured?
  • How will you ensure the pilot is as realistic as possible?
  • Identify when and where pilots will be run
Timeline
 
Activities List
 
Clearly articulate the steps and tasks necessary to test the proposed solutions
  • Action
  • Owner
  • Process it affects
  • Schedule
Determine how pilots will be conducted and length of time
  • Pilots should be run for as long as it takes to understand if the process is stable or not
Determine result tracking needs
 
Budget and resources required
 
Identify team members, equipment and material to be used
 
Plan appropriate training and communications for the pilot
  • Documentation required
Monitoring plan and operational definitions of measurements to collect and goals to be met.
 
How will you evaluate results?
 
How will you monitor and deal with issues during the pilot?
  • Minimise disruptions

Run the pilot

  • Collect data on internal and external factors that may be influencing the process
  • Expose the pilot to as broad a range of inputs and process conditions as possible

After the pilot

  • Collect and evaluate pilot results
    • Analyse the gaps between the predicted performance and the actual performance
  • Root cause the gaps to determine why and if solution changes are needed
  • Analyse the pilot plan. What worked? What didn't? What had to be added or changed?
    • Communicate pilot results
  • Create a summary of the strategy used to pilot the solution and communicate the results achieved
    • Solicit stakeholder feedback

Adjust the pilot plan for any problems or learnings that occurred during the pilot. Also adjust for the difference in the scale of the roll-out.

Do you need to run another (improved) pilot(s) or are you ready to apply the solution?

Example

See GOLEANSIXSIGMA's Pilot Checklist Template