Hey! Please no squeeza da banana!
Because when-a you do dat,
Den-a you make her flat….
So please no squeeza da banana!
If you squeeza, Officer, please,
Squeeza da coconut!
I used to love going to Pop pop’s house, and hearing him sing that song.
UC Davis’s Pilot-to-Purchase Project points out, “Public school districts indicate reliance on pilots to make purchasing decisions about education technology products.” And certainly Education Technology publishers rely on pilots as a critical part of the selling process.
A pilot is an evaluation of whether a product will be purchased. A pilot is neither a randomized controlled study nor a Beta test. While randomized controlled studies may have a place, they take too long and cost too much to be used for purchasing decisions. And while a pilot will provide valuable feedback for an Ed Tech provider, the product should already be ready-to-purchase.
But how do we prevent pilots from going flat, like the squeezed banana? How do we make them more meaningful than squeezing a coconut?
- Go into the pilot with criteria for success and a method for measuring that success
- Establish and maintain communications between the EdTech provider, teachers, and student throughout the process.
- Avoid technology problems, and, when they do occur, solve them rapidly.
- Teachers need to be comfortable with the product and technology
- Give the pilot enough time
Sounds easy right? Here’s a hint: it’s not, but worthwhile projects rarely are.
Mathematica Policy Research and SRI International have produced a tool to help educators and EdTech providers keep pilots on track, The Ed Tech Rapid Cycle Evaluation Coach, which is in Beta right now. Some of the key questions for planning:
- What is the problem you are trying to solve or opportunity you wish to pursue?
- Who should be involved in the selectio process?
- What are the technical requirements?
- How much training is required and how will it be delivered?
- What’s the indended implementation approach?
- What kind of data/reporting tools do you need?
And questions for the evaluation:
- Do intended users find the resource engaging?
- Is it accessible for all target users?
- How easy is the technology to use?
- Is the effect of the pilot above a threshold?
No more squeezing bananas, let’s make pilots meaningful. If you’re interested in conducting or planning pilots, let’s talk further.