Successful pilot projects gate investment in your company and repeatable sales, how can you figure out if your pilot is on the right track early with minimal direct feedback from your customer
Andy is Executive Director of the Georgia Cleantech Innovation Hub, driving climate tech innovation in Georgia and the Southeast. With over a decade of experience spanning battery storage, solar, smart grid and refrigerants, he connects entrepreneurs with resources to accelerate sustainable solutions. Andy’s work combines expertise in growth strategy and alliance building to support the region’s transition to a sustainable future.
Hey there,
Sometimes it is easy read your customer’s signals, but often, it is not. Getting an early read on how your pilot is faring and whether you are on track to meeting its objectives can be the difference between success and “death by pilot”.
The journey from proof-of-concept to an adoption-ready cleantech solution is fraught with challenges. Promising innovations fail for myriad reasons, but companies often get trapped at the pilot stage because they misread the signals that they are getting about their solution and the company they are working with. As someone who’s been on both sides of pilots with cleantech startups, I’ve seen how important it is to interpret the signals that you are getting from you pilot partner, because misinterpretation leads to frustration, wasted effort, and sometimes overly ambitious projections that can be delivered upon
Today, we are sharing a few insights to help you discern what some leading indicators are telling you about the odds of your pilot’s success, including:
- The make up of your ideal project champion (lead)
- Why who drafts the pilot’s performance expectations matters
- The people are you are meeting with may impact your project
Leading Indicators to Help You Assess Your Cleantech Pilot Success
Conflict that is not productive, metrics that are going sideways, and apathy are easy to recognize signs that your pilot project is headed in the wrong direction. Strong executive engagement, willing promotion, and solid results are, on the other hand, signs that you are hitting a home run in your pilot. Before the easy-to-read signs appear, how do you accurately evaluate how your pilot project is faring? There are a few signs that will not only tell you if you’re on track, but also highlight specific areas to lean into or seek improvement.
Let’s dive into a few leading indicators of pilot project success:
1. Your pilot champion is an up-and-coming leader and is paired with an experienced program manager
Pilots are risky for both parties. While the scope of pilots is limited compared to large scale rollouts, the amount of resources that both you and your customer put into the effort rarely meets the economic value of the project. Therefore, you need your champion to be someone that is viewed as an up-and-coming leader at your pilot partner. This person should be able to articulate to you a long-term vision and how this project fits in, and show a willingness to take calculated risk to make it happen. Your champion may not have the experience that a seasoned executive has, and they may only have one or fewer successful pilots under their belt, but they have the drive to solve problems and the mentors that can help them muster the resources to help the pilot effort along. Importantly, the champion needs to be supported by a program manager with specific experience working with startups and working with the internal pilot stakeholder organizations (e.g., technical, procurement, operations, legal, etc.). The program manager will help you navigate the project specific hurdles and allow your champion to focus on the hurdles related to long term adoption.
What happens if you don’t have both? All pilots have a champion, if you don’t have one, that is a major red flag. However, propensity for risk-taking is what is usually lacking in a champion, especially if your champion is an experienced executive. If that is that case, it is imperative that your team finds and recruits a risk-taker to pilot partner’s team. Without this role filled, the road to pilot success for your company is long. Similarly, if your pilot partner’s program manager that does not have the experience or gravitas, you need to either fill that gap with experience from your own team, think of your PM as a “shadow PM” for your pilot partner. A PM that can ask the right questions of your partner can overcome a lack of experience in your counterpart’s PM. You can also ask your pilot champion to bring in a new PM. If your pitch is well-reasoned and well-delivered, their willingness to augment their pilot team can say a lot about how bought in they are on pilot success.
2. Your partner is writing the first draft of the pilot performance metrics
As easy as it is to say that you should draft measures that “lock-in” purchase post pilot, e.g., meeting performance metrics or production milestones, there are few if any pilot partners that will do that during pilot contracting. The next best thing is to have your pilot customer, ideally with your guidance, draft the performance metrics for the pilot. This is beneficial for a few reasons.
- First, drafting helps build ownership and teases out what success really is for your pilot partner. This does not relieve you from having your own goals and objectives, you should absolutely have your own strong hypothesis as to what the expected and stretch performance metrics should be. While the performance criteria that your client develops will naturally be one-sided, focused on your company’s implementation and execution, you should also push your customer to have metrics related to their deployment of your technology.
- Next, it will help you assess what both their understanding of your technology and what their buying criteria may be. Information asymmetry is a significant and overlooked problem in early stage sales and technology adoption. Having your partner express what they think is both realistic for success of your pilot will give you a leg up on understanding what they may eventually expect from your technology in order for it to be purchased. Aside from any gaps in performance expectations that may exist, be cognizant as to the level of thought put into the metrics your client proposes on implementation. If they don’t put much thought into these, add some time both to your pilot implementation and sales process.
- Lastly, it may help provide you with some indirect competitive intelligence. The companies that you want to run pilots with are the same companies that your competitors want to run pilots with. The metrics that your pilot customer states will likely be a aggregation of performance from the status quo solution, your competitors, and from the base techno-economics required for long term success.
What if you client does not take the pen? While not ideal, it is fine for your company to take a first stab at performance metrics for your pilot project. However, you want real feedback from your pilot partner on these. In addition to having a syndication plan that gets input from the key stakeholder departments at your client, you should slightly overstate your expectations for the metrics that you expect from your customer, but never understand your own expectations. Why? Your customer will engage and negotiate you down on their metrics; this gets you the engagement you need. However, you should know better than any what your company is highly likely to deliver on and will also show the progress and performance that you need for future commercial success. Understating or overstating your capabilities should to be avoided for successful pilots.
3. You have open lines of communication with procurement
Startups don’t like interacting with procurement organizations, and vice versa. Procurement processes are not set up to deal with companies that don’t have well-established processes, lack the data they need, are not prepared to negotiate on multiple terms outside of pricing. Startup should aim to have exploratory conversations with their pilot customers around the expectations and requirements of the procurement process. A pilot champion may know the steps, but typically they are not the right person to be having procurement discussions with; you really need to be speaking directly with a person that manages the procurement process. Prior to having any discussion with procurement, you should have prepared yourself with a discussion with a coach or mentor beforehand, especially if you are unfamiliar with how these organizations work. Inexperienced companies that ask naive questions, send up unnecessary red flags with pilot partners. As with any conversation, prepare yourself by understanding what you will be asked for from someone that was once a practitioner.
What if you are not having those discussions? It can seem to be a little forward for you to ask your pilot champion to introduce to procurement, but it shouldn’t. The best way to frame this ask in the pilot goals and objectives discussions. A success metric could be achieving a specific requirement of the procurement process in addition to technical or business model demonstration. This opens the door for you to have those exploratory conversations with procurement. Also, it is important for you to realize that your pilot champion may (or may not) recognize that procurement can be a formidable blocker to successful execution of your project. Your champion may not want to give you that access willingly, however if it is framed in the goals of the project then you are much more likely to have success in transitioning from pilot to commercial sales.
Now, let’s dive into some leading indicators of pilot project failure that are also less obvious.
4. Your pilot champion is managing performance expectations at a first setback
Every project lead needs to manage expectations from time-to-time, however, if your project champion starts managing expectations at the first signs trouble, your pilot is at significant risk. Your startup will begin any pilot with little or no reputation at your client, and it will need to build believers fast. An early yellow flag from your project champion will make the hill that your company needs to climb to earn respect steeper and longer. Instead of immediately moving to manage expectations, you want your project champion to first roll up their sleeves and dig into problem solving with your team. As a startup leader, you should invite them to do so. The best project champions should be motivated by the career opportunities that success brings, and not be satisfied by mitigating risk, especially early in a project. If it turns out that root cause problem solving uncovers that the pilot requires more time or resources than first estimated, you want your project champion to request the resources needed, but hold the line on the performance expectations. Meeting your the performance expectations of the pilot that make your product commercially viable is of central importance, and you need your pilot counterparts to manage toward those with you.
What can you do to avoid this? A frank conversation early in a project about how the pilot team will work is recommended. Setting the expectation that the your team will be open with its results and challenges and quick to engage your partner in problem solving when something is not going to plan can help you navigate the potential for your partner to lose faith early. Also depending on the degree of rapport that you have with the pilot champion and as leader of your company, you can state the what success means for your company beyond the potential for future commercial success. You want to your partner to understand that meeting the performance objectives of the pilot are a culmination of a significant effort and success validates the effort, consistency and resilience of your team. Doing this goes a long way in making your project champion a member of your team and increases the probability that they also take as much ownership in its success as you do.
5. You are regularly attending meetings with 10+ attendees regarding your pilot project
Pilot projects are sized to prove performance of a technology in real-life context, without requiring extensive oversight. These projects should not require regular large, resource intensive meetings to manage progress or problem solve. Yes, kick-offs, performance reviews will involve larger audiences, but day-to-day management should not. Regular large meetings can be indicative of a couple of things which could be harbingers of issues going forward. First, these meetings could be signaling that your partner does not have sufficient expertise to implement your solution. This is not necessarily a knock on your partner, your solution could be too difficult to deploy causing them to reach for additional resources. If large meetings are becoming the norm for your pilot team, you should have dig into the root-cause before the project goes sideways. Second, your pilot could be over-scoped. A pilot is for all intents and purposes an experiment, you should be focusing on proving or disproving hypotheses that are critical for advancing your technical and business model development. If you are trying to do too much in a pilot, it will show in the long run with results that . The motivation to test many variables in a single pilot is completely understandable as the “death by pilot” adage is pervasive amongst cleantech startups. However, consider that a few, right-sized pilots with progressive metrics at multiple customers may put you in a better position to advance your technical and commercial objectives than a difficult to execute a “everything and the kitchen sink” pilot.
What can you do to avoid this? As hinted at above, much of this can be avoided in good project scoping and not being afraid to more than on pilot. A right-sizing effort should focus first on the two most important proof-points and the related metrics that you are trying to get out of the pilot for commercial viability. There are undoubtedly more elements for product that you need and want to test, and those can be layered into a pilot but ensure that those are contingent on first demonstrating the central proof-points. Doing this will help you to avoid having too broad of a focus that makes implementation more risky and more likely to get bogged down in conversations that are not critical to the project’s main objective.
Correctly interpreting leading indicators can help startup leaders to navigate the pilot process and manage toward success. These are only a few of those leading indicators; we would love to hear in the comments from the community on other leading indicators that startups and their partners have encountered and more importantly how they used them to lead to pilot success stories.
Fresh Links, Tools, and Resources
If you graduating from a university and losing access to its resources or the “garage R&D center” is no longer meeting your needs, here is a not exhaustive list of maker and prototyping spaces.
Weekly Resource List:
Mass Collective (364 Nelson Street SW Atlanta, GA 30313) — An inclusive makerspace that provides affordable access to tools, technologies, and teachers in a variety of disciplines.
Freeside (3043 Commerce Way Atlanta, GA 30354) — Freeside Atlanta is a 501(c)3 non-profit makerspace built entirely by volunteers. The Freeside community has built the space with tools and equipment for electronics, 3D printing, woodworking, metalworking, art, design, classes, and events.
Roswell Firelabs (1601 Holcomb Bridge Rd, Roswell, GA 30076) — Makerspace hosting 4,200+ sq. ft. of workshop and classroom space intending to build community through making, enabling innovation at every level, and promoting personal growth through hands-on experience.
Curiosity Lab (147 Technology Pkwy, Peachtree Corners, GA 30092) — A 5G enabled living laboratory for startups and established companies to deploy and prove out developing technologies in a real-world testing environment with no roadblocks, joined by the next generation of intelligent mobility and smart city technology.
The Maker Station (869 Pickens Industrial Drive North East, Suite# 1, Marietta, GA) — 100% volunteer-driven makerspace serving local makers in Smyrna, Marietta, Kennesaw, Acworth, and surrounding metro areas.
No Comments