Technology Demonstration and Market Stimulation Challenges

Challenge and prize competitions are one path that federal agencies take to drive innovation and solve mission-centric problems—whether technical, scientific, or creative. One type of competition is technology demonstration and market stimulation prizes, competitions that result in fully developed solutions to address market failures, solve significant problems facing society, or catalyze and demonstrate breakthrough technical innovations.

Definition

Tips for Challenge Managers

• Deconstruct the problem. Can it be broken into more specific milestones or targets? These component milestones could each be the basis of an incentive prize, or they could represent phases of one multi-stage competition.
• Get stakeholder input. For complex issues, consider enlisting assistance with selecting a prize type and designing the target criteria and rules. Consult a diverse set of experts and stakeholders through targeted interviews. Note that the America COMPETES prize authority encourages agencies to consult widely within and outside the federal government when selecting topics for prize competitions and throughout the lifecycle of a prize. One way to enlist input and ideas in prize design is to host one or more workshops to provide an opportunity for brainstorming and building out prize concepts. Agencies may also empanel advisory committees. Some prize administration vendors, including several on GSA Schedule 541 4G, provide services to assist federal agencies with designing prizes through expert consultations and facilitated workshops.
• Complete a landscape survey. Federal agencies should also consider completing a review of the current companies, research trends, and existing solutions related to the target challenge, either directly or by engaging a partner, vendor, or consultant to complete a report or survey of the business, scientific, and technical landscape related to the target challenge. This step ensures that the target set in the competition is achievable yet sufficiently audacious, and allows agencies to ensure they are not offering a challenge in an area where solutions already exist.
• Identify federal resources. A key step in prize design is to identify resources that could be made available to entrants in the competition that would improve or strengthen submissions, such as open government data or access to technical resources. Federal resources can also assist with competition judging and validation of winning solutions. For example, in the Wendy Schmidt Oil Cleanup X Challenge, a Department of Interior testing facility was used to host physical and laboratory testing of finalist prototype designs for high-performing oil cleanup equipment, and in the Progressive Insurance Automotive X PRIZE, Argonne National Laboratory provided dynamometer testing of the super-efficient finalist vehicles. Federal employees with subject matter expertise often serve as members of competition judging panels as well, directly assisting with evaluation and selection of winning solutions. Federal agencies can also identify resources that can be used to promote winning solutions, such as existing communities, networks, websites, forums, and events.
• Define competition structure. Once a prize concept has been identified, define the competition structure, including timeline, phases, and stages. Decide if the competition will be single-stage, where submissions are judged once to select winners, or whether it will be multi-stage, with one or more down-select points where entrants are narrowed down to a set of qualified teams, semi-finalists, or finalists. Determine how long each of these phases and stages lasts.
• Establish success criteria. Consult with experts in the relevant fields to establish the success metrics/criteria that will be used to select a winner—what does an entrant in the competition have to do to win? These target criteria could be objective and measurable—for example, in the Wendy Schmidt Oil Cleanup X Challenge, winning teams needed to achieve at least 2,500 gallons per minute oil recovery rate with at least 70% efficiency of oil to water recovered. Alternately, some competitions use subjective criteria that are assessed by a judging panel. A combination of objective and subjective criteria can also be used. For multi-stage competitions, each stage may use different criteria for selecting which entrants move forward in the competition.
• Define judging procedures and validation protocols. Based on the target success criteria you select for each phase of the competition, establish a judging plan. Recruit judges who are experts or influential in the challenge subject area to add authority to the judging. This expertise can come from professional experience, education, or current/prior roles. Consider a mix of judges from different parts of the country and different sectors—including public, private, philanthropic, media, and academia. Judges can also be helpful in promoting the challenge within their networks. Do not select judges who have personal or financial conflicts of interest. Note that the prize authority in America COMPETES exempts judging panels from the federal Advisory Committee Act (FACA). Establish a judging plan that will allow for each submission to receive review by multiple judges, and include a plan for how to deal with a large number of submissions, in case efforts to recruit entrants are highly successful. In some cases, judging will involve testing a prototype or product in field conditions. Think about when, where, and how to test and validate the performance of the entries, and consider government laboratories and testing facilities that might be applicable.
• Determine eligibility. Based on goals for the competition and any restrictions of the authority being used to conduct the competition, decide on what types of individuals and organizations are eligible to compete. Under America COMPETES prize authority, eligibility is limited to U.S.-based companies and U.S. citizens and permanent residents, and federal entities and federal employees are not eligible to compete if acting within the scope of their mission or employment. Other authorities may allow international winners. In some competitions, eligibility is limited to particular groups, such as age groups, student status, types of companies, regional/geographic restrictions, or other eligibility requirements.
• Define intellectual property provisions. Agencies can consider multiple approaches to intellectual property provisions for prize competitions, upon approval of agency counsel and the signed consent of the submitting participants. Options to consider include leaving ownership of intellectual property in the hands of the solvers, transferring license for use or ownership to the government or the public domain, requiring some type of open source license, or providing a license for use (and potentially for derivative works) to the government, among other options.
• Identify incentives. Determine the monetary and non-monetary incentives that are available to encourage solvers to enter the competition. To determine the level of monetary prize purse, consider: 1) available budget, 2) prize purses in similar competitions, 3) likely costs for solvers to enter and compete, 4) other market forces, such as cost of current best-of-class products or costs of commercialization, 5) amounts meaningful to attract attention of the media and other key stakeholders, and 6) quantity of prizes to award (such as “seed funding” awards to finalists after an initial competition stage, awards to first, second, and third place winners, or awards to winners in multiple categories. Note that the prize authority in America COMPETES caps the monetary awards at $50,000,000 per prize award. Non-monetary incentives can also motivate teams to compete and include incentives such as: recognition at an awards ceremony or other events; vouchers for access to business consulting, laboratories, or testing resources; media exposure; placement in ads; exposure to a celebrity judging panel or industry experts; or commitments to deploy the winning solution in a real-world context post-competition.
• Draft competition guidelines and rules. Work with agency general counsel to establish the rules and requirements for the competition, including specifics on competition timeline, incentives (prizes), payment, liability, insurance, how to enter/qualify, submission requirements, eligibility, intellectual property, judging panel, judging process, judging criteria, and other key requirements. In addition to the official rules and legal requirements for the competition, consider creating a plain language summary of the competition timeline, structure, criteria, and judging process, including graphics and images if helpful. Especially for complex, large competitions, consider publishing the draft competition guidelines and rules for a public comment period to allow key stakeholders to raise questions and provide feedback on the competition rules. Create any templates or forms needed for submissions from entrants.
• Establish competition operating budget. In addition to funding for the prize purse, prize administration usually requires funding for activities including vendor support; marketing, outreach, collateral, and advertising; travel to key competition events such as competition launch; industry meetings; field testing; awards ceremony; and cost of events.
• Define outreach plan. Determine who your target audiences are, including solvers, key industry stakeholders, partners, and the media. Establish a communications and outreach plan that goes beyond posting the competition on Challenge.gov to include steps for reaching key audiences, such as through attendance at industry events, webinars, advertising, media outreach, industry association and university networks. Engage the public through online and social media and other complementary programs such as citizen science, education programs, and public voting. The communications and outreach plan for a prize competition should include steps to be taken during each phase of the competition: pre-launch, competition, announcement of winners, and post-competition. Particularly for longer competitions, plan for how to keep enthusiasm and public interest high while the competition is underway prior to the announcement of the winners. Consider how to maintain the enthusiasm and motivation in the community that forms around the competition after the prize is awarded.
• Identify key program risks and create mitigation plans. Identify key risks to the success of your prize competition and create mitigation plans that will help to reduce any negative impacts of those risks. Common risks in prize competitions include: how to handle too few/many entries; safety plans for physical field testing; enforcement of intellectual property violations; and the potential for no winning solutions.
• Select partners and vendors. Identify partners that can support the prize through monetary or non-monetary means. Note that the prize authority in America COMPETES allows for prizes to be co-funded by multiple federal agencies and by philanthropic and private-sector partners. Identify partners that can contribute key non-monetary resources to the competition, such as testing facilities, data, marketing support, and personnel. Identify other federal agencies with interest in partnering on the funding and administration of the competition given shared goals. Finally, consider selecting a prize design and administration vendor. These contractors can provide services to help federal agencies design, implement, publicize, and evaluate challenges. Prepare a statement of work for these vendors.

Resources

Examples

Technology Demonstration and Market Stimulation Organizations