Open evaluation of new product concepts at the front end of innovation: objectives and contingency factors

The proliferation of innovation contests has fostered community-based idea evaluation as an alternative to expert juries to filter and select new product concepts at the fuzzy front end of corporate R&D innovation. We refer to this phenomenon as open evaluation, as all registered participants can engage in jury activities like voting, rating, and commenting. While previous research on innovation contests and user engagement includes participant-based evaluation, the investigative focus so far has not been on this phenomenon. Access to jury activities in open evaluation practice contradicts innovation theory, which recommends careful selection procedures to establish expert juries for assessing new product concepts. Additionally, little is known about contingency factors that influence the performance and acceptance of open evaluation's results. To address these two questions on the objectives and contingency factors for open evaluation of new product concepts, this study applies exploratory multiple-case research of open evaluation in nine innovation contests. Data collection encompassed expert interviews and complementary sources of evidence. Results indicate that firms pursue six distinct objectives to support participant-based generation and selection of new concepts. In addition, eight contingency factors influence the performance of open evaluation and the acceptance of its results. Finally, results showed open evaluation output to efficiently complement jury decisions in filtering and selecting ideas for new product development.