Let's Embed Your Knowledge into AI by Trial and Error Instead of Annotation.

The large annotation cost has always remained a challenge for machine learning applications in ubiquitous computing. Programmatic weak supervision (PWS) is a promising solution, but an efficient method and system for domain experts to build it interactively from scratch has not yet been developed. This paper shows the necessity of not only increasing PWS sources but also modifying or removing them and the method to achieve this. Also, we developed a prototype system to build artificial intelligence through trial and error with PWS. The user study with 15 users verified the necessity and effects of modifying and removing PWS sources through this method.