Matching of hand-drawn flowchart, pseudocode, and english description using transfer learning

An algorithm plays an important role when solving a problem. It is challenging to comprehend for computer novices or machines. Therefore, a textual explanation is provided to illustrate the algorithm. To understand an algorithm, a method needs to be devised to find or generate the corresponding text description and vice versa. This paper matches an algorithm in a variety of forms, such as pseudocode and hand-drawn flowchart, with the illustrative text written in English to facilitate a thorough understanding of the algorithm. The experiment includes a proposed set of rules for generating pseudocode from a hand-drawn flowchart and a proposed S-DistilBERT-based transfer learning method to determine the similarity match score between multiple forms of algorithm and text description. Basic block and line identification, as well as OCR-ization, are used to characterize the hand-drawn flowcharts. The experimental result show that we can generate the equivalent pseudocode in 8 5 % cases, and our fine-tuned S-DistilBERT model can accommodate the matching text for the existing pseudocode with 7 5 . 5 9 % and the generated pseudocode with 7 4 . 5 7 % accuracy. We also find the appropriate description from an algorithm in the top five matches in 30 out of 50 cases. The rules are found to be adequate for non-recursive flowcharts.