MagicToon: A 2D-to-3D creative cartoon modeling system with mobile AR

VR, pp. 195-204, 2017.

Cited by: 8|Bibtex|Views66
EI
Other Links: dblp.uni-trier.de|academic.microsoft.com
Weibo:
Our results have shown that MagicToon has relative advantages compared with both traditional sketch-based modeling systems and augmented reality coloring books

Abstract:

We present MagicToon, an interactive modeling system with mobile augmented reality (AR) that allows children to build 3D cartoon scenes creatively from their own 2D cartoon drawings on paper. Our system consists of two major components: an automatic 2D-to-3D cartoon model creator and an interactive model editor to construct more complicat...More

Code:

Data:

0
Introduction
  • Cartoon painting with pens and paper is a natural activity and an important experience for children to practice and express their creative skills.
  • There are several AR coloring book systems either from academic or industry, such as colAR [7], Crayola Color Alive [8], Disney’s Color and Play [11] and Chromville [6]
  • These systems can recognize the colors users paint on paper and use similar colors on 3D models in AR context.
  • This kind of systems limits the creativity of children because children cannot create their own personalized cartoon drawings and 3D models within the system
Highlights
  • Cartoon painting with pens and paper is a natural activity and an important experience for children to practice and express their creative skills
  • The drawings were created by different users: the Cherry (Figure 10a) and the Tree (Figure 10c) were drawn by one colleague who had no 2D artistic experience, the Bunny (Figure 10b) was drawn by another colleague with five years 3D modeling experience and the Bear (Figure 10d) was drawn by a twelve years old child
  • We found that coloring and modeling in augmented reality context gave them a novel way to interact with virtual models when using MagicToon
  • We found that female enjoyed in coloring activities much more than male and became frustrated more when using RigMesh compared with male
  • We present MagicToon, a creative 2D-to-3D personalized modeling system on mobile devices
  • Our results have shown that MagicToon has relative advantages compared with both traditional sketch-based modeling systems and augmented reality coloring books
Methods
  • 43 participants were participated in the user study, 18 male and 25 female.
  • These participants were aging from 10 to 13 years.
  • None of the subjects had significant 3D modeling experience, 2D artistic experience or AR experience.
  • Figure 11 shows two subjects were using the system to model objects and author cartoon scenes.
  • Figure 12 shows more results created by the subjects
Results
  • In order to get performance statistics of the model creator, the authors tested MagicToon on a 1.3GHz iPad Mini 2 to model four example cartoon drawings (Figure 10).
  • Figure shows the vote results of the first three additional questions.
  • Figure shows the average scores of the last two questions.
  • Figure shows the percentages of the first ranking frequencies of the three systems in the last two additional questions, in other words, how many female/male gave the highest rankings to the system.
  • Note that Figure 14, Figure and Figure show differences between the results from male and female
Conclusion
  • The authors present MagicToon, a creative 2D-to-3D personalized modeling system on mobile devices.
  • The authors have implemented an interactive model editor that supports to author cartoon scenes in AR environments, giving full play to children’s innovations.
  • The authors' results have shown that MagicToon has relative advantages compared with both traditional sketch-based modeling systems and AR coloring books.
  • A possible extension of the system is to support storytelling functionalities in the future, such as adding dialogs and sounds to the virtual cartoon characters
Summary
  • Introduction:

    Cartoon painting with pens and paper is a natural activity and an important experience for children to practice and express their creative skills.
  • There are several AR coloring book systems either from academic or industry, such as colAR [7], Crayola Color Alive [8], Disney’s Color and Play [11] and Chromville [6]
  • These systems can recognize the colors users paint on paper and use similar colors on 3D models in AR context.
  • This kind of systems limits the creativity of children because children cannot create their own personalized cartoon drawings and 3D models within the system
  • Methods:

    43 participants were participated in the user study, 18 male and 25 female.
  • These participants were aging from 10 to 13 years.
  • None of the subjects had significant 3D modeling experience, 2D artistic experience or AR experience.
  • Figure 11 shows two subjects were using the system to model objects and author cartoon scenes.
  • Figure 12 shows more results created by the subjects
  • Results:

    In order to get performance statistics of the model creator, the authors tested MagicToon on a 1.3GHz iPad Mini 2 to model four example cartoon drawings (Figure 10).
  • Figure shows the vote results of the first three additional questions.
  • Figure shows the average scores of the last two questions.
  • Figure shows the percentages of the first ranking frequencies of the three systems in the last two additional questions, in other words, how many female/male gave the highest rankings to the system.
  • Note that Figure 14, Figure and Figure show differences between the results from male and female
  • Conclusion:

    The authors present MagicToon, a creative 2D-to-3D personalized modeling system on mobile devices.
  • The authors have implemented an interactive model editor that supports to author cartoon scenes in AR environments, giving full play to children’s innovations.
  • The authors' results have shown that MagicToon has relative advantages compared with both traditional sketch-based modeling systems and AR coloring books.
  • A possible extension of the system is to support storytelling functionalities in the future, such as adding dialogs and sounds to the virtual cartoon characters
Tables
  • Table1: Mesh statistics for examples
  • Table2: Timing statistics for examples (ms)
Download tables as Excel
Related work
  • 2.1 AR Books

    AR has shown great potential to enhance entertainment and education for children. Zund et al [29] proposed the concept of Augmented Creativity as employing AR on modern mobile devices to enhance real-world education, opening new interaction possibilities. MagicBook [4] is one of the earliest systems that combined virtual digital content with a book in real world. In MagicBook, children can see three-dimensional virtual models appearing out of the book pages through a handheld augmented reality display. Clark and Dunser [7] presented a new type of interactive AR book experience. In their prototype, users can color the pages of a book and the system automatically maps the colored results to virtual pop-up scenes and 3D models. There is also a commercial product derived from their work called QuiverVision1. Since then, many other coloring book products, such as Crayola Color Alive [8], Chromville [6] and Disney’s Color and Play [11], have shown immense potential in this field. Magnenat et al [20] proposed a method that can detect and track the drawing process alive, enhancing the overall drawing experience further. There is one main limitation among these works that the 3D content cannot be created by users entirely. Although users can color 3D content, they always need to prepare printed line arts provided by the manufacturers. The systems do not enable them to create their own virtual character models.
Funding
  • This work is supported in part by the National Natural Science Foundation of China (No 61173105 and 61373085) and the National High Technology Research and Development Program of China (No 2015AA016404)
Reference
  • S. Arroyave-Tobon, G. Osorio-Gomez, and J. F. Cardona-McCormick. Air-modelling: a tool for gesture-based solid modelling in context during early design stages in ar environments. Computers in Industry, 66, 2015.
    Google ScholarLocate open access versionFindings
  • I. Baran and J. Popovic. Automatic rigging and animation of 3d characters. In ACM Transactions on Graphics (TOG), volume 26, 2007.
    Google ScholarLocate open access versionFindings
  • O. Bergig, N. Hagbi, J. El-Sana, and M. Billinghurst. In-place 3d sketching for authoring and augmenting mechanical systems. In Mixed and Augmented Reality, 2009. ISMAR 2009. 8th IEEE International Symposium on, 2009.
    Google ScholarLocate open access versionFindings
  • M. Billinghurst, H. Kato, and I. Poupyrev. Magicbook: Transitioning between reality and virtuality. In CHI ’01 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’01, 2001.
    Google ScholarLocate open access versionFindings
  • P. Borosan, M. Jin, D. DeCarlo, Y. Gingold, and A. Nealen. Rigmesh: Automatic rigging for part-based shape modeling and deformation. ACM Trans. Graph., 31(6), 2012.
    Google ScholarLocate open access versionFindings
  • Chromville. Chromville. https://chromville.com/.
    Findings
  • A. Clark and A. Dunser. An interactive augmented reality coloring book. In 3D User Interfaces (3DUI), 2012 IEEE Symposium on, 2012.
    Google ScholarLocate open access versionFindings
  • Crayola. Crayola color alive. http://www.crayola.com/.
    Findings
  • E. S. De Lima, B. Feijo, S. D. Barbosa, A. L. Furtado, A. E. Ciarlini, and C. T. Pozzer. Draw your own story: Paper and pencil interactive storytelling. Entertainment Computing, 5(1), 2014.
    Google ScholarLocate open access versionFindings
  • V. Developer. Sdk, unity extension vuforia–2.8, 2014.
    Google ScholarFindings
  • Disney. Color and play. http://www.onlycoloringpages.com/.
    Findings
  • P. Fiala and N. Adamo-Villani. Arpm: an augmented reality interface for polygonal modeling. In Mixed and Augmented Reality, 2005. Proceedings. Fourth IEEE and ACM International Symposium on, 2005.
    Google ScholarLocate open access versionFindings
  • D. Geiger, H. Pao, and N. Rubin. Salient and multiple illusory surfaces. In Computer Vision and Pattern Recognition, 1998. Proceedings. 1998 IEEE Computer Society Conference on, 1998.
    Google ScholarLocate open access versionFindings
  • Y. Gingold, T. Igarashi, and D. Zorin. Structured annotations for 2dto-3d modeling. In ACM SIGGRAPH Asia 2009 Papers, SIGGRAPH Asia ’09, 2009.
    Google ScholarLocate open access versionFindings
  • N. Hagbi, R. Grasset, O. Bergig, M. Billinghurst, and J. El-Sana. Inplace sketching for content authoring in augmented reality games. In Virtual Reality Conference (VR), 2010 IEEE, 2010.
    Google ScholarLocate open access versionFindings
  • S. G. Hart. Nasa-task load index (nasa-tlx); 20 years later. In Proceedings of the human factors and ergonomics society annual meeting, volume 50, 2006.
    Google ScholarLocate open access versionFindings
  • S. G. Hart and L. E. Staveland. Development of nasa-tlx (task load index): Results of empirical and theoretical research. Advances in psychology, 52, 1988.
    Google ScholarLocate open access versionFindings
  • W. Hurst and J. Dekker. Tracking-based interaction for object creation in mobile augmented reality. In Proceedings of the 21st ACM international conference on Multimedia, 2013.
    Google ScholarLocate open access versionFindings
  • T. Igarashi, S. Matsuoka, and H. Tanaka. Teddy: A sketching interface for 3d freeform design. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’99, 1999.
    Google ScholarLocate open access versionFindings
  • S. Magnenat, D. T. Ngo, F. Zund, M. Ryffel, G. Noris, G. Rothlin, A. Marra, M. Nitti, P. Fua, M. Gross, et al. Live texturing of augmented reality characters from colored drawings. Visualization and Computer Graphics, IEEE Transactions on, 21(11), 2015.
    Google ScholarLocate open access versionFindings
  • A. Nealen, T. Igarashi, O. Sorkine, and M. Alexa. Fibermesh: Designing freeform surfaces with 3d curves. In ACM SIGGRAPH 2007 Papers, SIGGRAPH ’07, 2007.
    Google ScholarLocate open access versionFindings
  • L. Olsen, F. F. Samavati, and J. A. Jorge. Naturasketch: Modeling from images and natural sketches. Computer Graphics and Applications, IEEE, 31(6), 2011.
    Google ScholarLocate open access versionFindings
  • S. Oviatt. Human-centered design meets cognitive load theory: designing interfaces that help people think. In Proceedings of the 14th annual ACM international conference on Multimedia, 2006.
    Google ScholarLocate open access versionFindings
  • R. Schmidt, T. Isenberg, P. Jepp, K. Singh, and B. Wyvill. Sketching, scaffolding, and inking: A visual history for interactive 3d modeling. In Proceedings of the 5th International Symposium on Nonphotorealistic Animation and Rendering, NPAR ’07, 2007.
    Google ScholarLocate open access versionFindings
  • R. Schmidt, B. Wyvill, M. C. Sousa, and J. A. Jorge. Shapeshop: Sketch-based solid modeling with blobtrees. In ACM SIGGRAPH 2007 Courses, SIGGRAPH ’07, 2007.
    Google ScholarLocate open access versionFindings
  • J. R. Shewchuk. Delaunay refinement algorithms for triangular mesh generation. Computational geometry, 22(1), 2002.
    Google ScholarLocate open access versionFindings
  • J. Vollmer, R. Mencl, and H. Mueller. Improved laplacian smoothing of noisy surface meshes. In Computer Graphics Forum, volume 18, 1999.
    Google ScholarLocate open access versionFindings
  • T. Zhang and C. Y. Suen. A fast parallel algorithm for thinning digital patterns. Communications of the ACM, 27(3), 1984.
    Google ScholarLocate open access versionFindings
  • F. Zund, M. Ryffel, S. Magnenat, A. Marra, M. Nitti, M. Kapadia, G. Noris, K. Mitchell, M. Gross, and R. W. Sumner. Augmented creativity: Bridging the real and virtual worlds to enhance creative play. In SIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications, SA ’15, 2015.
    Google ScholarLocate open access versionFindings
Full Text
Your rating :
0

 

Tags
Comments