Introducing 

Prezi AI.

Your new presentation assistant.

Refine, enhance, and tailor your content, source relevant images, and edit visuals quicker than ever before.

Loading…
Transcript

Hand-drawn doodle classification

Arpit Kumawat (2015UCP1524)

Rohit Deegwal (2015UCP1542)

Vedant Pankaj (2015UCP1674)

Introduction

Hand-Drawn Doodle Recognition System is a Machine Learning model based on Convolutional Neural Network supported by a Web Application to classify hand-drawn images of doodles across 345 categories

Introduction

Motivation

  • Computers having the ability to understand our quick line drawings will allow for broader forms of expression and communication

  • Using words can be limited when communicating across cultures and literacy levels, drawings can bridge those divides

  • If successful, this model can be applied for a variety of interesting tasks and applications that require computers to understand our quick line drawings or doodles

Objectives

  • Our goal is to use machine learning techniques to develop an efficient system to recognize labels of hand-drawn images/doodles

  • Input to our algorithm is an image and we use Convolutional Neural Networks (CNNs) to find an accurate predicted output for a class

  • The model then can be further used for any language learning/image search related purposes

research gap

Research Gap

  • Research and some articles are currently available for only few classes of objects

  • No system available that involves a large set of such drawing recognition

  • Development of a machine learning model for the entire dataset and finding out the conflicts or problems associated would help in its real life applications

methodology & procedure

  • Dataset - Quick Draw Dataset

  • Deep Learning Technique - Convolutional Neural Network

  • Web Application - A python-flask based web server and interface

  • Approach Selection & Tools Used - Python, Keras, Numpy, Pandas, Google Colab, Tensorflow

Methodology

DAtaset

Dataset

  • Google’s QuickDraw is the world’s largest doodling dataset, consisting of 50 million hand-drawn images across 345 categories ( about 39GB)

  • Timestamped vectors, tagged with metadata and shared by Google for developers, researchers to explore and learn

  • Available as json, binary and numpy files - numpy array files with drawings rendered into a 28x28 grayscale bitmap

approach & tools

Approach & tools

  • Compared different algorithms like knn , logistic regression and results among them all on the dataset fetching range of different accuracies

  • Few published articles mentioned the potential use of Convolutional Neural Networks for the process

  • CNN are best suited for such small hand-drawn images with the chance to bag the maximum accuracy through customizing CNN to our needs

TOOLS USED FOR IMPLEMENTATION

Tools

learning model Architecture

learning model architecture

web application architecture

web app architecture

results

results

The Experiment environment consists of four different training phases of the proposed model done by choosing following number of classes at once :

  • 3 classes
  • 10 classes
  • 50 classes
  • 345 classes

Accuracy tables

Accuracy Tables

3 class classification

10 class classification

345 class classification

50 class classification

Accuracy & loss curves for 345 class classification

Accuracy & loss Graphs

conclusion & applications

Conclusion & Applications

  • The developed system performs well on the overall set except in case of highly conflicting classes

  • Can help with learning of basically anything with visual or pictorial references, most challenging of all, Languages

  • Can also be extended to people who are specially-abled because drawing and pictorial representation is an underrated but efficient mode of communication

  • Image searching can can simplify the search based on the image provided

references

References

  • Google Quick Draw Dataset https://github.com/googlecreativelab/quickdraw-dataset

  • Convolutional Neural Network Notes https://colab.research.google.com/drive/1H_CCNqv5z8P5bRD8uqMxg4LmwQiOLwMH#scrollTo=L62geNmwXHMU

  • “Basics to deploy a keras model as web app” https://github.com/mtobeiyf/keras-flask-deploy-webap

  • Drawing: A New Way To Search (Computer Vision) by Nguyet Minh Phu, Connie Xiao, Jervis Muindi Department of Computer Science Stanford University

Learn more about creating dynamic, engaging presentations with Prezi