Hand-drawn doodle classification
Arpit Kumawat (2015UCP1524)
Rohit Deegwal (2015UCP1542)
Vedant Pankaj (2015UCP1674)
Introduction
Hand-Drawn Doodle Recognition System is a Machine Learning model based on Convolutional Neural Network supported by a Web Application to classify hand-drawn images of doodles across 345 categories
Introduction
Motivation
- Computers having the ability to understand our quick line drawings will allow for broader forms of expression and communication
- Using words can be limited when communicating across cultures and literacy levels, drawings can bridge those divides
- If successful, this model can be applied for a variety of interesting tasks and applications that require computers to understand our quick line drawings or doodles
Objectives
- Our goal is to use machine learning techniques to develop an efficient system to recognize labels of hand-drawn images/doodles
- Input to our algorithm is an image and we use Convolutional Neural Networks (CNNs) to find an accurate predicted output for a class
- The model then can be further used for any language learning/image search related purposes
research gap
Research Gap
- Research and some articles are currently available for only few classes of objects
- No system available that involves a large set of such drawing recognition
- Development of a machine learning model for the entire dataset and finding out the conflicts or problems associated would help in its real life applications
methodology & procedure
- Dataset - Quick Draw Dataset
- Deep Learning Technique - Convolutional Neural Network
- Web Application - A python-flask based web server and interface
- Approach Selection & Tools Used - Python, Keras, Numpy, Pandas, Google Colab, Tensorflow
Methodology
DAtaset
Dataset
- Google’s QuickDraw is the world’s largest doodling dataset, consisting of 50 million hand-drawn images across 345 categories ( about 39GB)
- Timestamped vectors, tagged with metadata and shared by Google for developers, researchers to explore and learn
- Available as json, binary and numpy files - numpy array files with drawings rendered into a 28x28 grayscale bitmap
approach & tools
Approach & tools
- Compared different algorithms like knn , logistic regression and results among them all on the dataset fetching range of different accuracies
- Few published articles mentioned the potential use of Convolutional Neural Networks for the process
- CNN are best suited for such small hand-drawn images with the chance to bag the maximum accuracy through customizing CNN to our needs
TOOLS USED FOR IMPLEMENTATION
Tools
learning model Architecture
learning model architecture
web application architecture
web app architecture
results
results
The Experiment environment consists of four different training phases of the proposed model done by choosing following number of classes at once :
- 3 classes
- 10 classes
- 50 classes
- 345 classes
Accuracy tables
Accuracy Tables
Accuracy & loss curves for 345 class classification
Accuracy & loss Graphs
conclusion & applications
Conclusion & Applications
- The developed system performs well on the overall set except in case of highly conflicting classes
- Can help with learning of basically anything with visual or pictorial references, most challenging of all, Languages
- Can also be extended to people who are specially-abled because drawing and pictorial representation is an underrated but efficient mode of communication
- Image searching can can simplify the search based on the image provided
references
References
- Google Quick Draw Dataset https://github.com/googlecreativelab/quickdraw-dataset
- Convolutional Neural Network Notes https://colab.research.google.com/drive/1H_CCNqv5z8P5bRD8uqMxg4LmwQiOLwMH#scrollTo=L62geNmwXHMU
- “Basics to deploy a keras model as web app” https://github.com/mtobeiyf/keras-flask-deploy-webap
- Drawing: A New Way To Search (Computer Vision) by Nguyet Minh Phu, Connie Xiao, Jervis Muindi Department of Computer Science Stanford University