​WHAT IS MACHINE LEARNING?

It is akin to feeding data into a system or plotting data onto a graph and drawing the line of best fit. Keep adding more data and the line of best fit, only gets better and better and we learn to perfect it. The learning is not just to get the line of best fit, but to also understand what features or tasks are being plotted. Example: you reckon temperature and humidity are features to be studied to predict possibility of rain. And you start plotting data on these two features. Like any student who burns the midnight oil and to do well in his test, performance of machine learning will improve with more data (or experience if you will).

WHAT IS DEEP LEARNING?

It is a kind of machine learning. Deep Learning views concepts than just features. It sees rain as a nested hierarchy of concepts. Each concept defined in relation to simpler concepts. So Deep Learning understands the concept of dark cloudy evenings, nip in the air. These complex abstract representations can be broken down into simpler less abstract representations of dip in temperature, humidity and wind. And decides which features would help it best predict rain. Dark clouds, but a windy morning may not give you rain. The wind will take the cloud away.

​Just like 4 lines don't always represent a square, unless they are connected, perpendicular and of equal length. Just like a wolf is not a dog, but close enough in features.

AI: Machine & Deep Learning

Deep Learning gets into action and fetches a program from the billions in it's repository and starts renaming file names. See the last 2 rows in RED. As we read earlier, it is next to impossible to interpret which program was pulled up by Deep Learning to create OUTPUT for the thousand odd documents listed. That is the interpretation (Black Box) drawback with Deep Learning. So we save time in writing a program ourselves. Deep Learning does it for you. And please try and avoid typo errors like "3.0.1 Pricing_OTC_FFD_V2.0" sample OUTPUT, where you have a typo error for 'FDD'. Such noise in sample data adds to time to train Deep Learning. There are now new methods of writing programs on the fly, where Deep Learning picks up the INPUT and OUTPUT for 1st row, and writes an algorithm for file name conversion. Picks the 2nd row and runs it past the algorithm, learning in the process and fine-tuning the program. And so on, till it gets the pattern and gets rolling for the entire list. The key advantage of such a learning exercise is that we know how it worked it's way thru the data, unlike the typical Black Box drawback.

SO WHAT HAVE WE LEARNT?
Once all the Deep Learning is done, can the Business Analyst begin to understand the data presented and analyze it? As Business Analyst you understand parameters, features and data sets. You will change them around and expect Deep Learning to return it's analysis. All you need to do is to enter problem type, select features, threshold levels etc. and voila! Visual Analytics Tools are now available that allow business analysts to do just that. There are now numerous options to integrate Business Layer with Deep Learning to analyze and use data. You can embed analytics into the algorithm, so that a Business Analyst with no knowledge of machine learning can use the analysis. Or you could have the business analysis application layer integrate with such analytics tool on deep learning. So Deep Learning does the analysis. Your business application is integrated with Deep Learning analytics and you can see the business view of the analysis. If your business application cannot integrate with Deep Learning analytics tool, then Wrapper API, provided by various programming languages, can be used to access visual analytics tool. And of course we such deep learning analytics for image processing, voice recognition, text processing available as a service on cloud by various service providers

AI: Machine & Deep Learning  - Learning to learn

You provide the list of file names (INPUT column- shown below) and provide some sample OUTPUT file names expected. See the 1st 3 rows in the table below.

LEARNING TO LEARN

"If you hear hoofbeats, think of horses not zebras". An aphorism coined by Dr. Theodore Woodward, professor at the University of Maryland School of Medicine, in 1940, to describe a very simple, yet fundamental principle in medicine. Learning maybe understood in "Knowing That", "Knowing How" and "Knowing Why". I "know that" swimming pool has water, neck deep. To go across, I must jump in and swim. "Know That" is what can be referred to as Factual. "Knowing How" to swim, keep your arms out in front, face in water, kick you legs, good body rotation as you move your hands thru water and easy breathing. "Knowing How" is what we refer to as Experiential. And just like we pass on these basics to our kids and leave their brains to figure out the rest (e.g. how to increase speed, how to breathe, how to float), we pass on such basics (data) to machines and thru their neural network (similar to how brain functions- passing electrical signals from neuron to another). We pass images and few characteristics of these images to their neural network and soon enough they start recognizing images and learn the patterns that are formed- "Knowing Why". So where does this lead us? Your mobile phone will analyze your speech patterns and recognize Alzheimer's. Your cars steering wheel will pick-up that you are drowsy and feeling sleepy. Your treadmill will send the your heart rate report to your doctor. So we will actually be picked up by one algorithm to another.

WRITING A DEEP LEARNING PROGRAM

If you are at the edge of your seat and rearing to begin coding for deep learning? Wait! There are several Open Source Deep Learning Frameworks available today to start programming. Frameworks that support various programming languages. Example: Framework called Neon. You can program in Python. Similarly a Framework called Caffe, where you can program in C++. Now here's why I asked you to wait! Apart from having knowledge of Frameworks and programming language you need to understand models like CNN (Convolutional Neural Networks) and Recurrent Neural Networks (RNN). You should have good knowledge of statistics, mathematics and machine learning. Note: the coding in Deep Learning is a few lines of code, unlike the thousands of lines you write for - say an e-commerce application. However what is key is your ability to understand the concepts behind the code, so that you can write a good analytical model. Hence the term Data Scientist, for Deep Learning. And not programmer or coder. Hold on! Don't close the browser yet! The good news is, now we have open source tools that provide visual coding, so that the less experienced data scientists (coders and programmers really) can pick-up and learn the maths and models to build deep networks. Again if all this is too daunting, you can dip into services provided by AWS, Google, Microsoft and fish out pre-trained models for your business. Services like Text-To-Speech, Image Recognition, BoT framework etc.


CAN DEEP LEARNING WRITE PROGRAMS
Even today most of us programmers and coders write code step-by-step, following the algorithm. If deep learning can do tonnes of stuff, why do we need to write programs- why can't deep learning do that as well? Here's how you need to help Deep Learning to kick-start programming. Provide Deep Learning with sample inputs and sample outputs expected. And it will get cracking on the rest. Let us take example of a Knowledge Management (KM) Lead who is uploading documents into KM portal. He receives project documents from a project and the file names are all over the place or not aligned with KM prescribed naming convention. Let us for argument sake assume there a couple of thousand such documents, where file names need to be changed.

GETTING STARTED WITH MACHINE LEARNING

Here's a possible route of getting started with machine learning:

  • Learn a language: say Python or R
  • Refresh your knowledge of statistics- descriptive and inferential statistics
  • Identify features, get a clean set of data for your hypothesis
  • Learn machine learning: algorithms
  • Compete in Kaggle to land the message, that you - the Data Scientist - has arrived


GETTING STARTED WITH DEEP LEARNING

Here's a possible route of getting started with Deep Learning:

  • You guessed it right! Learn machine learning first
  • Get a good hardware (4GB- GPU, 4GB RAM, a good CPU)
  • Google up reading material on deep learning to warm up
  • Then select where you want to go with Deep Learning (speech/audio, Natural Language Processing, vision etc.)
  • And last but not the least Kaggle it!

DOT1 Solutions Private Limited

DEEP LEARNING VS MACHINE LEARNING

If you want deep learning, you need lot of data. Else machine learning is good enough. If you want to know how you mobile phone was able to recognize Alzheimer's, and understand the decision tree - if you will - then deep learning can be a Black Box. You may not know how it did it. Deep learning needs higher computer power. It uses GPUs (Graphic Processing Unit) - the processor used to manage and boost performance of video and images. Machine learning will do with your existing regular desktop and laptops. You need to enter the features that will cause a rain - temperature, humidity etc. for machine to learn, analyze and predict. However in deep learning, It moves to higher level features- recognizing edges that could be a face, shapes that add up to a face, and then the face itself. Problem solving is a step by step process for Machine Learning. Example: to detect an object in a video or image, it will first detect an object and then recognize the object. So machine learning Box-Ons to objects like SVM (Linear Support Vector Machines) with HOG (Histogram of Oriented Gradients). Deep Learning, on the other hand, solves problems end-to-end. It goes YOLO (You Only Look Once) and give location and name of object on the video or image. Deep learning takes higher time to learn than machine learning. But less time to test.