In my last blog post, I detailed the implementation of machine learning models in iOS applications using the Core ML and Vision frameworks. As you probably remember from the tutorial, I implemented the Inception v3 model to give the app the ability to classify 1,000 common objects in the world. While it is true that you can easily download the model from a Github repository, have you ever wonder where it came from? In this blog post, I will introduce the “brain” behind the Inception v3 model––an artificial neural network (ANN).
Before we get into the details, allow me to briefly discuss the origin of ANN. The development of neural networks was inspired by biological neural networks (i.e. animal brains). The human brain, for instance, is made of billions of interconnected neurons that receive electrical signals via dendrites and output electrical signals with axons (Shiffman). Similar to the human brain, an ANN contains a network of “neurons” that receives inputs and produce and pass-on outputs to the next hidden layers before reaching the final output layer. Because of this characteristic, ANNs are considered as connectionist models.
It is important to note that ANNs are inherently different from the good old rule-based programming because they have the ability to learn, that is the ability to adapt their structures according to the input data. Each of lines connecting the neurons as shown in the illustration above, or “dendrites” if you will, can be associated with a weight, “a number that controls the signal between the two neurons,” (Shiffman). When data is fed to the network, the weights are able to adjust themselves to reduce the error between the predicted output and the expected output value provided in the dataset. The act of feeding datasets to the neural network is called training.
Certain inputs can produce null results, which is why there is usually a bias input to prevent null results. Activation functions in neurons are predefined formulas used to compute the output values given the inputs. When a neural network is being trained, datasets are fed to the network continuously and repeatedly. After each training iteration, a test is performed with a test dataset to compute an average error percentage of the network. This process repeats until the error becomes minimal. Note that in certain cases, usually due to the quality of the training dataset, the training error might not decrease even after thousands of iterations.
ANNs are being widely used to perform tasks that are “easy for humans but difficult for computers.” Because ANNs excel in classification problems, they are used in image classification problems. The so-called convolutional neural networks (CNN or ConvNet) are developed and are able to achieve less than 0.5% of error rate in applications like MNIST handwritten digit recognition.
In my next blog post, I will give a simple example of an artificial neural network that predicts whether a data point (x, y) is above or below a line in a 2D coordinate system given the x and y values. This blog concludes my blog posts for the first semester of the 2017-2018 school year. Thank you for reading my blogs and see you next year.
Ainley, Nathaniel. “‘Computer Thoughts’ Projected onto a Giant Dome via Neural Network.” Vice Creators, 28 Jan. 2016, creators.vice.com/en_uk/article/pgqvzn/dome-projection-mapping-via-neural-network. Accessed 11 Dec. 2017.
Eindhoven University of Technology. “Efficient mapping of the training of Convolutional Neural Networks to a CUDA-based cluster.” Parallel Architecture Research Eindhoven, parse.ele.tue.nl/education/cluster2. Accessed 11 Dec. 2017.
Lerner, K. Lee. “Nerve Impulses and Conduction of Impulses.” The Gale Encyclopedia of Science, edited by K. Lee Lerner and Brenda Wilmoth Lerner, 3rd ed., vol. 4, Gale, 2004, pp. 2731-2732. Gale Virtual Reference Library, http://link.galegroup.com/apps/doc/CX3418501554/GVRL?u=west66701&sid=GVRL&xid=74e8bad6. Accessed 10 Dec. 2017.
Lerner, K. Lee, and Christine Miner Minderovic. “Neuron.” The Gale Encyclopedia of Science, edited by K. Lee Lerner and Brenda Wilmoth Lerner, 3rd ed., vol. 4, Gale, 2004, pp. 2740-2744. Gale Virtual Reference Library, http://link.galegroup.com/apps/doc/CX3418501557/GVRL?u=west66701&sid=GVRL&xid=7c5a0796. Accessed 10 Dec. 2017.
Shiffman, Daniel. “Chapter 10. Neural Networks.” The Nature of Code, natureofcode.com/book/chapter-10-neural-networks/. Accessed 1 Dec. 2017.
Thanapitak, Surachoke. Figure 2.1: The structure of a neuron. ResearchGate, Jan. 2012, http://www.researchgate.net/figure/303389501_fig1_Figure-21-The-structure-of-a-neuron. Accessed 1 Dec. 2017.
XenonStack. “Overview of Artificial Neural Networks and its Applications.” Hackernoon, Medium, hackernoon.com/overview-of-artificial-neural-networks-and-its-applications-2525c1addff7. Accessed 1 Dec. 2017.