Not Hotdog and Transfer Learning

One of the funniest episodes in the latest season of Silicon Valley featured the creation of Jian Yang’s Not Hotdog app. As the app's name implies, Not Hotdog is an app to determine if an image contains a hotdog or not. The best part about it is that Not Hotdog is actually available for download in the App Store or Play Store.

hotdog ni aljur

Interestingly, the developer of the app wrote a blog post about how they developed the app. The developer discussed on the capability to run a neural network on the phone and the difficulty to pull it off properly. One highlight of the post was the use of transfer learning to speed up the learning process.

Transfer learning is a technique that retrains an existing model to cater to a more specific problem or domain. Some examples of transfer learning only retrains the final neuron layer while leaving the rest of the network untouched.

In the case of the Not Hotdog app, the blog post tried transfer learning using different models like Inception, VGG, and SqueezeNet with good results. Unfortunately, due to technical constraints of being on a mobile app, the developer opted to use a new architecture called MobileNets.

Inception and Lechon

The deep learning model Inception is a deep convolutional neural network architecture developed by Google for the ImageNet Large Visual Recognition Challenge, an annual competition where research teams submit programs that classify and detect objects and scenes. Inception-v3 reaches an astounding 3.43% top-5 error rate, an improvement from 4.9% of BN-Inception-v2 and 6.67% of Inception (GoogLeNet). In comparison, a human (well, Andrej Karpathy) got 5.1% top-5 error rate.

In this post, the concept of the Not Hotdog app was imitated to build a neural network that can identify lechon, a popular Filipino dish. Unlike the app, the “Uy Lechon!” neural network did not have the limitations of being in a mobile app. For the purpose of exploration, the neural network was run on a Macbook Pro.

For preparation, 360 images of lechon baboy (pork) were manually (yes, manually!) downloaded from Google Images. These images were used as the training data for the model. The Inception-v3 model and transfer learning were utilized so that the model does not need to be trained and tuned from scratch. Bazel was also used to retrain the last layer of the neural network.

A few problems

There was a hitch encountered while building the “Uy Lechon!” model. Identifying lechon was pretty easy for the model, but the neural network still needed to know what was not lechon--that is, the negative case.

Not Hotdog encountered the exact same problem as well. The Not Hotdog developers solved it by feeding a lot of not hotdog images then adjusted the model weights in favor of the hotdog.

In “Uy Lechon!” ‘s case, manually selecting various foods that are not lechon would take a lot time. There were food images datasets available for download, however. Unfortunately during the creation of the neural network, only mobile data internet was available and downloading a dataset with that size might not be economically feasible.

Solution

This problem was solved by using the flower dataset example by Google. That dataset was not that big compared when compared to the food images dataset. To balance the training data skew, only 360 images were used in the flower dataset. Yes, flowers might not be food but they were also not lechon. This worked to improve the negative case for detecting lechon.

The training started immediately after the lechon and not lechon images were properly segregated. In training the model, 4,000 steps were run by the algorithm, which took around 7 minutes long. Finally, a Python notebook was created to try out the “Uy Lechon” model.

Results

After the training, the “Uy Lechon” model was tested using 10 different images. Check out the results below:

  • Test 1: Lechon being cooked
  • Expected result: Lechon
  • Model output: Lechon
  • Test 2: Another lechon being cooked
  • Expected result: Lechon
  • Model output: Lechon

The first two were both images of lechon. Fortunately, the model identified both of them correctly.

  • Test 3: Pork sinigang (soup)
  • Expected result: Not lechon
  • Model output: Not lechon
  • Test 4: Pinakbet (vegetable)
  • Expected result: Not lechon
  • Model output: Not lechon
  • Test 5: Rooster
  • Expected result: Not lechon
  • Model output: Not lechon

These 3 tests determined if the "Uy Lechon" model can properly identify negative cases. They were also properly identified by the model.

  • Test 6: Lechon in a table
  • Expected result: Lechon
  • Model output: Lechon
  • Test 7: Bread with a lechon-like shape
  • Expected result: Not lechon?
  • Model output: Lechon

Test 7 was interesting. The model identified it as a lechon, but the image was just a bread with a shape of a lechon. Can it still be considered as a lechon?

  • Test 8: Hotdog
  • Expected result: Not lechon
  • Model output: Lechon

The model wrongly identified the image of a hotdog as a lechon. This was funny because the Not Hotdog app also identifed lechon as a hotdog.

  • Test 9: Human arm
  • Expected result: Not lechon
  • Model output: Not lechon
  • Test 10: Lechon manok (roasted chicken)
  • Expected result: Not lechon?
  • Model output: Lechon

The last test was also interesting. Technically, lechon manok can still be considered as lechon, but they were not part of the training data (only lechon baboy). The model probably misidentified it because both lechon manok and baboy shared almost the same color and style.

These tests can be viewed in a Python notebook at GitHub.

Sources