You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Our objective in this small project is to build an image classification API from scratch.
19
+
We are going to go through all the steps needed to achieve this objective :
20
+
21
+
* Data annotation (with Unsplash API + Labelme )
22
+
23
+
* Model Training ( With Tensorflow )
24
+
25
+
* Making the API ( With Uvicorn and FastApi )
26
+
27
+
* Deploying the API on a remote server ( With Docker and Google Cloud Platform )
28
+
29
+
## Data Annotation :
30
+
31
+
One of the most important parts of any machine learning project is the quality and quantity of the annotated data. It is one of the key factors that will influence the quality of the predictions when the API is deployed.
32
+
33
+
In this project we will try to classify an input image into four classes :
34
+
35
+
* City
36
+
37
+
* Beach
38
+
39
+
* Sunset
40
+
41
+
* Trees/Forest
42
+
43
+
I choose those classes because it is easy to find tons of images representing them online. We use those classes to define a multi-label classification problem :
44
+
45
+
](https://cdn-images-1.medium.com/max/2000/1*buGA2Qk4KXqJMq5Xu5gffg.png)*Examples of inputs and targets / Images from [https://unsplash.com/](https://unsplash.com/)*
46
+
47
+
Now that we have defined the problem we want to solve we need to get a sufficient amount of labeled samples for training and evaluation.
48
+
To do that we will first use the [Unsplash](https://unsplash.com/) API to get URLs of images given multiple search queries.
49
+
50
+
# First install [https://github.com/yakupadakli/python-unsplash](https://github.com/yakupadakli/python-unsplash)
51
+
# Unsplash API [https://unsplash.com/documentation](https://unsplash.com/documentation)
We would try to get image URLs that are related to our target classes plus some other random images that will serve as negative examples.
81
+
82
+
The next step is to go through all the images and assign a set of labels to each one of them, like what is shown in the figure above. For this it is always easier to use annotations tools that are designed for this task like [LabelMe,](https://github.com/wkentaro/labelme) it is a python library that you can run easily from the command line:
83
+
84
+
labelme . -flags labels.txt
85
+
86
+
*Labelme user interface*
87
+
88
+
Using Labelme I labeled around a thousand images and made the urls+labels available here: [https://github.com/CVxTz/ToyImageClassificationDataset](https://github.com/CVxTz/ToyImageClassificationDataset)
89
+
90
+
## Model
91
+
92
+
Now that we have the labeled samples we can try building a classifier using Tensorflow. We will use MobileNet_V2 as the backbone of the classifier since it is fast and less likely to over-fit given the tiny amount of labeled samples we have, you can easily use it by importing it from keras_applications :
93
+
94
+
from tensorflow.keras.applications import MobileNetV2
Since it is a multi-label classification problem with four classes, we will have an output layer of four neurons with the Sigmoid activation ( given an example, we can have multiple neurons active or no neuron active as the target)
99
+
100
+
### Transfer learning
101
+
102
+
One commonly used trick to tackle the lack of labeled samples is to use transfer learning. It is when you transfer some of the weights learned from a source task ( like image classification with a different set of labels) to your target task as the starting point of your training. This allows for a better initialization compared to starting from random and allows for reusing some of the representations learned on the source task for our multi-label classification.
103
+
104
+
Here we will transfer the weights that resulted from training in ImageNet. Doing this is very easy when using Tensorflow+Keras for MobileNet_V2, you just need to specify weights=”imagenet” when creating an instance of MobileNetV2
Another trick to improve performance when having a small set of annotated samples is to do data augmentation. It is the process of applying random perturbations that preserve the label information ( a picture of a city after the perturbations still looks like a city ). Some common transformations are vertical mirroring, salt and pepper noise or blurring.
111
+
112
+
](https://cdn-images-1.medium.com/max/3710/1*BNhj5p5uTfwF9yaeRDuBUQ.png)*Data augmentation examples / Image from [https://unsplash.com/](https://unsplash.com/)*
113
+
114
+
To achieve this we use a python package called imgaug and define a sequence of transformation along with their amplitude :
115
+
116
+
sometimes = **lambda **aug: iaa.Sometimes(0.1, aug)
We split the dataset into two folds, training and validation and use the binary_crossentropy as our target along with the binary_accuracy as the evaluation metric.
135
+
136
+
We run the training from the command line after updating some configuration files :
137
+
138
+
# data_config.yaml for defnining the classes and input size**
We end up with a validation binary accuracy of **94%**
163
+
164
+
## Making the API
165
+
166
+
We will be using FastAPI to expose a predictor through an easy to use API that can take as input an image file and outputs a JSON with the classification scores for each class.
167
+
168
+
First, we need to write a Predictor class that can easily load a tensorflow.keras model and have a method to classify an image that is in the form of a file object.
pred = self.model.predict(arr[np.newaxis, ...]).ravel().tolist()
206
+
pred = [round(x, 3) **for **x **in **pred]
207
+
**return **{k: v **for **k, v **in **zip(self.targets, pred)}
208
+
209
+
**def **predict_from_file(self, file_object):
210
+
arr = read_from_file(file_object)
211
+
**return **self.predict_from_array(arr)
212
+
213
+
We can use a configuration file to instantiate a predictor object that has all the parameters to do predictions and will download the model from the GitHub repository of the project :
 on [Unsplash](https://unsplash.com/s/photos/city?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)](https://cdn-images-1.medium.com/max/5606/1*uPdQuS0Y4Esz5vYZk13Npw.jpeg)*Photo by [Antonio Resendiz](https://unsplash.com/@antonioresendiz_?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText) on [Unsplash](https://unsplash.com/s/photos/city?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)*
248
+
249
+
Uploading the image above gives up the following output :
250
+
251
+
{**
252
+
**"beach":** 0**,**
253
+
**"city":** 0**.**999**,**
254
+
**"sunset":** 0**.**005**,**
255
+
**"trees":** 0
256
+
**}
257
+
258
+
Which is the expected output!
259
+
260
+
We can also send a request via curl and time it :
261
+
262
+
time curl -X POST "[http://127.0.0.1:8080/scorefile/](http://127.0.0.1:8000/scorefile/)" -H "accept: application/json" -H "Content-Type: multipart/form-data" -F "file=[@antonio](http://twitter.com/antonio)-resendiz-VTLqQe4Ej8I-unsplash.jpg;type=image/jpeg"
We finally run the container while mapping the port 8080 of container to that of the host :
298
+
299
+
sudo docker run -p 8080:8080 img_classif .
300
+
301
+
### Deploying on a remote server
302
+
303
+
I tried to do this on an ec2 instance from AWS but the ssh command line was clunky and the terminal would freeze at the last command, no idea why. So I decided to do the deployment using Google Cloud Platform’s App engine. Link to a more detailed tutorial on the subject [here](https://blog.machinebox.io/deploy-docker-containers-in-google-cloud-platform-4b921c77476b).
304
+
305
+
* Create a google cloud platform account
306
+
307
+
* install gcloud
308
+
309
+
* create project project_id
310
+
311
+
* clone [https://github.com/CVxTz/FastImageClassification](https://github.com/CVxTz/FastImageClassification) and call :
In this project, we built and deployed machine-learning powered image classification API from scratch using Tensorflow, Docker, FastAPI and Google Cloud Platform‘s App Engine. All those tools made the whole process straightforward and relatively easy. The next step would be to explore questions related to security and performance when handling a large number of queries.
0 commit comments