Motivation: currently, I have been working on a Machine Learning project in which I need to deploy not one but multiple Keras models into our production. Our project uses Django as the backend framework, and therefore, I needed to find a way to load Keras models into Django project correctly.
So, the challenges are:
1. How to load a Keras model in Django project?
2. How to load multiple models in the same project?
This blog will tell you how I have solved those 2 challenged and successfully deployed the cool models into production. Let's get started!
- How to load a Keras model in Django project?
To be honest, in my first naïve implementation, I loaded the model inside my API view function. In consequence, the model will be loaded every time the API is called. This is a really bad practice because the loading of keras model is not the fastest process in the world (as the model size might goes up to hundreds of Megs). Therefore, this is a heavy, slow, and unnecessary process. If you are interested in my first implementation, the code is as follow:
Although the above code was not qualified for production use (due to the loading time issue), it is still worth noting that there are a couple of key points that I needed to do in order to use the model:
- Each model needs to be loaded a long with its graph (see _load_model_from_path(path) function). And the graph I used is graph = tf.get_default_graph().
- Also, model.predict() needs to be called inside graph.as_default().
After having keras model callable in Django, I needed to figure out a proper way to preload keras model in advance, so that I won't need to load every time the API is called. Fortunately, the idea is simple: load model as global variable. However, there comes to another question: Where should we load that global variable? The short answer is: the loading can be done in the project settings.py file since that is the file that Django has to always load when the server starts (alternatively, you can put in other files/modules as well, just make sure that the selected file/module is called when the server starts). The code is as follow:
2. How to load multiple models in the same project?
Now, we have our modal preloaded. It seems really cool. The next question is how to load multiple models in one project? Easy, right? (just call load_model() from keras.models as many times as you want 🤔). Unfortunately, the actual implementation wasn’t that straight forward (at least for me, I faced a lot of tensorflow/keras error messages, mainly because of the incorrect graph issues). At the end, I figured out the correct way to do is: create default graph for each model, and use that correspondent graph when model.predict() is called.
Since the models need to be loaded when the server starts, the server startup time will be slower than usual, but you will notice the significant improvement when sending to the prediction APIs.
That's how I developed my solution. What's your? Please feel free to share your ideas/improvements.
Happy coding machine learning!
