For an ML lifecycle ML Serving is also important and what if there is a way to easily deploy your models on production on the cloud using efficient tools without worrying about vendor lock in?

Today we will be learning how to take a model train it on a dataset and achieve 90% accuracy which is pretty simple.

Also we will then talk about how to create an API Endpoint for it using FastAPI.
And we will talk about how to improve it's accuracy and reducing the API latency which is crucial for high volume API request handling.

Finally we will talk about how to scale and API using K8s pods and how to set it up on AWS Amazon Elastic Kubernetes Service.

We will finally test it out and see if this can handle 1 million requests per day and we will do an API testing for it.

So Let's get started ...

For the sake of training a model we are taking a simple Standford IMDB dataset and training a BERT model with this dataset.

I provided an access to a Jupyter Notebook that have achieved a 91% accuracy in on the PyTorch model which means, during inferencing the model will be 91% accurate on the dataset. To be accurate we can use trained models from hugging face itself.

#use this code to draw inference from this hugging face model trained on IMDB dataset
from transformers import BertForSequenceClassification, BertTokenizer, TextClassificationPipeline
model_path = "JiaqiLee/imdb-finetuned-bert-base-uncased"
tokenizer = BertTokenizer.from_pretrained(model_path)
model = BertForSequenceClassification.from_pretrained(model_path, num_labels=2)
pipeline = TextClassificationPipeline(model=model, tokenizer=tokenizer)
print(pipeline("I like the movie, it was awesome"))

We can simply create a FastAPI endpoint for this, and start deploying it on K8s right?

Creating a FastAPI Endpoint

So let's do the easiest job in the world.

#using huggingface model

from fastapi import FastAPI
from pydantic import BaseModel
from transformers import BertForSequenceClassification, BertTokenizer, TextClassificationPipeline

app = FastAPI()

# Load the model and tokenizer
model_path = "JiaqiLee/imdb-finetuned-bert-base-uncased"
tokenizer = BertTokenizer.from_pretrained(model_path)
model = BertForSequenceClassification.from_pretrained(model_path, num_labels=2)
pipeline = TextClassificationPipeline(model=model, tokenizer=tokenizer)

class TextInput(BaseModel):
    text: str

@app.post("/classify/")
def classify_text(input: TextInput):
    result = pipeline(input.text)
    return {"label": result[0]['label'], "score": result[0]['score']}

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

This code will setup a Fast Api endpoint `/classify` for you. you can try by sending an API request to this endpoint and try for yourself.

I have tested this on postman and the API latency came around 120-160 ms which is very high for serving 1 million requests per day.

So there are a couple methods to reduce to API latency, here are some of the ways we can optimize the API latency time

• Download the Pytorch model instead of calling the HF API

• Use BertTokenizerFast

• Use ONNX (Open Neural Network Exchange) Model

• Reduce Model Size (no implementation here)

• Use Asynchronous FastAPI (no implementation here)

I have implemented a few methods to reduce API Latency which shows significant reduction in API Response Time.

Using the downloaded Pytorch Model

Just download the Pytorch model contents and replace the model path in the previous code to the directory where you saved the model.

This reduced the API latency by 10-12 ms and shows that we are moving in the right direction.

#replace the model_path with the directory of your saved model
model_path = "./model"

Using BertTokenizerFast for faster API response times

#rest of the imports 
from transformers import BertTokenizerFast
.
.
.
tokenizer = BertTokenizerFast.from_pretrained(model_path)
#rest of the code
.
.

This claims to reduce API latency drastically but in my case this has been successful to a small extent, where the latency reduced around another 10-12 ms bringing down the overall latency to 90 ms.

Using ONNX models

ONNX is so famous that I don't need to introduce you to the world of ONNX, it has integration with Pytorch and you can easily convert a Pytorch Model to an ONNX model. Given below is a code implementation.

from transformers import BertForSequenceClassification, BertTokenizer
import torch

model_path = "JiaqiLee/imdb-finetuned-bert-base-uncased"
tokenizer = BertTokenizer.from_pretrained(model_path)
model = BertForSequenceClassification.from_pretrained(model_path, num_labels=2)

# Create a dummy input for tracing
dummy_input = tokenizer("This is a dummy input", return_tensors="pt")

# Export the model to ONNX format
torch.onnx.export(model,
                  (dummy_input['input_ids'], dummy_input['attention_mask']),
                  "model.onnx",
                  input_names=["input_ids", "attention_mask"],
                  output_names=["output"],
                  dynamic_axes={"input_ids": {0: "batch_size"},
                                "attention_mask": {0: "batch_size"},
                                "output": {0: "batch_size"}})

print("Model successfully converted to ONNX format and saved as 'model.onnx'.")

We can use the saved model and run it, and also we can create a similar API endpoint of it using FastAPI and run it.

#running an onnx model .

import onnxruntime as ort
from transformers import BertTokenizer
import numpy as np

model_path = "./model"
tokenizer = BertTokenizer.from_pretrained(model_path)

# Load the ONNX model
onnx_model_path = "model.onnx"
ort_session = ort.InferenceSession(onnx_model_path)

# Check the expected sequence length
expected_seq_len = ort_session.get_inputs()[0].shape[1]

# Define a function to perform inference
def classify_text(text):
    inputs = tokenizer(text, return_tensors="np", padding='max_length', truncation=True, max_length=expected_seq_len)
    ort_inputs = {k: np.array(v) for k, v in inputs.items() if k in ["input_ids", "attention_mask"]}
    ort_outs = ort_session.run(None, ort_inputs)
    label_id = ort_outs[0].argmax(axis=1)[0]
    labels = ["NEGATIVE", "POSITIVE"]  # Assuming binary classification
    return {"label": labels[label_id], "score": float(ort_outs[0].max())}

# Test the function with a sample input
sample_text = "The movie was fantastic! I really enjoyed it."
result = classify_text(sample_text)
print(result)

#expected output
{'label': 'POSITIVE', 'score': 2.6588470935821533}

This has reduce the model API latency by a drastic amount of 30-40 ms from 90-100ms which is huge.

We can further optimize the ONNX model or quantize the model(which reduces the performance) to reduce the API response time.

Code for optimizing the model

import onnx
from onnxruntime.transformers import optimizer

# Load the ONNX model
onnx_model_path = "model.onnx"
onnx_model = onnx.load(onnx_model_path)

# Optimize the ONNX model
optimized_model = optimizer.optimize_model(onnx_model, model_type='bert')

# Save the optimized model
optimized_model_path = "model2.onnx"
optimized_model.save_model_to_file(optimized_model_path)

print("Model has been further optimized and saved.")

This will reduce the model response time and API Response time by another 10-12 ms which will finally bring our API response time in around 20-30 ms

Here is an awesome repo to help you with optimization , have a look - https://github.com/ELS-RD/transformer-deploy

transformer-deploy by Lefebvre Dalloz

Now that we have brought down the API response time, let's focus on Model Serving.
I have used Ray Serve for my own purpose, which needs a separate introduction of its own but we will continue with simply serving this model with AWS (EKS) and use Kubernetes Horizontal Pod Autoscaler.
The HPA automatically scales the number of pods in a deployment or replication controller based on observed CPU utilization (or other select metrics).

Setup your AWS EKS and containerize your FastAPI Application and push it on Docker Hub (you can use AWS too or anything else)

You can use my Docker Image for example - subhro2084/fastapi-app general | Docker Hub

After you dockerize your fastapi image, make sure you get the endpoint right and it works. Only then upload it on dockerhub.

If your EKS cluster is created, fire your AWS cloud shell terminal and get started with these steps, try a little bit doesn't works at one go (it works on my machine 🙂 feelings ....yk)

OKAY, Let's get started 😊

First, have a look around how much is ONE MILLION REQUESTS PER DAY??

At it's peak you should be able to handle 25 RPS, and you have to keep your pods ready till that. The following infra was tested on 40 RPS which it handles easily.

Create a file fastapi-app.yaml and add this content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fastapi-app
  labels:
    app: fastapi-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: fastapi-app
  template:
    metadata:
      labels:
        app: fastapi-app
    spec:
      containers:
      - name: fastapi-app
        image: subhro2084/fastapi-app:latest #depends on what image you are using
        ports:
        - containerPort: 8000
        resources:
          requests:
            memory: "512Mi" #depends on the type of machine you have used
            cpu: "250m"
          limits:
            memory: "1Gi"
            cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
  name: fastapi-service
spec:
  selector:
    app: fastapi-app
  type: LoadBalancer
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8000
---
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: fastapi-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: fastapi-app
  minReplicas: 1
  maxReplicas: 10
  targetCPUUtilizationPercentage: 50

You need to save this YAML file in your AWS Cloudshell and apply the configuration:
kubectl apply -f fastapi-app.yaml

Verify the deployment using -

kubectl get pods Ensure your pods are running kubectl get hpa to monitor the HPA status

Test the Application ->

curl -X POST "http://<EXTERNAL-IP-OF-YOUR-AWS-POD>/classify/" -H "accept: application/json" -H "Content-Type: application/json" -d '{"text": "I love you"}'

You can use K6 which is famous for load testing and run it locally.

Use this script.js file and save it

import http from 'k6/http';
import { check, sleep } from 'k6';

export let options = {
  stages: [
    { duration: '5m', target: 40 }, // Ramp-up to 25 RPS over 5 minutes
    { duration: '8h', target: 40 }, // Stay at 25 RPS for 8 hours (office hours)
    { duration: '5m', target: 5 }, // Ramp-down to 5 RPS over 5 minutes
    { duration: '15h', target: 5 }, // Stay at 5 RPS for 15 hours (non-office hours)
  ],
};

export default function () {
  let res = http.post('http://a4ee008522b7547aaaa5c7a338f1123d-1605357505.ap-south-1.elb.amazonaws.com/classify/', JSON.stringify({ text: 'I love you' }), {
    headers: { 'Content-Type': 'application/json' },
  });
  check(res, { 'status was 200': (r) => r.status === 200 });
  sleep(1);
}

Run it on your machine using k6 run --out web-dashboard script.js

We will get a similar report like this -

• Iterate and change the number of pods.

• Adjust the Horizontal Pod Autoscalers (HPAs).

• Use better machines.

• Opt for smaller model sizes.

• Implement asynchronous API.

These are some of the steps we can take to reduce the API limit, this a work under progress , I will still need to add a wonderful tool called Ray Serve which runs on top of K8s, which changes the game.

Thanks for reading till here❤️❤️, if you like this, please tell how to improve, here are my socials links if you find I made an embarrassing mistake here 😎.

Twitter🐤 , Linkedin🫠

The point being ML code you write on your local machine is just a small part of the whole process it takes to create an enterprise grade software that can actually solve real world problems.

A glimpse of how enterprise production solution looks like -

Challenges Enterprise face in deploying ML solutions

• Data Collection

• Deploying and Reproducing the model in production

• Model Monitoring

• Keeping model relevant by adopting to changing business scenarios

• Communicate and interpret model output to various stakeholders.

Kubeflow allows you to make the deployment of machine learning workflows on Kubernetes simple, composable, portable, and scalable.

How to setup Kubeflow in Windows Subsystem for Linux Locally

The official website for Kubeflow currently does not have a clear doc on how to set it up Windows, so I will be discussing it in this blog.

To install Kubeflow locally, you need a few tools.

We’ll assume you have knowledge of Docker, as it is basically a prerequisite for working with Kubernetes. If you are unfamiliar with Docker, check out this tutorial to get you up to speed, then come back here.

Since Kubeflow runs wherever K8s runs, we can just deploy a K8s cluster locally and try to run KubeFlow.
Additionally, you’ll need the following tools:

• kubectl, which is a command-line tool to manage your K8s cluster

• kustomize to configure applications using YAML

Minikube Installation

Minikube will help setup a cluster locally on your machine. In terms of terminology, think of your computer as a single node responsible for housing the pods. These pods are where your application containers operate. The management of these pods is carried out by a deployment, which outlines the ideal condition for your Kubernetes application.

The initial command fetches and downloads the necessary binary, whereas the subsequent command facilitates its installation to the designated location.

curl -LO <https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64> sudo install minikube-linux-amd64 /usr/local/bin/minikube

What is kubectl?

kubectl is a versatile command-line utility designed to manage Kubernetes clusters efficiently. Often likened to a "Swiss Army knife" for its multipurpose functionality, it stands as an essential instrument in the administration of cluster resources.

sudo snap install kubectl --classic

What is Kustomize

Kubernetes operations are predominantly governed through an extensive array of YAML files. To streamline customization, kustomize offers a powerful solution. This tool enables the modification of raw, template-free YAML files without altering the original documents, ensuring they remain intact and operational as is.

Finally!!

Now that you have all the prerequisite software and packages installed, it is now time to install Kubeflow.

Follow these steps:

1. Clone the manifests repo from the Kubeflow team:

git clone <https://github.com/kubeflow/manifests.git>

1. Change to the repo directory:

cd manifests

1. Build and apply the YAML files for all Kubeflow components:

while ! kustomize build example | awk '!/well-defined/' | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 10; done

1. Wait for everything to settle out.

You can check to see if everything has settled out by running:

kubectl get pods -A

This will list all pods across all namespaces.

NAMESPACE         NAME                                     READY  STATUS    RESTARTS      AGE
auth              dex-7ff46847-sqxzj                       1/1    Running   0             10h
cert-manager      cert-manager-7fb78674d7-nllnn            1/1    Running   0             10h
cert-manager      cert-manager-cainjector-5dfc946d84-m6f7  1/1    Running   0             10h
cert-manager      cert-manager-webhook-8744b7588-cvzzm     1/1    Running   0             10h
istio-system      authservice-0                            1/1    Running   0             10h
istio-system      cluster-local-gateway-675bb7b74-49x27    1/1    Running   0             10h
istio-system      istio-ingressgateway-c7fdd4bf6-z68qt     1/1    Running   0             10h
istio-system      istiod-6995577d4-7h6zv                   1/1    Running   0             10h
knative-eventing  eventing-controller-86647cbc5b-62tl4     1/1    Running   0             10h
knative-eventing  eventing-webhook-6f48bb5f4c-c5ljb        1/1    Running   0             10h
knative-serving   activator-855b695596-zrfrr               2/2    Running   0             10h
knative-serving   autoscaler-7cbddfc9f7-gjckn              2/2    Running   0             10h
knative-serving   controller-6657c556fd-q728z              2/2    Running   0             10h
knative-serving   domain-mapping-544987775c-bffh5          2/2    Running   0             10h
knative-serving   domainmapping-webhook-6b48bdc856-bmllz   2/2    Running   0             10h
knative-serving   net-istio-controller-6fbdbd9959-bmglm    2/2    Running   0             10h
knative-serving   net-istio-webhook-7d4879cd7f-xwsl5       2/2    Running   0             10h
knative-serving   webhook-665c977469-rw6v6                 2/2    Running   0             10h
kube-system       coredns-787d4945fb-mgpsr                 1/1    Running   1 (10h ago)   10h
kube-system       etcd-minikube                            1/1    Running   2 (52s ago)   10h
kube-system       kube-apiserver-minikube                  1/1    Running   1 (10h ago)   10h
kube-system       kube-controller-manager-minikube         1/1    Running   2 (8h ago)    10h
kube-system       kube-proxy-l4tvb                         1/1    Running   1 (10h ago)   10h
kube-system       kube-scheduler-minikube                  1/1    Running   1 (10h ago)   10h
kube-system       nvidia-device-plugin-daemonset-cd6h8     1/1    Running   0             10h
kube-system       storage-provisioner                      1/1    Running   2 (10h ago)   10h
kubeflow          admission-webhook-deployment-6d48f6f745  1/1    Running   53 (10h ago)  10h
kubeflow          cache-server-6b44c46d47-lvcqr            2/2    Running   0             10h
kubeflow          centraldashboard-f966d7897-ltjhn         2/2    Running   0             10h
kubeflow          jupyter-web-app-deployment-795dcd4c9b-r  2/2    Running   0             10h
kubeflow          katib-controller-746969dc99-2fz29        1/1    Running   53 (10h ago)  10h
kubeflow          katib-db-manager-5ddbffd67-w429n         1/1    Running   0             10h
kubeflow          katib-mysql-66c8cdff4f-mrhz9             1/1    Running   0             10h
kubeflow          katib-ui-58b54d465f-kxmv2                2/2    Running   1 (10h ago)   10h
kubeflow          kserve-controller-manager-96b896c66-84v  2/2    Running   0             10h
kubeflow          kserve-models-web-app-9fbcd79f5-xksvx    2/2    Running   0             10h
kubeflow          kubeflow-pipelines-profile-controller-6  1/1    Running   0             10h
kubeflow          metacontroller-0                         1/1    Running   0             10h
kubeflow          metadata-envoy-deployment-7b49bdb748-tn  1/1    Running   0             10h
kubeflow          metadata-grpc-deployment-6d744c66bb-fkt  2/2    Running   3 (10h ago)   10h
kubeflow          metadata-writer-5bfdbf79b7-b5trj         2/2    Running   0             10h
kubeflow          minio-549846c488-x7jj6                   2/2    Running   0             10h
kubeflow          ml-pipeline-86d69497fc-mvtb9             2/2    Running   53 (10h ago)  10h
kubeflow          ml-pipeline-persistenceagent-5789446f9c  2/2    Running   0             10h
kubeflow          ml-pipeline-scheduledworkflow-fb9fbd76b  2/2    Running   0             10h
kubeflow          ml-pipeline-ui-74fcbdddd9-sm7dd          2/2    Running   0             10h
kubeflow          ml-pipeline-viewer-crd-bdf696cb9-97tks   2/2    Running   1 (10h ago)   10h
kubeflow          ml-pipeline-visualizationserver-845d745  2/2    Running   0             10h
kubeflow          mysql-5f968h4688-dlgv4                   2/2    Running   0             10h
kubeflow          notebook-controller-deployment-576df594  2/2    Running   2 (10h ago)   10h
kubeflow          profiles-deployment-7bc6469cdd-r5vzw     3/3    Running   53 (10h ago)  10h
kubeflow          tensorboard-controller-deployment-84954  3/3    Running   1 (10h ago)   10h
kubeflow          tensorboards-web-app-deployment-74bc589  2/2    Running   0             10h
kubeflow          training-operator-7c5456c65-fsqdr        1/1    Running   0             10h
kubeflow          volumes-web-app-deployment-86dddc89d4-8  2/2    Running   0             10h
kubeflow          workflow-controller-56cc57796-gjtd9      2/2    Running   1 (10h ago)   10h

How do I setup a Kubeflow Dashboard?

The dashboard is accessed via http requests routed through the istio-ingressgateway service in the istio-system namespace. To forward the port, you use kubectl:

kubectl port-forward svc/istio-ingressgateway -n istio-system 8080:80

This tells your cluster to listen on port 8080 locally and forward it to the service on port 80. You can then reach the dashboard at http://localhost:8080.

The default username is user@example.com and the password is 12341234.

How do I stop minikube?

You can stop everything you’re running by stopping minikube:

minikube stop

If you want to delete your Kubeflow cluster, run:

minikube delete

Congratulations!! You've done it.

There will be a lot of errors that you will encounter while setting it up, so please refer to the Kubeflow Docs.

All AI chatbots right now like Bard or ChatGPT are based on a architecture called Transformers. It is really useful thanks to a special mechanism called self attention that helps the models to look back into a set of sequential inputs and perform some pretty insane text completion based on the input you feed it.
But for these models even counting and basic arithmetic are a big problem.

Also it hallucinated on larger contexts when you can provide really long contexts like PDF documents (research papers). The exact details are missed and these chatbots give you generalized and oversimplified answers.

To address this problem, researchers made MAMBA (Selective Structured State Space Sequence Models) model. It is basically LSTM + Transformers.

The current SOTA AI models like ChatGPT get exponentially more expensive to train and run the bigger they are. Like Zuck casually mentioned to buy 600,000 H100s GPUs to train LLMs by the end of 2024. And that goes without saying that these models are not suitable for scaling up even more.

This is majorly because of it's own attention mechanism where they need to note the positions of all texts in the whole context which makes longer texts harder to work with.

So some big brain researchers from CMU and Princeton dug up and old architecture called state space models and refine it to create something called the S4 model, short for structured state space sequence models and implemented it into something called Mamba.

First, it not only solves the scaling problem that the transformers have, where the computation doesn't scale exponentially and only linearly, but second, it is not using the attention mechanism and can still recall any details you provided within full precision. For a more technical perspective, it is the first alternative model architecture to achieve benchmarks that surpasses the strongest transformer recipe.

For example if you are going to a party - everyone needs to meet everyone else, all relations between each other. That's how transformers are working.
But think of a situation when you are attending a party and everyone just knows the party host, so you just have to meet the host, as he/she knows everyone and their relations among each other - that's how Mamba works.

Mamba save you so much more time when you increase the amount of people attending the party. Fundamentally the S4 model is a completely different architecture from the transformers.

Mamba is more similar to LSTM and recurrent models. LSTM needs the output from the previous hidden state and the global input to generate the next prediction. Hence every layers needs to wait for it's previous layer to finish it's prediction to proceede, which makes it extremely slow.

In the S4 models Mamba uses, each hidden state is only dependent on the global input, so there's no wasting time waiting for the result from the last layer and whatnot. On top of having non linearity between the hidden states, it makes the calculations insanely fast. We can just finish all the matrix multiplications at once.

This improved the quadratic scaling that the transformers has from

$$from O(n)^2 to O(n)$$

Let's talk about similar models like Vision Mamba in another blog, stay tuned.

Refer to this GitHub Repository for latest updates in Mamba Papers

These seem like ridiculous questions but with the advancements in Sentiment Analysis in Machine Learning, our machines are getting closer to answering these questions.

Let's get straight into examples!

If I would have asked you to rate a sentence out of 10, with 0 being negative and 10 being positive. How would you do that?

For example, have a look at these pictures -

Pretty positive right? Let's give it around 10!

Now have a look at this! Positive, but liking a movie is not as great as loving a movie! We know this right?

Now here's one more. This clearly states a very negative sentence, so that's totally 0.

Sentiment Analysis is simply using Machine Learning to teach Computers to do just this! Extract the sentiments out of our sentences or the emotions behind it!

The most pressing question is what's the use of it?

Well, there are many uses of it but one of the major things Sentiment Analysis will be used in is Controlling the Radicalization of the Internet. This is a very powerful tool that can be used by democracies around the world to make the internet a safer place! But also, at the same time this can be used to press the voices of some people that the Government doesn't like! So that's a Double Edged sword, you can use it the way you like it. Okay now let's get to Coding!

The Code!

We will be using Jupyter Notebooks to build our Machine Learning Model.

Installing and Importing Dependencies

Installing PyTorch- To know more about it refer to my next blog

!pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113

Installing some other stuff!

!pip install transformers requests beautifulsoup4 pandas numpy

Transformers helps us to easily import, download and install our NLP model.

Requests are going to help us in making a request to the site we will be scraping for the testing data.

Beautiful Soup as you all know is used for web scraping to extract the data that we actually need - is taught in basics of python programming.

Pandas help us to structure the data in a format that makes it actually easy for us to work with.

NumPy helps us to work with arrays - we will understand its needs when we deep dive into it.

Importing the dependencies.

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
import requests
from bs4 import BeautifulSoup
import re

Importing the tokenizer from Transformers will help us to pass through a string and convert that into a sequence of numbers that we can pass through our NLP model.

Auto Model for Sequence Classification - This will give us the architecture from transformers to be able to load in our NLP model.

Next, we are importing PyTorch, requests, Beautiful Soup. Re is used for creating a regex function to be able to extract a specific comment we want.

Setting up the Model

tokenizer = AutoTokenizer.from_pretrained('nlptown/bert-base-multilingual-uncased-sentiment')

model = AutoModelForSequenceClassification.from_pretrained('nlptown/bert-base-multilingual-uncased-sentiment')

In the first line we are loading in our tokenizer and then in the next line we are loading in our model.

So, the first line says about creating a tokenizer that is coming from a pre trained model. The words within quotes are the link of the model that is present in the Hugging Face website.

In the next line we are setting up our model = Auto model Sequence Classification which we imported earlier and we are using the from pretrained method to be able to load the pretrained model.

Encode and Calculate Sentiment

tokens = tokenizer. Encode ('I hated this, absolutely the worst', return_tensocould've)

So, the Tokenizer has just converted the tokens into numbers.

I will share more about this in the next blog.

Introduction to Time Series Forecasting

Time Series Forecasting Using Deep AR and Gluton TS

In this blog article, we will introduce you to time series forecasting using two deep learning architectures: DeepAR and GluonTS. We will cover the basics of each approach and show you how to get started with each one.

DeepAR is a neural network architecture for time series forecasting that is based on recurrent neural networks (RNNs). DeepAR has been shown to outperform other traditional time series forecasting methods, such as autoregressive moving average (ARMA) models, in terms of accuracy.

GluonTS is a deep learning toolkit that makes it easy to build time series models. GluonTS provides ready-to-use components that can be used to build complex models with minimal effort. In addition, GluonTS comes with pre-trained models that can be fine-tuned for your specific problem.

So, let's get started with our introduction to time series forecasting using DeepAR and GluonTS!

Bias-Variance Tradeoff and Overfitting

Bias-variance tradeoff is a fundamental problem in machine learning. It occurs when our models are too simple (high bias) or too complex (high variance). This tradeoff is often referred to as the “dilemma” because we can never have both low bias and low variance.

Overfitting occurs when our models are too complex. This causes them to learn the noise in the data rather than the signal. Overfitting is a major problem in machine learning because it leads to poor generalization performance on out-of-sample data.

The goal of any machine learning algorithm is to find a balance between bias and variance so that we can minimize both and achieve good generalization performance. However, this is often easier said than done. In practice, we usually have to sacrifice one for the other. For example, if we want low bias, we might have to accept high variance. Or if we want low variance, we might have to accept high bias.

The deep AR model proposed by Google DeepMind addresses this issue by using a deep neural network to automatically learn the appropriate level of complexity for the time series data. This results in a model with much lower variance and better generalization performance.

Gluton TS is another approach that tries to address the issue of overfitting in time series forecasting. It does this by first training a base model on the data and then training a second model that learns how to correct the errors made by the

Fixed Effect Model

A fixed effect model is a statistical model that estimates the effects of one or more variables on a response variable. The model includes a fixed intercept and slope for each predictor variable. Fixed effect models are used when the predictor variables are fixed, such as in an experiment.

In time series forecasting, a fixed effect model can be used to identify the trend and seasonality in the data. The model can also be used to forecast future values of the response variable.

ARMA Model

An ARMA model is a statistical model that combines an autoregressive (AR) model with a moving average (MA) model. These models are used to forecast future values of time series data, such as stock prices, based on past values.

The ARMA model is a generalization of the AR and MA models, which are both special cases of the ARMA model. The ARMA model is also known as the Box-Jenkins model, after the statisticians who developed it.

The ARMA model is specified by two parameters: the order of the AR part of the model, and the order of the MA part of the model. For example, an ARMA(1,1) model would be an autoregressivemodel with one lag and a moving average model with one lag.

The coefficients in an ARMA model are estimated using maximum likelihood estimation (MLE). Once estimated, the fitted values from the model can be used to forecast future values of the time series data.

Autoregressive Neural Network (AR) Method for ARMA Models

The autoregressive neural network (AR) model is a type of artificial neural network that can be used for time series forecasting. The AR model is based on the idea that the past values of a time series can be used to predict future values.

The AR model can be used to forecast time series data such as stock prices, economic indicators, and weather patterns. The AR model is a type of recurrent neural network (RNN), which means that it can handle data with temporal dependencies.

The AR model is trained using a training dataset, which contains historical data points. The model then makes predictions for future data points based on the patterns it has learned from the training dataset.

Gluton TS is a deep learning platform that can be used to train autoregressive models. Gluton TS offers an easy-to-use interface and powerful tools for data preprocessing, model training, and prediction tuning.

GlutonTS for AR Models

If you're looking to get started with time series forecasting using deep learning, then you'll want to check out GlutonTS. GlutonTS is a library for training and deploying autoregressive (AR) models.

AR models are a type of neural network that are well-suited for time series data. They are able to capture the dependencies between successive timesteps in a series, making them ideal for forecasting tasks.

GlutonTS makes it easy to train and deploy AR models. It provides a simple API for building models and training on time series data. It also includes support for popular machine learning frameworks such as TensorFlow and PyTorch.

If you're interested in time series forecasting, then you should definitely check out GlutonTS. It's a great way to get started with deep learning for this task. Happy learning!

Conclusion

We've discussed how deep AR and Gluton TS can be used for time series forecasting. Deep AR is a powerful tool which allows us to model complex non-linear relations, while Gluton TS helps make the process of data preparation and feature engineering easier. Together, these two tools provide a powerful combination for accurate predictions in time series forecasting applications. We hope this article has shown you just how useful they can be in helping you achieve reliable forecasts.

As a developer, you may be familiar with the term "image detection." Image detection is a process in which a computer can detect objects in an image or video. This process is becoming increasingly popular in the field of computer vision, as it allows machines to identify objects in images or videos with great accuracy.

In this article, I'm going to take you on a journey to uncover the magic of image detection with Python, OpenCV, and TensorFlow. We'll start with a brief introduction to image detection and then delve into the overviews of OpenCV, TensorFlow, object detection, and the various object detection algorithms. Then, we'll discuss how to use Python for image detection and explore deep learning and object detection. Finally, we'll look at some of the object detection models and algorithms and explore the various applications of object detection in the real world.

Introduction to Object Detection

Image detection is the process of detecting objects in an image or video. It is an important part of computer vision, a field of artificial intelligence that deals with understanding images and videos. Image detection can be used for a variety of purposes, such as security, medical imaging, and autonomous vehicles.

Image detection involves identifying objects in an image or video. The objects can be anything, from cars and people to animals and buildings. The first step in the process is to extract the features (edges, contours, etc.) of the objects in the image. Then, a machine learning algorithm is used to identify the objects in the image.

The main types of image detection algorithms are classification, localization, and object detection. Classification algorithms are used to classify objects in an image into different categories. Localization algorithms are used to locate the position of an object in an image. Finally, object detection algorithms are used to identify objects and their locations in an image.

Overview of OpenCV

OpenCV (Open Source Computer Vision Library) is a popular library for image processing and computer vision. It is open source and has been around since 2000. It is written in C++, but can be used with Python, Java, and other programming languages.

OpenCV is used for a variety of tasks, such as facial recognition, object tracking, and object detection. It has a large collection of algorithms and tools for image processing and computer vision. It also has a good set of tutorials and documentation which makes it easier to learn and use.

Overview of Object Detection

Object detection is the process of identifying objects in an image or video. It is a form of computer vision which is used to detect objects in images or videos. It is an important part of artificial intelligence as it allows machines to recognize and locate objects in images and videos.

Object detection algorithms can be used for a variety of tasks, such as facial recognition, object tracking, and autonomous vehicle navigation. There are various object detection algorithms, such as classification, localization, and object detection. Classification algorithms are used to classify objects in an image into different categories. Localization algorithms are used to locate the position of an object in an image. Finally, object detection algorithms are used to identify objects and their locations in an image.

Types of Object Detection Algorithms

There are various types of object detection algorithms, each of which has its own advantages and disadvantages. Some of the most popular object detection algorithms are:

• Haar-Cascade: This is an old algorithm which uses a cascade of Haar features to detect objects in an image. It is used for facial recognition and object detection.

• HOG (Histogram of Oriented Gradients): This is a feature descriptor which is used to detect objects in an image. It is used in object detection and facial recognition.

• SSD (Single Shot Detection): This is a deep learning-based algorithm which uses a single neural network to detect objects in an image. It is used for object detection and facial recognition.

• YOLO (You Only Look Once): This is a deep learning-based algorithm which uses a single neural network to detect objects in an image. It is used for object detection and facial recognition.

• R-CNN (Region-based Convolutional Neural Network): This is a deep learning-based algorithm which uses a region-based convolutional neural network to detect objects in an image. It is used for object detection and facial recognition.

• Fast R-CNN: This is a deep learning-based algorithm which uses a region-based convolutional neural network to detect objects in an image. It is used for object detection and facial recognition.

• Faster R-CNN: This is a deep learning-based algorithm which uses a region-based convolutional neural network to detect objects in an image. It is used for object detection and facial recognition.

One of the main challenges in object detection is the large variability in object appearance and scale. To handle this, many object detection models use feature pyramids, which allow the model to detect objects at multiple scales.

In TensorFlow, the Object Detection API provides a collection of pre-trained models that are trained on the COCO dataset (Common Objects in Context) and are able to detect a wide range of objects. To use one of these models, you will need to install the Object Detection API and download the model checkpoint file.

Here is an example of how to use the Object Detection API to detect objects in an image:

import tensorflow as tf
from object_detection.utils import label_map_util
from object_detection.utils import visualization_utils as vis_util

# Load the label map
label_map = label_map_util.load_labelmap('path/to/label_map.pbtxt')
categories = label_map_util.convert_label_map_to_categories(label_map, max_num_classes=90, use_display_name=True)
category_index = label_map_util.create_category_index(categories)

# Load the model
detection_model = tf.keras.models.load_model('path/to/model.h5')

# Load the image
image = tf.keras.preprocessing.image.load_img('path/to/image.jpg')
image_np = tf.keras.preprocessing.image.img_to_array(image)

# Expand the dimensions of the image
image_np_expanded = np.expand_dims(image_np, axis=0)

# Perform object detection
output_dict = detection_model(image_np_expanded)

# Get the detections
detections = output_dict['detection_boxes']
scores = output_dict['detection_scores']

# Visualize the detections
vis_util.visualize_boxes_and_labels_on_image_array(
    image_np,
    detections,
    scores,
    category_index,
    instance_masks=output_dict.get('detection_masks'),
    use_normalized_coordinates=True,
    min_score_thresh=0.8,
    max_boxes_to_draw=20
)

# Display the image
plt.figure(figsize=(12, 8))
plt.imshow(image_np)
plt.show()

Object Detection Applications

Object detection is used for a variety of applications, such as facial recognition, object tracking, and autonomous vehicle navigation. It is also used for security purposes, such as detecting intruders in a building or detecting suspicious behavior in a crowd.

Object detection can also be used to create object recognition applications, such as a smartphone app that can recognize objects in an image. It can also be used to create medical imaging applications, such as an application which can detect tumors in an X-ray or MRI scan.

Conclusion

In this article, I have taken you on a journey to uncover the magic of image detection with Python, OpenCV, and TensorFlow. We started with an introduction to image detection and then discussed the overviews of OpenCV, TensorFlow, object detection, and the various object detection algorithms. Then, we discussed how to use Python for image detection and explored deep learning and object detection. Finally, we looked at some of the object detection models and algorithms and explored the various applications of object detection in the real world.

Object detection is an important part of computer vision and artificial intelligence. It is a powerful tool which can be used for a variety of tasks, such as facial recognition, object tracking, and autonomous vehicle navigation. With the help of Python, OpenCV, and TensorFlow, image detection can be used to solve real-world problems.

So, if you're looking to get started with image detection, then this article has provided you with the necessary information to get started. Now, all you have to do is take the plunge and start exploring the magic of image detection with Python, OpenCV, and TensorFlow!

CTA: Real time object detection computer vision

Image classification and deep learning are two powerful tools that can help you reach new heights in data science. By combining these two technologies, you can create powerful and accurate models that can solve complex problems. In this blog, we’ll discuss what image classification is, the benefits of image classification, image classification with deep learning, and best practices for image classification.

What is Image Classification

Image classification is a computer vision technique that uses machine learning algorithms to identify and classify objects in an image. It can be used for a wide range of applications, from facial recognition to medical diagnosis. Image classification utilizes a variety of algorithms, including convolutional neural networks (CNNs), support vector machines (SVMs), and random forests.

In image classification, images are presented to a computer algorithm, which then performs a series of computations to identify objects in the image. The algorithm then assigns a label or class to each object in the image. For example, an algorithm may be trained to recognize cats, dogs, and birds in a given image.

The process of image classification can be broken down into two parts: feature extraction and classification. Feature extraction is the process of extracting the objects in an image that are relevant to the task at hand. This involves identifying edges, lines, shapes, and other features that can be used to identify an object. The classification process is the process of assigning labels to the extracted features. This can be done manually or with the help of a machine learning algorithm.

Benefits of Image Classification

Image classification has many benefits, including:

• Improved accuracy: Image classification algorithms can be more accurate than human annotators, as they are trained to recognize specific features.

• Faster processing: Image classification algorithms can process images faster than humans, as they don’t need to manually search for objects in an image.

• Cost savings: Automated image classification algorithms can save companies time and money by eliminating the need for manual annotation.

• Automation: Image classification algorithms can be used to automate tasks such as facial recognition, object detection, and medical diagnosis.

• Scalability: Image classification algorithms can be scaled to process large amounts of data quickly and accurately.

Tensorflow Transfer Learning

TensorFlow is a popular open-source machine learning library that can be used for image classification. It provides a variety of tools and algorithms that can be used to train and deploy image classification models. One of the most powerful tools available in TensorFlow is transfer learning.

Transfer learning is a technique that allows you to take a model that has already been trained on a large dataset and fine-tune it for your own dataset. This can be useful if you don’t have access to a large dataset, or if you want to improve the accuracy of your model. Transfer learning can also save you time, as you don’t have to train a new model from scratch.

Image Classification with Deep Learning Tools

There are a variety of deep learning tools available that can be used for image classification. These tools make it easy to train and deploy image classification models without having to write your own code. Some of the most popular deep learning tools include TensorFlow, Keras, Caffe, and PyTorch.

TensorFlow is a popular open-source machine learning library that can be used for image classification. It provides a variety of tools and algorithms that can be used to train and deploy image classification models.

Keras is another popular deep learning library that can be used for image classification. It provides a high-level API that makes it easy to train and deploy image classification models.

Caffe is a deep learning framework that can be used for image classification. It provides a flexible architecture that makes it easy to deploy and scale image classification models.

PyTorch is a popular deep learning library that can be used for image classification. It provides a library of algorithms and tools that make it easy to train and deploy image classification models.

Demonstration

To start, we will need to install TensorFlow 2 and any other required libraries. You can install TensorFlow by running the following command:

pip install tensorflow

Next, we will need to download and extract a pre-trained model. TensorFlow provides a number of pre-trained models in its TensorFlow Hub library. For this example, we will use the MobileNetV2 model, which has been trained on the ImageNet dataset. You can download and extract the model by running the following code:

import tensorflow as tf
import tensorflow_hub as hub

model = tf.keras.Sequential([
    hub.KerasLayer("https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4",
                   input_shape=(224, 224, 3))
])

Next, we will need to download and prepare our dataset. For this example, we will use the Cats vs Dogs dataset from Kaggle. This dataset contains 25,000 images of cats and dogs, split into training and validation sets.

After downloading and extracting the dataset, we can use the following code to load the images and prepare them for training:

import os
import numpy as np
import matplotlib.pyplot as plt

# Load the images
X = []
Y = []
for filename in os.listdir("train"):
    label = filename.split(".")[0]
    if label == "cat":
        Y.append(0)
    else:
        Y.append(1)
    image = plt.imread("train/" + filename)
    X.append(image)
X = np.array(X)
Y = np.array(Y)

# Split the data into train and validation sets
from sklearn.model_selection import train_test_split
X_train, X_val, Y_train, Y_val = train_test_split(X, Y, test_size=0.2)

Here's another example!

from keras.applications import VGG16
from keras.preprocessing.image import ImageDataGenerator
from keras.layers import Dense, Dropout, Flatten
from keras import Model
from keras import optimizers

# Load the VGG16 model
base_model = VGG16(weights='imagenet', include_top=False, input_shape=(224, 224, 3))

# Add a new classifier layer on top of the base model
x = base_model.output
x = Flatten()(x)
x = Dense(1024, activation='relu')(x)
x = Dropout(0.5)(x)
predictions = Dense(10, activation='softmax')(x)
model = Model(inputs=base_model.input, outputs=predictions)

# Freeze the base model layers
for layer in base_model.layers:
    layer.trainable = False

# Compile the model
model.compile(optimizer=optimizers.SGD(lr=0.0001, momentum=0.9), loss='categorical_crossentropy', metrics=['accuracy'])

# Train the model using the ImageDataGenerator
train_datagen = ImageDataGenerator(rescale=1./255,
                                   shear_range=0.2,
                                   zoom_range=0.2,
                                   horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
        'data/train',
        target_size=(224, 224),
        batch_size=32,
        class_mode='categorical')

validation_generator = test_datagen.flow_from_directory(
        'data/val',
        target_size=(224, 224),
        batch_size=32,
        class_mode='categorical')

model.fit_generator(
        train_generator,
        steps_per_epoch=100,
        epochs=10,
        validation_data=validation_generator,
        validation_steps=50)

Tips for Image Classificatoin

There are a few tips that can help you improve the accuracy of your image classification models:

• Use data augmentation: Data augmentation is a technique that can be used to increase the amount of data available for training. This can be useful if you don’t have access to a large dataset.

• Utilize transfer learning: Transfer learning is a powerful technique that can be used to improve the accuracy of your model.

• Use a balanced dataset: It’s important to use a balanced dataset when training your model. This means that the dataset should contain an equal number of samples for each class.

• Use a large dataset: The larger the dataset, the better the accuracy of your model.

Best Practices for Image Classification

When it comes to creating accurate image classification models, there are a few best practices that you should follow:

• Choose the right algorithm: Not all algorithms are suitable for all tasks. Make sure to choose an algorithm that is suitable for your task.

• Use a balanced dataset: As mentioned above, it’s important to use a balanced dataset when training your model.

• Utilize transfer learning: Transfer learning can be a powerful tool for improving the accuracy of your model.

• Use data augmentation: Data augmentation can be used to increase the amount of data available for training.

• Tune your hyperparameters: Hyperparameters are parameters that control the behaviour of your model. Tuning these parameters can help you improve the accuracy of your model.

Challenges of Image Classification

Image classification is a complex task that can be difficult to get right. Some of the challenges that you may face when creating an image classification model include:

• Overfitting: Overfitting occurs when the model starts to learn the noise in the data instead of the underlying patterns. This can lead to poor performance on new data.

• Unbalanced datasets: Unbalanced datasets can lead to biased models that don’t generalize well.

• Poorly labeled data: Poorly labeled data can lead to inaccurate models.

• Poorly chosen algorithms: Choosing the wrong algorithm for your task can lead to poor performance.

• Lack of data: If you don’t have access to enough data, it can be difficult to train an accurate model.

Conclusion

Image classification and deep learning can be powerful tools for solving complex problems. By combining these two technologies, you can create powerful and accurate models that can identify objects in an image with high accuracy. In this blog, we discussed what image classification is, the benefits of image classification, image classification with deep learning, and best practices for image classification. We also discussed some of the challenges that you may face when creating an image classification model.

If you’re looking to create an accurate image classification model, it’s important to choose the right algorithm, use a balanced dataset, and utilize transfer learning. And remember to use data augmentation to increase the amount of data available for training. With these tips, you can create powerful and accurate image classification models that can solve complex problems.

Image classification and deep learning are two powerful tools that can help you reach new heights in data science. So, what are you waiting for? Start experimenting with image classification and deep learning today and see what you can create!

Let me start with a very familiar example. It's Google Search!

Have you seen the Documentary The Social Dilemma in Netflix? Go watch it!

The documentary tells you the story about how big tech companies collect data from you and use it to show you advertisements. But this is just the tip of Ice Berg. They predict your next move! When you go to google and type "climate change is" you will see the results are different for different people across the world.

Your Twitter, LinkedIn and Instagram feed are different from mine! And that's okay because you and I may not like the same thing. Well, but how did Instagram or LinkedIn learn that? Well, that's Machine Learning! They learnt from your previous actions and the accounts you followed, the number of times you chatted with someone or the amount of time you took to look in a certain post. Everything is tracked and recorded!

What is Machine Learning made off??

Well Machine Learning, as the name suggests, is a computer program that is made up of a number of algorithms which predict the next move based on the previous data given to it.

Let's start with an example!

Suppose somehow, I built a Machine Learning Model to predict Cats. The model is 70% accurate i.e. If I show the model a picture of a cat, it will be able to tell me with 70% accuracy whether the image is of a cat or not! So, I train the model a bit more and it learns from the data I feed to it. Now the model is 99% accurate and now it can detect a cat from any image. This is just a small example of how ML models work.

In real life, ML models have a variety of use cases. It can be used to predict new chemical combinations for new medicines, stock prices, you next search in google, or the next post you're going to see in Instagram

Now let's talk about types of Machine Learning

They are generally of 3 types...

1. Supervised Learning
2. Unsupervised Learning
3. Reinforcement Learning

Well, there can be a lot of types that can be mentioned like Semi-Supervised or Inductive Learning, but that requires a separate blog to talk about, hence I am not mentioning them here.

So, coming up with Supervised Learning! What is it?

So Supervised Learning as the name suggests is Machine Learning under supervision. Which means the ML model is trained under labelled data which contains correct inputs and its desired output. The ML model learns over time to predict the correct results. Remember, it knows what is correct only on the basis of the fact that we teach them what is right and what is wrong. If I teach slapping is Good and Loving is Bad, it will learn that! The ML model will later classify slapping as a good act! And that's Supervised Learning

Example of Supervised Learning:

Your Outlook or Gmail account that detects and puts certain emails into the spam folder is an example of Supervised Learning. It has been trained previously and it further learns from the Spam emails that get filtered out. It's a continuous process.

Next is Unsupervised Learning

So Unsupervised Learning is based on Machine Learning Algorithms that clusters data sets which are generally unlabeled and hence this does not require any human intervention, hence "Unsupervised Machine Learning.

Let's Take an Example

If I give my ML model some images of a cat, and on the basis of the algorithm in it, the model predicts that the cat has a tail, 4 legs and 2 eyes. And they discovered it without any human intervention!

Finally, we have Reinforcement Learning

RL is an algorithm to train ML models to make a sequence of decisions. The whole process is generally based on a reward-based system where we give a reward to if it takes the right decision and we punish it if they take some wrong decision.

Example:

This is a very famous example of RL where we teach a rocket to land perfectly. It gets a reward for every small step it performs and the final goal is to reach maximum reward and land the rocket in the desired position.