Annotate AI datasets with ChatGPT

Annotations are important but costly

Data sets are important in applying AI to business and society.
Data sets are both a competitive advantage and a barrier to AI utilization.
The AI ​​model itself is widely available for free, and it is not difficult to use.
So what is the difference in business competitiveness and the barrier to social adoption? Computing power (GPU) and data sets.
There are not that many datasets that are publicly available, and if you want to actually use AI for specific tasks in business or society, it is essential to create a dataset for that purpose.
However, dataset annotation (the process of adding labels to the collected data to inform AI or quantifying the locations of objects) is often an unremarkable, time-consuming, and costly manual process.
Since data is separated using human judgment (and the results are taught to AI to give it similar capabilities), it is only natural that human intervention is required.
(It seems that there are not many published cases where this has been successfully done.)
However, in certain input-output situations, it is possible to annotate image discrimination datasets using ChatGPT, which can adapt to the purpose on a par with humans. .

Annotation of image discrimination dataset using ChatGPT Case study

This time, the students were asked to sort whether the image was a landscape of Paris or a landscape of Kyoto.

Well, this would be easy enough for a human to do. You might think so, but when you have, say, 20,000 pieces, sorting them becomes a hassle. Also, this is just an example; in reality, you may need to classify into ten or more classes.

So let’s let ChatGPT do the troublesome work for us.

manner

Installing openai libraries.

pip install openai

I’ll have one sorted for now.

Which city is the scenery in this image from? Paris, Kyoto, please answer in English with only the city name. If you do not fall into either category, please answer other. Capitalize the city name and do not include periods.

Send this message to the gpt-4-vision API.

import base64
import requests

text = "Which of the following cities does the scenery in this image come from? Paris, Kyoto, please answer in English with only the city name. If you do not fall into either category, please answer "other". Capitalize the city name and do not include periods."

# OpenAI API Key
api_key = "Your_API_Key"

# Path to your image
image_path = "paris0.jpg"

def ask_gpt(text, image_path, api_key):
  
  with open(image_path, "rb") as image_file:
    base64_image = base64.b64encode(image_file.read()).decode('utf-8')

  headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {api_key}"
  }

  payload = {
    "model": "gpt-4-vision-preview",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": text
          },
          {
            "type": "image_url",
            "image_url": {
              "url": f"data:image/jpeg;base64,{base64_image}"
            }
          }
        ]
      }
    ],
    "max_tokens": 300
  }

  response = requests.post("https://api.openai.com/v1/chat/completions", headers=headers, json=payload)
  city = response.json()["choices"][0]["message"]["content"]
  
  return city

city = ask_gpt(text, image_path, api_key)
print(city)

Paris

It will respond if you ask it briefly, such as “What city is it in?”, but since I want to use it for labeling in an image classification task, I’ll ask it to respond in one word from the options.

have a lot sorted

Loop questions to gpt about the contents of the image directory and divide the images into folders according to the results.

import os
import shutil

image_dir = "images"
image_list = os.listdir(image_dir)

for image_file in image_list:
  image_path = os.path.join(image_dir, image_file)
  city = ask_gpt(text, image_path, api_key)
  
  dest_dir = city
  dest_path = os.path.join(dest_dir, image_file)
  shutil.move(image_path, dest_path)

This level of classification is perfect.

Paris

Kyoto

However, as of March 2024, the exact location of the object cannot be answered, so annotations for Object Detection and Segmentation tasks will not be possible.

However, when it comes to just sorting image classification data, it can be extremely troublesome (and expensive to have someone do it for you), so I would be grateful if ChatGPT could do it for me.

🐣

I’m a freelance engineer.
Work consultation
Please feel free to contact us with a brief development description.
rockyshikoku@gmail.com

I am creating applications using machine learning and AR technology.

I send machine learning / AR related information.

GitHub

Twitter
Medium