Food Recognition Benchmark 2022
🍕 Food Recognition Benchmark: Data Exploration & Baseline
Dataset exploration and `detectron2` baseline training code
🍕 Food Recognition Benchmark¶
Credits: This notebook is fork of notebook created by @shubhamai for previous iteration of the challenge. You can find the original notebook here.
Problem Statement¶
Detecting & Segmenting various kinds of food from an image. For ex. Someone got into new restaurent and get a food that he has never seen, well our DL model is in rescue, so our DL model will help indentifying which food it is from the class our model is being trained on!
Dataset¶
We will be using data from Food Recognition Challenge - A benchmark for image-based food recognition challange which is running since 2020.
https://www.aicrowd.com/challenges/food-recognition-benchmark-2022#datasets
We have a total of 39k training images with 3k validation set and 4k public-testing set. All the images are RGB and annotations exist in MS-COCO format.
Evaluation¶
The evaluation metrics is IOU aka. Intersection Over Union ( more about that later ).
The actualy metric is computed by averaging over all the precision and recall values for IOU which greater than 0.5.
https://www.aicrowd.com/challenges/food-recognition-challenge#evaluation-criteria
What does this notebook contains?¶
Setting our Workspace 💼
Data Exploration 🧐
- Reading Dataset
- Data Visualisations
Image Visulisation 🖼️
- Reading Images
Creating our Dataset 🔨
- Fixing the Dataset
- Creating our dataset
Creating our Model 🏭
- Creating R-CNN Model
- Setting up hyperparameters
Training the Model 🚂
- Setting up Tensorboard
- Start Training!
Evaluating the model 🧪
- Evaluating our Model
Testing the Model 💯
- Testing the Model
Submitting our predictions 📝
Generate More Data + Some tips & tricks 💡
Setting our Workspace 💼¶
In this section we will be downloading our dataset, unzipping it & downliading detectron2 library and importing all libraries that we will be using
Downloading & Unzipping our Dataset¶
In [1]:
# Login to AIcrowd
!pip install aicrowd-cli > /dev/null
!aicrowd login
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. google-colab 1.0.0 requires requests~=2.23.0, but you have requests 2.26.0 which is incompatible. datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible. Please login here: https://api.aicrowd.com/auth/NnwCEee07iHXtuxG0ByOQmZge6Jb-VfXvENxWAjJhls /usr/bin/xdg-open: 851: /usr/bin/xdg-open: www-browser: not found /usr/bin/xdg-open: 851: /usr/bin/xdg-open: links2: not found /usr/bin/xdg-open: 851: /usr/bin/xdg-open: elinks: not found /usr/bin/xdg-open: 851: /usr/bin/xdg-open: links: not found /usr/bin/xdg-open: 851: /usr/bin/xdg-open: lynx: not found /usr/bin/xdg-open: 851: /usr/bin/xdg-open: w3m: not found xdg-open: no method available for opening 'https://api.aicrowd.com/auth/NnwCEee07iHXtuxG0ByOQmZge6Jb-VfXvENxWAjJhls' API Key valid Saved API Key successfully!
In [2]:
# List dataset for this challenge
!aicrowd dataset list -c food-recognition-benchmark-2022
# Download dataset
!aicrowd dataset download -c food-recognition-benchmark-2022
Datasets for challenge #962 ┌───┬────────────────────────────────┬────────────────────────────────┬────────┐ │ # │ Title │ Description │ Size │ ├───┼────────────────────────────────┼────────────────────────────────┼────────┤ │ 0 │ public_validation_set_2.0.tar… │ Validation Dataset (contains │ 59M │ │ │ │ 1000 images and 498 │ │ │ │ │ categories, with annotations) │ │ │ 1 │ public_test_release_2.0.tar.gz │ [Public] Testing Dataset │ 197M │ │ │ │ (contains 3000 images and 498 │ │ │ │ │ categories, without │ │ │ │ │ annotations) │ │ │ 2 │ public_training_set_release_2… │ Training Dataset (contains │ 2.14GB │ │ │ │ 39962 images and 498 │ │ │ │ │ categories) │ │ └───┴────────────────────────────────┴────────────────────────────────┴────────┘ public_validation_set_2.0.tar.gz: 100% 62.4M/62.4M [00:05<00:00, 11.6MB/s] public_test_release_2.0.tar.gz: 100% 207M/207M [00:12<00:00, 16.2MB/s] public_training_set_release_2.0.tar.gz: 100% 2.30G/2.30G [02:38<00:00, 14.5MB/s]
In [3]:
# Create data directory
!mkdir -p data/ data/train data/val data/test
!cp *test* data/test && cd data/test && echo "Extracting test dataset" && tar -xvf *test* > /dev/null
!cp *val* data/val && cd data/val && echo "Extracting val dataset" && tar -xvf *val* > /dev/null
!cp *train* data/train && cd data/train && echo "Extracting train dataset" && tar -xvf *train* > /dev/null
Extracting test dataset Extracting val dataset Extracting train dataset
So, the data directory is something like this:
Importing Necessary Libraries¶
In [4]:
# Making sure that we are using GPUs
!nvidia-smi
Tue Dec 21 06:43:52 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.44 Driver Version: 460.32.03 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 00000000:00:04.0 Off | 0 |
| N/A 33C P8 27W / 149W | 0MiB / 11441MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
In [1]:
# Colab has cuda 11.1 pre-installed nowadays, downgrading to 1.9 for detectron2
!pip install -U torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html
!pip install cython pyyaml==5.1
!pip install -U pycocotools
import torch, torchvision
print(torch.__version__, torch.cuda.is_available())
!gcc --version
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) Looking in links: https://download.pytorch.org/whl/torch_stable.html Requirement already satisfied: torch==1.9.0+cu111 in /usr/local/lib/python3.7/dist-packages (1.9.0+cu111) Requirement already satisfied: torchvision==0.10.0+cu111 in /usr/local/lib/python3.7/dist-packages (0.10.0+cu111) Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch==1.9.0+cu111) (3.10.0.2) Requirement already satisfied: pillow>=5.3.0 in /usr/local/lib/python3.7/dist-packages (from torchvision==0.10.0+cu111) (7.1.2) Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torchvision==0.10.0+cu111) (1.19.5) WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) Requirement already satisfied: cython in /usr/local/lib/python3.7/dist-packages (0.29.24) Requirement already satisfied: pyyaml==5.1 in /usr/local/lib/python3.7/dist-packages (5.1) WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) Requirement already satisfied: pycocotools in /usr/local/lib/python3.7/dist-packages (2.0.3) Requirement already satisfied: setuptools>=18.0 in /usr/local/lib/python3.7/dist-packages (from pycocotools) (57.4.0) Requirement already satisfied: cython>=0.27.3 in /usr/local/lib/python3.7/dist-packages (from pycocotools) (0.29.24) Requirement already satisfied: matplotlib>=2.1.0 in /usr/local/lib/python3.7/dist-packages (from pycocotools) (3.2.2) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.1.0->pycocotools) (3.0.6) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.1.0->pycocotools) (0.11.0) Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.1.0->pycocotools) (2.8.2) Requirement already satisfied: numpy>=1.11 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.1.0->pycocotools) (1.19.5) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.1.0->pycocotools) (1.3.2) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib>=2.1.0->pycocotools) (1.15.0) WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) 1.9.0+cu111 True gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
In [2]:
!nvidia-smi
Tue Dec 21 07:20:35 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.44 Driver Version: 460.32.03 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 00000000:00:04.0 Off | 0 |
| N/A 33C P8 27W / 149W | 3MiB / 11441MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
In [3]:
# install detectron2:
!pip install -U detectron2==0.6+cu111 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/index.html
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) Looking in links: https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/index.html Collecting detectron2==0.6+cu111 Downloading https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/detectron2-0.6%2Bcu111-cp37-cp37m-linux_x86_64.whl (6.9 MB) |████████████████████████████████| 6.9 MB 811 kB/s Requirement already satisfied: tqdm>4.29.0 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (4.62.3) Requirement already satisfied: pycocotools>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (2.0.3) Requirement already satisfied: Pillow>=7.1 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (7.1.2) Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (3.2.2) Requirement already satisfied: tensorboard in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (2.7.0) Requirement already satisfied: fvcore<0.1.6,>=0.1.5 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (0.1.5.post20211023) Requirement already satisfied: iopath<0.1.10,>=0.1.7 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (0.1.9) Requirement already satisfied: yacs>=0.1.8 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (0.1.8) Requirement already satisfied: black==21.4b2 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (21.4b2) Requirement already satisfied: pydot in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (1.3.0) Requirement already satisfied: hydra-core>=1.1 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (1.1.1) Requirement already satisfied: tabulate in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (0.8.9) Requirement already satisfied: cloudpickle in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (1.3.0) Requirement already satisfied: termcolor>=1.1 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (1.1.0) Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (0.16.0) Requirement already satisfied: omegaconf>=2.1 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (2.1.1) Requirement already satisfied: mypy-extensions>=0.4.3 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (0.4.3) Requirement already satisfied: typed-ast>=1.4.2 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (1.5.1) Requirement already satisfied: regex>=2020.1.8 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (2021.11.10) Requirement already satisfied: pathspec<1,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (0.9.0) Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (3.10.0.2) Requirement already satisfied: click>=7.1.2 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (7.1.2) Requirement already satisfied: toml>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (0.10.2) Requirement already satisfied: appdirs in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (1.4.4) Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from fvcore<0.1.6,>=0.1.5->detectron2==0.6+cu111) (1.19.5) Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from fvcore<0.1.6,>=0.1.5->detectron2==0.6+cu111) (5.1) Requirement already satisfied: importlib-resources in /usr/local/lib/python3.7/dist-packages (from hydra-core>=1.1->detectron2==0.6+cu111) (5.4.0) Requirement already satisfied: antlr4-python3-runtime==4.8 in /usr/local/lib/python3.7/dist-packages (from hydra-core>=1.1->detectron2==0.6+cu111) (4.8) Requirement already satisfied: portalocker in /usr/local/lib/python3.7/dist-packages (from iopath<0.1.10,>=0.1.7->detectron2==0.6+cu111) (2.3.2) Requirement already satisfied: setuptools>=18.0 in /usr/local/lib/python3.7/dist-packages (from pycocotools>=2.0.2->detectron2==0.6+cu111) (57.4.0) Requirement already satisfied: cython>=0.27.3 in /usr/local/lib/python3.7/dist-packages (from pycocotools>=2.0.2->detectron2==0.6+cu111) (0.29.24) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2==0.6+cu111) (3.0.6) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2==0.6+cu111) (0.11.0) Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2==0.6+cu111) (2.8.2) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2==0.6+cu111) (1.3.2) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->detectron2==0.6+cu111) (1.15.0) Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from importlib-resources->hydra-core>=1.1->detectron2==0.6+cu111) (3.6.0) Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (1.0.1) Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (0.12.0) Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (0.37.0) Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (2.26.0) Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (0.4.6) Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (0.6.1) Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (1.8.0) Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (1.35.0) Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (3.3.6) Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (1.42.0) Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (3.17.3) Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->detectron2==0.6+cu111) (4.2.4) Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->detectron2==0.6+cu111) (4.8) Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->detectron2==0.6+cu111) (0.2.8) Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2==0.6+cu111) (1.3.0) Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard->detectron2==0.6+cu111) (4.8.2) Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->detectron2==0.6+cu111) (0.4.8) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6+cu111) (2.10) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6+cu111) (2021.10.8) Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6+cu111) (2.0.8) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6+cu111) (1.24.3) Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2==0.6+cu111) (3.1.1) WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) Installing collected packages: detectron2 Attempting uninstall: detectron2 WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) Found existing installation: detectron2 0.6+cu101 Uninstalling detectron2-0.6+cu101: Successfully uninstalled detectron2-0.6+cu101 WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages) Successfully installed detectron2-0.6+cu111
In [4]:
# You may need to restart your runtime prior to this, to let your installation take effect
# Some basic setup:
# Setup detectron2 logger
import detectron2
from detectron2.utils.logger import setup_logger
setup_logger()
# import some common libraries
import numpy as np
import pandas as pd
import cv2
import json
from tqdm.notebook import tqdm
# import some common detectron2 utilities
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog
from detectron2.utils.visualizer import ColorMode
from detectron2.data.datasets import register_coco_instances
from detectron2.engine import DefaultTrainer
from detectron2.evaluation import COCOEvaluator, inference_on_dataset
from detectron2.data import build_detection_test_loader
# For reading annotations file
from pycocotools.coco import COCO
# utilities
from pprint import pprint # For beautiful print!
from collections import OrderedDict
import os
# For data visualisation
import matplotlib.pyplot as plt
import plotly.graph_objects as go
import plotly.express as px
from google.colab.patches import cv2_imshow
Data Exploration 🧐¶
In this section we are going to read our dataset & doing some data visualisations
Reading Data¶
In [5]:
# Reading annotations.json
TRAIN_ANNOTATIONS_PATH = "data/train/annotations.json"
TRAIN_IMAGE_DIRECTIORY = "data/train/images/"
VAL_ANNOTATIONS_PATH = "data/val/annotations.json"
VAL_IMAGE_DIRECTIORY = "data/val/images/"
train_coco = COCO(TRAIN_ANNOTATIONS_PATH)
loading annotations into memory... Done (t=8.87s) creating index... index created!
In [6]:
# Reading the annotation files
with open(TRAIN_ANNOTATIONS_PATH) as f:
train_annotations_data = json.load(f)
with open(VAL_ANNOTATIONS_PATH) as f:
val_annotations_data = json.load(f)
In [7]:
train_annotations_data['annotations'][0]
Out[7]:
{'area': 5059.0,
'bbox': [39.5, 39.5, 167.0, 92.0],
'category_id': 1352,
'id': 184135,
'image_id': 131094,
'iscrowd': 0,
'segmentation': [[115.0,
206.5,
98.0,
204.5,
74.5,
182.0,
65.0,
167.5,
47.5,
156.0,
39.5,
137.0,
39.5,
130.0,
51.0,
118.5,
62.00000000000001,
112.5,
76.0,
113.5,
121.5,
151.0,
130.5,
169.0,
131.5,
185.0,
128.5,
195.0]]}
Data Format 🔍¶
Our COCO data format is something like this -
"info": {...},
"categories": [...],
"images": [...],
"annotations": [...],In which categories is like this
[
{'id': 2578,
'name': 'water',
'name_readable': 'Water',
'supercategory': 'food'},
{'id': 1157,
'name': 'pear',
'name_readable': 'Pear',
'supercategory': 'food'},
...
{'id': 1190,
'name': 'peach',
'name_readable': 'Peach',
'supercategory': 'food'}
]Info is empty ( not sure why )
images is like this
[
{'file_name': '065537.jpg',
'height': 464,
'id': 65537,
'width': 464},
{'file_name': '065539.jpg',
'height': 464,
'id': 65539,
'width': 464},
...
{'file_name': '069900.jpg',
'height': 391,
'id': 69900,
'width': 392},
]Annotations is like this
{'area': 44320.0,
'bbox': [86.5, 127.49999999999999, 286.0, 170.0],
'category_id': 2578,
'id': 102434,
'image_id': 65537,
'iscrowd': 0,
'segmentation': [[235.99999999999997,
372.5,
169.0,
372.5,
...
368.5,
264.0,
371.5]]}In [9]:
# Reading all classes
category_ids = train_coco.loadCats(train_coco.getCatIds())
category_names = [_["name_readable"] for _ in category_ids]
print("## Categories\n-", "\n- ".join(category_names))
## Categories - Bread, wholemeal - Jam - Water - Bread, sourdough - Banana - Soft cheese - Ham, raw - Hard cheese - Cottage cheese - Bread, half white - Coffee, with caffeine - Fruit salad - Pancakes - Tea - Salmon, smoked - Avocado - Spring onion / scallion - Ristretto, with caffeine - Ham - Egg - Bacon, frying - Chips, french fries - Juice, apple - Chicken - Tomato, raw - Broccoli - Shrimp, boiled - Beetroot, steamed, without addition of salt - Carrot, raw - Chickpeas - French salad dressing - Pasta, Hörnli - Sauce, cream - Meat balls - Pasta - Tomato sauce - Cheese - Pear - Cashew nut - Almonds - Lentils - Mixed vegetables - Peanut butter - Apple - Blueberries - Cucumber - Cocoa powder - Greek Yaourt, yahourt, yogourt ou yoghourt - Maple syrup (Concentrate) - Buckwheat, grain peeled - Butter - Herbal tea - Mayonnaise - Soup, vegetable - Wine, red - Wine, white - Green bean, steamed, without addition of salt - Sausage - Pizza, Margherita, baked - Salami - Mushroom - (bread, meat substitute, lettuce, sauce) - Tart - Tea, verveine - Rice - White coffee, with caffeine - Linseeds - Sunflower seeds - Ham, cooked - Bell pepper, red, raw - Zucchini - Green asparagus - Tartar sauce - Lye pretzel (soft) - Cucumber, pickled - Curry, vegetarian - Yaourt, yahourt, yogourt ou yoghourt, natural - Soup of lentils, Dahl (Dhal) - Soup, cream of vegetables - Balsamic vinegar - Salmon - Salt cake (vegetables, filled) - Bacon - Orange - Pasta, noodles - Cream - Cake, chocolate - Pasta, spaghetti - Black olives - Parmesan - Spaetzle - Salad, lambs' ear - Salad, leaf / salad, green - Potatoes steamed - White cabbage - Halloumi - Beetroot, raw - Bread, grain - Applesauce, unsweetened, canned - Cheese for raclette - Mushrooms - Bread, white - Curds, natural, with at most 10% fidm - Bagel (without filling) - Quiche, with cheese, baked, with puff pastry - Soup, potato - Bouillon, vegetable - Beef, sirloin steak - Taboulé, prepared, with couscous - Eggplant - Bread - Turnover with meat (small meat pie, empanadas) - Mungbean sprouts - Mozzarella - Pasta, penne - Lasagne, vegetable, prepared - Mandarine - Kiwi - French beans - Tartar (meat) - Spring roll (fried) - Pork, chop - Caprese salad (Tomato Mozzarella) - Leaf spinach - Roll of half-white or white flour, with large void - Pasta, ravioli, stuffing - Omelette, plain - Tuna - Dark chocolate - Sauce (savoury) - Dried raisins - Ice tea - Kaki - Macaroon - Smoothie - Crêpe, plain - Chicken nuggets - Chili con carne, prepared - Veggie burger - Cream spinach - Cod - Chinese cabbage - Hamburger (Bread, meat, ketchup) - Soup, pumpkin - Sushi - Chestnuts - Coffee, decaffeinated - Sauce, soya - Balsamic salad dressing - Pasta, twist - Bolognaise sauce - Leek - Fajita (bread only) - Potato-gnocchi - Beef, cut into stripes (only meat) - Rice noodles/vermicelli - Tea, ginger - Tea, green - Bread, whole wheat - Onion - Garlic - Hummus - Pizza, with vegetables, baked - Beer - Glucose drink 50g - Chicken, wing - Ratatouille - Peanut - High protein pasta (made of lentils, peas, ...) - Cauliflower - Quiche, with spinach, baked, with cake dough - Green olives - Brazil nut - Eggplant caviar - Bread, pita - Pasta, wholemeal - Sauce, pesto - Oil - Couscous - Sauce, roast - Prosecco - Crackers - Bread, toast - Shrimp / prawn (small) - Panna cotta - Romanesco - Water with lemon juice - Espresso, with caffeine - Egg, scrambled, prepared - Juice, orange - Ice cubes - Braided white loaf - Emmental cheese - Croissant, wholegrain - Hazelnut-chocolate spread(Nutella, Ovomaltine, Caotina) - Tomme - Water, mineral - Hazelnut - Bacon, raw - Bread, nut - Black Forest Tart - Soup, Miso - Peach - Figs - Beef, filet - Mustard, Dijon - Rice, Basmati - Mashed potatoes, prepared, with full fat milk, with butter - Dumplings - Pumpkin - Swiss chard - Red cabbage - Spinach, raw - Naan (indien bread) - Chicken curry (cream/coconut milk. curry spices/paste)) - Crunch Müesli - Biscuits - Bread, French (white flour) - Meatloaf - Fresh cheese - Honey - Vegetable mix, peas and carrots - Parsley - Brownie - Dairy ice cream - Tea, black - Carrot cake - Fish fingers (breaded) - Salad dressing - Dried meat - Chicken, breast - Mixed salad (chopped without sauce) - Feta - Praline - Tea, peppermint - Walnut - Potato salad, with mayonnaise yogurt dressing - Kebab in pita bread - Kolhrabi - Alfa sprouts - Brussel sprouts - Bacon, cooking - Gruyère - Bulgur - Grapes - Pork, escalope - Chocolate egg, small - Cappuccino - Zucchini, stewed, without addition of fat, without addition of salt - Crisp bread, Wasa - Bread, black - Perch fillets (lake) - Rosti - Mango - Sandwich (ham, cheese and butter) - Müesli - Spinach, steamed, without addition of salt - Fish - Risotto, without cheese, cooked - Milk Chocolate with hazelnuts - Cake (oblong) - Crisps - Pork - Pomegranate - Sweet corn, canned - Flakes, oat - Greek salad - Cantonese fried rice - Sesame seeds - Bouillon - Baked potato - Fennel - Meat - Bread, olive - Croutons - Philadelphia - Mushroom, (average), stewed, without addition of fat, without addition of salt - Bell pepper, red, stewed, without addition of fat, without addition of salt - White chocolate - Mixed nuts - Breadcrumbs (unspiced) - Fondue - Sauce, mushroom - Tea, spice - Strawberries - Tea, rooibos - Pie, plum, baked, with cake dough - Potatoes au gratin, dauphinois, prepared - Capers - Vegetables - Bread, wholemeal toast - Red radish - Fruit tart - Beans, kidney - Sauerkraut - Mustard - Country fries - Ketchup - Pasta, linguini, parpadelle, Tagliatelle - Chicken, cut into stripes (only meat) - Cookies - Sun-dried tomatoe - Bread, Ticino - Semi-hard cheese - Margarine - Porridge, prepared, with partially skimmed milk - Soya drink (soy milk) - Juice, multifruit - Popcorn salted - Chocolate, filled - Milk chocolate - Bread, fruit - Mix of dried fruits and nuts - Corn - Tête de Moine - Dates - Pistachio - Celery - White radish - Oat milk - Cream cheese - Bread, rye - Witloof chicory - Apple crumble - Goat cheese (soft) - Grapefruit, pomelo - Risotto, with mushrooms, cooked - Blue mould cheese - Biscuit with Butter - Guacamole - Pecan nut - Tofu - Cordon bleu, from pork schnitzel, fried - Paprika chips - Quinoa - Kefir drink - M&M's - Salad, rocket - Bread, spelt - Pizza, with ham, with mushrooms, baked - Fruit coulis - Plums - Beef, minced (only meat) - Pizza, with ham, baked - Pineapple - Soup, tomato - Cheddar - Tea, fruit - Rice, Jasmin - Seeds - Focaccia - Milk - Coleslaw (chopped without sauce) - Pastry, flaky - Curd - Savoury puff pastry stick - Sweet potato - Chicken, leg - Croissant - Sour cream - Ham, turkey - Processed cheese - Fruit compotes - Cheesecake - Pasta, tortelloni, stuffing - Sauce, cocktail - Croissant with chocolate filling - Pumpkin seeds - Artichoke - Champagne - Grissini - Sweets / candies - Brie - Wienerli (Swiss sausage) - Syrup (diluted, ready to drink) - Apple pie - White bread with butter, eggs and milk - Savoury puff pastry - Anchovies - Tuna, in oil, drained - Lemon pie - Meat terrine, paté - Coriander - Falafel (balls) - Berries - Latte macchiato, with caffeine - Faux-mage Cashew, vegan chers - Beans, white - Sugar Melon - Mixed seeds - Hamburger - Hamburger bun - Oil & vinegar salad dressing - Soya Yaourt, yahourt, yogourt ou yoghourt - Chocolate milk, chocolate drink - Celeriac - Chocolate mousse - Cenovis, yeast spread - Thickened cream (> 35%) - Meringue - Lamb, chop - Shrimp / prawn (large) - Beef - Lemon - Croque monsieur - Chives - Chocolate cookies - Birchermüesli, prepared, no sugar added - Fish crunchies (battered) - Muffin - Savoy cabbage, steamed, without addition of salt - Pine nuts - Chorizo - Chia grains - Frying sausage - French pizza from Alsace, baked - Chocolate - Cooked sausage - Grits, polenta, maize flour - Gummi bears, fruit jellies, Jelly babies with fruit essence - Wine, rosé - Coca Cola - Raspberries - Roll with pieces of chocolate - Goat, (average), raw - Lemon Cake - Coconut milk - Rice, wild - Gluten-free bread - Pearl onions - Buckwheat pancake - Bread, 5-grain - Light beer - Sugar, glazing - Tzatziki - Butter, herb - Ham croissant - Corn crisps - Lentils green (du Puy, du Berry) - Cocktail - Rice, whole-grain - Veal sausage - Cervelat - Sorbet - Aperitif, with alcohol, apérol, Spritz - Dips - Corn Flakes - Peas - Tiramisu - Apricots - Cake, marble - Lamb - Lasagne, meat, prepared - Coca Cola Zero - Cake, salted - Dough (puff pastry, shortcrust, bread, pizza dough) - Rice waffels - Sekt - Brioche - Vegetable au gratin, baked - Mango dried - Processed meat, Charcuterie - Mousse - Sauce, sweet & sour - Basil - Butter, spread, puree almond - Pie, apricot, baked, with cake dough - Rusk, wholemeal - Beef, roast - Vanille cream, cooked, Custard, Crème dessert - Pasta in conch form - Nuts - Sauce, carbonara - Fig, dried - Pasta in butterfly form, farfalle - Minced meat - Carrot, steamed, without addition of salt - Ebly - Damson plum - Shoots - Bouquet garni - Coconut - Banana cake - Waffle - Apricot, dried - Sauce, curry - Watermelon, fresh - Sauce, sweet-salted (asian) - Pork, roast - Blackberry - Smoked cooked sausage of pork and beef meat sausag - bean seeds - Italian salad dressing - White asparagus - Pie, rhubarb, baked, with cake dough - Tomato, stewed, without addition of fat, without addition of salt - Cherries - Nectarine
In [10]:
# Getting all categoriy with respective to their total images
no_images_per_category = {}
for n, i in enumerate(train_coco.getCatIds()):
imgIds = train_coco.getImgIds(catIds=i)
label = category_names[n]
no_images_per_category[label] = len(imgIds)
img_info = pd.DataFrame(train_coco.loadImgs(train_coco.getImgIds()))
no_images_per_category = OrderedDict(sorted(no_images_per_category.items(), key=lambda x: -1*x[1]))
# Top 30 categories, based on number of images
i = 0
for k, v in no_images_per_category.items():
print(k, v)
i += 1
if i > 30:
break
Water 2928 Salad, leaf / salad, green 2002 Bread, white 1891 Tomato, raw 1865 Butter 1601 Carrot, raw 1482 Bread, wholemeal 1452 Coffee, with caffeine 1406 Rice 1024 Egg 1015 Mixed vegetables 892 Apple 892 Jam 797 Cucumber 742 Wine, red 728 Banana 654 Cheese 646 Potatoes steamed 644 Bell pepper, red, raw 549 Hard cheese 547 Espresso, with caffeine 547 Tea 516 Bread, whole wheat 504 Mixed salad (chopped without sauce) 498 Avocado 480 White coffee, with caffeine 470 Tomato sauce 466 Wine, white 430 Broccoli 421 Strawberries 412 Pasta, spaghetti 398
Data Visualisations¶
In [15]:
fig = go.Figure([go.Bar(x=list(no_images_per_category.keys())[:50], y=list(no_images_per_category.values())[:50])])
fig.update_layout(
title="No of Image per class",)
fig.show()
fig = go.Figure([go.Bar(x=list(no_images_per_category.keys())[50:200], y=list(no_images_per_category.values())[50:200])])
fig.update_layout(
title="No of Image per class",)
fig.show()
fig = go.Figure([go.Bar(x=list(no_images_per_category.keys())[200:], y=list(no_images_per_category.values())[200:])])
fig.update_layout(
title="No of Image per class",)
fig.show()
In [16]:
pprint(f"Average number of image per class : { sum(list(no_images_per_category.values())) / len(list(no_images_per_category.values())) }")
pprint(f"Highest number of image per class is : { list(no_images_per_category.keys())[0]} of { list(no_images_per_category.values())[0] }")
pprint(f"Lowest number of image per class is : Veggie Burger of { sorted(list(no_images_per_category.values()))[0] }")
'Average number of image per class : 141.359437751004' 'Highest number of image per class is : Water of 2928' 'Lowest number of image per class is : Veggie Burger of 12'
In [17]:
fig = go.Figure(data=[go.Pie(labels=list(no_images_per_category.keys()), values=list(no_images_per_category.values()),
hole=.3, textposition='inside', )], )
fig.update_layout(
title="No of Image per class ( In pie )",)
fig.show()
In [18]:
fig = go.Figure()
fig.add_trace(go.Histogram(x=img_info['height']))
fig.add_trace(go.Histogram(x=img_info['width']))
# Overlay both histograms
fig.update_layout(barmode='stack', title="Histogram of Image width & height",)
fig.show()
Image Visulisation 🖼️¶
In this section we are going to do imaghe visualisations!
In [19]:
print(img_info)
id file_name width height 0 131094 131094.jpg 480 480 1 131097 131097.jpg 391 390 2 131098 131098.jpg 391 390 3 131100 131100.jpg 391 390 4 131101 131101.jpg 391 390 ... ... ... ... ... 39957 131017 131017.jpg 480 480 39958 131021 131021.jpg 464 464 39959 131053 131053.jpg 391 390 39960 131066 131066.jpg 464 464 39961 131071 131071.jpg 464 464 [39962 rows x 4 columns]
In [20]:
len(train_annotations_data['annotations'][n]['segmentation']), len(train_annotations_data['annotations'][n]['bbox'])
Out[20]:
(1, 4)
In [11]:
img_no = 7
annIds = train_coco.getAnnIds(imgIds=train_annotations_data['images'][img_no]['id'])
anns = train_coco.loadAnns(annIds)
# load and render the image
plt.imshow(plt.imread(TRAIN_IMAGE_DIRECTIORY+train_annotations_data['images'][img_no]['file_name']))
plt.axis('off')
# Render annotations on top of the image
train_coco.showAnns(anns)
In [22]:
w, h = 15, 15 # Setting width and height of every image
rows, cols = 5, 5 # Setting the number of image rows & cols
fig = plt.figure(figsize=(15, 15)) # Making the figure with size
plt.title("Images")
plt.axis('off')
# Going thought every cell in rows and cols
for i in range(1, cols * rows+1):
annIds = train_coco.getAnnIds(imgIds=img_info['id'][i])
anns = train_coco.loadAnns(annIds)
fig.add_subplot(rows, cols, i)
# Show the image
img = plt.imread(TRAIN_IMAGE_DIRECTIORY+img_info['file_name'][i])
for i in anns:
[x,y,w,h] = i['bbox']
cv2.rectangle(img, (int(x), int(y)), (int(x+h), int(y+w)), (255,0,0), 2)
plt.imshow(img)
# Render annotations on top of the image
train_coco.showAnns(anns)
# Setting the axis off
plt.axis("off")
# Showing the figure
plt.show()
Creating our Dataset 🔨¶
In this section we are goind to fix out dataset first ( because there is some issues with dataset ( size mismatch ) & creating our dataset to put into the model
Fixing the Data¶
In [23]:
np.array(train_annotations_data['annotations'][n]['segmentation']).shape , np.array(train_annotations_data['annotations'][n]['bbox']).shape
Out[23]:
((1, 38), (4,))
In [12]:
# Function for taking a annotation & directiory of images and returning new annoation json with fixed image size info
def fix_data(annotations, directiory):
for n, i in enumerate(tqdm((annotations['images']))):
img = cv2.imread(directiory+i["file_name"])
if img.shape[0] != i['height']:
annotations['images'][n]['height'] = img.shape[0]
print(i["file_name"])
print(annotations['images'][n], img.shape)
if img.shape[1] != i['width']:
annotations['images'][n]['width'] = img.shape[1]
print(i["file_name"])
print(annotations['images'][n], img.shape)
return annotations
train_annotations_data = fix_data(train_annotations_data, TRAIN_IMAGE_DIRECTIORY)
with open('data/train/new_ann.json', 'w') as f:
json.dump(train_annotations_data, f)
008618.jpg
{'id': 8618, 'file_name': '008618.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008618.jpg
{'id': 8618, 'file_name': '008618.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008619.jpg
{'id': 8619, 'file_name': '008619.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008619.jpg
{'id': 8619, 'file_name': '008619.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008620.jpg
{'id': 8620, 'file_name': '008620.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008620.jpg
{'id': 8620, 'file_name': '008620.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008621.jpg
{'id': 8621, 'file_name': '008621.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008621.jpg
{'id': 8621, 'file_name': '008621.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008626.jpg
{'id': 8626, 'file_name': '008626.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008626.jpg
{'id': 8626, 'file_name': '008626.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008627.jpg
{'id': 8627, 'file_name': '008627.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008627.jpg
{'id': 8627, 'file_name': '008627.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008628.jpg
{'id': 8628, 'file_name': '008628.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008628.jpg
{'id': 8628, 'file_name': '008628.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008817.jpg
{'id': 8817, 'file_name': '008817.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008817.jpg
{'id': 8817, 'file_name': '008817.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008864.jpg
{'id': 8864, 'file_name': '008864.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008864.jpg
{'id': 8864, 'file_name': '008864.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008869.jpg
{'id': 8869, 'file_name': '008869.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008869.jpg
{'id': 8869, 'file_name': '008869.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008919.jpg
{'id': 8919, 'file_name': '008919.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008919.jpg
{'id': 8919, 'file_name': '008919.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008934.jpg
{'id': 8934, 'file_name': '008934.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008934.jpg
{'id': 8934, 'file_name': '008934.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
011967.jpg
{'id': 11967, 'file_name': '011967.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
011967.jpg
{'id': 11967, 'file_name': '011967.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
012045.jpg
{'id': 12045, 'file_name': '012045.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
012045.jpg
{'id': 12045, 'file_name': '012045.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
013989.jpg
{'id': 13989, 'file_name': '013989.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
013989.jpg
{'id': 13989, 'file_name': '013989.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
017312.jpg
{'id': 17312, 'file_name': '017312.jpg', 'width': 3264, 'height': 3264} (3264, 2448, 3)
017312.jpg
{'id': 17312, 'file_name': '017312.jpg', 'width': 2448, 'height': 3264} (3264, 2448, 3)
021923.jpg
{'id': 21923, 'file_name': '021923.jpg', 'width': 4128, 'height': 4128} (4128, 2322, 3)
021923.jpg
{'id': 21923, 'file_name': '021923.jpg', 'width': 2322, 'height': 4128} (4128, 2322, 3)
023295.jpg
{'id': 23295, 'file_name': '023295.jpg', 'width': 4128, 'height': 4128} (4128, 2322, 3)
023295.jpg
{'id': 23295, 'file_name': '023295.jpg', 'width': 2322, 'height': 4128} (4128, 2322, 3)
023296.jpg
{'id': 23296, 'file_name': '023296.jpg', 'width': 4128, 'height': 4128} (4128, 2322, 3)
023296.jpg
{'id': 23296, 'file_name': '023296.jpg', 'width': 2322, 'height': 4128} (4128, 2322, 3)
023975.jpg
{'id': 23975, 'file_name': '023975.jpg', 'width': 4128, 'height': 4128} (4128, 2322, 3)
023975.jpg
{'id': 23975, 'file_name': '023975.jpg', 'width': 2322, 'height': 4128} (4128, 2322, 3)
023976.jpg
{'id': 23976, 'file_name': '023976.jpg', 'width': 4128, 'height': 4128} (4128, 2322, 3)
023976.jpg
{'id': 23976, 'file_name': '023976.jpg', 'width': 2322, 'height': 4128} (4128, 2322, 3)
049396.jpg
{'id': 49396, 'file_name': '049396.jpg', 'width': 4608, 'height': 4608} (4608, 3456, 3)
049396.jpg
{'id': 49396, 'file_name': '049396.jpg', 'width': 3456, 'height': 4608} (4608, 3456, 3)
050167.jpg
{'id': 50167, 'file_name': '050167.jpg', 'width': 4032, 'height': 4032} (4032, 1960, 3)
050167.jpg
{'id': 50167, 'file_name': '050167.jpg', 'width': 1960, 'height': 4032} (4032, 1960, 3)
053873.jpg
{'id': 53873, 'file_name': '053873.jpg', 'width': 4608, 'height': 4608} (4608, 3456, 3)
053873.jpg
{'id': 53873, 'file_name': '053873.jpg', 'width': 3456, 'height': 4608} (4608, 3456, 3)
053875.jpg
{'id': 53875, 'file_name': '053875.jpg', 'width': 4608, 'height': 4608} (4608, 3456, 3)
053875.jpg
{'id': 53875, 'file_name': '053875.jpg', 'width': 3456, 'height': 4608} (4608, 3456, 3)
053879.jpg
{'id': 53879, 'file_name': '053879.jpg', 'width': 4608, 'height': 4608} (4608, 3456, 3)
053879.jpg
{'id': 53879, 'file_name': '053879.jpg', 'width': 3456, 'height': 4608} (4608, 3456, 3)
056091.jpg
{'id': 56091, 'file_name': '056091.jpg', 'width': 4608, 'height': 4608} (4608, 3456, 3)
056091.jpg
{'id': 56091, 'file_name': '056091.jpg', 'width': 3456, 'height': 4608} (4608, 3456, 3)
In [13]:
def fix_data_val(annotations, directiory):
for n, i in enumerate(tqdm((annotations['images']))):
img = cv2.imread(directiory+i["file_name"])
if img.shape[0] != i['height']:
print(n)
annotations['images'][n]['height'] = img.shape[0]
print(i["file_name"])
print(annotations['images'][n], img.shape)
if img.shape[1] != i['width']:
annotations['images'][n]['width'] = img.shape[1]
print(i["file_name"])
print(annotations['images'][n], img.shape)
return annotations
val_annotations_data = fix_data_val(val_annotations_data, VAL_IMAGE_DIRECTIORY)
with open('data/val/new_ann.json', 'w') as f:
json.dump(val_annotations_data, f)
In [14]:
dict_addres = val_annotations_data['images'][748]
plt.imread(VAL_IMAGE_DIRECTIORY+dict_addres['file_name']).shape[:2], (dict_addres['height'], dict_addres['width'])
Out[14]:
((918, 919), (918, 919))
Loading Dataset¶
In [15]:
train_annotations_path = 'data/train/new_ann.json'
train_images_path = 'data/train/images'
val_annotations_path = 'data/val/new_ann.json'
val_images_path = 'data/val/images'
In [16]:
register_coco_instances("training_dataset", {},train_annotations_path, train_images_path)
register_coco_instances("validation_dataset", {},val_annotations_path, VAL_IMAGE_DIRECTIORY)
Creating our Model 🏭¶
We are going to make an Faster R-CNN Model using Detectron2, and setting up hyperpamaters to train our model
Creating R-CNN Model¶
In [17]:
cfg = get_cfg()
# Check the model zoo and use any of the models ( from detectron2 github repo)
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.DATASETS.TRAIN = ("training_dataset",)
cfg.DATASETS.TEST = ()
cfg.DATALOADER.NUM_WORKERS = 2
# Loading pre trained weights
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
Setting up hyperparameters¶
In [18]:
# No. of Batchs
cfg.SOLVER.IMS_PER_BATCH = 10
# Learning Rate:
cfg.SOLVER.BASE_LR = 0.0025
# No of Interations
cfg.SOLVER.MAX_ITER = 50001
# Images per batch (Batch Size)
cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 128
# No of Categories(Classes) present
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 498
cfg.OUTPUT_DIR = "logs/"
os.makedirs(cfg.OUTPUT_DIR, exist_ok=True)
In [25]:
trainer = DefaultTrainer(cfg)
trainer.resume_or_load(resume=False)
[12/21 07:39:41 d2.engine.defaults]: Model: GeneralizedRCNN( (backbone): FPN( (fpn_lateral2): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1)) (fpn_output2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (fpn_lateral3): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1)) (fpn_output3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (fpn_lateral4): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1)) (fpn_output4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (fpn_lateral5): Conv2d(2048, 256, kernel_size=(1, 1), stride=(1, 1)) (fpn_output5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (top_block): LastLevelMaxPool() (bottom_up): ResNet( (stem): BasicStem( (conv1): Conv2d( 3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05) ) ) (res2): Sequential( (0): BottleneckBlock( (shortcut): Conv2d( 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv1): Conv2d( 64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05) ) (conv2): Conv2d( 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05) ) (conv3): Conv2d( 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) ) (1): BottleneckBlock( (conv1): Conv2d( 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05) ) (conv2): Conv2d( 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05) ) (conv3): Conv2d( 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) ) (2): BottleneckBlock( (conv1): Conv2d( 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05) ) (conv2): Conv2d( 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05) ) (conv3): Conv2d( 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) ) ) (res3): Sequential( (0): BottleneckBlock( (shortcut): Conv2d( 256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) (conv1): Conv2d( 256, 128, kernel_size=(1, 1), stride=(2, 2), bias=False (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05) ) (conv2): Conv2d( 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05) ) (conv3): Conv2d( 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) ) (1): BottleneckBlock( (conv1): Conv2d( 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05) ) (conv2): Conv2d( 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05) ) (conv3): Conv2d( 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) ) (2): BottleneckBlock( (conv1): Conv2d( 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05) ) (conv2): Conv2d( 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05) ) (conv3): Conv2d( 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) ) (3): BottleneckBlock( (conv1): Conv2d( 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05) ) (conv2): Conv2d( 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05) ) (conv3): Conv2d( 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) ) ) (res4): Sequential( (0): BottleneckBlock( (shortcut): Conv2d( 512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05) ) (conv1): Conv2d( 512, 256, kernel_size=(1, 1), stride=(2, 2), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv2): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv3): Conv2d( 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05) ) ) (1): BottleneckBlock( (conv1): Conv2d( 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv2): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv3): Conv2d( 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05) ) ) (2): BottleneckBlock( (conv1): Conv2d( 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv2): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv3): Conv2d( 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05) ) ) (3): BottleneckBlock( (conv1): Conv2d( 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv2): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv3): Conv2d( 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05) ) ) (4): BottleneckBlock( (conv1): Conv2d( 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv2): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv3): Conv2d( 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05) ) ) (5): BottleneckBlock( (conv1): Conv2d( 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv2): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05) ) (conv3): Conv2d( 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05) ) ) ) (res5): Sequential( (0): BottleneckBlock( (shortcut): Conv2d( 1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05) ) (conv1): Conv2d( 1024, 512, kernel_size=(1, 1), stride=(2, 2), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) (conv2): Conv2d( 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) (conv3): Conv2d( 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05) ) ) (1): BottleneckBlock( (conv1): Conv2d( 2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) (conv2): Conv2d( 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) (conv3): Conv2d( 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05) ) ) (2): BottleneckBlock( (conv1): Conv2d( 2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) (conv2): Conv2d( 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05) ) (conv3): Conv2d( 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05) ) ) ) ) ) (proposal_generator): RPN( (rpn_head): StandardRPNHead( (conv): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1) (activation): ReLU() ) (objectness_logits): Conv2d(256, 3, kernel_size=(1, 1), stride=(1, 1)) (anchor_deltas): Conv2d(256, 12, kernel_size=(1, 1), stride=(1, 1)) ) (anchor_generator): DefaultAnchorGenerator( (cell_anchors): BufferList() ) ) (roi_heads): StandardROIHeads( (box_pooler): ROIPooler( (level_poolers): ModuleList( (0): ROIAlign(output_size=(7, 7), spatial_scale=0.25, sampling_ratio=0, aligned=True) (1): ROIAlign(output_size=(7, 7), spatial_scale=0.125, sampling_ratio=0, aligned=True) (2): ROIAlign(output_size=(7, 7), spatial_scale=0.0625, sampling_ratio=0, aligned=True) (3): ROIAlign(output_size=(7, 7), spatial_scale=0.03125, sampling_ratio=0, aligned=True) ) ) (box_head): FastRCNNConvFCHead( (flatten): Flatten(start_dim=1, end_dim=-1) (fc1): Linear(in_features=12544, out_features=1024, bias=True) (fc_relu1): ReLU() (fc2): Linear(in_features=1024, out_features=1024, bias=True) (fc_relu2): ReLU() ) (box_predictor): FastRCNNOutputLayers( (cls_score): Linear(in_features=1024, out_features=499, bias=True) (bbox_pred): Linear(in_features=1024, out_features=1992, bias=True) ) (mask_pooler): ROIPooler( (level_poolers): ModuleList( (0): ROIAlign(output_size=(14, 14), spatial_scale=0.25, sampling_ratio=0, aligned=True) (1): ROIAlign(output_size=(14, 14), spatial_scale=0.125, sampling_ratio=0, aligned=True) (2): ROIAlign(output_size=(14, 14), spatial_scale=0.0625, sampling_ratio=0, aligned=True) (3): ROIAlign(output_size=(14, 14), spatial_scale=0.03125, sampling_ratio=0, aligned=True) ) ) (mask_head): MaskRCNNConvUpsampleHead( (mask_fcn1): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1) (activation): ReLU() ) (mask_fcn2): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1) (activation): ReLU() ) (mask_fcn3): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1) (activation): ReLU() ) (mask_fcn4): Conv2d( 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1) (activation): ReLU() ) (deconv): ConvTranspose2d(256, 256, kernel_size=(2, 2), stride=(2, 2)) (deconv_relu): ReLU() (predictor): Conv2d(256, 498, kernel_size=(1, 1), stride=(1, 1)) ) ) ) [12/21 07:39:45 d2.data.datasets.coco]: Loading data/train/new_ann.json takes 4.09 seconds. WARNING [12/21 07:39:45 d2.data.datasets.coco]: Category ids in annotations are not in [1, #categories]! We'll apply a mapping for you. [12/21 07:39:46 d2.data.datasets.coco]: Loaded 39962 images in COCO format from data/train/new_ann.json [12/21 07:39:47 d2.data.build]: Removed 0 images with no usable annotations. 39962 images left. [12/21 07:39:50 d2.data.dataset_mapper]: [DatasetMapper] Augmentations used in training: [ResizeShortestEdge(short_edge_length=(640, 672, 704, 736, 768, 800), max_size=1333, sample_style='choice'), RandomFlip()] [12/21 07:39:50 d2.data.build]: Using training sampler TrainingSampler [12/21 07:39:50 d2.data.common]: Serializing 39962 elements to byte tensors and concatenating them all ... [12/21 07:39:51 d2.data.common]: Serialized dataset takes 117.68 MiB WARNING [12/21 07:39:51 d2.solver.build]: SOLVER.STEPS contains values larger than SOLVER.MAX_ITER. These values will be ignored.
Skip loading parameter 'roi_heads.box_predictor.cls_score.weight' to the model due to incompatible shapes: (81, 1024) in the checkpoint but (499, 1024) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.cls_score.bias' to the model due to incompatible shapes: (81,) in the checkpoint but (499,) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (320, 1024) in the checkpoint but (1992, 1024) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (320,) in the checkpoint but (1992,) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.mask_head.predictor.weight' to the model due to incompatible shapes: (80, 256, 1, 1) in the checkpoint but (498, 256, 1, 1) in the model! You might want to double check if this is expected. Skip loading parameter 'roi_heads.mask_head.predictor.bias' to the model due to incompatible shapes: (80,) in the checkpoint but (498,) in the model! You might want to double check if this is expected. Some model parameters or buffers are not found in the checkpoint: roi_heads.box_predictor.bbox_pred.{bias, weight} roi_heads.box_predictor.cls_score.{bias, weight} roi_heads.mask_head.predictor.{bias, weight}
Training the Model 🚂¶
Setting up Tensorboard & finally training our model!
Setting up Tensorboard¶
In [20]:
%load_ext tensorboard
In [21]:
#%tensorboard --logdir logs
Start Training!¶
In [ ]:
trainer.train()
Evaluating the model 🧪¶
After training is done, we will evaluate our model to see how it's performing in the unseen images!
In [ ]:
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.01
cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")
evaluator = COCOEvaluator("validation_dataset", cfg, False, output_dir=cfg.OUTPUT_DIR)
val_loader = build_detection_test_loader(cfg, "validation_dataset")
valResults = inference_on_dataset(trainer.model, val_loader, evaluator)
WARNING [12/20 22:27:53 d2.data.datasets.coco]: Category ids in annotations are not in [1, #categories]! We'll apply a mapping for you. [12/20 22:27:53 d2.data.datasets.coco]: Loaded 1000 images in COCO format from data/val/new_ann.json [12/20 22:27:53 d2.data.build]: Distribution of instances among all 498 categories: | category | #instances | category | #instances | category | #instances | |:-------------:|:-------------|:-------------:|:-------------|:-------------:|:-------------| | beetroot-st.. | 5 | carrot-stea.. | 2 | mushroom-av.. | 2 | | savoy-cabba.. | 0 | zucchini-st.. | 0 | vanille-cre.. | 0 | | shrimp-boiled | 3 | spinach-ste.. | 2 | spinach-raw | 2 | | green-bean-.. | 1 | sweet-corn-.. | 2 | watermelon-.. | 5 | | pizza-with-.. | 1 | pizza-with-.. | 1 | pizza-with-.. | 2 | | bell-pepper.. | 1 | applesauce-.. | 1 | quiche-with.. | 2 | | quiche-with.. | 1 | risotto-wit.. | 0 | potato-sala.. | 1 | | wienerli-sw.. | 5 | curds-natur.. | 2 | tuna-in-oil.. | 0 | | tomato-stew.. | 2 | sweet-potato | 3 | country-fries | 3 | | potato-gnoc.. | 1 | potatoes-st.. | 8 | chips-frenc.. | 9 | | rosti | 2 | vegetable-m.. | 1 | coleslaw-ch.. | 0 | | vegetables | 7 | mixed-veget.. | 13 | ratatouille | 3 | | mixed-salad.. | 12 | leaf-spinach | 8 | witloof-chi.. | 2 | | salad-rocket | 4 | salad-leaf-.. | 62 | salad-lambs.. | 6 | | artichoke | 0 | eggplant | 7 | avocado | 9 | | french-beans | 8 | cucumber-pi.. | 2 | cucumber | 26 | | capers | 1 | pumpkin | 3 | bell-pepper.. | 19 | | tomato-raw | 60 | zucchini | 9 | red-radish | 8 | | beetroot-raw | 3 | white-radish | 0 | carrot-raw | 27 | | celeriac | 0 | cauliflower | 7 | broccoli | 10 | | chinese-cab.. | 4 | kolhrabi | 3 | romanesco | 2 | | brussel-spr.. | 2 | red-cabbage | 1 | sauerkraut | 2 | | white-cabbage | 0 | mushroom | 6 | mushrooms | 4 | | peas | 2 | corn | 3 | spring-onio.. | 2 | | garlic | 1 | leek | 5 | pearl-onions | 0 | | onion | 13 | fennel | 3 | swiss-chard | 3 | | mungbean-sp.. | 0 | green-aspar.. | 2 | white-aspar.. | 0 | | shoots | 1 | alfa-sprouts | 0 | celery | 2 | | bean-seeds | 0 | beans-kidney | 3 | beans-white | 1 | | chickpeas | 10 | lentils | 2 | pineapple | 3 | | apple | 18 | pomegranate | 3 | apricots | 2 | | banana | 7 | berries | 2 | pear | 5 | | blackberry | 0 | dates | 3 | strawberries | 12 | | figs | 3 | fruit-salad | 1 | grapefruit-.. | 3 | | blueberries | 13 | raspberries | 7 | kaki | 2 | | cherries | 3 | kiwi | 6 | mandarine | 5 | | mango | 1 | sugar-melon | 4 | nectarine | 3 | | orange | 9 | peach | 3 | plums | 7 | | grapes | 10 | dried-raisins | 1 | lemon | 2 | | damson-plum | 1 | peanut-butter | 3 | chestnuts | 4 | | seeds | 1 | pumpkin-seeds | 1 | pine-nuts | 2 | | sunflower-s.. | 2 | mixed-seeds | 0 | almonds | 4 | | nuts | 0 | walnut | 4 | cashew-nut | 3 | | peanut | 2 | hazelnut | 4 | coconut | 1 | | brazil-nut | 2 | pecan-nut | 1 | mixed-nuts | 7 | | pistachio | 2 | linseeds | 1 | sesame-seeds | 2 | | green-olives | 2 | black-olives | 8 | milk | 6 | | kefir-drink | 1 | coconut-milk | 3 | soya-drink-.. | 1 | | soya-yaourt.. | 3 | cottage-che.. | 4 | curd | 0 | | blue-mould-.. | 1 | brie | 1 | cheddar | 1 | | emmental-ch.. | 4 | feta | 8 | fondue | 5 | | fresh-cheese | 6 | gruyere | 8 | semi-hard-c.. | 2 | | halloumi | 3 | hard-cheese | 12 | cheese | 17 | | mozzarella | 8 | parmesan | 11 | philadelphia | 4 | | cheese-for-.. | 1 | cream-cheese | 1 | processed-c.. | 1 | | tete-de-moine | 3 | tomme | 1 | soft-cheese | 6 | | mousse | 0 | panna-cotta | 0 | tiramisu | 1 | | cream | 5 | sour-cream | 3 | thickened-c.. | 0 | | dairy-ice-c.. | 4 | sorbet | 1 | flakes-oat | 1 | | rice-noodle.. | 1 | bulgur | 2 | couscous | 2 | | ebly | 1 | grits-polen.. | 5 | quinoa | 3 | | rice | 24 | rice-basmati | 3 | rice-jasmin | 0 | | rice-whole-.. | 2 | rice-wild | 1 | spaetzle | 2 | | pasta | 13 | pasta-hornli | 1 | pasta-in-bu.. | 1 | | pasta-lingu.. | 4 | pasta-in-co.. | 1 | pasta-noodles | 2 | | pasta-penne | 6 | pasta-ravio.. | 2 | pasta-spagh.. | 5 | | pasta-twist | 3 | pasta-torte.. | 0 | pasta-whole.. | 1 | | bagel-witho.. | 1 | bread-frenc.. | 2 | bread | 12 | | bread-5-grain | 0 | bread-spelt | 1 | bread-fruit | 2 | | bread-half-.. | 1 | bread-grain | 8 | bread-nut | 0 | | bread-olive | 2 | bread-pita | 2 | bread-rye | 2 | | bread-whole.. | 23 | bread-sourd.. | 3 | bread-black | 2 | | bread-ticino | 1 | bread-toast | 10 | bread-whole.. | 4 | | bread-whole.. | 36 | bread-white | 39 | brioche | 2 | | roll-of-hal.. | 2 | hamburger-bun | 2 | roll-with-p.. | 2 | | white-bread.. | 6 | focaccia | 2 | croissant | 7 | | croissant-w.. | 1 | lye-pretzel.. | 3 | braided-whi.. | 13 | | crisp-bread.. | 0 | breadcrumbs.. | 1 | rice-waffels | 3 | | grissini | 1 | rusk-wholem.. | 1 | corn-flakes | 0 | | crunch-muesli | 3 | muesli | 4 | dough-puff-.. | 0 | | pastry-flaky | 0 | meat | 3 | minced-meat | 0 | | beef | 2 | beef-roast | 1 | beef-sirloi.. | 2 | | beef-filet | 0 | beef-minced.. | 1 | beef-cut-in.. | 3 | | pork | 1 | pork-roast | 1 | pork-chop | 0 | | pork-escalope | 0 | lamb | 0 | lamb-chop | 1 | | chicken | 6 | chicken-bre.. | 3 | chicken-wing | 0 | | chicken-cut.. | 2 | chicken-leg | 1 | frying-saus.. | 1 | | cervelat | 1 | chicken-nug.. | 2 | chorizo | 1 | | meatloaf | 2 | hamburger | 4 | dried-meat | 3 | | veal-sausage | 1 | processed-m.. | 0 | salami | 6 | | cooked-saus.. | 0 | ham | 5 | ham-cooked | 5 | | ham-raw | 6 | ham-turkey | 1 | smoked-cook.. | 0 | | bacon | 0 | bacon-frying | 3 | bacon-cooking | 3 | | bacon-raw | 1 | meat-terrin.. | 2 | sausage | 2 | | veggie-burger | 2 | tofu | 6 | fish | 1 | | cod | 1 | salmon | 7 | anchovies | 2 | | tuna | 0 | shrimp-praw.. | 0 | shrimp-praw.. | 3 | | fish-crunch.. | 0 | fish-finger.. | 3 | egg | 21 | | oil | 1 | butter | 41 | butter-herb | 1 | | margarine | 1 | praline | 1 | jam | 21 | | honey | 5 | sugar-glazing | 0 | maple-syrup.. | 1 | | dark-chocol.. | 8 | milk-chocol.. | 4 | white-choco.. | 0 | | chocolate | 2 | chocolate-f.. | 0 | cocoa-powder | 0 | | hazelnut-ch.. | 5 | m-m-s | 4 | chocolate-e.. | 1 | | sweets-cand.. | 1 | gummi-bears.. | 0 | apple-pie | 1 | | brownie | 0 | cake-oblong | 3 | lemon-cake | 0 | | crepe-plain | 6 | fruit-tart | 1 | cake-marble | 1 | | cake-chocol.. | 2 | muffin | 4 | omelette-pl.. | 1 | | carrot-cake | 3 | black-fores.. | 3 | tart | 1 | | waffle | 0 | croissant-w.. | 1 | cookies | 2 | | biscuits | 3 | macaroon | 2 | meringue | 3 | | biscuit-wit.. | 1 | chocolate-c.. | 2 | juice-apple | 4 | | juice-multi.. | 1 | juice-orange | 0 | smoothie | 0 | | coca-cola | 0 | coca-cola-z.. | 4 | ice-tea | 0 | | syrup-dilut.. | 7 | tea | 7 | cappuccino | 5 | | espresso-wi.. | 14 | coffee-with.. | 27 | coffee-deca.. | 2 | | latte-macch.. | 1 | white-coffe.. | 17 | ristretto-w.. | 5 | | tea-green | 2 | tea-black | 0 | tea-verveine | 0 | | tea-fruit | 0 | tea-spice | 0 | tea-ginger | 0 | | herbal-tea | 4 | tea-pepperm.. | 2 | tea-rooibos | 1 | | ice-cubes | 0 | water | 77 | water-mineral | 5 | | aperitif-wi.. | 0 | cocktail | 1 | champagne | 1 | | prosecco | 1 | sekt | 1 | wine-rose | 3 | | wine-red | 8 | wine-white | 16 | beer | 5 | | light-beer | 2 | sauce-savoury | 1 | sauce-roast | 1 | | sauce-carbo.. | 0 | sauce-cockt.. | 1 | sauce-curry | 1 | | sauce-pesto | 2 | sauce-mushr.. | 2 | sauce-cream | 7 | | sauce-sweet.. | 1 | ketchup | 7 | bolognaise-.. | 2 | | tomato-sauce | 4 | dips | 0 | salad-dress.. | 3 | | balsamic-sa.. | 4 | french-sala.. | 9 | italian-sal.. | 2 | | oil-vinegar.. | 1 | guacamole | 3 | mayonnaise | 2 | | tartar-sauce | 1 | tzatziki | 3 | basil | 2 | | coriander | 1 | bouquet-garni | 2 | parsley | 1 | | chives | 3 | cenovis-yea.. | 0 | sauce-soya | 1 | | mustard | 5 | mustard-dijon | 0 | balsamic-vi.. | 0 | | soup-vegeta.. | 3 | soup-cream-.. | 2 | soup-potato | 1 | | soup-pumpkin | 3 | soup-miso | 1 | soup-tomato | 2 | | bouillon | 2 | bouillon-ve.. | 1 | falafel-balls | 3 | | savoury-puf.. | 0 | savoury-puf.. | 2 | corn-crisps | 0 | | crackers | 1 | croutons | 1 | crisps | 3 | | popcorn-sal.. | 3 | croque-mons.. | 6 | spring-roll.. | 2 | | ham-croissant | 0 | salt-cake-v.. | 0 | hamburger-b.. | 4 | | cordon-bleu.. | 2 | lasagne-mea.. | 3 | mashed-pota.. | 4 | | pizza-margh.. | 4 | sandwich-ha.. | 0 | sushi | 5 | | kebab-in-pi.. | 2 | pancakes | 0 | hummus | 13 | | greek-salad | 1 | dumplings | 0 | apricot-dried | 1 | | chocolate-m.. | 0 | cheesecake | 1 | caprese-sal.. | 1 | | chili-con-c.. | 1 | taboule-pre.. | 0 | perch-fille.. | 0 | | risotto-wit.. | 2 | salmon-smoked | 11 | pie-apricot.. | 2 | | eggplant-ca.. | 1 | apple-crumble | 0 | egg-scrambl.. | 6 | | oat-milk | 1 | lemon-pie | 0 | glucose-dri.. | 5 | | goat-averag.. | 0 | pie-rhubarb.. | 0 | chicken-cur.. | 0 | | pie-plum-ba.. | 2 | potatoes-au.. | 3 | buckwheat-g.. | 0 | | birchermues.. | 7 | fajita-brea.. | 2 | mango-dried | 2 | | lentils-gre.. | 5 | naan-indien.. | 0 | butter-spre.. | 2 | | chocolate-m.. | 0 | water-with-.. | 5 | sun-dried-t.. | 1 | | gluten-free.. | 1 | fruit-coulis | 1 | greek-yaour.. | 3 | | cake-salted | 2 | soup-of-len.. | 0 | fig-dried | 5 | | turnover-wi.. | 1 | lasagne-veg.. | 2 | sauce-sweet.. | 1 | | french-pizz.. | 3 | fruit-compo.. | 0 | vegetable-a.. | 2 | | porridge-pr.. | 0 | curry-veget.. | 0 | bread-meat-.. | 2 | | tartar-meat | 1 | chia-grains | 0 | faux-mage-c.. | 4 | | milk-chocol.. | 4 | yaourt-yaho.. | 8 | paprika-chips | 0 | | banana-cake | 3 | cream-spinach | 0 | cantonese-f.. | 1 | | goat-cheese.. | 1 | buckwheat-p.. | 1 | meat-balls | 2 | | high-protei.. | 1 | mix-of-drie.. | 1 | baked-potato | 7 | | | | | | | | | total | 1830 | | | | | [12/20 22:27:53 d2.data.common]: Serializing 1000 elements to byte tensors and concatenating them all ... [12/20 22:27:53 d2.data.common]: Serialized dataset takes 1.58 MiB [12/20 22:27:53 d2.evaluation.evaluator]: Start inference on 1000 images [12/20 22:27:56 d2.evaluation.evaluator]: Inference done 11/1000. 0.2827 s / img. ETA=0:04:48 [12/20 22:28:01 d2.evaluation.evaluator]: Inference done 28/1000. 0.2851 s / img. ETA=0:04:45 [12/20 22:28:06 d2.evaluation.evaluator]: Inference done 45/1000. 0.2858 s / img. ETA=0:04:40 [12/20 22:28:11 d2.evaluation.evaluator]: Inference done 62/1000. 0.2868 s / img. ETA=0:04:37 [12/20 22:28:16 d2.evaluation.evaluator]: Inference done 79/1000. 0.2871 s / img. ETA=0:04:32 [12/20 22:28:21 d2.evaluation.evaluator]: Inference done 96/1000. 0.2869 s / img. ETA=0:04:27 [12/20 22:28:26 d2.evaluation.evaluator]: Inference done 113/1000. 0.2875 s / img. ETA=0:04:23 [12/20 22:28:32 d2.evaluation.evaluator]: Inference done 130/1000. 0.2877 s / img. ETA=0:04:18 [12/20 22:28:37 d2.evaluation.evaluator]: Inference done 147/1000. 0.2880 s / img. ETA=0:04:13 [12/20 22:28:42 d2.evaluation.evaluator]: Inference done 164/1000. 0.2883 s / img. ETA=0:04:08 [12/20 22:28:47 d2.evaluation.evaluator]: Inference done 181/1000. 0.2882 s / img. ETA=0:04:03 [12/20 22:28:52 d2.evaluation.evaluator]: Inference done 198/1000. 0.2883 s / img. ETA=0:03:58 [12/20 22:28:57 d2.evaluation.evaluator]: Inference done 215/1000. 0.2884 s / img. ETA=0:03:53 [12/20 22:29:02 d2.evaluation.evaluator]: Inference done 232/1000. 0.2888 s / img. ETA=0:03:49 [12/20 22:29:07 d2.evaluation.evaluator]: Inference done 249/1000. 0.2891 s / img. ETA=0:03:44 [12/20 22:29:12 d2.evaluation.evaluator]: Inference done 266/1000. 0.2891 s / img. ETA=0:03:39 [12/20 22:29:17 d2.evaluation.evaluator]: Inference done 283/1000. 0.2891 s / img. ETA=0:03:34 [12/20 22:29:23 d2.evaluation.evaluator]: Inference done 300/1000. 0.2890 s / img. ETA=0:03:29 [12/20 22:29:28 d2.evaluation.evaluator]: Inference done 317/1000. 0.2890 s / img. ETA=0:03:24 [12/20 22:29:33 d2.evaluation.evaluator]: Inference done 334/1000. 0.2890 s / img. ETA=0:03:18 [12/20 22:29:38 d2.evaluation.evaluator]: Inference done 351/1000. 0.2893 s / img. ETA=0:03:13 [12/20 22:29:43 d2.evaluation.evaluator]: Inference done 368/1000. 0.2898 s / img. ETA=0:03:09 [12/20 22:29:48 d2.evaluation.evaluator]: Inference done 385/1000. 0.2897 s / img. ETA=0:03:04 [12/20 22:29:53 d2.evaluation.evaluator]: Inference done 401/1000. 0.2906 s / img. ETA=0:02:59 [12/20 22:29:58 d2.evaluation.evaluator]: Inference done 418/1000. 0.2904 s / img. ETA=0:02:54 [12/20 22:30:04 d2.evaluation.evaluator]: Inference done 435/1000. 0.2905 s / img. ETA=0:02:49 [12/20 22:30:09 d2.evaluation.evaluator]: Inference done 451/1000. 0.2909 s / img. ETA=0:02:45 [12/20 22:30:14 d2.evaluation.evaluator]: Inference done 468/1000. 0.2910 s / img. ETA=0:02:40 [12/20 22:30:19 d2.evaluation.evaluator]: Inference done 485/1000. 0.2909 s / img. ETA=0:02:35 [12/20 22:30:24 d2.evaluation.evaluator]: Inference done 502/1000. 0.2909 s / img. ETA=0:02:30 [12/20 22:30:29 d2.evaluation.evaluator]: Inference done 519/1000. 0.2909 s / img. ETA=0:02:25 [12/20 22:30:34 d2.evaluation.evaluator]: Inference done 536/1000. 0.2907 s / img. ETA=0:02:19 [12/20 22:30:40 d2.evaluation.evaluator]: Inference done 553/1000. 0.2905 s / img. ETA=0:02:14 [12/20 22:30:45 d2.evaluation.evaluator]: Inference done 570/1000. 0.2905 s / img. ETA=0:02:09 [12/20 22:30:50 d2.evaluation.evaluator]: Inference done 587/1000. 0.2906 s / img. ETA=0:02:04 [12/20 22:30:55 d2.evaluation.evaluator]: Inference done 604/1000. 0.2906 s / img. ETA=0:01:59 [12/20 22:31:00 d2.evaluation.evaluator]: Inference done 621/1000. 0.2906 s / img. ETA=0:01:54 [12/20 22:31:05 d2.evaluation.evaluator]: Inference done 638/1000. 0.2906 s / img. ETA=0:01:49 [12/20 22:31:10 d2.evaluation.evaluator]: Inference done 655/1000. 0.2905 s / img. ETA=0:01:44 [12/20 22:31:16 d2.evaluation.evaluator]: Inference done 672/1000. 0.2904 s / img. ETA=0:01:38 [12/20 22:31:21 d2.evaluation.evaluator]: Inference done 689/1000. 0.2904 s / img. ETA=0:01:33 [12/20 22:31:26 d2.evaluation.evaluator]: Inference done 706/1000. 0.2903 s / img. ETA=0:01:28 [12/20 22:31:31 d2.evaluation.evaluator]: Inference done 723/1000. 0.2902 s / img. ETA=0:01:23 [12/20 22:31:36 d2.evaluation.evaluator]: Inference done 740/1000. 0.2902 s / img. ETA=0:01:18 [12/20 22:31:41 d2.evaluation.evaluator]: Inference done 757/1000. 0.2902 s / img. ETA=0:01:13 [12/20 22:31:46 d2.evaluation.evaluator]: Inference done 774/1000. 0.2901 s / img. ETA=0:01:08 [12/20 22:31:52 d2.evaluation.evaluator]: Inference done 792/1000. 0.2900 s / img. ETA=0:01:02 [12/20 22:31:57 d2.evaluation.evaluator]: Inference done 808/1000. 0.2902 s / img. ETA=0:00:57 [12/20 22:32:02 d2.evaluation.evaluator]: Inference done 825/1000. 0.2901 s / img. ETA=0:00:52 [12/20 22:32:07 d2.evaluation.evaluator]: Inference done 842/1000. 0.2901 s / img. ETA=0:00:47 [12/20 22:32:12 d2.evaluation.evaluator]: Inference done 859/1000. 0.2900 s / img. ETA=0:00:42 [12/20 22:32:17 d2.evaluation.evaluator]: Inference done 876/1000. 0.2900 s / img. ETA=0:00:37 [12/20 22:32:22 d2.evaluation.evaluator]: Inference done 893/1000. 0.2901 s / img. ETA=0:00:32 [12/20 22:32:27 d2.evaluation.evaluator]: Inference done 910/1000. 0.2900 s / img. ETA=0:00:27 [12/20 22:32:32 d2.evaluation.evaluator]: Inference done 926/1000. 0.2899 s / img. ETA=0:00:22 [12/20 22:32:37 d2.evaluation.evaluator]: Inference done 943/1000. 0.2898 s / img. ETA=0:00:17 [12/20 22:32:43 d2.evaluation.evaluator]: Inference done 960/1000. 0.2897 s / img. ETA=0:00:12 [12/20 22:32:48 d2.evaluation.evaluator]: Inference done 977/1000. 0.2897 s / img. ETA=0:00:06 [12/20 22:32:53 d2.evaluation.evaluator]: Inference done 994/1000. 0.2896 s / img. ETA=0:00:01 [12/20 22:32:55 d2.evaluation.evaluator]: Total inference time: 0:05:00.808809 (0.302320 s / img per device, on 1 devices) [12/20 22:32:55 d2.evaluation.evaluator]: Total inference pure compute time: 0:04:48 (0.289574 s / img per device, on 1 devices) [12/20 22:32:55 d2.evaluation.coco_evaluation]: Preparing results for COCO format ... [12/20 22:32:55 d2.evaluation.coco_evaluation]: Saving results to /content/logs/coco_instances_results.json [12/20 22:32:55 d2.evaluation.coco_evaluation]: Evaluating predictions ... Loading and preparing results... DONE (t=0.01s) creating index... index created! Running per image evaluation... Evaluate annotation type *bbox* DONE (t=9.96s). Accumulating evaluation results... DONE (t=2.67s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.003 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.008 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.002 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.015 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.003 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.007 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.009 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.009 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.015 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.009 [12/20 22:33:08 d2.evaluation.coco_evaluation]: Evaluation results for bbox: | AP | AP50 | AP75 | APs | APm | APl | |:-----:|:------:|:------:|:-----:|:-----:|:-----:| | 0.301 | 0.825 | 0.164 | 0.000 | 1.515 | 0.318 | [12/20 22:33:08 d2.evaluation.coco_evaluation]: Per-category bbox AP: | category | AP | category | AP | category | AP | |:------------------------------------------------------------------------|:-------|:-----------------------------------------------------------------|:-------|:-------------------------------------------------------------------------|:-------| | beetroot-steamed-without-addition-of-salt | 0.000 | carrot-steamed-without-addition-of-salt | 0.000 | mushroom-average-stewed-without-addition-of-fat-without-addition-of-salt | 0.000 | | savoy-cabbage-steamed-without-addition-of-salt | nan | zucchini-stewed-without-addition-of-fat-without-addition-of-salt | nan | vanille-cream-cooked-custard-creme-dessert | nan | | shrimp-boiled | 0.000 | spinach-steamed-without-addition-of-salt | 0.000 | spinach-raw | 0.000 | | green-bean-steamed-without-addition-of-salt | 0.000 | sweet-corn-canned | 0.000 | watermelon-fresh | 0.000 | | pizza-with-ham-with-mushrooms-baked | 0.000 | pizza-with-ham-baked | 0.000 | pizza-with-vegetables-baked | 0.000 | | bell-pepper-red-stewed-without-addition-of-fat-without-addition-of-salt | 0.000 | applesauce-unsweetened-canned | 0.000 | quiche-with-cheese-baked-with-puff-pastry | 0.000 | | quiche-with-spinach-baked-with-cake-dough | 0.000 | risotto-with-mushrooms-cooked | nan | potato-salad-with-mayonnaise-yogurt-dressing | 0.000 | | wienerli-swiss-sausage | 0.000 | curds-natural-with-at-most-10-fidm | 0.000 | tuna-in-oil-drained | nan | | tomato-stewed-without-addition-of-fat-without-addition-of-salt | 0.000 | sweet-potato | 0.000 | country-fries | 0.000 | | potato-gnocchi | 0.000 | potatoes-steamed | 0.000 | chips-french-fries | 0.000 | | rosti | 0.000 | vegetable-mix-peas-and-carrots | 0.000 | coleslaw-chopped-without-sauce | nan | | vegetables | 0.000 | mixed-vegetables | 0.000 | ratatouille | 0.000 | | mixed-salad-chopped-without-sauce | 6.172 | leaf-spinach | 0.000 | witloof-chicory | 0.000 | | salad-rocket | 0.000 | salad-leaf-salad-green | 21.202 | salad-lambs-ear | 0.000 | | artichoke | nan | eggplant | 0.000 | avocado | 0.000 | | french-beans | 0.000 | cucumber-pickled | 0.000 | cucumber | 0.000 | | capers | 0.000 | pumpkin | 0.000 | bell-pepper-red-raw | 0.000 | | tomato-raw | 0.000 | zucchini | 0.000 | red-radish | 0.000 | | beetroot-raw | 0.000 | white-radish | nan | carrot-raw | 1.015 | | celeriac | nan | cauliflower | 0.000 | broccoli | 0.000 | | chinese-cabbage | 0.000 | kolhrabi | 0.000 | romanesco | 0.000 | | brussel-sprouts | 0.000 | red-cabbage | 0.000 | sauerkraut | 0.000 | | white-cabbage | nan | mushroom | 0.000 | mushrooms | 0.000 | | peas | 0.000 | corn | 0.000 | spring-onion-scallion | 0.000 | | garlic | 0.000 | leek | 0.000 | pearl-onions | nan | | onion | 0.000 | fennel | 0.000 | swiss-chard | 0.000 | | mungbean-sprouts | nan | green-asparagus | 0.000 | white-asparagus | nan | | shoots | 0.000 | alfa-sprouts | nan | celery | 0.000 | | bean-seeds | nan | beans-kidney | 0.000 | beans-white | 0.000 | | chickpeas | 0.000 | lentils | 0.000 | pineapple | 0.000 | | apple | 10.218 | pomegranate | 0.000 | apricots | 0.000 | | banana | 13.809 | berries | 0.000 | pear | 0.000 | | blackberry | nan | dates | 0.000 | strawberries | 0.000 | | figs | 0.000 | fruit-salad | 0.000 | grapefruit-pomelo | 0.000 | | blueberries | 0.000 | raspberries | 0.000 | kaki | 0.000 | | cherries | 0.000 | kiwi | 0.000 | mandarine | 24.480 | | mango | 0.000 | sugar-melon | 0.000 | nectarine | 0.000 | | orange | 0.000 | peach | 0.000 | plums | 0.000 | | grapes | 0.000 | dried-raisins | 0.000 | lemon | 0.000 | | damson-plum | 0.000 | peanut-butter | 0.000 | chestnuts | 0.000 | | seeds | 0.000 | pumpkin-seeds | 0.000 | pine-nuts | 0.000 | | sunflower-seeds | 0.000 | mixed-seeds | nan | almonds | 0.000 | | nuts | nan | walnut | 0.000 | cashew-nut | 0.000 | | peanut | 0.000 | hazelnut | 0.000 | coconut | 0.000 | | brazil-nut | 0.000 | pecan-nut | 0.000 | mixed-nuts | 0.000 | | pistachio | 0.000 | linseeds | 0.000 | sesame-seeds | 0.000 | | green-olives | 0.000 | black-olives | 0.000 | milk | 0.000 | | kefir-drink | 0.000 | coconut-milk | 0.000 | soya-drink-soy-milk | 0.000 | | soya-yaourt-yahourt-yogourt-ou-yoghourt | 0.000 | cottage-cheese | 0.000 | curd | nan | | blue-mould-cheese | 0.000 | brie | 0.000 | cheddar | 0.000 | | emmental-cheese | 0.000 | feta | 0.000 | fondue | 0.000 | | fresh-cheese | 0.000 | gruyere | 0.000 | semi-hard-cheese | 0.000 | | halloumi | 0.000 | hard-cheese | 0.000 | cheese | 0.000 | | mozzarella | 0.000 | parmesan | 0.000 | philadelphia | 0.000 | | cheese-for-raclette | 0.000 | cream-cheese | 0.000 | processed-cheese | 0.000 | | tete-de-moine | 0.000 | tomme | 0.000 | soft-cheese | 0.000 | | mousse | nan | panna-cotta | nan | tiramisu | 0.000 | | cream | 0.000 | sour-cream | 0.000 | thickened-cream-35 | nan | | dairy-ice-cream | 0.000 | sorbet | 0.000 | flakes-oat | 0.000 | | rice-noodles-vermicelli | 0.000 | bulgur | 0.000 | couscous | 0.000 | | ebly | 0.000 | grits-polenta-maize-flour | 0.000 | quinoa | 0.000 | | rice | 0.000 | rice-basmati | 0.000 | rice-jasmin | nan | | rice-whole-grain | 0.000 | rice-wild | 0.000 | spaetzle | 0.000 | | pasta | 0.000 | pasta-hornli | 0.000 | pasta-in-butterfly-form-farfalle | 0.000 | | pasta-linguini-parpadelle-tagliatelle | 0.000 | pasta-in-conch-form | 0.000 | pasta-noodles | 0.000 | | pasta-penne | 0.000 | pasta-ravioli-stuffing | 0.000 | pasta-spaghetti | 0.000 | | pasta-twist | 0.000 | pasta-tortelloni-stuffing | nan | pasta-wholemeal | 0.000 | | bagel-without-filling | 0.000 | bread-french-white-flour | 0.000 | bread | 0.000 | | bread-5-grain | nan | bread-spelt | 0.000 | bread-fruit | 0.000 | | bread-half-white | 0.000 | bread-grain | 0.000 | bread-nut | nan | | bread-olive | 0.000 | bread-pita | 0.000 | bread-rye | 0.000 | | bread-whole-wheat | 0.000 | bread-sourdough | 0.000 | bread-black | 0.000 | | bread-ticino | 0.000 | bread-toast | 0.000 | bread-wholemeal-toast | 0.000 | | bread-wholemeal | 0.545 | bread-white | 5.438 | brioche | 0.000 | | roll-of-half-white-or-white-flour-with-large-void | 0.000 | hamburger-bun | 0.000 | roll-with-pieces-of-chocolate | 0.000 | | white-bread-with-butter-eggs-and-milk | 0.000 | focaccia | 0.000 | croissant | 0.000 | | croissant-wholegrain | 0.000 | lye-pretzel-soft | 0.000 | braided-white-loaf | 0.000 | | crisp-bread-wasa | nan | breadcrumbs-unspiced | 0.000 | rice-waffels | 0.000 | | grissini | 0.000 | rusk-wholemeal | 0.000 | corn-flakes | nan | | crunch-muesli | 0.000 | muesli | 0.000 | dough-puff-pastry-shortcrust-bread-pizza-dough | nan | | pastry-flaky | nan | meat | 0.000 | minced-meat | nan | | beef | 0.000 | beef-roast | 0.000 | beef-sirloin-steak | 0.000 | | beef-filet | nan | beef-minced-only-meat | 0.000 | beef-cut-into-stripes-only-meat | 0.000 | | pork | 0.000 | pork-roast | 0.000 | pork-chop | nan | | pork-escalope | nan | lamb | nan | lamb-chop | 0.000 | | chicken | 0.000 | chicken-breast | 0.000 | chicken-wing | nan | | chicken-cut-into-stripes-only-meat | 0.000 | chicken-leg | 0.000 | frying-sausage | 0.000 | | cervelat | 0.000 | chicken-nuggets | 0.000 | chorizo | 0.000 | | meatloaf | 0.000 | hamburger | 0.000 | dried-meat | 0.000 | | veal-sausage | 0.000 | processed-meat-charcuterie | nan | salami | 0.000 | | cooked-sausage | nan | ham | 0.000 | ham-cooked | 0.000 | | ham-raw | 0.000 | ham-turkey | 0.000 | smoked-cooked-sausage-of-pork-and-beef-meat-sausag | nan | | bacon | nan | bacon-frying | 0.000 | bacon-cooking | 0.000 | | bacon-raw | 0.000 | meat-terrine-pate | 0.000 | sausage | 0.000 | | veggie-burger | 0.000 | tofu | 0.000 | fish | 0.000 | | cod | 0.000 | salmon | 0.000 | anchovies | 0.000 | | tuna | nan | shrimp-prawn-small | nan | shrimp-prawn-large | 0.000 | | fish-crunchies-battered | nan | fish-fingers-breaded | 0.000 | egg | 0.988 | | oil | 0.000 | butter | 0.642 | butter-herb | 0.000 | | margarine | 0.000 | praline | 0.000 | jam | 0.000 | | honey | 0.000 | sugar-glazing | nan | maple-syrup-concentrate | 0.000 | | dark-chocolate | 0.000 | milk-chocolate | 0.000 | white-chocolate | nan | | chocolate | 0.000 | chocolate-filled | nan | cocoa-powder | nan | | hazelnut-chocolate-spread-nutella-ovomaltine-caotina | 0.000 | m-m-s | 0.000 | chocolate-egg-small | 0.000 | | sweets-candies | 0.000 | gummi-bears-fruit-jellies-jelly-babies-with-fruit-essence | nan | apple-pie | 0.000 | | brownie | nan | cake-oblong | 0.000 | lemon-cake | nan | | crepe-plain | 0.000 | fruit-tart | 0.000 | cake-marble | 0.000 | | cake-chocolate | 0.000 | muffin | 0.000 | omelette-plain | 0.000 | | carrot-cake | 0.000 | black-forest-tart | 0.000 | tart | 0.000 | | waffle | nan | croissant-with-chocolate-filling | 0.000 | cookies | 0.000 | | biscuits | 0.000 | macaroon | 0.000 | meringue | 0.000 | | biscuit-with-butter | 0.000 | chocolate-cookies | 0.000 | juice-apple | 0.000 | | juice-multifruit | 0.000 | juice-orange | nan | smoothie | nan | | coca-cola | nan | coca-cola-zero | 0.000 | ice-tea | nan | | syrup-diluted-ready-to-drink | 0.000 | tea | 0.000 | cappuccino | 0.000 | | espresso-with-caffeine | 0.769 | coffee-with-caffeine | 5.884 | coffee-decaffeinated | 0.000 | | latte-macchiato-with-caffeine | 0.000 | white-coffee-with-caffeine | 0.932 | ristretto-with-caffeine | 0.000 | | tea-green | 0.000 | tea-black | nan | tea-verveine | nan | | tea-fruit | nan | tea-spice | nan | tea-ginger | nan | | herbal-tea | 0.000 | tea-peppermint | 0.000 | tea-rooibos | 0.000 | | ice-cubes | nan | water | 22.046 | water-mineral | 0.000 | | aperitif-with-alcohol-aperol-spritz | nan | cocktail | 0.000 | champagne | 0.000 | | prosecco | 0.000 | sekt | 0.000 | wine-rose | 0.000 | | wine-red | 2.094 | wine-white | 5.914 | beer | 0.000 | | light-beer | 0.000 | sauce-savoury | 0.000 | sauce-roast | 0.000 | | sauce-carbonara | nan | sauce-cocktail | 0.000 | sauce-curry | 0.000 | | sauce-pesto | 0.000 | sauce-mushroom | 0.000 | sauce-cream | 0.000 | | sauce-sweet-sour | 0.000 | ketchup | 0.000 | bolognaise-sauce | 0.000 | | tomato-sauce | 0.000 | dips | nan | salad-dressing | 0.000 | | balsamic-salad-dressing | 0.000 | french-salad-dressing | 0.000 | italian-salad-dressing | 0.000 | | oil-vinegar-salad-dressing | 0.000 | guacamole | 0.000 | mayonnaise | 0.000 | | tartar-sauce | 0.000 | tzatziki | 0.000 | basil | 0.000 | | coriander | 0.000 | bouquet-garni | 0.000 | parsley | 0.000 | | chives | 0.000 | cenovis-yeast-spread | nan | sauce-soya | 0.000 | | mustard | 0.000 | mustard-dijon | nan | balsamic-vinegar | nan | | soup-vegetable | 0.000 | soup-cream-of-vegetables | 0.000 | soup-potato | 0.000 | | soup-pumpkin | 0.000 | soup-miso | 0.000 | soup-tomato | 0.000 | | bouillon | 0.000 | bouillon-vegetable | 0.000 | falafel-balls | 0.000 | | savoury-puff-pastry | nan | savoury-puff-pastry-stick | 0.000 | corn-crisps | nan | | crackers | 0.000 | croutons | 0.000 | crisps | 0.000 | | popcorn-salted | 0.000 | croque-monsieur | 0.000 | spring-roll-fried | 0.000 | | ham-croissant | nan | salt-cake-vegetables-filled | nan | hamburger-bread-meat-ketchup | 0.000 | | cordon-bleu-from-pork-schnitzel-fried | 0.000 | lasagne-meat-prepared | 0.000 | mashed-potatoes-prepared-with-full-fat-milk-with-butter | 0.000 | | pizza-margherita-baked | 0.000 | sandwich-ham-cheese-and-butter | nan | sushi | 0.000 | | kebab-in-pita-bread | 0.000 | pancakes | nan | hummus | 0.000 | | greek-salad | 0.000 | dumplings | nan | apricot-dried | 0.000 | | chocolate-mousse | nan | cheesecake | 0.000 | caprese-salad-tomato-mozzarella | 0.000 | | chili-con-carne-prepared | 0.000 | taboule-prepared-with-couscous | nan | perch-fillets-lake | nan | | risotto-without-cheese-cooked | 0.000 | salmon-smoked | 0.000 | pie-apricot-baked-with-cake-dough | 0.000 | | eggplant-caviar | 0.000 | apple-crumble | nan | egg-scrambled-prepared | 0.000 | | oat-milk | 0.000 | lemon-pie | nan | glucose-drink-50g | 0.000 | | goat-average-raw | nan | pie-rhubarb-baked-with-cake-dough | nan | chicken-curry-cream-coconut-milk-curry-spices-paste | nan | | pie-plum-baked-with-cake-dough | 0.000 | potatoes-au-gratin-dauphinois-prepared | 0.000 | buckwheat-grain-peeled | nan | | birchermuesli-prepared-no-sugar-added | 0.000 | fajita-bread-only | 0.000 | mango-dried | 0.000 | | lentils-green-du-puy-du-berry | 0.000 | naan-indien-bread | nan | butter-spread-puree-almond | 0.000 | | chocolate-milk-chocolate-drink | nan | water-with-lemon-juice | 0.000 | sun-dried-tomatoe | 0.000 | | gluten-free-bread | 0.000 | fruit-coulis | 0.000 | greek-yaourt-yahourt-yogourt-ou-yoghourt | 0.000 | | cake-salted | 0.000 | soup-of-lentils-dahl-dhal | nan | fig-dried | 0.000 | | turnover-with-meat-small-meat-pie-empanadas | 0.000 | lasagne-vegetable-prepared | 0.000 | sauce-sweet-salted-asian | 0.000 | | french-pizza-from-alsace-baked | 0.000 | fruit-compotes | nan | vegetable-au-gratin-baked | 0.000 | | porridge-prepared-with-partially-skimmed-milk | nan | curry-vegetarian | nan | bread-meat-substitute-lettuce-sauce | 0.000 | | tartar-meat | 0.000 | chia-grains | nan | faux-mage-cashew-vegan-chers | 0.000 | | milk-chocolate-with-hazelnuts | 0.000 | yaourt-yahourt-yogourt-ou-yoghourt-natural | 0.000 | paprika-chips | nan | | banana-cake | 0.000 | cream-spinach | nan | cantonese-fried-rice | 0.000 | | goat-cheese-soft | 0.000 | buckwheat-pancake | 0.000 | meat-balls | 0.000 | | high-protein-pasta-made-of-lentils-peas | 0.000 | mix-of-dried-fruits-and-nuts | 0.000 | baked-potato | 0.000 | Loading and preparing results... DONE (t=0.16s) creating index... index created! Running per image evaluation... Evaluate annotation type *segm* DONE (t=9.27s). Accumulating evaluation results... DONE (t=2.70s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.006 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.010 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.007 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.002 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.006 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.014 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.016 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.016 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.010 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.017 [12/20 22:33:21 d2.evaluation.coco_evaluation]: Evaluation results for segm: | AP | AP50 | AP75 | APs | APm | APl | |:-----:|:------:|:------:|:-----:|:-----:|:-----:| | 0.600 | 1.017 | 0.663 | 0.000 | 0.195 | 0.642 | [12/20 22:33:21 d2.evaluation.coco_evaluation]: Per-category segm AP: | category | AP | category | AP | category | AP | |:------------------------------------------------------------------------|:-------|:-----------------------------------------------------------------|:-------|:-------------------------------------------------------------------------|:-------| | beetroot-steamed-without-addition-of-salt | 0.000 | carrot-steamed-without-addition-of-salt | 0.000 | mushroom-average-stewed-without-addition-of-fat-without-addition-of-salt | 0.000 | | savoy-cabbage-steamed-without-addition-of-salt | nan | zucchini-stewed-without-addition-of-fat-without-addition-of-salt | nan | vanille-cream-cooked-custard-creme-dessert | nan | | shrimp-boiled | 0.000 | spinach-steamed-without-addition-of-salt | 0.000 | spinach-raw | 0.000 | | green-bean-steamed-without-addition-of-salt | 0.000 | sweet-corn-canned | 0.000 | watermelon-fresh | 0.000 | | pizza-with-ham-with-mushrooms-baked | 0.000 | pizza-with-ham-baked | 0.000 | pizza-with-vegetables-baked | 0.000 | | bell-pepper-red-stewed-without-addition-of-fat-without-addition-of-salt | 0.000 | applesauce-unsweetened-canned | 0.000 | quiche-with-cheese-baked-with-puff-pastry | 0.000 | | quiche-with-spinach-baked-with-cake-dough | 0.000 | risotto-with-mushrooms-cooked | nan | potato-salad-with-mayonnaise-yogurt-dressing | 0.000 | | wienerli-swiss-sausage | 0.000 | curds-natural-with-at-most-10-fidm | 0.000 | tuna-in-oil-drained | nan | | tomato-stewed-without-addition-of-fat-without-addition-of-salt | 0.000 | sweet-potato | 0.000 | country-fries | 0.000 | | potato-gnocchi | 0.000 | potatoes-steamed | 0.000 | chips-french-fries | 0.000 | | rosti | 0.000 | vegetable-mix-peas-and-carrots | 0.000 | coleslaw-chopped-without-sauce | nan | | vegetables | 0.000 | mixed-vegetables | 0.000 | ratatouille | 0.000 | | mixed-salad-chopped-without-sauce | 4.664 | leaf-spinach | 0.000 | witloof-chicory | 0.000 | | salad-rocket | 0.000 | salad-leaf-salad-green | 24.273 | salad-lambs-ear | 0.000 | | artichoke | nan | eggplant | 0.000 | avocado | 0.000 | | french-beans | 0.000 | cucumber-pickled | 0.000 | cucumber | 0.000 | | capers | 0.000 | pumpkin | 0.000 | bell-pepper-red-raw | 0.000 | | tomato-raw | 0.000 | zucchini | 0.000 | red-radish | 0.000 | | beetroot-raw | 0.000 | white-radish | nan | carrot-raw | 4.958 | | celeriac | nan | cauliflower | 0.000 | broccoli | 0.000 | | chinese-cabbage | 0.000 | kolhrabi | 0.000 | romanesco | 0.000 | | brussel-sprouts | 0.000 | red-cabbage | 0.000 | sauerkraut | 0.000 | | white-cabbage | nan | mushroom | 0.000 | mushrooms | 0.000 | | peas | 0.000 | corn | 0.000 | spring-onion-scallion | 0.000 | | garlic | 0.000 | leek | 0.000 | pearl-onions | nan | | onion | 0.000 | fennel | 0.000 | swiss-chard | 0.000 | | mungbean-sprouts | nan | green-asparagus | 0.000 | white-asparagus | nan | | shoots | 0.000 | alfa-sprouts | nan | celery | 0.000 | | bean-seeds | nan | beans-kidney | 0.000 | beans-white | 0.000 | | chickpeas | 0.000 | lentils | 0.000 | pineapple | 0.000 | | apple | 24.733 | pomegranate | 0.000 | apricots | 0.000 | | banana | 38.713 | berries | 0.000 | pear | 0.000 | | blackberry | nan | dates | 0.000 | strawberries | 0.000 | | figs | 0.000 | fruit-salad | 0.000 | grapefruit-pomelo | 0.000 | | blueberries | 0.000 | raspberries | 0.000 | kaki | 0.000 | | cherries | 0.000 | kiwi | 0.000 | mandarine | 35.804 | | mango | 0.000 | sugar-melon | 0.000 | nectarine | 0.000 | | orange | 0.000 | peach | 0.000 | plums | 0.000 | | grapes | 0.000 | dried-raisins | 0.000 | lemon | 0.000 | | damson-plum | 0.000 | peanut-butter | 0.000 | chestnuts | 0.000 | | seeds | 0.000 | pumpkin-seeds | 0.000 | pine-nuts | 0.000 | | sunflower-seeds | 0.000 | mixed-seeds | nan | almonds | 0.000 | | nuts | nan | walnut | 0.000 | cashew-nut | 0.000 | | peanut | 0.000 | hazelnut | 0.000 | coconut | 0.000 | | brazil-nut | 0.000 | pecan-nut | 0.000 | mixed-nuts | 0.000 | | pistachio | 0.000 | linseeds | 0.000 | sesame-seeds | 0.000 | | green-olives | 0.000 | black-olives | 0.000 | milk | 0.000 | | kefir-drink | 0.000 | coconut-milk | 0.000 | soya-drink-soy-milk | 0.000 | | soya-yaourt-yahourt-yogourt-ou-yoghourt | 0.000 | cottage-cheese | 0.000 | curd | nan | | blue-mould-cheese | 0.000 | brie | 0.000 | cheddar | 0.000 | | emmental-cheese | 0.000 | feta | 0.000 | fondue | 0.000 | | fresh-cheese | 0.000 | gruyere | 0.000 | semi-hard-cheese | 0.000 | | halloumi | 0.000 | hard-cheese | 0.000 | cheese | 0.000 | | mozzarella | 0.000 | parmesan | 0.000 | philadelphia | 0.000 | | cheese-for-raclette | 0.000 | cream-cheese | 0.000 | processed-cheese | 0.000 | | tete-de-moine | 0.000 | tomme | 0.000 | soft-cheese | 0.000 | | mousse | nan | panna-cotta | nan | tiramisu | 0.000 | | cream | 0.000 | sour-cream | 0.000 | thickened-cream-35 | nan | | dairy-ice-cream | 0.000 | sorbet | 0.000 | flakes-oat | 0.000 | | rice-noodles-vermicelli | 0.000 | bulgur | 0.000 | couscous | 0.000 | | ebly | 0.000 | grits-polenta-maize-flour | 0.000 | quinoa | 0.000 | | rice | 0.000 | rice-basmati | 0.000 | rice-jasmin | nan | | rice-whole-grain | 0.000 | rice-wild | 0.000 | spaetzle | 0.000 | | pasta | 0.000 | pasta-hornli | 0.000 | pasta-in-butterfly-form-farfalle | 0.000 | | pasta-linguini-parpadelle-tagliatelle | 0.000 | pasta-in-conch-form | 0.000 | pasta-noodles | 0.000 | | pasta-penne | 0.000 | pasta-ravioli-stuffing | 0.000 | pasta-spaghetti | 0.000 | | pasta-twist | 0.000 | pasta-tortelloni-stuffing | nan | pasta-wholemeal | 0.000 | | bagel-without-filling | 0.000 | bread-french-white-flour | 0.000 | bread | 0.000 | | bread-5-grain | nan | bread-spelt | 0.000 | bread-fruit | 0.000 | | bread-half-white | 0.000 | bread-grain | 0.000 | bread-nut | nan | | bread-olive | 0.000 | bread-pita | 0.000 | bread-rye | 0.000 | | bread-whole-wheat | 0.354 | bread-sourdough | 0.000 | bread-black | 0.000 | | bread-ticino | 0.000 | bread-toast | 0.000 | bread-wholemeal-toast | 0.000 | | bread-wholemeal | 3.144 | bread-white | 15.266 | brioche | 0.000 | | roll-of-half-white-or-white-flour-with-large-void | 0.000 | hamburger-bun | 0.000 | roll-with-pieces-of-chocolate | 0.000 | | white-bread-with-butter-eggs-and-milk | 0.000 | focaccia | 0.000 | croissant | 0.000 | | croissant-wholegrain | 0.000 | lye-pretzel-soft | 0.000 | braided-white-loaf | 0.000 | | crisp-bread-wasa | nan | breadcrumbs-unspiced | 0.000 | rice-waffels | 0.000 | | grissini | 0.000 | rusk-wholemeal | 0.000 | corn-flakes | nan | | crunch-muesli | 0.000 | muesli | 0.000 | dough-puff-pastry-shortcrust-bread-pizza-dough | nan | | pastry-flaky | nan | meat | 0.000 | minced-meat | nan | | beef | 0.000 | beef-roast | 0.000 | beef-sirloin-steak | 0.000 | | beef-filet | nan | beef-minced-only-meat | 0.000 | beef-cut-into-stripes-only-meat | 0.000 | | pork | 0.000 | pork-roast | 0.000 | pork-chop | nan | | pork-escalope | nan | lamb | nan | lamb-chop | 0.000 | | chicken | 0.000 | chicken-breast | 0.000 | chicken-wing | nan | | chicken-cut-into-stripes-only-meat | 0.000 | chicken-leg | 0.000 | frying-sausage | 0.000 | | cervelat | 0.000 | chicken-nuggets | 0.000 | chorizo | 0.000 | | meatloaf | 0.000 | hamburger | 0.000 | dried-meat | 0.000 | | veal-sausage | 0.000 | processed-meat-charcuterie | nan | salami | 0.000 | | cooked-sausage | nan | ham | 0.000 | ham-cooked | 0.000 | | ham-raw | 0.000 | ham-turkey | 0.000 | smoked-cooked-sausage-of-pork-and-beef-meat-sausag | nan | | bacon | nan | bacon-frying | 0.000 | bacon-cooking | 0.000 | | bacon-raw | 0.000 | meat-terrine-pate | 0.000 | sausage | 0.000 | | veggie-burger | 0.000 | tofu | 0.000 | fish | 0.000 | | cod | 0.000 | salmon | 0.000 | anchovies | 0.000 | | tuna | nan | shrimp-prawn-small | nan | shrimp-prawn-large | 0.000 | | fish-crunchies-battered | nan | fish-fingers-breaded | 0.000 | egg | 4.055 | | oil | 0.000 | butter | 0.770 | butter-herb | 0.000 | | margarine | 0.000 | praline | 0.000 | jam | 0.000 | | honey | 0.000 | sugar-glazing | nan | maple-syrup-concentrate | 0.000 | | dark-chocolate | 0.000 | milk-chocolate | 0.000 | white-chocolate | nan | | chocolate | 0.000 | chocolate-filled | nan | cocoa-powder | nan | | hazelnut-chocolate-spread-nutella-ovomaltine-caotina | 0.000 | m-m-s | 0.000 | chocolate-egg-small | 0.000 | | sweets-candies | 0.000 | gummi-bears-fruit-jellies-jelly-babies-with-fruit-essence | nan | apple-pie | 0.000 | | brownie | nan | cake-oblong | 0.000 | lemon-cake | nan | | crepe-plain | 0.000 | fruit-tart | 0.000 | cake-marble | 0.000 | | cake-chocolate | 0.000 | muffin | 0.000 | omelette-plain | 0.000 | | carrot-cake | 0.000 | black-forest-tart | 0.000 | tart | 0.000 | | waffle | nan | croissant-with-chocolate-filling | 0.000 | cookies | 0.000 | | biscuits | 0.000 | macaroon | 0.000 | meringue | 0.000 | | biscuit-with-butter | 0.000 | chocolate-cookies | 0.000 | juice-apple | 0.000 | | juice-multifruit | 0.000 | juice-orange | nan | smoothie | nan | | coca-cola | nan | coca-cola-zero | 0.000 | ice-tea | nan | | syrup-diluted-ready-to-drink | 0.000 | tea | 0.000 | cappuccino | 0.000 | | espresso-with-caffeine | 5.758 | coffee-with-caffeine | 12.123 | coffee-decaffeinated | 0.000 | | latte-macchiato-with-caffeine | 0.000 | white-coffee-with-caffeine | 7.710 | ristretto-with-caffeine | 0.000 | | tea-green | 0.000 | tea-black | nan | tea-verveine | nan | | tea-fruit | nan | tea-spice | nan | tea-ginger | nan | | herbal-tea | 0.000 | tea-peppermint | 0.000 | tea-rooibos | 0.000 | | ice-cubes | nan | water | 26.454 | water-mineral | 0.000 | | aperitif-with-alcohol-aperol-spritz | nan | cocktail | 0.000 | champagne | 0.000 | | prosecco | 0.000 | sekt | 0.000 | wine-rose | 0.000 | | wine-red | 18.478 | wine-white | 16.528 | beer | 0.000 | | light-beer | 0.000 | sauce-savoury | 0.000 | sauce-roast | 0.000 | | sauce-carbonara | nan | sauce-cocktail | 0.000 | sauce-curry | 0.000 | | sauce-pesto | 0.000 | sauce-mushroom | 0.000 | sauce-cream | 0.000 | | sauce-sweet-sour | 0.000 | ketchup | 0.000 | bolognaise-sauce | 0.000 | | tomato-sauce | 0.000 | dips | nan | salad-dressing | 0.000 | | balsamic-salad-dressing | 0.000 | french-salad-dressing | 0.000 | italian-salad-dressing | 0.000 | | oil-vinegar-salad-dressing | 0.000 | guacamole | 0.000 | mayonnaise | 0.000 | | tartar-sauce | 0.000 | tzatziki | 0.000 | basil | 0.000 | | coriander | 0.000 | bouquet-garni | 0.000 | parsley | 0.000 | | chives | 0.000 | cenovis-yeast-spread | nan | sauce-soya | 0.000 | | mustard | 0.000 | mustard-dijon | nan | balsamic-vinegar | nan | | soup-vegetable | 0.000 | soup-cream-of-vegetables | 0.000 | soup-potato | 0.000 | | soup-pumpkin | 0.000 | soup-miso | 0.000 | soup-tomato | 0.000 | | bouillon | 0.000 | bouillon-vegetable | 0.000 | falafel-balls | 0.000 | | savoury-puff-pastry | nan | savoury-puff-pastry-stick | 0.000 | corn-crisps | nan | | crackers | 0.000 | croutons | 0.000 | crisps | 0.000 | | popcorn-salted | 0.000 | croque-monsieur | 0.000 | spring-roll-fried | 0.000 | | ham-croissant | nan | salt-cake-vegetables-filled | nan | hamburger-bread-meat-ketchup | 0.000 | | cordon-bleu-from-pork-schnitzel-fried | 0.000 | lasagne-meat-prepared | 0.000 | mashed-potatoes-prepared-with-full-fat-milk-with-butter | 0.000 | | pizza-margherita-baked | 0.000 | sandwich-ham-cheese-and-butter | nan | sushi | 0.000 | | kebab-in-pita-bread | 0.000 | pancakes | nan | hummus | 0.000 | | greek-salad | 0.000 | dumplings | nan | apricot-dried | 0.000 | | chocolate-mousse | nan | cheesecake | 0.000 | caprese-salad-tomato-mozzarella | 0.000 | | chili-con-carne-prepared | 0.000 | taboule-prepared-with-couscous | nan | perch-fillets-lake | nan | | risotto-without-cheese-cooked | 0.000 | salmon-smoked | 0.000 | pie-apricot-baked-with-cake-dough | 0.000 | | eggplant-caviar | 0.000 | apple-crumble | nan | egg-scrambled-prepared | 0.000 | | oat-milk | 0.000 | lemon-pie | nan | glucose-drink-50g | 0.000 | | goat-average-raw | nan | pie-rhubarb-baked-with-cake-dough | nan | chicken-curry-cream-coconut-milk-curry-spices-paste | nan | | pie-plum-baked-with-cake-dough | 0.000 | potatoes-au-gratin-dauphinois-prepared | 0.000 | buckwheat-grain-peeled | nan | | birchermuesli-prepared-no-sugar-added | 0.000 | fajita-bread-only | 0.000 | mango-dried | 0.000 | | lentils-green-du-puy-du-berry | 0.000 | naan-indien-bread | nan | butter-spread-puree-almond | 0.000 | | chocolate-milk-chocolate-drink | nan | water-with-lemon-juice | 0.000 | sun-dried-tomatoe | 0.000 | | gluten-free-bread | 0.000 | fruit-coulis | 0.000 | greek-yaourt-yahourt-yogourt-ou-yoghourt | 0.000 | | cake-salted | 0.000 | soup-of-lentils-dahl-dhal | nan | fig-dried | 0.000 | | turnover-with-meat-small-meat-pie-empanadas | 0.000 | lasagne-vegetable-prepared | 0.000 | sauce-sweet-salted-asian | 0.000 | | french-pizza-from-alsace-baked | 0.000 | fruit-compotes | nan | vegetable-au-gratin-baked | 0.000 | | porridge-prepared-with-partially-skimmed-milk | nan | curry-vegetarian | nan | bread-meat-substitute-lettuce-sauce | 0.000 | | tartar-meat | 0.000 | chia-grains | nan | faux-mage-cashew-vegan-chers | 0.000 | | milk-chocolate-with-hazelnuts | 0.000 | yaourt-yahourt-yogourt-ou-yoghourt-natural | 0.000 | paprika-chips | nan | | banana-cake | 0.000 | cream-spinach | nan | cantonese-fried-rice | 0.000 | | goat-cheese-soft | 0.000 | buckwheat-pancake | 0.000 | meat-balls | 0.000 | | high-protein-pasta-made-of-lentils-peas | 0.000 | mix-of-dried-fruits-and-nuts | 0.000 | baked-potato | 0.000 |
In [ ]:
cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.01
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 498
cfg.DATASETS.TEST = ("validation_dataset", )
predictor = DefaultPredictor(cfg)
In [ ]:
val_metadata = MetadataCatalog.get("val_dataset")
im = cv2.imread("data/val/images/085220.jpg")
outputs = predictor(im)
v = Visualizer(im[:, :, ::-1],
metadata=val_metadata,
scale=2,
instance_mode=ColorMode.IMAGE_BW
)
out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
cv2_imshow(out.get_image()[:, :, ::-1])
In [ ]:
#!bash <(curl -sL https://gitlab.aicrowd.com/jyotish/food-recognition-challenge-detectron2-baseline/raw/master/utils/submit-colab.sh)
Data Argumentation¶
In [ ]:
!pip install --upgrade fastai
Requirement already satisfied: fastai in /usr/local/lib/python3.7/dist-packages (2.5.3) Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from fastai) (1.1.5) Requirement already satisfied: torch<1.11,>=1.7.0 in /usr/local/lib/python3.7/dist-packages (from fastai) (1.10.1) Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from fastai) (1.4.1) Requirement already satisfied: torchvision>=0.8.2 in /usr/local/lib/python3.7/dist-packages (from fastai) (0.11.2) Requirement already satisfied: pillow>6.0.0 in /usr/local/lib/python3.7/dist-packages (from fastai) (7.1.2) Requirement already satisfied: pip in /usr/local/lib/python3.7/dist-packages (from fastai) (21.1.3) Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from fastai) (5.1) Requirement already satisfied: spacy<4 in /usr/local/lib/python3.7/dist-packages (from fastai) (2.2.4) Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from fastai) (1.0.1) Requirement already satisfied: fastprogress>=0.2.4 in /usr/local/lib/python3.7/dist-packages (from fastai) (1.0.0) Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from fastai) (2.26.0) Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from fastai) (3.2.2) Requirement already satisfied: fastdownload<2,>=0.0.5 in /usr/local/lib/python3.7/dist-packages (from fastai) (0.0.5) Requirement already satisfied: fastcore<1.4,>=1.3.22 in /usr/local/lib/python3.7/dist-packages (from fastai) (1.3.27) Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from fastai) (21.3) Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from fastprogress>=0.2.4->fastai) (1.19.5) Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (1.0.6) Requirement already satisfied: blis<0.5.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (0.4.1) Requirement already satisfied: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (0.8.2) Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (57.4.0) Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (2.0.6) Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (4.62.3) Requirement already satisfied: catalogue<1.1.0,>=0.0.7 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (1.0.0) Requirement already satisfied: thinc==7.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (7.4.0) Requirement already satisfied: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (1.1.3) Requirement already satisfied: srsly<1.1.0,>=1.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (1.0.5) Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (3.0.6) Requirement already satisfied: importlib-metadata>=0.20 in /usr/local/lib/python3.7/dist-packages (from catalogue<1.1.0,>=0.0.7->spacy<4->fastai) (4.8.2) Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.20->catalogue<1.1.0,>=0.0.7->spacy<4->fastai) (3.10.0.2) Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.20->catalogue<1.1.0,>=0.0.7->spacy<4->fastai) (3.6.0) Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (2.0.8) Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (2.10) Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (1.24.3) Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (2021.10.8) Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (3.0.6) Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (2.8.2) Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (0.11.0) Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (1.3.2) Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->fastai) (1.15.0) Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas->fastai) (2018.9) Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fastai) (3.0.0) Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fastai) (1.1.0)
In [ ]:
from fastai.vision.core import *
from fastai.vision.utils import *
from fastai.vision.augment import *
from fastai.data.core import *
from fastai.data.transforms import *
In [ ]:
images, lbl_bbox = get_annotations('data/train/annotations.json')\
In [ ]:
idx=14
coco_fn,bbox = 'data/train/images/'+images[idx],lbl_bbox[idx]
def _coco_bb(x): return TensorBBox.create(bbox[0])
def _coco_lbl(x): return bbox[1]
In [ ]:
coco_dsrc = Datasets([coco_fn]*10, [PILImage.create, [_coco_bb,], [_coco_lbl, MultiCategorize(add_na=True)]], n_inp=1)
coco_tdl = TfmdDL(coco_dsrc, bs=9, after_item=[BBoxLabeler(), PointScaler(), ToTensor(), Resize(256)],
after_batch=[IntToFloatTensor(), *aug_transforms()])
coco_tdl.show_batch(max_n=9)
Content
Comments
You must login before you can post a comment.