ObjectDetection-CoreML

supporting models: YOLOv8, YOLOv5, YOLOv3, MobileNetV2+SSDLite

This project is Object Detection on iOS with Core ML.
If you are interested in iOS + Machine Learning, visit here you can see various DEMOs.

Requirements

• Xcode 10.3+
• iOS 13.0+
• Swift 4.2

How To Build and Run the Project

1. Clone the project

git clone https://github.com/tucan9389/ObjectDetection-CoreML

2. Prepare Core ML model

• You can download COCO models or another model from here

Or if you want to make and use model with custom dataset,

1. follow roboflow tutorial from scratch or yolov5 repo's tutorial
2. and convert the .pt model to .mlmodel model with our issue.

3. Add the model to the project

By default, the project uses the yolov8s model. If you want to use another model, you can replace the model file in the project.

4. Set model name properly in ViewController.swift

5. Build and Run

How To Run with your own model

1. Convert your model to Core ML

At this moment(23.04.08), there is error when converting yolov8 models to Core ML. Once ultralytics/ultralytics#1791 is merged, you can use the following steps. (Or you can use this PR alternatively.)

Pre-requirements

pip install ultralytics
pip install coremltools

Option 1) With shell

yolo export model=yolov8n.pt format=coreml nms

Option 2) With python script

# mian.py
from ultralytics import YOLO

if __name__ == '__main__':
    model = YOLO("yolov8n.pt", task='detect')  # load a pretrained model
    model.overrides['nms'] = True
    success = model.export(format="coreml")  # export the model to CoreML format

# in terminal
python main.py
# then you can see the `.mlpackage` or `.mlmodel` file in your current directory
# (btw you can check your current directory with `pwd` command)

2. Follow the steps above from Step 3

Models

Model Matadata

Model Size, Minimum iOS Version, Download Link

Model	Size (MB)	Minimum iOS Version	Download Link	Trained Dataset
yolov8n.mlmodel	12.7	iOS14	Link
yolov8s.mlmodel	44.7	iOS14	Link
yolov8m.mlmodel	52.1	iOS14	Link
yolov8l.mlmodel	87.8	iOS14	Link
yolov8x.mlmodel	272.9	iOS14	Link
yolov5n.mlmodel	7.52	iOS13	Link	COCO
yolov5s.mlmodel	28.0	iOS13	Link	COCO
yolov5m.mlmodel	81.2	iOS13	Link	COCO
yolov5l.mlmodel	178.0	iOS13	Link	COCO
yolov5x.mlmodel	331.0	iOS13	Link	COCO
yolov5n6.mlmodel	12.8	iOS13	Link	COCO
yolov5s6.mlmodel	48.5	iOS13	Link	COCO
yolov5m6.mlmodel	137.0	iOS13	Link	COCO
yolov5l6.mlmodel	293.0	iOS13	Link	COCO
yolov5x6.mlmodel	537.0	iOS13	Link	COCO
YOLOv3.mlmodel	248.4	iOS12	Link	COCO
YOLOv3FP16.mlmodel	124.2	iOS12	Link	COCO
YOLOv3Int8LUT.mlmodel	62.2	iOS12	Link	COCO
YOLOv3Tiny.mlmodel	35.5	iOS12	Link	COCO
YOLOv3TinyFP16.mlmodel	17.8	iOS12	Link	COCO
YOLOv3TinyInt8LUT.mlmodel	8.9	iOS12	Link	COCO
MobileNetV2_SSDLite.mlmodel	9.3	iOS12	Link	COCO
ObjectDetector.mlmodel	63.7	iOS12	Link	6 Label Dataset

Trained Dataset Infos

COCO Dataset

• https://github.com/ultralytics/yolov5/blob/9da6d0f9f5bc37fa386b7b82d2a963f94650949a/data/coco.yaml#L17-L97

6 Label Dataset(Apple's DEMO)

• Bagel
• Banana
• Coffee
• Croissant
• Egg
• Waffle

Performance

Build Setting:
Xcoede > Build Settings > Apple Clang - Code Generation > Optimization Level > Fastest [-O3]

Infernece Time (ms)

Model vs. Device	14 Pro	13 Pro	12 Pro	11 Pro	XS	XS Max	XR	X	7+	7
yolov8n	15
yolov8s	29
yolov8m	37
yolov8l	45
yolov8x	51
yolov5n			24
yolov5s			29
yolov5m			39
yolov5l			38
yolov5x			69
yolov5n6			24
yolov5s6			34
yolov5m6			39
yolov5l6			41
yolov5x6			57
YOLOv3			45	83	108	93	100	356	569	561
YOLOv3FP16			44	84	104	89	101	348	572	565
YOLOv3Int8LUT			53	86	101	92	100	337	575	572
YOLOv3Tiny			36	44	46	41	47	106	165	168
YOLOv3TinyFP16			33	44	51	41	44	103	165	167
YOLOv3TinyInt8LUT			39	44	45	39	39	106	160	161
MobileNetV2_SSDLite			17	18	31	31	31	109	141	134
ObjectDetector			13	18	24	26	23	63	86	84

Total Time (ms)

Model vs. Device	14 Pro	13 Pro	12 Pro		11 Pro	XS	XS Max	XR	X	7+	7
yolov8n	15
yolov8s	31
yolov8m	39
yolov8l	47
yolov8x	52
yolov5n			26
yolov5s			31
yolov5m			41
yolov5l			39
yolov5x			72
yolov5n6			25
yolov5s6			36
yolov5m6			41
yolov5l6			42
yolov5x6			59
YOLOv3			46	84	108	93	100	357	569	561
YOLOv3FP16			45	85	104	89	101	348	572	565
YOLOv3Int8LUT			54	86	102	92	102	338	576	573
YOLOv3Tiny			37	45	46	42	48	106	166	169
YOLOv3TinyFP16			35	45	51	41	44	104	165	167
YOLOv3TinyInt8LUT			41	45	45	39	40	107	160	161
MobileNetV2_SSDLite			19	19	32	31	32	109	142	134
ObjectDetector			14	18	25	26	23	64	87	85

FPS

Model vs. Device	14 Pro	13 Pro	12 Pro		11 Pro	XS	XS Max	XR	X	7+	7
yolov8n	38
yolov8s	14
yolov8m	14
yolov8l	14
yolov8x	13
yolov5n			19
yolov5s			14
yolov5m			13
yolov5l			14
yolov5x			7
yolov5n6			19
yolov5s6			14
yolov5m6			13
yolov5l6			14
yolov5x6			13
YOLOv3			12	9	8	10	9	2	1	1
YOLOv3FP16			13	9	9	10	8	2	1	1
YOLOv3Int8LUT			14	9	9	10	9	2	1	1
YOLOv3Tiny			14	14	21	22	20	8	5	5
YOLOv3TinyFP16			14	14	19	23	21	9	5	5
YOLOv3TinyInt8LUT			11	14	21	24	23	8	5	5
MobileNetV2_SSDLite			19	29	23	23	23	8	6	6
ObjectDetector			17	29	23	23	24	14	10	11

See also

• motlabs/awesome-ml-demos-with-ios
  : The challenge using machine learning model created from tensorflow on iOS
• Machine Learning - Models - Apple Developer
• hollance/coreml-survival-guide
• vonholst/SSDMobileNet_CoreML
  : iOS project for object detection(SSDMobileNet V1) using Core ML.
• ultralytics/ultralytics
  : YOLOv8 repository
• ultralytics/yolov5
  : YOLOv5 repository