Run yolov5s on Android device!

Related tags

App yolov5s_android
Overview

yolov5s_android 🚀

The implementation of yolov5s on android for the yolov5s export contest.
Download the latest android apk from release and install your device.

Environment

  • Host Ubuntu18.04
  • Docker
    • Tensorflow 2.4.0
    • PyTorch 1.7.0
    • OpenVino 2021.3
  • Android App
    • Android Studio 4.2.1
    • minSdkVersion 28
    • targetSdkVersion 29
    • TfLite 2.4.0
  • Android Device
    • Xiaomi Mi11 (Storage 128GB/ RAM8GB)
    • OS MUI 12.5.8

We use docker container for host evaluation and model conversion.

git clone --recursive https://github.com/lp6m/yolov5s_android
cd yolov5s_android
docker build ./ -f ./docker/Dockerfile  -t yolov5s_android
docker run -it --gpus all -v `pwd`:/workspace yolov5s_anrdoid bash

Files

  • ./app
    • Android application.
    • To build application by yourself, copy ./tflite_model/*.tflite to app/tflite_yolov5_test/app/src/main/assets/, and build on Android Studio.
    • The app can perform inference with various configurations of input size, inference accuracy, and model accuracy.
    • For 'Open Directory Mode', save the detected bounding boxes results as a json file in coco format.
    • Realtime deteciton from camera image (precision and input size is fixed to int8/320). Achieved FPS is about 15FPS on Mi11.
    • NOTE Please select image/directory as an absolute path from 'Device'. The app does not support select image/directory from 'Recent' in some devices.
  • ./benchmark
  • ./convert_model
    • Model conversion guide and model quantization script.
  • ./docker
    • Dockerfile for the evaluation and model conversion environment.
  • ./host
    • detect.py : Run detection for image with TfLite model on host environment.
    • evaluate.py: Run evaluation with coco validation dataset and inference results.
  • ./tflite_model
    • Converted TfLite Model.

Performance

Latency

These results are measured on Xiaomi Mi11.
Please refer benchmark/README.md about the detail of benchmark command.
The latency does not contain the pre/post processing time and data transfer time.

float32 model

delegate 640x640 [ms] 320x320 [ms]
None (CPU) 249 61
NNAPI (qti-gpu, fp32) 156 112
NNAPI (qti-gpu, fp16) 92 79

int8 model

We tried to accelerate the inference process by using NNAPI (qti-dsp) and offload calculation to Hexagon DSP, but it doesn't work for now. Please see here in detail.

delegate 640x640 [ms] 320x320 [ms]
None (CPU) 95 23
NNAPI (qti-default) Not working Not working
NNAPI (qti-dsp) Not working Not working

Accuracy

Please refer host/README.md about the evaluation method.
We set conf_thresh=0.25 and iou_thresh=0.45 for nms parameter.

device, model, delegate 640x640 mAP 320x320 mAP
host GPU (Tflite + PyTorch, fp32) 27.8 26.6
host CPU (Tflite + PyTorch, int8) 26.6 25.5
NNAPI (qti-gpu, fp16) 28.5 26.8
CPU (int8) 27.2 25.8

Model conversion

This project focuses on obtaining a tflite model by model conversion from PyTorch original implementation, rather than doing own implementation in tflite.
We convert models in this way: PyTorch -> ONNX -> OpenVino -> TfLite.
To convert the model from OpenVino to TfLite, we use openvino2tensorflow. Please refer convert_model/README.md about the model conversion.

Comments
  • Custom model & dataset

    Custom model & dataset

    Thanks for your good work. I have a customized model at two detection scale trained with four class.. How do I modify the code to fit the application? Moreso, how to set up --tfds_root for quantize.py?

    opened by larrywal-express 7
  • Cannot deploy yolov5s.pt(v6.1) on android mobile phone

    Cannot deploy yolov5s.pt(v6.1) on android mobile phone

    I have convered yolov5s.pt to yolov5s_fp32_640.tflite, but I can't deploy it on android mobile phone. The version of yolov5s.pt is 6.1.

    step1: python export.py --weights weights/yolov5s.pt --include onnx --simplify (the image_size of export.py is [640,640]). step2: python3 /opt/intel/openvino_2021.4.752/deployment_tools/model_optimizer/mo.py --input_model yolov5s.onnx --input_shape [1,3,640,640] --output_dir ./openvino --data_type FP32 --output Conv_198,Conv_248,Conv_298 step3: openvino2tensorflow --model_path ./openvino/yolov5s.xml --model_output_path tflite --output_pb --output_saved_model --output_no_quant_float32_tflite step4: rename file as "yolov5s_fp32_640.tflite", and put it into android/app/src/main/assets

    but it can't run. “Cannot copy from a TensorFlowLite tensor (Identity) with shape [1,20,20,255] to a java object with shape [1,80,80,255]”

    opened by qiujianchen 4
  • Run inference in GUI gives me `Inference failed`

    Run inference in GUI gives me `Inference failed`

    I managed to build my app and import a model but when I press RUN INFERENCE I get:

    Inference failed : Cannot copy from a TensorFlowLite tensor (StatufulPartitionedCall:2) with shape [1, 20, 20, 51] to a Java object with shape [1, 40 , 40, 51]
    
    opened by mikel-brostrom 3
  • node wrong

    node wrong

    opened by ppogg 3
  • RuntimeError: Max and min for dynamic tensors should be recorded during calibration: Failed for tensor inputs Empty min/max for tensor inputs

    RuntimeError: Max and min for dynamic tensors should be recorded during calibration: Failed for tensor inputs Empty min/max for tensor inputs

    I run command in colab: !python3 quantize.py --image_dir /contnet/dataset/coco128/coco128/images/train2017/ --calib_num 93 --input_size 640

    the folder contains *.jpg & *.txt

    opened by yspisgod 2
  • How can I adjust the output of my TFLite exported model in order to make it work with your app

    How can I adjust the output of my TFLite exported model in order to make it work with your app

    Hi!

    I tried my own tflite model trained with yolov5 on your app, but get the flow error:

    Cannot copy from a TensorFlowLite tensor (StatefulPartitionedCall:0) with shape [1, 6300, 7] to a Java object with shape [1, 40, 40, 255].

    your models have 4 outputs, but my models only have 3 outputs, what should i do to make my model work.

    Thanks!

    opened by ppl520zfl 1
  • no node named Conv_253

    no node named Conv_253

    when i run python3 /opt/intel/openvino_2021.3.394/deployment_tools/model_optimizer/mo.py --input_model yolov5s.onnx --input_shape [1,3,640,640] --output_dir ./openvino --data_type FP32 --output Conv_253,Conv_302,Conv_351

    it seems that "-output" is unnecessary

    opened by yspisgod 1
  • TfliteRunner custom context

    TfliteRunner custom context

    How to change TfliteRunner for working with custom Activity in manifest. I'am using MAppication class for starting app. MainActivity open in MApplication image

    opened by alexmihalyk23 1
  • When the screen is horizontal, the detection box is inaccurate

    When the screen is horizontal, the detection box is inaccurate

    I tried to run this project on the Android tablet, and found that the default horizontal screen was inaccurate. Then I switched back to the phone and rotated it to the horizontal screen, and the same inaccuracy occurred. I wonder if there is any good way to support horizontal screen detection

    opened by Chopin68 1
  • Higher confidence but smaller bbox size

    Higher confidence but smaller bbox size

    I have an interesting inference output. The image below is using conf=0.84 and IoU=0.30 image First, the bboxes are smaller than the objects. Second, somehow when I have more than one bbox to the same object as shown in the image for bbox yellow and blue, note that in the yellow case, the higher prediction 93.82 the bbox is smaller than the 89.70 that has a bigger bbox. The same happens with blue case where the higher prediction 92.96 the bbox is smaller than the 91.48 has a bigger bbox. If I decrease IoU or increase the confidence, I will get only one bbox for yellow and blue with higher conf but the ones removed had a better bbox size when compared with their objects. Why is the algorithm assigning lower conf to bboxes with a larger size (near the object size) and higher conf to bboxes with smaller sizes? Any hint on this?

    opened by zenetio 1
Releases(v1.0.2)
Owner
null
Android Studio project wrapper around the Elixir TodoApp Desktop app to run on Android including the Erlang runtime

TodoApp Android: An Android Sample App This Android Studio project wraps the Desktop Sample App to run on an Android phone. How to build & run Install

elixir-desktop 78 Dec 10, 2022
This component allows you to interact with remote machine via SSH. You can execute commands or run scripts located on that machine

th2 act ssh (1.2.0) Overview Provides user with the ability to execute specified scripts or commands with the parameter he needs Custom resources for

th2 1 Dec 18, 2022
An App based on MVVM architecture to track & store a user's runs using Google Maps, with options to view & sort the runs as per the user's choice along the with option to run the app in background.

An App based on MVVM architecture to track & store a user's runs using Google Maps, with options to view & sort the runs as per the user's choice along the with option to run the app in background.

Harshit Maheshwari 1 Jun 9, 2022
Source code of JekyllEx Android App which can manage your Jekyll blog directly from your Android device!

JekyllEx Android App Built with ❤︎ by Gourav Khunger ?? Introduction JekyllEx is an Android App that allows you to manage a Jekyll Blog directly from

JekyllEx 24 Nov 8, 2022
Using Android device to uninstall iPhone apps

Using Android device to uninstall iPhone apps

chi 23 Nov 6, 2022
Android library for finding connected devices on same WiFi network. It can provide IP Address, device name, MAC Address and vendor names.

Android WiFi Tools Android library for finding connected devices on the same WiFi network. It can provide IP Addresses, device names, MAC Address and

Tej Magar 5 Nov 16, 2022
Manage your torrents from your Android device with Transdroid.

All popular clients are supported: µTorrent, Transmission, rTorrent, Vuze, Deluge, BitTorrent 6, qBittorrent, and many more. You can view and manage running torrents and individual files. Adding is easy via the integrated search or RSS feeds (full version required). Monitor progress using the home screen widget or background alarm service.

Eric Kok 1.1k Jan 5, 2023
On-Device ID Card & Passport & Driver License Recognition SDK for Android

ID Card, Passport, Driver License Recognition SDK for Android On-Device & Offline, Robust, ID Card, Passport, Driver License OCR SDK for Android   ??

FaceOnLive 224 Dec 31, 2022
Tournant is a Gourmand/Gourmet recipe parser which serves your dishes on your Android device

Tournant Gourmand/Gourmet recipe parser Website: tournant.zimbelstern.eu Tournant serves your favorite dishes on your Android device! With Tournant, y

null 5 Dec 23, 2022
Sanctuary relies on the Android Work Profile APIs to create a self-contained work profile on a user's personal device.

Sanctuary relies on the Android Work Profile APIs to create a self-contained work profile on a user's personal device. Managed apps, data, and management policies are restricted to the work profile, keeping them secure and separate from personal data while maintaining user privacy.

Jonathan Odul 1 Dec 15, 2021
Bitcoin Wallet - a standalone Bitcoin payment app for your Android device

Bitcoin Wallet app for your Android device. Standalone Bitcoin node, no centralized backend required Google Colab

DE MINING 0 Feb 25, 2022
Monero PocketNode - A Monero node for your Android Device

Monero PocketNode - A Monero node for your Android Device

CRYPTO GRAMPY™ 24 Dec 23, 2022
Helps to find your surrounding's light value (for android only) using device light sensor

react-native-ambient-light-sensor Helps to find your surrounding's light value (in lux unit) (for android only) using device light sensor Installation

Shayan Chatterjee 4 Aug 26, 2022
An Android mobile app for viewing device screen in your web browser

Screen Stream over HTTP An Android mobile app for viewing device screen in your web browser. Developed by Dmitriy Krivoruchko · If there are any issue

Dmitriy Krivoruchko 1.1k Dec 31, 2022
AdpPad is a GUI application to use adb. adb is a command line tool to control android device.

AdbPad AdpPad is a GUI application to use adb. adb is a command line tool to control android device. Demo.mp4 ✨ Feature Observe a connected android de

katz 31 Dec 30, 2022
Scans all the SMS of an Android device for any known domain that is related with the spyware Predator by Cytrox / Intellexa.

Συχνές ερωτήσεις και απαντήσεις Τι είναι το Predator; Ποιο είναι το ρίσκο μου να έχω δεχθεί επίθεση με το Predator; Πώς δουλεύει η εφαρμογή SMS Predat

Andronikos Koutroumpelis 14 Dec 6, 2022
A library for Android to get the market name of the device.

GoogleSupportedDevices A library for Android to get the market name of the device. Getting Started Include the library in your build.gradle. implement

David González 7 Dec 4, 2022
Communicating between Wear OS and Android device using the OpWear module and a sample of displaying real-time camera on the watch and sending commands to the mobile by Wear OS.

OpWear-Cam Communicating between Wear OS and Android device using the OpWear module and a sample of displaying real-time camera on the watch and sendi

AmirHosseinAghajari 6 Nov 8, 2022
An app to view libraries used in apps in your device.

An app to view libraries used in apps in your device.

Absinthe 9 Dec 24, 2022