Skip to content

Latest commit

 

History

History
110 lines (79 loc) · 3.7 KB

README.md

File metadata and controls

110 lines (79 loc) · 3.7 KB
Raw

PyTorch ResNex101 32x16d inference

Description

This document has instructions for running ResNex101 32x16d inference using Intel-optimized PyTorch.

Bare Metal

General setup

Follow link to install Conda and build Pytorch, IPEX, TorchVison and Jemalloc.

Model Specific Setup

  • Set Jemalloc Preload for better performance

    The jemalloc should be built from the General setup section.

    export LD_PRELOAD="path/lib/libjemalloc.so":$LD_PRELOAD
export MALLOC_CONF="oversize_threshold:1,background_thread:true,metadata_thp:auto,dirty_decay_ms:9000000000,muzzy_decay_ms:9000000000"

  • Set IOMP preload for better performance

    IOMP should be installed in your conda env from the General setup section.

    export LD_PRELOAD=path/lib/libiomp5.so:$LD_PRELOAD

  • Set ENV to use AMX if you are using SPR

    export DNNL_MAX_CPU_ISA=AVX512_CORE_AMX

Quick Start Scripts

Script name Description
inference_realtime.sh Runs multi instance realtime inference using 4 cores per instance with synthetic data for the specified precision (fp32, avx-fp32, int8, avx-int8, bf16, or bf32).
inference_throughput.sh Runs multi instance batch inference using 1 instance per socket with synthetic data for the specified precision (fp32, avx-fp32m int8, avx-int8, bf16, or bf32).
accuracy.sh Measures the inference accuracy (providing a DATASET_DIR environment variable is required) for the specified precision (fp32, avx-fp32, int8, avx-int8, bf16, or bf32).

Note: The avx-int8 and avx-fp32 precisions run the same scripts as int8 and fp32, except that the DNNL_MAX_CPU_ISA environment variable is unset. The environment variable is otherwise set to DNNL_MAX_CPU_ISA=AVX512_CORE_AMX.

Datasets

ImageNet

The ImageNet validation dataset is used to run ResNex101 32x16d accuracy tests.

Download and extract the ImageNet2012 dataset from http://www.image-net.org/, then move validation images to labeled subfolders, using the valprep.sh shell script

A after running the data prep script, your folder structure should look something like this:

imagenet
└── val
    ├── ILSVRC2012_img_val.tar
    ├── n01440764
    │   ├── ILSVRC2012_val_00000293.JPEG
    │   ├── ILSVRC2012_val_00002138.JPEG
    │   ├── ILSVRC2012_val_00003014.JPEG
    │   ├── ILSVRC2012_val_00006697.JPEG
    │   └── ...
    └── ...

The folder that contains the val directory should be set as the DATASET_DIR (for example: export DATASET_DIR=/home/<user>/imagenet).

Run the model

Follow the instructions above to setup your bare metal environment, download and preprocess the dataset, and do the model specific setup. Once all the setup is done, the Model Zoo can be used to run a quickstart script. Ensure that you have enviornment variables set to point to the dataset directory and an output directory.

# Clone the model zoo repo and set the MODEL_DIR
git clone https://github.com/IntelAI/models.git
cd models
export MODEL_DIR=$(pwd)

# Env vars
export DATASET_DIR=<path_to_Imagenet_Dataset>
export OUTPUT_DIR=<Where_to_save_log>
export PRECISION=<precision to run>

# Run a quickstart script
cd ${MODEL_DIR}/quickstart/image_recognition/pytorch/resnext-32x16d/inference/cpu
bash inference_realtime.sh

License

LICENSE