Dynamic Shape with a Custom Node¶

Introduction¶

This guide shows how to configure a simple Directed Acyclic Graph (DAG) with a custom node that performs input resizing before passing input data to the model.

The node below is provided as a demonstration. See instructions for how to build and use the custom node: Image Transformation.

To run inference with this setup, we will use the following:

Example client in Python face_detection.py that can be used to request inference on with the desired input shape.
An example face_detection_retail_0004 model.

When using the face_detection_retail_0004 model with the face_detection.py script, images are loaded and resized to the desired width and height. Then the output from the server is processed and inference results are displayed with bounding boxes drawn around the detected faces.

Steps¶

Clone OpenVINO Model Server GitHub repository and enter model_server directory.

git clone https://github.com/openvinotoolkit/model_server.git
cd model_server

Download the Pretrained Model¶

Download the model files and store them in the models directory:

mkdir -p models/face_detection/1
curl https://storage.openvinotoolkit.org/repositories/open_model_zoo/2022.1/models_bin/3/face-detection-retail-0004/FP32/face-detection-retail-0004.bin https://storage.openvinotoolkit.org/repositories/open_model_zoo/2022.1/models_bin/3/face-detection-retail-0004/FP32/face-detection-retail-0004.xml -o models/face_detection/1/face-detection-retail-0004.bin -o models/face_detection/1/face-detection-retail-0004.xml

Pull the Latest Model Server Image¶

Pull the latest version of OpenVINO Model Server from Docker Hub :

docker pull openvino/model_server:latest

Build a Custom Node¶

Go to the custom node examples directory:
```
cd src/custom_nodes
```

Build the custom node:

# replace to 'redhat` if using UBI base image
export BASE_OS=ubuntu

make BASE_OS=${BASE_OS} NODES=image_transformation

Copy the custom node to the models repository:

cp lib/${BASE_OS}/libcustom_node_image_transformation.so ../../models/libcustom_node_image_transformation.so

Create Model Server Configuration File¶

Go to the models directory:

cd ../../models

Create a new file named config.json in the models directory:

echo '{
    "model_config_list": [
        {
            "config": {
                "name": "face_detection_retail",
                "base_path": "/models/face_detection"
            }
        }
    ],
    "custom_node_library_config_list": [
        {"name": "image_transformation",
            "base_path": "/models/libcustom_node_image_transformation.so"}
    ],
    "pipeline_config_list": [
        {
            "name": "face_detection",
            "inputs": ["data"],
            "nodes": [
                {
                    "name": "image_transformation_node",
                    "library_name": "image_transformation",
                    "type": "custom",
                    "params": {
                        "target_image_width": "300",
                        "target_image_height": "300",

                        "mean_values": "[123.675,116.28,103.53]",
                        "scale_values": "[58.395,57.12,57.375]",

                        "original_image_color_order": "BGR",
                        "target_image_color_order": "BGR",

                        "original_image_layout": "NCHW",
                        "target_image_layout": "NCHW"
                    },
                    "inputs": [
                        {"image": {
                                "node_name": "request",
                                "data_item": "data"}}],
                    "outputs": [
                        {"data_item": "image",
                            "alias": "transformed_image"}]
                },
                {
                    "name": "face_detection_node",
                    "model_name": "face_detection_retail",
                    "type": "DL model",
                    "inputs": [
                        {"data":
                            {
                             "node_name": "image_transformation_node",
                             "data_item": "transformed_image"
                            }
                        }
                    ],
                    "outputs": [
                        {"data_item": "detection_out",
                         "alias": "face_detection_output"}
                    ]
                }
            ],
            "outputs": [
                {"detection_out": {
                        "node_name": "face_detection_node",
                        "data_item": "face_detection_output"}}
            ]
        }
    ]
}' >> config.json

Start Model Server Container with Downloaded Model¶

Start the container with the image pulled in the previous step and mount <models_dir> :

docker run --rm -d -v ${PWD}:/models -p 9000:9000 openvino/model_server:latest --config_path /models/config.json --port 9000

Run the Client¶

cd ../demos/face_detection/python
virtualenv .venv
. .venv/bin/activate
pip install -r ../../common/python/requirements.txt
mkdir results_500x500

python face_detection.py --grpc_port 9000 --width 500 --height 500 --input_images_dir ../../common/static/images/people --output_dir results_500x500 --model_name face_detection

mkdir results_600x400

python face_detection.py --grpc_port 9000 --width 600 --height 400 --input_images_dir ../../common/static/images/people --output_dir results_600x400 --model_name face_detection

Results of running the client will be available in directories specified in --output_dir