Sample autogenerated report

This section contains a CI sample report for running inference on a compiled model.

The CI is set up as follows:

  • Environment: Github Actions

  • Task: dog and cat breeds classification based on the Oxford-IIIT Pet Dataset,

  • Training framework: TensorFlow,

  • Compiler framework: TVM,

  • Target: CPU (using LLVM target on TVM).

Pet Dataset classification using TVM-compiled TensorFlow model

Commands used

Note

This section was generated using:

python -m kenning.scenarios.inference_tester \
    --modelwrapper-cls \
        kenning.modelwrappers.classification.tensorflow_pet_dataset.TensorFlowPetDatasetMobileNetV2 \
    --dataset-cls \
        kenning.datasets.pet_dataset.PetDataset \
    --measurements \
        ./build/local-cpu-tvm-tensorflow-classification.json \
    --compiler-cls \
        kenning.compilers.tvm.TVMCompiler \
    --runtime-cls \
        kenning.runtimes.tvm.TVMRuntime \
    --model-path \
        kenning:///models/classification/tensorflow_pet_dataset_mobilenetv2.h5 \
    --model-framework \
        keras \
    --target \
        llvm \
    --compiled-model-path \
        ./build/compiled-model.tar \
    --opt-level \
        3 \
    --save-model-path \
        ./build/compiled-model.tar \
    --target-device-context \
        cpu \
    --dataset-root \
        ./build/pet-dataset/ \
    --inference-batch-size \
        1 \
    --verbosity \
        INFO

General information for build.local-cpu-tvm-tensorflow-classification.json

Model framework:

  • tensorflow ver. 2.9.3

Compiler framework:

  • tvm ver. 0.11.1

Input JSON:

{
    "dataset": {
        "type": "kenning.datasets.pet_dataset.PetDataset",
        "parameters": {
            "classify_by": "breeds",
            "image_memory_layout": "NHWC",
            "dataset_root": "build/pet-dataset",
            "inference_batch_size": 1,
            "download_dataset": true,
            "force_download_dataset": false,
            "external_calibration_dataset": null,
            "split_fraction_test": 0.2,
            "split_fraction_val": null,
            "split_seed": 1234
        }
    },
    "model_wrapper": {
        "type": "kenning.modelwrappers.classification.tensorflow_pet_dataset.TensorFlowPetDatasetMobileNetV2",
        "parameters": {
            "model_path": "/home/runner/.kenning/models/classification/tensorflow_pet_dataset_mobilenetv2.h5"
        }
    },
    "runtime": {
        "type": "kenning.runtimes.tvm.TVMRuntime",
        "parameters": {
            "save_model_path": "/home/runner/work/kenning/kenning/build/compiled-model.tar",
            "target_device_context": "cpu",
            "target_device_context_id": 0,
            "runtime_use_vm": false,
            "disable_performance_measurements": false
        }
    },
    "optimizers": [
        {
            "type": "kenning.compilers.tvm.TVMCompiler",
            "parameters": {
                "model_framework": "keras",
                "target": "llvm",
                "target_host": null,
                "opt_level": 3,
                "libdarknet_path": "/usr/local/lib/libdarknet.so",
                "compile_use_vm": false,
                "output_conversion_function": "default",
                "conv2d_data_layout": "",
                "conv2d_kernel_layout": "",
                "use_fp16_precision": false,
                "use_int8_precision": false,
                "use_tensorrt": false,
                "dataset_percentage": 0.25,
                "compiled_model_path": "/home/runner/work/kenning/kenning/build/compiled-model.tar"
            }
        }
    ]
}

Inference performance metrics for build.local-cpu-tvm-tensorflow-classification.json

Inference time

Bokeh Application