Sample autogenerated report¶
This section contains a CI sample report for running inference on a compiled model.
The CI is set up as follows:
Environment: Github Actions
Task: dog and cat breeds classification based on the Oxford-IIIT Pet Dataset,
Training framework: TensorFlow,
Compiler framework: TVM,
Target: CPU (using LLVM target on TVM).
Pet Dataset classification using TVM-compiled TensorFlow model¶
Commands used¶
Note
This section was generated using:
python -m kenning.scenarios.inference_tester \
kenning.modelwrappers.classification.tensorflow_pet_dataset.TensorFlowPetDatasetMobileNetV2 \
kenning.datasets.pet_dataset.PetDataset \
./build/local-cpu-tvm-tensorflow-classification.json \
--compiler-cls \
kenning.compilers.tvm.TVMCompiler \
--runtime-cls \
kenning.runtimes.tvm.TVMRuntime \
--model-path \
./kenning/resources/models/classification/tensorflow_pet_dataset_mobilenetv2.h5 \
--model-framework \
keras \
--target \
llvm \
--compiled-model-path \
./build/compiled-model.tar \
--opt-level \
3 \
--save-model-path \
./build/compiled-model.tar \
--target-device-context \
cpu \
--dataset-root \
./build/pet-dataset/ \
--inference-batch-size \
1 \
--download-dataset \
--verbosity \
INFO
python -m kenning.scenarios.render_report \
Pet Dataset classification using TVM-compiled TensorFlow model \
docs/source/generated/local-cpu-tvm-tensorflow-classification.md \
--root-dir \
docs/source/ \
--img-dir \
docs/source/generated/img \
--report-types \
performance \
classification \
--measurements \
build/local-cpu-tvm-tensorflow-classification.json
General information for build.local-cpu-tvm-tensorflow-classification.json¶
Model framework:
tensorflow ver. 2.9.3
Compiler framework:
tvm ver. 0.11.1
Input JSON:
{
"dataset": {
"type": "kenning.datasets.pet_dataset.PetDataset",
"parameters": {
"classify_by": "breeds",
"image_memory_layout": "NHWC",
"dataset_root": "build/pet-dataset",
"inference_batch_size": 1,
"download_dataset": true,
"external_calibration_dataset": null
}
},
"model_wrapper": {
"type": "kenning.modelwrappers.classification.tensorflow_pet_dataset.TensorFlowPetDatasetMobileNetV2",
"parameters": {
"model_path": "kenning/resources/models/classification/tensorflow_pet_dataset_mobilenetv2.h5"
}
},
"runtime": {
"type": "kenning.runtimes.tvm.TVMRuntime",
"parameters": {
"save_model_path": "build/compiled-model.tar",
"target_device_context": "cpu",
"target_device_context_id": 0,
"runtime_use_vm": false,
"disable_performance_measurements": true
}
},
"optimizers": [
{
"type": "kenning.compilers.tvm.TVMCompiler",
"parameters": {
"model_framework": "keras",
"target": "llvm",
"target_host": null,
"opt_level": 3,
"libdarknet_path": "/usr/local/lib/libdarknet.so",
"compile_use_vm": false,
"output_conversion_function": "default",
"conv2d_data_layout": "",
"conv2d_kernel_layout": "",
"use_fp16_precision": false,
"use_int8_precision": false,
"use_tensorrt": false,
"dataset_percentage": 0.25,
"compiled_model_path": "build/compiled-model.tar"
}
}
]
}