Sample AutoML report¶

This section contains a sample AutoML report generated during CI.

The CI is set up as follows:

Environment: Github Actions
Task: Anomaly Detection
AutoML framework: Auto-PyTorch
Compiler framework: TFLite,
Platform: MAX32690 Evaluation Kit

AutoML statistics¶

Optimized metric: f1
The number of generated models: 51
The number of trained and evaluated models: 38
The number of successful training processes: 46
The number of models that caused a crash: 0
The number of models that failed due to the timeout: 1
The number of models that failed due to the too large size: 4
The number of models that failed due to incompatibility: 0

Training overview¶

Bokeh Plot

Figure 12 Loss value during AutoML training process¶

Bokeh Plot

Figure 13 Comparison of loss value across models¶

Summary of generated models¶

Bokeh Plot

Figure 14 Metrics of models trained by AutoML flow¶

Table 5 Summary of generated models’ parameters¶
Model ID	Number of layers	Optimized model size [KB]	Total parameters	Trainable parameters
3	7	15.78515625	2815	2814
4	10	49.8828125	11623	11622
5	17	33.10546875	7498	7497
6	21	33.703125	7613	7612
7	27	61.05078125	14094	14093
8	14	50.42578125	11841	11840
9	21	35.796875	7834	7833
10	17	37.453125	8732	8731
11	23	54.33984375	11691	11690
12	21	30.58984375	7656	7655
13	17	33.16015625	7799	7798
14	12	37.78125	8227	8226
15	10	42.41796875	10264	10263
16	13	45.55859375	10455	10454
17	8	19.86328125	4364	4363
18	27	47.53125	10720	10719
19	21	35.453125	8204	8203
20	9	17.77734375	3190	3189
21	13	32.6171875	7850	7849
22	12	29.12109375	5864	5863
23	15	34.7578125	8009	8008
24	19	45.32421875	10190	10189
25	11	30.05859375	6573	6572
26	25	57.24609375	13239	13238
27	13	23.9140625	5428	5427
28	11	30.703125	7744	7743
29	13	52.5625	13700	13699
30	21	32.3671875	7593	7592
31	11	44.21875	9580	9579
32	9	22.546875	5868	5867
33	17	50.02734375	12217	12216
34	12	51.5546875	13437	13436
35	19	31.24609375	6620	6619
36	12	40.328125	8949	8948
37	17	36.046875	7788	7787
38	19	47.15625	11050	11049
39	13	28.4375	6345	6344
40	11	42.58984375	10169	10168
41	9	39.98828125	9895	9894
42	9	12.5859375	2351	2350
43	17	20.21875	3853	3852
44	21	42.87890625	9632	9631
45	15		11264	11263

Classification comparison¶

Comparison of inference time, F1 score and model size¶

Bokeh Plot

Figure 15 Model size, speed and quality summary. The F1 score of the model is presented on Y axis. The inference time of the model is presented on X axis. The size of the model is represented by the size of its point.¶

Table 6 Comparison of model inference time, accuracy and size¶
Model name	Mean Inference time [s]	Size [MB]	F1 score
automl_conf_0	0.003192	0.043	0.571429
automl_conf_1	0.000834	0.032	0.571429
automl_conf_2	0.000834	0.032	0.571429
automl_conf_3	0.000949	0.037	0.533333
automl_conf_4	0.000410	0.016	0.533333

Detailed metrics comparison¶

Bokeh Plot

Figure 16 Radar chart representing the accuracy, precision and recall for models¶

Table 7 Summary of classification metrics for models¶
Model name	Accuracy	Mean precision	Mean sensitivity	G-mean	ROC AUC	F1 score
automl_conf_0	0.976000	0.825137	0.745868	0.704179	0.745868	0.571429
automl_conf_1	0.976000	0.825137	0.745868	0.704179	0.745868	0.571429
automl_conf_2	0.976000	0.825137	0.745868	0.704179	0.745868	0.571429
automl_conf_3	0.972000	0.777484	0.743802	0.702710	0.743802	0.533333
automl_conf_4	0.972000	0.777484	0.743802	0.702710	0.743802	0.533333

Inference comparison¶

Performance metrics¶

Bokeh Application

Figure 17 Plot represents changes of inference time over time for all models.¶

Table 8 Summary of inference time metrics for models¶
Model name	Maximum [s]	Median [s]	Standard deviation [s]	Minimum [s]	Mean [s]
automl_conf_0	0.003539	0.003176	0.000141	0.002773	0.003192
automl_conf_1	0.000917	0.000832	0.000014	0.000801	0.000834
automl_conf_2	0.000921	0.000833	0.000012	0.000807	0.000834
automl_conf_3	0.001023	0.000946	0.000014	0.000915	0.000949
automl_conf_4	0.000503	0.000408	0.000013	0.000376	0.000410

Mean comparison plots¶

Bokeh Plot

Figure 18 Violin chart representing distribution of values for performance metrics for models¶

Table 9 Performance metric for models¶
Model name	Inference time [s]
automl_conf_0	0.003192
automl_conf_1	0.000834
automl_conf_2	0.000834
automl_conf_3	0.000949
automl_conf_4	0.000410

Last update: 2026-08-21