Sample AutoML report

This section contains a sample AutoML report generated during CI.

The CI is set up as follows:

AutoML statistics

  • Optimized metric: f1

  • The number of generated models: 62

  • The number of trained and evaluated models: 43

  • The number of successful training processes: 53

  • The number of models that caused a crash: 0

  • The number of models that failed due to the timeout: 1

  • The number of models that failed due to the too large size: 8

  • The number of models that failed due to incompatibility: 0

Training overview

Bokeh Plot

Figure 12 Loss value during AutoML training process

Bokeh Plot

Figure 13 Comparison of loss value across models

Summary of generated models

Bokeh Plot

Figure 14 Metrics of models trained by AutoML flow

Table 5 Summary of generated models’ parameters

Model ID

Number of layers

Optimized model size [KB]

Total parameters

Trainable parameters

3

7

15.5546875

2815

2814

4

10

49.73046875

11623

11622

5

17

33.08203125

7498

7497

6

21

33.4921875

7613

7612

7

27

61.02734375

14094

14093

8

14

50.16015625

11841

11840

9

21

35.60546875

7834

7833

10

17

37.4296875

8732

8731

11

23

54.3359375

11691

11690

12

21

30.37890625

7656

7655

13

17

33.13671875

7799

7798

14

12

37.5625

8227

8226

15

10

42.421875

10264

10263

16

13

45.5546875

10455

10454

17

8

19.8671875

4364

4363

18

27

47.5078125

10720

10719

19

10

42.234375

10052

10051

20

19

45.875

10707

10706

21

11

33.0234375

7276

7275

22

21

33.0234375

8266

8265

23

23

44.55859375

10318

10317

24

17

31.94921875

6520

6519

25

10

36.109375

7850

7849

26

14

53.28125

12688

12687

27

13

57.70703125

14126

14125

28

21

24.35546875

4676

4675

29

11

26.8515625

5762

5761

30

9

41.14453125

9488

9487

31

11

47.01171875

10084

10083

32

10

40.78125

9333

9332

33

17

49.8125

13830

13829

34

7

28.078125

6002

6001

35

17

54.1328125

13244

13243

36

10

14.88671875

2302

2301

37

12

53.57421875

13471

13470

38

13

21.20703125

4534

4533

39

9

39.09765625

10783

10782

40

10

33.23046875

8566

8565

41

25

64.33203125

14531

14530

42

11

27.7578125

5542

5541

43

14

44.0546875

8630

8629

44

19

46.0546875

10194

10193

45

14

54.5

12633

12632

46

19

42.7578125

9670

9669

47

25

37.8671875

7278

7277

48

10

40.92578125

8541

8540

49

25

41.97265625

8491

8490

50

21

36.33984375

8069

8068

51

13

28.80859375

7269

7268

52

21

25.45703125

4973

4972

53

11

33.83203125

8420

8419

Classification comparison

Comparison of inference time, F1 score and model size

Bokeh Plot

Figure 15 Model size, speed and quality summary. The F1 score of the model is presented on Y axis. The inference time of the model is presented on X axis. The size of the model is represented by the size of its point.

Table 6 Comparison of model inference time, accuracy and size

Model name

Mean Inference time [s]

Size [MB]

F1 score

workspace.automl-results.1234_46_5.0.measurements.json

0.003294

0.044

0.333333

workspace.automl-results.1234_24_1.6666666666666665.measurements.json

0.001036

0.034

0.235294

workspace.automl-results.1234_24_5.0.measurements.json

0.001037

0.034

0.235294

workspace.automl-results.1234_12_5.0.measurements.json

0.000842

0.032

0.250000

workspace.automl-results.1234_36_5.0.measurements.json

0.000592

0.017

0.250000

Detailed metrics comparison

Bokeh Plot

Figure 16 Radar chart representing the accuracy, precision and recall for models

Table 7 Summary of classification metrics for models

Model name

Accuracy

Mean precision

Mean sensitivity

G-mean

ROC AUC

F1 score

workspace.automl-results.1234_46_5.0.measurements.json

0.952000

0.731557

0.618697

0.496839

0.618697

0.333333

workspace.automl-results.1234_24_1.6666666666666665.measurements.json

0.948000

0.679592

0.577031

0.405667

0.577031

0.235294

workspace.automl-results.1234_24_5.0.measurements.json

0.948000

0.679592

0.577031

0.405667

0.577031

0.235294

workspace.automl-results.1234_12_5.0.measurements.json

0.952000

0.729675

0.579132

0.406529

0.579132

0.250000

workspace.automl-results.1234_36_5.0.measurements.json

0.952000

0.729675

0.579132

0.406529

0.579132

0.250000

Inference comparison

Performance metrics

Bokeh Application

Figure 17 Plot represents changes of inference time over time for all models.

Table 8 Summary of inference time metrics for models

Model name

Median [s]

Mean [s]

Standard deviation [s]

Maximum [s]

Minimum [s]

workspace.automl-results.1234_46_5.0.measurements.json

0.003289

0.003294

0.000181

0.003731

0.002794

workspace.automl-results.1234_24_1.6666666666666665.measurements.json

0.001035

0.001036

0.000007

0.001098

0.001016

workspace.automl-results.1234_24_5.0.measurements.json

0.001035

0.001037

0.000011

0.001149

0.001007

workspace.automl-results.1234_12_5.0.measurements.json

0.000841

0.000842

0.000011

0.000925

0.000822

workspace.automl-results.1234_36_5.0.measurements.json

0.000590

0.000592

0.000014

0.000687

0.000554

Mean comparison plots

Bokeh Plot

Figure 18 Violin chart representing distribution of values for performance metrics for models

Table 9 Performance metric for models

Model name

Inference time [s]

workspace.automl-results.1234_46_5.0.measurements.json

0.003294

workspace.automl-results.1234_24_1.6666666666666665.measurements.json

0.001036

workspace.automl-results.1234_24_5.0.measurements.json

0.001037

workspace.automl-results.1234_12_5.0.measurements.json

0.000842

workspace.automl-results.1234_36_5.0.measurements.json

0.000592
Last update: 2026-05-29