Skip to main content

Table 3 Comparison of FastCAT and CVAT by user. This table shows our comparison of the well-known CVAT annotation tool to our new annotation tool FastCAT in terms of quality of annotation and annotation speed. The quality metric is the F1-score. We count a TP if the drawn box matches the ground truth box more than 70 %

From: Fast machine learning annotation in the medical domain: a semi-automated video annotation tool for gastroenterologists

 

Quality (%)

Speed (SPF)

Total time (min)

Medical Experience

CVAT

FastCat

CVAT

FastCat

CVAT

FastCat

User 1

99.30

99.50

7.33

3.71

48.78

25.30

Low

User 2

98.85

98.90

3.47

1.88

23.38

13.70

Low

User 3

97.97

98.51

4.59

1.53

31.28

11.17

Low

User 4

98.93

99.75

5.12

2.57

33.96

16.53

Middle

User 5

98.53

98.83

5.41

2.49

37.00

18.10

Middle

User 6

98.52

99.23

4.04

3.24

27.90

24.95

Low

User 7

99.45

99.30

5.20

2.70

35.01

21.28

Middle

User 8

99.35

99.08

5.25

2.86

33.90

19.57

Low

User 9

99.12

98.54

4.12

2.25

27.12

14.99

Low

User 10

98.93

99.48

5.63

2.76

37.53

19.89

Low

Mean

98.98

99.03

5.79

2.93

33.59

18.55

Low