Log in Help
Print
Home 〉 sale 〉 rnti-09 〉 cross-validation 〉 stars 〉 root 〉 train_test.txt
 
Configuration File=file:/home/saggion/temp/ml-opinion/cross-validation/stars/root/config.xml


*************************
A new session for NLP learning is starting.

No chunk length statistics list file in initialisation phrase.
Pre-processing the 89 documents...
Learning starts.
For the information about this learning see the log file /home/saggion/temp/ml-opinion/cross-validation/stars/root/savedFiles/logFileForNLPLearning.save
The number of threads used is 1
** Training mode:
time for NLP features: 3
time for fv: 1
time for filtering: 0
time for NLP training: 2
This learning session finished!.
Pre-processing the 22 documents...
Learning starts.
For the information about this learning see the log file /home/saggion/temp/ml-opinion/cross-validation/stars/root/savedFiles/logFileForNLPLearning.save
The number of threads used is 1
** Application mode for document from 0 to 22(not included):
This learning session finished!.


------------------------------------------------

annotation sets:
*Output*
*Key*

For the document: review-11_517.xml_0045B
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (12.0, 0.0, 6.0, 3.0);  (precision, recall, F1)= (0.6666667, 0.8, 0.72727275);  Lenient: (0.6666667, 0.8, 0.72727275)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (12.0, 0.0, 6.0, 3.0);  (precision, recall, F1)= (0.6666667, 0.8, 0.72727275);  Lenient: (0.6666667, 0.8, 0.72727275)

For each pair of annotators, and for each label:
(0,1), label= 3_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 2.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 3.0, 0.0);  (precision, recall, F1)= (0.25, 1.0, 0.4);  Lenient: (0.25, 1.0, 0.4)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (11.0, 0.0, 1.0, 3.0);  (precision, recall, F1)= (0.9166667, 0.78571427, 0.84615386);  Lenient: (0.9166667, 0.78571427, 0.84615386)


For the document: review-10_218.xml_0045C
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: review-10_90.xml_0045D
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: review-11_135.xml_0045E
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.5, 0.5, 0.5);  Lenient: (0.5, 0.5, 0.5)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.5, 0.5, 0.5);  Lenient: (0.5, 0.5, 0.5)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (1.0, 0.5, 0.6666667);  Lenient: (1.0, 0.5, 0.6666667)


For the document: review-11_269.xml_0045F
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:

For the document: review-10_233.xml_00460
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (15.0, 0.0, 6.0, 4.0);  (precision, recall, F1)= (0.71428573, 0.7894737, 0.75000006);  Lenient: (0.71428573, 0.7894737, 0.75000006)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (15.0, 0.0, 6.0, 4.0);  (precision, recall, F1)= (0.71428573, 0.7894737, 0.75000006);  Lenient: (0.71428573, 0.7894737, 0.75000006)

For each pair of annotators, and for each label:
(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 4.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (15.0, 0.0, 2.0, 4.0);  (precision, recall, F1)= (0.88235295, 0.7894737, 0.83333343);  Lenient: (0.88235295, 0.7894737, 0.83333343)


For the document: review-10_172.xml_00461
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: review-10_484.xml_00462
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (13.0, 0.0, 2.0, 0.0);  (precision, recall, F1)= (0.8666667, 1.0, 0.92857146);  Lenient: (0.8666667, 1.0, 0.92857146)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (13.0, 0.0, 2.0, 0.0);  (precision, recall, F1)= (0.8666667, 1.0, 0.92857146);  Lenient: (0.8666667, 1.0, 0.92857146)

For each pair of annotators, and for each label:
(0,1), label= 2_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 3_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (2.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.6666667, 1.0, 0.8);  Lenient: (0.6666667, 1.0, 0.8)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (10.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: review-10_556.xml_00463
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (14.0, 0.0, 3.0, 1.0);  (precision, recall, F1)= (0.8235294, 0.93333334, 0.875);  Lenient: (0.8235294, 0.93333334, 0.875)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (14.0, 0.0, 3.0, 1.0);  (precision, recall, F1)= (0.8235294, 0.93333334, 0.875);  Lenient: (0.8235294, 0.93333334, 0.875)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 2.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (14.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (1.0, 0.93333334, 0.9655172);  Lenient: (1.0, 0.93333334, 0.9655172)


For the document: review-10_54.xml_00464
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (7.0, 0.0, 4.0, 3.0);  (precision, recall, F1)= (0.6363636, 0.7, 0.6666667);  Lenient: (0.6363636, 0.7, 0.6666667)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (7.0, 0.0, 4.0, 3.0);  (precision, recall, F1)= (0.6363636, 0.7, 0.6666667);  Lenient: (0.6363636, 0.7, 0.6666667)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 3.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (6.0, 0.0, 1.0, 3.0);  (precision, recall, F1)= (0.85714287, 0.6666667, 0.75);  Lenient: (0.85714287, 0.6666667, 0.75)


For the document: review-10_294.xml_00465
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (17.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.9444444, 0.9444444, 0.9444444);  Lenient: (0.9444444, 0.9444444, 0.9444444)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (17.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.9444444, 0.9444444, 0.9444444);  Lenient: (0.9444444, 0.9444444, 0.9444444)

For each pair of annotators, and for each label:
(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (2.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.6666667, 1.0, 0.8);  Lenient: (0.6666667, 1.0, 0.8)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (15.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (1.0, 0.9375, 0.9677419);  Lenient: (1.0, 0.9375, 0.9677419)


For the document: review-10_7.xml_00466
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (22.0, 0.0, 2.0, 1.0);  (precision, recall, F1)= (0.9166667, 0.95652175, 0.9361702);  Lenient: (0.9166667, 0.95652175, 0.9361702)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (22.0, 0.0, 2.0, 1.0);  (precision, recall, F1)= (0.9166667, 0.95652175, 0.9361702);  Lenient: (0.9166667, 0.95652175, 0.9361702)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (22.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.95652175, 0.95652175, 0.95652175);  Lenient: (0.95652175, 0.95652175, 0.95652175)


For the document: review-11_64.xml_00467
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 3.0, 2.0);  (precision, recall, F1)= (0.5, 0.6, 0.54545456);  Lenient: (0.5, 0.6, 0.54545456)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 3.0, 2.0);  (precision, recall, F1)= (0.5, 0.6, 0.54545456);  Lenient: (0.5, 0.6, 0.54545456)

For each pair of annotators, and for each label:
(0,1), label= 3_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 1.0, 2.0);  (precision, recall, F1)= (0.75, 0.6, 0.6666667);  Lenient: (0.75, 0.6, 0.6666667)


For the document: review-10_260.xml_00468
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:

For the document: review-11_443.xml_00469
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (10.0, 0.0, 13.0, 9.0);  (precision, recall, F1)= (0.4347826, 0.5263158, 0.47619045);  Lenient: (0.4347826, 0.5263158, 0.47619045)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (10.0, 0.0, 13.0, 9.0);  (precision, recall, F1)= (0.4347826, 0.5263158, 0.47619045);  Lenient: (0.4347826, 0.5263158, 0.47619045)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 2.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 3_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 4.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 7.0, 0.0);  (precision, recall, F1)= (0.125, 1.0, 0.22222222);  Lenient: (0.125, 1.0, 0.22222222)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (9.0, 0.0, 0.0, 9.0);  (precision, recall, F1)= (1.0, 0.5, 0.6666667);  Lenient: (1.0, 0.5, 0.6666667)


For the document: review-11_327.xml_0046A
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (4.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.8, 0.8, 0.8000001);  Lenient: (0.8, 0.8, 0.8000001)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (4.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.8, 0.8, 0.8000001);  Lenient: (0.8, 0.8, 0.8000001)

For each pair of annotators, and for each label:
(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (4.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (1.0, 0.8, 0.88888896);  Lenient: (1.0, 0.8, 0.88888896)


For the document: review-11_186.xml_0046B
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: review-10_261.xml_0046C
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:

For the document: review-11_374.xml_0046D
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (12.0, 0.0, 3.0, 1.0);  (precision, recall, F1)= (0.8, 0.9230769, 0.85714287);  Lenient: (0.8, 0.9230769, 0.85714287)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (12.0, 0.0, 3.0, 1.0);  (precision, recall, F1)= (0.8, 0.9230769, 0.85714287);  Lenient: (0.8, 0.9230769, 0.85714287)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.75, 1.0, 0.85714287);  Lenient: (0.75, 1.0, 0.85714287)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 2.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (9.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (1.0, 0.9, 0.9473684);  Lenient: (1.0, 0.9, 0.9473684)


For the document: review-11_28.xml_0046E
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: review-10_413.xml_0046F
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (23.0, 0.0, 2.0, 2.0);  (precision, recall, F1)= (0.92, 0.92, 0.92);  Lenient: (0.92, 0.92, 0.92)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (23.0, 0.0, 2.0, 2.0);  (precision, recall, F1)= (0.92, 0.92, 0.92);  Lenient: (0.92, 0.92, 0.92)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (4.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

(0,1), label= 2_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (2.0, 0.0, 2.0, 0.0);  (precision, recall, F1)= (0.5, 1.0, 0.6666667);  Lenient: (0.5, 1.0, 0.6666667)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (16.0, 0.0, 0.0, 2.0);  (precision, recall, F1)= (1.0, 0.8888889, 0.94117653);  Lenient: (1.0, 0.8888889, 0.94117653)


For the document: review-10_118.xml_00470
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (2.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (2.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (2.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


Macro averaged over 22 documents:

For each pair of annotators, each type and each label:
Annotation type *comment*
For each pair of annotators
For pair (Output,Key):   (correct, paritalCorrect, spurious, missing)= (7.2727275, 0.0, 2.2272727, 1.3636364);  (precision, recall, F1)= (0.6147003, 0.6542348, 0.63304156);  Lenient: (0.6147003, 0.6542348, 0.63304156)

pairAnns=(Output,Key), type=comment, label=1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.4090909, 0.0, 0.4090909, 0.0);  (precision, recall, F1)= (0.17045455, 0.18181819, 0.17532468);  Lenient: (0.17045455, 0.18181819, 0.17532468)

pairAnns=(Output,Key), type=comment, label=2_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.045454547, 0.0, 0.045454547, 0.0);  (precision, recall, F1)= (0.045454547, 0.045454547, 0.045454547);  Lenient: (0.045454547, 0.045454547, 0.045454547)

pairAnns=(Output,Key), type=comment, label=3_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.045454547, 0.0, 0.3181818, 0.0);  (precision, recall, F1)= (0.045454547, 0.045454547, 0.045454547);  Lenient: (0.045454547, 0.045454547, 0.045454547)

pairAnns=(Output,Key), type=comment, label=4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.4090909, 0.0, 1.1818181, 0.0);  (precision, recall, F1)= (0.14583334, 0.27272728, 0.17676769);  Lenient: (0.14583334, 0.27272728, 0.17676769)

pairAnns=(Output,Key), type=comment, label=5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (6.3636365, 0.0, 0.27272728, 1.3636364);  (precision, recall, F1)= (0.65284926, 0.5571863, 0.5953047);  Lenient: (0.65284926, 0.5571863, 0.5953047)

Overall pairs:   (correct, paritalCorrect, spurious, missing)= (7.2727275, 0.0, 2.2272727, 1.3636364);  (precision, recall, F1)= (0.6147003, 0.6542348, 0.63304156);  Lenient: (0.6147003, 0.6542348, 0.63304156)

Overall pairs and types:   (correct, paritalCorrect, spurious, missing)= (7.2727275, 0.0, 2.2272727, 1.3636364);  (precision, recall, F1)= (0.6147003, 0.6542348, 0.63304156);  Lenient: (0.6147003, 0.6542348, 0.63304156)