Log in Help
Print
Home 〉 sale 〉 rnti-09 〉 cross-validation 〉 stars 〉 senti 〉 train_test.txt
 
Pre-processing the 89 documents...
Learning starts.
For the information about this learning see the log file /home/saggion/temp/ml-opinion/cross-validation/stars/senti/savedFiles/logFileForNLPLearning.save
The number of threads used is 1
** Training mode:
time for NLP features: 1
time for fv: 0
time for filtering: 0
time for NLP training: 1
This learning session finished!.
Pre-processing the 22 documents...
Learning starts.
For the information about this learning see the log file /home/saggion/temp/ml-opinion/cross-validation/stars/senti/savedFiles/logFileForNLPLearning.save
The number of threads used is 1
** Application mode for document from 0 to 22(not included):
This learning session finished!.


------------------------------------------------

annotation sets:
*Output*
*Key*

For the document: review-11_517.xml_00155
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (11.0, 0.0, 7.0, 7.0);  (precision, recall, F1)= (0.6111111, 0.6111111, 0.6111111);  Lenient: (0.6111111, 0.6111111, 0.6111111)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (11.0, 0.0, 7.0, 7.0);  (precision, recall, F1)= (0.6111111, 0.6111111, 0.6111111);  Lenient: (0.6111111, 0.6111111, 0.6111111)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 3.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 3_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 2.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 4.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (11.0, 0.0, 1.0, 4.0);  (precision, recall, F1)= (0.9166667, 0.73333335, 0.8148148);  Lenient: (0.9166667, 0.73333335, 0.8148148)


For the document: review-10_218.xml_00156
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: review-10_90.xml_00157
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: review-11_135.xml_00158
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.5, 0.5, 0.5);  Lenient: (0.5, 0.5, 0.5)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.5, 0.5, 0.5);  Lenient: (0.5, 0.5, 0.5)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (1.0, 0.5, 0.6666667);  Lenient: (1.0, 0.5, 0.6666667)


For the document: review-11_269.xml_00159
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:

For the document: review-10_233.xml_0015A
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (17.0, 0.0, 4.0, 4.0);  (precision, recall, F1)= (0.8095238, 0.8095238, 0.8095238);  Lenient: (0.8095238, 0.8095238, 0.8095238)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (17.0, 0.0, 4.0, 4.0);  (precision, recall, F1)= (0.8095238, 0.8095238, 0.8095238);  Lenient: (0.8095238, 0.8095238, 0.8095238)

For each pair of annotators, and for each label:
(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 4.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (17.0, 0.0, 0.0, 4.0);  (precision, recall, F1)= (1.0, 0.8095238, 0.8947368);  Lenient: (1.0, 0.8095238, 0.8947368)


For the document: review-10_172.xml_0015B
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: review-10_484.xml_0015C
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (10.0, 0.0, 5.0, 5.0);  (precision, recall, F1)= (0.6666667, 0.6666667, 0.6666667);  Lenient: (0.6666667, 0.6666667, 0.6666667)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (10.0, 0.0, 5.0, 5.0);  (precision, recall, F1)= (0.6666667, 0.6666667, 0.6666667);  Lenient: (0.6666667, 0.6666667, 0.6666667)

For each pair of annotators, and for each label:
(0,1), label= 2_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 3_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 3.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (10.0, 0.0, 0.0, 5.0);  (precision, recall, F1)= (1.0, 0.6666667, 0.8);  Lenient: (1.0, 0.6666667, 0.8)


For the document: review-10_556.xml_0015D
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (15.0, 0.0, 2.0, 2.0);  (precision, recall, F1)= (0.88235295, 0.88235295, 0.88235295);  Lenient: (0.88235295, 0.88235295, 0.88235295)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (15.0, 0.0, 2.0, 2.0);  (precision, recall, F1)= (0.88235295, 0.88235295, 0.88235295);  Lenient: (0.88235295, 0.88235295, 0.88235295)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.5, 1.0, 0.6666667);  Lenient: (0.5, 1.0, 0.6666667)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (14.0, 0.0, 0.0, 2.0);  (precision, recall, F1)= (1.0, 0.875, 0.93333334);  Lenient: (1.0, 0.875, 0.93333334)


For the document: review-10_54.xml_0015E
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (7.0, 0.0, 4.0, 3.0);  (precision, recall, F1)= (0.6363636, 0.7, 0.6666667);  Lenient: (0.6363636, 0.7, 0.6666667)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (7.0, 0.0, 4.0, 3.0);  (precision, recall, F1)= (0.6363636, 0.7, 0.6666667);  Lenient: (0.6363636, 0.7, 0.6666667)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 3.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (6.0, 0.0, 1.0, 3.0);  (precision, recall, F1)= (0.85714287, 0.6666667, 0.75);  Lenient: (0.85714287, 0.6666667, 0.75)


For the document: review-10_294.xml_0015F
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (15.0, 0.0, 3.0, 3.0);  (precision, recall, F1)= (0.8333333, 0.8333333, 0.8333333);  Lenient: (0.8333333, 0.8333333, 0.8333333)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (15.0, 0.0, 3.0, 3.0);  (precision, recall, F1)= (0.8333333, 0.8333333, 0.8333333);  Lenient: (0.8333333, 0.8333333, 0.8333333)

For each pair of annotators, and for each label:
(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 3.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (15.0, 0.0, 0.0, 3.0);  (precision, recall, F1)= (1.0, 0.8333333, 0.90909094);  Lenient: (1.0, 0.8333333, 0.90909094)


For the document: review-10_7.xml_00160
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (23.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.9583333, 1.0, 0.9787234);  Lenient: (0.9583333, 1.0, 0.9787234)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (23.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.9583333, 1.0, 0.9787234);  Lenient: (0.9583333, 1.0, 0.9787234)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (22.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.95652175, 1.0, 0.9777778);  Lenient: (0.95652175, 1.0, 0.9777778)


For the document: review-11_64.xml_00161
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (4.0, 0.0, 2.0, 2.0);  (precision, recall, F1)= (0.6666667, 0.6666667, 0.6666667);  Lenient: (0.6666667, 0.6666667, 0.6666667)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (4.0, 0.0, 2.0, 2.0);  (precision, recall, F1)= (0.6666667, 0.6666667, 0.6666667);  Lenient: (0.6666667, 0.6666667, 0.6666667)

For each pair of annotators, and for each label:
(0,1), label= 3_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (4.0, 0.0, 0.0, 2.0);  (precision, recall, F1)= (1.0, 0.6666667, 0.8);  Lenient: (1.0, 0.6666667, 0.8)


For the document: review-10_260.xml_00162
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:

For the document: review-11_443.xml_00163
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (10.0, 0.0, 13.0, 11.0);  (precision, recall, F1)= (0.4347826, 0.47619048, 0.45454544);  Lenient: (0.4347826, 0.47619048, 0.45454544)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (10.0, 0.0, 13.0, 11.0);  (precision, recall, F1)= (0.4347826, 0.47619048, 0.45454544);  Lenient: (0.4347826, 0.47619048, 0.45454544)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (2.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

(0,1), label= 3_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 4.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 8.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (8.0, 0.0, 1.0, 10.0);  (precision, recall, F1)= (0.8888889, 0.44444445, 0.5925926);  Lenient: (0.8888889, 0.44444445, 0.5925926)


For the document: review-11_327.xml_00164
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (4.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.8, 0.8, 0.8000001);  Lenient: (0.8, 0.8, 0.8000001)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (4.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.8, 0.8, 0.8000001);  Lenient: (0.8, 0.8, 0.8000001)

For each pair of annotators, and for each label:
(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (4.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (1.0, 0.8, 0.88888896);  Lenient: (1.0, 0.8, 0.88888896)


For the document: review-11_186.xml_00165
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: review-10_261.xml_00166
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:

For the document: review-11_374.xml_00167
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (10.0, 0.0, 5.0, 4.0);  (precision, recall, F1)= (0.6666667, 0.71428573, 0.68965524);  Lenient: (0.6666667, 0.71428573, 0.68965524)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (10.0, 0.0, 5.0, 4.0);  (precision, recall, F1)= (0.6666667, 0.71428573, 0.68965524);  Lenient: (0.6666667, 0.71428573, 0.68965524)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (3.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.75, 0.75, 0.75);  Lenient: (0.75, 0.75, 0.75)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 2.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (7.0, 0.0, 2.0, 2.0);  (precision, recall, F1)= (0.7777778, 0.7777778, 0.77777773);  Lenient: (0.7777778, 0.7777778, 0.77777773)


For the document: review-11_28.xml_00168
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: review-10_413.xml_00169
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (16.0, 0.0, 9.0, 9.0);  (precision, recall, F1)= (0.64, 0.64, 0.64);  Lenient: (0.64, 0.64, 0.64)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (16.0, 0.0, 9.0, 9.0);  (precision, recall, F1)= (0.64, 0.64, 0.64);  Lenient: (0.64, 0.64, 0.64)

For each pair of annotators, and for each label:
(0,1), label= 1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 4.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 2_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 4.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (16.0, 0.0, 0.0, 9.0);  (precision, recall, F1)= (1.0, 0.64, 0.7804878);  Lenient: (1.0, 0.64, 0.7804878)


For the document: review-10_118.xml_0016A
For the annotation type *comment*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*rating*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (2.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (2.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= 5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (2.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


Macro averaged over 22 documents:

For each pair of annotators, each type and each label:
Annotation type *comment*
For each pair of annotators
For pair (Output,Key):   (correct, paritalCorrect, spurious, missing)= (6.818182, 0.0, 2.6818182, 2.4545455);  (precision, recall, F1)= (0.5957183, 0.60455143, 0.5999657);  Lenient: (0.5957183, 0.60455143, 0.5999657)

pairAnns=(Output,Key), type=comment, label=1_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.45454547, 0.0, 0.36363637, 0.18181819);  (precision, recall, F1)= (0.2840909, 0.3068182, 0.2916667);  Lenient: (0.2840909, 0.3068182, 0.2916667)

pairAnns=(Output,Key), type=comment, label=2_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.09090909, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

pairAnns=(Output,Key), type=comment, label=3_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.36363637, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

pairAnns=(Output,Key), type=comment, label=4_Star_Review:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.5909091, 0.09090909);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

pairAnns=(Output,Key), type=comment, label=5_Star_Review:   (correct, paritalCorrect, spurious, missing)= (6.3636365, 0.0, 0.27272728, 2.1818182);  (precision, recall, F1)= (0.654409, 0.5187915, 0.57209855);  Lenient: (0.654409, 0.5187915, 0.57209855)

Overall pairs:   (correct, paritalCorrect, spurious, missing)= (6.818182, 0.0, 2.6818182, 2.4545455);  (precision, recall, F1)= (0.5957183, 0.60455143, 0.5999657);  Lenient: (0.5957183, 0.60455143, 0.5999657)

Overall pairs and types:   (correct, paritalCorrect, spurious, missing)= (6.818182, 0.0, 2.6818182, 2.4545455);  (precision, recall, F1)= (0.5957183, 0.60455143, 0.5999657);  Lenient: (0.5957183, 0.60455143, 0.5999657)