Log in Help
Print
Home 〉 sale 〉 rnti-09 〉 cross-validation 〉 thumbs 〉 compare_test_train.txt
 
***ALG1***

annotation sets:
*Key*
*Output*

For the document: ThumbsUp-ThumbsDown-12.html.xml_004DF
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-129.html.xml_004E0
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-119.html.xml_004E1
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-13.html.xml_004E2
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-114.html.xml_004E3
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-14.html.xml_004E4
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-151.html.xml_004E5
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-125.html.xml_004E6
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-16.html.xml_004E7
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-127.html.xml_004E8
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-152.html.xml_004E9
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-123.html.xml_004EA
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-140.html.xml_004EB
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-165.html.xml_004EC
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-112.html.xml_004ED
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-128.html.xml_004EE
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-113.html.xml_004EF
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-104.html.xml_004F0
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-163.html.xml_004F1
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-1.html.xml_004F2
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-18.html.xml_004F3
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-3.html.xml_004F4
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-11.html.xml_004F5
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 2.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 2.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 2.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-137.html.xml_004F6
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-17.html.xml_004F7
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-167.html.xml_004F8
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-143.html.xml_004F9
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-158.html.xml_004FA
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-19.html.xml_004FB
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-2.html.xml_004FC
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


Macro averaged over 30 documents:

For each pair of annotators, each type and each label:
Annotation type *p*
For each pair of annotators
For pair (Key,Output):   (correct, paritalCorrect, spurious, missing)= (0.76666665, 0.0, 0.26666668, 0.23333333);  (precision, recall, F1)= (0.76666665, 0.76666665, 0.76666665);  Lenient: (0.76666665, 0.76666665, 0.76666665)

pairAnns=(Key,Output), type=p, label=down:   (correct, paritalCorrect, spurious, missing)= (0.53333336, 0.0, 0.26666668, 0.0);  (precision, recall, F1)= (0.53333336, 0.53333336, 0.53333336);  Lenient: (0.53333336, 0.53333336, 0.53333336)

pairAnns=(Key,Output), type=p, label=up:   (correct, paritalCorrect, spurious, missing)= (0.23333333, 0.0, 0.0, 0.23333333);  (precision, recall, F1)= (0.23333333, 0.23333333, 0.23333333);  Lenient: (0.23333333, 0.23333333, 0.23333333)

Overall pairs:   (correct, paritalCorrect, spurious, missing)= (0.76666665, 0.0, 0.26666668, 0.23333333);  (precision, recall, F1)= (0.76666665, 0.76666665, 0.76666665);  Lenient: (0.76666665, 0.76666665, 0.76666665)

Overall pairs and types:   (correct, paritalCorrect, spurious, missing)= (0.76666665, 0.0, 0.26666668, 0.23333333);  (precision, recall, F1)= (0.76666665, 0.76666665, 0.76666665);  Lenient: (0.76666665, 0.76666665, 0.76666665)

***ALG2***
annotation sets:
*Output*
*Key*

For the document: ThumbsUp-ThumbsDown-12.html.xml_004BA
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-129.html.xml_004BB
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-119.html.xml_004BC
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-13.html.xml_004BD
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-114.html.xml_004BE
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-14.html.xml_004BF
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-151.html.xml_004C0
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-125.html.xml_004C1
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-16.html.xml_004C2
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-127.html.xml_004C3
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-152.html.xml_004C4
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-123.html.xml_004C5
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-140.html.xml_004C6
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-165.html.xml_004C7
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-112.html.xml_004C8
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-128.html.xml_004C9
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-113.html.xml_004CA
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-104.html.xml_004CB
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-163.html.xml_004CC
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-1.html.xml_004CD
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-18.html.xml_004CE
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-3.html.xml_004CF
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-11.html.xml_004D0
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 2.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 2.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 2.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-137.html.xml_004D1
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-17.html.xml_004D2
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


For the document: ThumbsUp-ThumbsDown-167.html.xml_004D3
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-143.html.xml_004D4
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-158.html.xml_004D5
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-19.html.xml_004D6
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (1.0, 0.0, 0.0, 0.0);  (precision, recall, F1)= (1.0, 1.0, 1.0);  Lenient: (1.0, 1.0, 1.0)


For the document: ThumbsUp-ThumbsDown-2.html.xml_004D7
For the annotation type *p*
Compute the IAA for 2 Annotator(s) and on 1 document(s).
Annotation feature=*thumbs*
Fmeasures averaged over 1 pairs of annotators.
  (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators:
(0,1):   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

For each pair of annotators, and for each label:
(0,1), label= down:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.0, 1.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

(0,1), label= up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 1.0, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)


Macro averaged over 30 documents:

For each pair of annotators, each type and each label:
Annotation type *p*
For each pair of annotators
For pair (Output,Key):   (correct, paritalCorrect, spurious, missing)= (0.53333336, 0.0, 0.46666667, 0.5);  (precision, recall, F1)= (0.53333336, 0.53333336, 0.53333336);  Lenient: (0.53333336, 0.53333336, 0.53333336)

pairAnns=(Output,Key), type=p, label=down:   (correct, paritalCorrect, spurious, missing)= (0.53333336, 0.0, 0.0, 0.5);  (precision, recall, F1)= (0.53333336, 0.53333336, 0.53333336);  Lenient: (0.53333336, 0.53333336, 0.53333336)

pairAnns=(Output,Key), type=p, label=up:   (correct, paritalCorrect, spurious, missing)= (0.0, 0.0, 0.46666667, 0.0);  (precision, recall, F1)= (0.0, 0.0, 0.0);  Lenient: (0.0, 0.0, 0.0)

Overall pairs:   (correct, paritalCorrect, spurious, missing)= (0.53333336, 0.0, 0.46666667, 0.5);  (precision, recall, F1)= (0.53333336, 0.53333336, 0.53333336);  Lenient: (0.53333336, 0.53333336, 0.53333336)

Overall pairs and types:   (correct, paritalCorrect, spurious, missing)= (0.53333336, 0.0, 0.46666667, 0.5);  (precision, recall, F1)= (0.53333336, 0.53333336, 0.53333336);  Lenient: (0.53333336, 0.53333336, 0.53333336)