Chapter 11
Profiling Processing Resources [#]

11.1 Overview [#]

There is a reporting tool for GATE processing resources. It reports the total time taken by processing resources and the time taken for each document to be processed by an application of type corpus pipeline.

GATE use log4j, a logging system, to write profiling informations in a file. The GATE profiling reporting tool uses the file generated by log4j and produces a report on the processing resources. It profiles JAPE grammars at the rule level, enabling the user precisely identify the performance bottlenecks. It also produces a report on the time taken to process each document.

This initial code for the reporting tool was written by Intelius employees Andrew Borthwick and Chirag Viradiya and generously released under the LGPL licence to be part of GATE.

11.1.1 Features

1. Ability to generate following two reports
   1. Report 1: Total time taken for each level of processing [Application, Multiphase transducer (MPT), processing resource (PR), and Phase], subtotalled at each level.
   2. Report 2: Top N most expensive documents in terms of total time taken by a given PR element for those documents (sorted in descending order of time taken by the processing element for that document).
2. Report 1 specific features
   1. A command line option to sort the time statistics in order of amount of time consumed by processing elements or in order of execution.
   2. A command line option to hide the processing elements which took 0 milliseconds for processing.
   3. Ability to generate HTML with a collapsible tree.
3. Report 2 specific features
   1. A command line option to specify the number of most expensive (in terms of processing time for given PR) documents to be included in the report.
   2. A command line option to specify the PR for which the processing time of a document is to be considered.
4. Features common to both reports
   1. Ability to generate report in different formats–as indented text or in HTML format.
   2. Ability to generate a report only on the log entries from the last logical run of GATE.
   3. All processing times reported in number of seconds and in terms of percentage (rounded to nearest 0.1%) of total time.
   4. A command line counterpart API method exposed for using the tool from within other java projects.
   5. The tool halts if the benchmark.txt file is modified while generating the report.

11.1.2 Limitations

Be aware that the profiling doesn’t support non corpus pipeline as application type. There is indeed no interest in profiling a non corpus pipeline that works on one or no document at all. To get meaningful results you should run your corpus pipeline on at least 10 documents.

11.2 Graphical User Interface [#]

The activation of the profiling and the creation of profiling reports are accessible from the ‘Tools’ menu in GATE with the submenu ‘Profiling reports’.

You can ‘Start recording’ and ‘Stop recording’ the processing resource at any time. The logging is cumulative so if you want to get a new report you must use the ‘Clear the log’ menu item.

Two types of reports are available: ‘Report on processing resources’ and ‘Report on document processed’. See the previous section for more information.

11.3 Command Line Interface [#]

Report1 Usage: java gate.util.reporting.PRTimeReporter [Options]

Options:

-i input file path (default: benchmark.txt in the execution directory)

-m print media - html/text (default: html)

-z supressZeroTimeEntries - true/false (default: true)

-s sorting order - exec_order/time_taken (default: exec_order)

-o output file path (default: report.html/txt in the system temporary directory)

-l logical start (not set by default)

-h show help

Note that supressZeroTimeEntries will be ignored if the sorting order is ‘time_taken’

Report 2 Usage: java gate.util.reporting.DocTimeReporter [Options]

Options:

-i input file path (default: benchmark.txt in the execution directory)

-m print media - html/text (default: html)

-d number of docs, use -1 for all docs (default: 10 docs)

-p processing resource name to be matched (default: all_prs)

-o output file path (default: report.html/txt in the system temporary directory)

-l logical start (not set by default)

-h show help

Examples

• Run report 1: Report on Total time taken by each processing element across corpus
  • java -cp ”gate/bin:gate/lib/GnuGetOpt.jar” gate.util.reporting.PRTimeReporter -i benchmark.txt -o report.txt -m text
• Run report 2: Report on Time taken by document within given corpus.
  • java -cp ”gate/bin:gate/lib/GnuGetOpt.jar” gate.util.reporting.DocTimeReporter -i benchmark.txt -o report.html -m html

11.4 Application Programming Interface [#]

11.4.1 Log4j.properties

This is required to direct the profiling information to the benchmark.txt file. The benchmark.txt generated by GATE will be used as input for GATE profiling report tool as input.

• # File appender that outputs only benchmark messages
• log4j.appender.benchmarklog=org.apache.log4j.RollingFileAppender
• log4j.appender.benchmarklog.Threshold=DEBUG
• log4j.appender.benchmarklog.File=benchmark.txt
• log4j.appender.benchmarklog.MaxFileSize=5MB
• log4j.appender.benchmarklog.MaxBackupIndex=1
• log4j.appender.benchmarklog.layout=org.apache.log4j.PatternLayout
• log4j.appender.benchmarklog.layout.ConversionPattern=%m%n
• # Configure the Benchmark logger so that it only goes to the benchmark log file
• log4j.logger.gate.util.Benchmark=DEBUG, benchmarklog
• log4j.additivity.gate.util.Benchmark=false

11.4.2 Enabling profiling

There are two ways to enable profiling of the processing resources:

1. In gate/build.properties, add the line: run.gate.enable.benchmark=true
2. In your Java code, use the method: Benchmark.setBenchmarkingEnabled(true)

11.4.3 Reporting tool

Report1

1. Instantiate the Class PRTimeReporter
   1. PRTimeReporter report1 = new PRTimeReporter();
2. Set the input benchmark file
   1. File benchmarkFile = new File(”benchmark.txt”);
   2. report1.setBenchmarkFile(benchmarkFile);
3. Set the output report File
   1. File reportFile = new File(”report.txt”); or
   2. File reportFile = new File(”report.html”);
   3. report1.setReportFile(reportFile);
4. Set the following optional parameters:
   1. PrintMedia : Generate report in html or text format (default: both)
      1. report1.setPrintMedia(PRTimeReporter.MEDIA_TEXT); or
      2. report1.setPrintMedia(PRTimeReporter.MEDIA_HTML);
   2. SortOrder: Sort in order of execution or descending order of time taken (default: exec_order)
      1. report1.setSortOrder(PRTimeReporter.SORT_TIME_TAKEN); or
      2. report1.setSortOrder(PRTimeReporter.SORT_EXEC_ORDER);
   3. SupressZeroTimeEntries: True/False (default: True). Parameter ignored if SortOrder specified is ‘SORT_TIME_TAKEN’
      1. report1.setSupressZeroTimeEntries(true);
   4. LogicalStart: A string indicating the logical start to be operated upon for generating reports
      1. report1.setLogicalStart(”InteliusPipelineStart”);
5. Generate the text/html report
   1. report1.executeReport();

Report2

1. Instantiate the Class DocTimeReporter
   1. DocTimeReporter report2 = new DocTimeReporter();
2. Set the input path for benchmark.txt
   1. File benchmarkFile = new File(”benchmark.txt”);
   2. report2.setBenchmarkFile(benchmarkFile);
3. Set the output report path
   1. File reportFile = new File(”report.txt”); or
   2. File reportFile = new File(”report.html”);
   3. report2.setReportFile(reportFile);
4. Set the following optional parameters:
   1. PrintMedia : Generate report in html or text format (default: both)
      1. report2.setPrintMedia(DocTimeReporter.MEDIA_TEXT); or
      2. report2.setPrintMedia(DocTimeReporter.MEDIA_HTML);
   2. NoOfDocs: Number of documents to be displayed in the report (default: 10 docs)
      1. report2.setNoOfDocs(2); // 2 docs or
      2. repor2.setNoOfDocs(DocTimeReporter.ALL_DOCS); // All documents
   3. SearchString: The PR name for which the documents to be considered (default: MATCH_ALL_PR_REGEX).
      1. report2.setSearchString(”HTML”); // match ALL PRS having HTML as substring
   4. LogicalStart: A string indicating the logical start to be operated upon for generating reports
      1. report2.setLogicalStart(”InteliusPipelineStart”);
5. Generate the text/html report
   1. report2.executeReport();