CoW - Completed Tasks

Note: add new completed tasks at the bottom (i.e. this is in chronological order, oldest first).

Created sins/versioning and adapted the SVNKit example code as a test suite. Successfully creates repositories and does all the usual SVN stuff on them.
Done using Appfuse 1 because 2 not ready yet
- Using Equinox or Appfuse 2 seed a webapp to host GATEWiki as part of the sins tree. This should be a maven multi-project build incorporating the webapp, yam and clone. Use Spring 2 / Spring MVC. Integrate with the cruise control build/test.
Choose a testing strategy, write some initial tests (including against the test instance), add it to the Cruise Control suite (i.e. the top-level ant build) and document the build/test arrangements in the developer guide.
Implement Spring/Acegi registration and login
Done using Hypersonic and DAOs: Allow registration of sandboxes; store the list in a ~/.sins.xml file.
TAO and MC sites: move to gateservice1
Reconfigure GATEWiki to use Jetty, and add a sins/gatewiki/jetty tree.
Investigated using Tomcat Virtual Directory: It is possible to set that pages are opened from different location and not only from Tomcat deploy (webapps) directory. The effort is adding a new .xml file where the context and a docBase is specified, so that Tomcat "knows" where to look under specified context. Different types of files could be opened (.html, .pdf, .jpg, .avi, .yam). Doesn't require Tomcat restart.
- Jetty: since configuration is done in jetty.xml requires restart.
GATE movies: add to DD's set, and add to SVN
Use log4j in gate.versioning
[GATEWiki] SVNKit integration (see also gatewiki.html#bean-interactions):
- Enumerate functions needed from SVNKit (e.g. getting a list of what has changed when updating) and implement example tests in sins/versioning. Finalise the Sandbox object model.
- SandboxManager: singleton that allows getting of Sandbox but is the only place Sandboxes are created; guarantees one Sandbox per disk path
- Sandbox: prototype bean created only by SandboxManager. All locking operations must be synchronised and atomic.
[YAM] Add logging and spring as in gate.versioning; iohandler use FSRs
[YAM] deal with threading issues in Yam and make it a spring bean
[GATEWiki]: Implemented representation of sandbox directory tree with Struts Menu Tag Library and a util class to work with file system. This repository tree is loaded during startup of app and set to be an attribute of ServletContext. Some problems found with Jetty when this repository is of bigger size (e.g. when I add current sins project to the registered sandboxes list). Tomcat handles this fine.
[YAM] implement new top-level API based on YamFile
- record generation runs and prevent concurrent identical translations
- implement naive "needsRegen"
- wiki links translated according to if they exist (if they don't then use the URL of a "create" page)
[GATEWiki]:
- Whenever asked to serve a PDF or HTML check for a more recent YAM and regenerate if necessary.
- When requested, relative to a particular directory or file, do an SVN update or commit, tracking the resultant changes and regenerating where necessary.
- Dynamicaly passing username/password from gatewiki to versioning:
  - Changed versioning method for adding sandboxes, so that now username and password are passed to enable svn authentification.
  - Changed gatewiki controller to pass the username and password when adding Sandbox.
  - SandboxModel table has now one more field which is reduntant but is needed and that is 'username' which is actually a username of a user who registered that sandbox.
  - We are currently assuming that users use their databases (with list of sandboxes only) localy - db is not shared. This means that only one user will register all sandboxes.
  - We also assume that svn username/password is the same for all registered sandboxes.
  - In order to user log in username/password for svn authentication, acegi password encryption feature is turned off.
[GATEWiki] experimented with xwiki
- building
  - svn co svn://svn.forge.objectweb.org/svnroot/xwiki/trunks-users
  - cd trunks-users/xwiki
  - ant standalone
  - cd release/xwikionjetty
  - . startxwiki.sh
  - http://localhost:8080/xwiki/bin/view/Main/WebHome
[GATEWiki] Investigate Grails <HC>
- grails app for wiki
  - grails 0.5 (head) install
    - svn co http://svn.codehaus.org/grails/trunk/grails/
    - set GRAILS_HOME, adjust PATH
    - it needs the install dir to be writeable
  - put in sins/clare svn
  - add sandbox domain class
  - made db persistent for development and production
[YAM] Yammering <HC>
- make it possible to ignore titles in included files
  - get tests working again
  - add a test for title ignore during include (includes2.yam)
  - add option and make it the default
- create doc/yam-syntax.yam, from yam-minimal.yam
- package.html etc.
- document the build/test process (include check-errors, bin/install)
[CLONE] Improvements of CLONE <DD>
- Delete punctuation from the canonical value of quoted chunks.
- Unparseable sentences are silently ignored now: terrible for the users! Work out how to apply rules in order with a final catch-all that returns a good warning (investigate to see if there is a smart solution to do this in JAPE).
- Investigate possible bug that a newline sometimes needs to follow the last sentence (couldn't reproduce).
- Support for setting different datatypes (string, date, double). 'Projects have date deadlines.' will create datatype property deadline with type date. 'Projects have numeric budget.' will create double data type for new property.
- Support for adding synonyms to classes not only to instances added.
- LabelIndex removed and Clie class refactored to handle methods that used to be in LabelIndex class. This reduces bug re. refreshing labels after introducing new ontology API.
- Changed the unit test to use Ontology.toString() method. This method creates sorted string out of the ontology so that each time this method is called for the same ontology it generates the same string.
- Make a text file to mention all possible combinations (sentences) that could be proccessed with clie. This text file is used in tests now.
- Implemented web service for clone, and basic webapp that uses this ws to answer user questions.
[CLONE] Querying with CLONE <DD>
- Querying ontology with simple questions like: how many projects are there? or what are the types of documents?/what is the deadline of sekt?/list authors of 'some paper'. Look at query-example.txt for some samples.
[CLONE]: separated query language application from clone. <DD>
[CoW] Controllable Wiki <HC>
- creation
  - using grails 0.5
  - grails create-app cow
  - adding to svn (http://grails.codehaus.org/Checking+Projects+into+SVN):
    - svn add cow; svn ci -m added cow; cd cow
    - svn propset svn:ignore "WEB-INF" web-app/
    - svn propset svn:ignore "core" plugins/
    - svn rm web-app/WEB-INF
    - svn rm plugins/core
    - svn commit -m "Fixing SVN"
    - grails upgrade
      (this re-seeds the non-versioned stuff)
- sandboxes
  - added sandbox domain class
  - to get default controller, view etc. used:
    - def scaffold = Sandbox in SandboxController
  - validated sandbox filename
- persistence settings
  - made db persistent for development and production
    - persistent data: edit grails-app/conf/DevelopmentDataSource.groovy to set HSQLDB to "file" and dbCreate to "update"
    - seems to have a 20 second write delay
- build
  - changed the ant build to cater for the new stuff
  - moved ~/.m2 into sins; now versioning and yam take their gate.jar dependency from sins/maven-repository (so that we can remove the need for installing any maven-related stuff in order to build, e.g. with CC) (see sins/build.xml)
  - upgraded gate jar to 4.0; installed it to local rep using:
    - mvn install:install-file -DgeneratePom=true - -Dmaven.repo.local=/home/hamish/sins/maven-repository - -Dfile=/home/hamish/gate/bin/gate.jar -DgroupId=gate - -DartifactId=gate -Dversion=4.0beta1 -Dpackaging=jar
[CoW] CoW: Controllable Wiki <HC>
- general
  - did grails install-templates: creates src/templates
- serving static pages as a component of dynamic pages.
  - used generate-all to get the controller and view code
  - played with sitemesh api and a view to render a static page
  - investigated the problems with url mappings
    - implemented a UrlRewriteFilter to change url paths to ? syntax
    - see also notes in gatewiki.html
  - upgraded to grails HEAD (0.5.5) and re-test
    - fixed startup .zip problem: delete .tar etc. in grails/dist
[CoW] CoW: Controllable Wiki <HC>
- serving static pages
  - changed the views so that links to pages ending in id numbers have a trailing /
  - changed "sandbox" to "wiki" with an action "area" so URLs look like /cow/wiki/area/4?path=...
  - added urlrewrite jar and WEB-INF/web.template.xml to SVN
  - found sitemesh bug on gate.ac.uk index.html and wrote workaround (gate.util.GFiles)
  - using NanoHTTPD.serveFile to construct responses for non-HTML and directory files
    - could in future use java activation to guess mime type and so on; see dani's code, e.g.: def fds = new javax.activation.FileDataSource(fileName);
    - made changes to NanoHTTPD to remove the / from the paths it returns
  - if contentType of response is set to "text/html" grails incorrectly tries to construct a view...
  - finally fixed the paths issues and got static files working properly
    - images now working but not in subdirs because when including the body of a page in a subdir, the relative links will inevitably be broken...
      - tried adding the path part of the ?page= spec, but of course this is not set for the relative links :-(
    - options:
      - go back to the hacked UrlMappingsFilter now the id/ thing has solved part (all?) of the problems that it had? (worked!)
        
        clean test in grails
        
        implement filter hack and re-test
        
        if succeeds try in cow
        
        turn off url rewrite filter
        
        change cow to use CowUrlMappings again
        
        submit JIRA with the patch
      - redirect the *.* urls and use CowUrlMappings to process the dir parts? (rejected)
- misc
  - installed firefox web dev add-on from xandra: https://addons.mozilla.org/en-US/firefox/addon/60
- css etc.:
  - modify src/templates before generating controllers and views
  - logo: grails-app/views/layouts/main.gsp link, and put the .png in web-app/images
[CoW] CoW: Controllable Wiki <HC>
- cleaned up static code and removed NanoHTTPD dependency
- serving static pages
  - delete sandbox
  - split WikiController to create WikiAdminController, and move the relevant views
  - add dirListingAllowed to Wiki
  - views: change wiki show to only show id
  - added jetty to sins and checked deployment
  - added new actions and menus, and a top-level, and a short description to Wiki
  - install versioning bean into cow (just add the spring config and make it a member of a controller and it should be autowired) (uses sb mgr for serialisation of access to dir trees)
  - first version of page edit action with YAM summary
- designed layout and made some examples for the CSS stuff
- documented the build process
[CLOnE]
- Created an ontology lookup component [DD, VT]
- Created an application integrating a Tokeniser, SentenceSplitter, POSTagger, Morph and the above mentioned ontology lookup component that annotates an input document with mentions of ontological resource (Using flexible gazetteer) [DD, VT]
- Developed a scoring strategy to combine specificity of the property and also the similarity of its name with existing properties [DD, VT]
[CoW] CoW: Controllable Wiki <HC>
- added WikiPage class in WikiController to simplify action logic
- fix bug with markup in .yams and filling text area for edit (use textarea)
- added contextPath stuff in YamFile and AbstractTranslator to support translation in a different directory to the rest of the sources
- finished preview (uses show view with preview set true and temp file POSTed on to save from the save link)
- finished complete edit/preview/save cycle (save gets a POST with the temp file name that contains the new version of the file, and, in workstation mode, just stuffs it on disk, calls YamFile.translate and redirects to show with a flash message)
- cleaned up save code and added more checks on WikiPage in WC actions
- added create page / make directory
[CoW] CoW: Controllable Wiki <HC>
- added dir list option to all html show views
- renamed WikiAdminController as AdminController
[YAM] Yammering <HC>
- added gate.yam.dependencies to sins/yam in groovy and get the ant build to add it to the YAM snapshot jar
- reverted to ant and cleaned up the build.xml
[CoW] CoW: Controllable Wiki <HC>
- added Canoo functional tests
  - tested admin, wiki page creation and edit, including the svn results
- new create stuff
  - added create new page and linked from directory list views
  - create of existing forwards to show
  - create of non-html/non-dir messages "use upload"
[CoW] security, authorisation and authentication (see also gatewiki.html#section-4.3.)
- Acegi based security prototype <JS>
  - Send a Acegi plugin patch to Grails list (support M:N relation between authorities and request maps) IN PROGRESS
  - Add initial data somehow, instead of invoking setup controller. In ApplicationBootStrap should be figured out if the DB is empty and populate it with some data. Details, welcome. (If you do Wiki.list and get nothing, that indicates that the DB contains no wikis, for example.)
  - Person - Authority. Add custom code on scaffolded GSP pages:
    - add/update person - display checkboxes for available authorities
    - delete authority - display warning that there are users with that role and that they should be edited first
  - Wiki - RequestMap. Implement 1:N relationship with cascade delete.
    - add/update wiki - display matrix ROLE name : R/W privileges. After submit, delete all request maps for particular wiki and add new ones in the way it works now
    - when you delete wiki, delete all request maps that belong to that wiki
    - do not provide Edit Request Map, since it has no sense. Request map lives as long as Wiki lives, so we should edit Wiki only
  - Add access denied page. Now Acegi plugin cannot find it in case that user is not allowed to see particular URL, therefore Error 404 occurs.
  - Use existing grails gsp tags to show Login/Logout links depending on if the user is authenticated or not. Details, welcome. (try the acegi taglib and g:if as needed)
  - Use existing grails gsp tag to find authority of authenticated user and display him particular link (e,g Admin in header). Details, welcome.
[Versioning] Versioning API <HC>
- changed the API to use File instead of FileSystemResource, because FSR doesn't add anything for normal use, and are a pain to get (requiring a Spring context)
[CoW] security, authorisation and authentication <JS, HC>
- integration log: followed james' instructions except:
  - didn't copy cow-acegi/views/wikiAdmin/list to cow/views/admin/show as it doesn't seem to be different from the old one
  - didn't copy the top bar toggle javascript stuff from main.gsp to wiki.gsp (although did copy to main.gsp, which now just serves the admin controller views) as james will integrate this directly with the trunk, and it needs adapting to the new cow CSS layout
  - didn't copy the main.css as the main changes there seem to support the top bar toggle stuff... james overwrote it
  - omitted welcome message, as I intend that the user name will be a link to an account management page
- fixed integration issues
  - in w/s mode acegi stuff should do nothing. (-Dgate.cow.mode=workstation)
  - changes to names of classes and fields (where these are from the Acegi Plugin code you can leave the names in the code the same but change them in the GSPs)
    - Requestmap to RequestMap
    - Emailshow to ShowEmail
    - Passwd to PassWord
    - Group to Role
    - ConfigAttribute to Roles
    - RequestMap URL to URI
    - UserName to LoginName
    - UserRealName to FirstName plus LastName
    - Authority to Role
  - UI supporting anon read (not just "don't set any readers").
  - just display the authority name, not the ROLE_... name
  - fixed acegi filter prob in workstation mode
- fixed: the newpage redirect to directory incorrectly strips the trailing / from the requestPath (HC)
- reorganised the admin areas and / page
[CoW] build, restructuring <HC>
- moved yam to cow
  - src/java/resources gets put in lib/cow-resources.jar
  - pick up yam/bin stuff
- made the ant build copy the test results for workstation and server mode and link from cow/index
- finished war, deploy, etc. and sins/build.xml
- clean checkout/cruise testing
- add the clean shutdown option to run-dev too
[CoW] config, DBs and other user data: CowDataArea (CDA) <HC>
- bootup: use doWithSpring to init the CDA
- ApplicationBootstrap: if no Wikis exist create 1 on help and 2 on main
[CoW] converters <VT>
- JSPWiki2Yam converter
- HTML converter
[CoW] grails 0.6
- moved CowUrlMappings to UrlMappings
- moved datasource stuff to DataSource
- moved log4j to Config
- updated the build
- acegi
  - fixed doWithSpring to replace cf method
  - commented out non-needed resources from acegi plugin.xml
  - changed the one line of AcegiGrailsPlugin that didn't use transformToGetter
[CoW] grails 1.0-RC1
- upgraded libraries to match grails versions
[CoW] sourceforge release
- set up new sourceforge project and moved the code there
- helpdocs.zip moved to src/java/resources so that can find them in the war deploy
- create new page in wiki root fixed
[CoW] tests
- webtest didn't work since canoo moved to a plugin
  - webtest now forks the VM, so need to set additional classpath entries and propagate gate.* system properties
  - also fixed a couple of bugs in the RunWebtest script (see also http://jira.codehaus.org/browse/GRAILSPLUGINS-169 and http://jira.codehaus.org/browse/GRAILSPLUGINS-168 and the sysproperty stuff in call-webtest.xml in the webtest plugin)
[CoW] security, authorisation and authentication <JS, HC>
- add user-level account creation, with
  - a captcha: http://grails.org/Simple+Captcha+Plugin
  - email notifications of account creation with "enable" link
[CoW] security, authorisation and authentication <JS>
- upgrade to acegi 0.2
- webtests for crud over acegi, user registration
[CoW] security <HC>
- to prevent embedded HTML or JavaScript attacks in wiki pages, the YAM translator needs to call org.springframework.web.util.HtmlUtils.htmlEscape(text.toString()) on all non-control text (ASTPlain, ASTVerbatim and ASTUrl)
[Cow] Update Aceg-plugin to 0.2 <JS>
- removed old acegi-plugin 0.1 and captcha-0.5 plugin(as acegi-plugin now includes it)
- New features of acegi-plugin 0.2 include some scripts to generate custom stuff:
  - grails create auth-domains User Role, which will create acegi domain classes into cow/grails-app/domain/ and local AcegiConfig file into cow/grails/conf
  - grails create-requestmap, which will generate Requestmap domain.
  - set loginUserDomainClass "User" and authorityDomainClass"Role" in the AcegiConfig
  - grails generate-manager, which generates all the controllers and views for acegi stuff.
  - grails generate-registration, which generate the controller and views for registration.
- added webtest for User, Role, Requestmap and register etc.
- modified the TestSuite to ask it to load the tests orderly as RequestmapTest must run before AdminTest
- updated web-app/plugins/acegi-0.1 to acegi-0.2
[CoW] tests <JS>
- user and registration-related webtests should only run in server mode
[CoW] additional wiki functions (e.g. wysiwyg edit, delete, rename)
- wysywig editing
  - currently it seems to delete the title; is this a converter problem?
  - tried fck editor plugin, but buggy
  - try cleaning, removing web-app and plugins versions of fck
  - add fck in web-app/js (2.4.3) and FckEditorTagLib from 0.5.6
  - try richTextEditor tag; works, use this instead of plugin to avoid plugin bugs and simplify maintenance
  - add tests for the html2yam translator and define the necessary fixes to the yam2html xslt for valy
- changes to the HtmlToYamConverter
  - html generation <HC>
    - rework the citation span to include the ids of the links: the cite key.
    - comments are lost, add cow-comment class
    - put span class cow-footnote-text plus id around footnote text and add the id to the footnote reference
    - put a class on the html element cow-html. comment: this is not valid html. meta tag added instead.
  - XSL/converter changes <VT>
    - deal with footnotes
    - image attributes aren't translated, e.g. '%image(test-image.png, ALT tag, 200, 200, left, 0)' becomes %image(test-image.png, ALT tag) instead of %image(test-image.png, ALT tag, 200, 200, left, 0)
    - it adds a new line before each list
    - it loses newlines within lists, but seemingly not in ordered lists? (not explicitly fixed, but can't replicate - perhaps dissapeared with other changes?)
    - the processEscapes methods should not rely on cow-escape but look for class cow-html on the html element (low priority - I've turned these methods off at present). Methods removed.
    - convert nbsp to %\
    - recognise citation URLs (span class cow-citation) and convert them to %cite(...) notation
    - recognise comments and convert to YAM notation
  - fix citation hardcoding <AR>
    - big.bib and citation links are hardcoded. Remove this, setting on YamFile from application config.
[CoW] various code cleanups <JS>
- Admin.save, Admin.update: factor out the common code into a method
- put all the auth-related code from CoW that's not in the acegi plugin into a class
- migrate the path analysis stuff to **: http://jira.codehaus.org/browse/GRAILS-1428
- move cow-test-area to "cow test area" to trap spaces in paths bugs
[CoW] security bugs <JS>
- fix exception when logging in with wrong password (add to webtests)
[CoW] security etc. <JS>
- all the acegi-related pages are now mapped in workstation mode to an "unavailable page"
- login link should not show up in workstation mode (change to plugin isNotLoggedIn tag)
[CoW] build <VT>
- put the normal classes dir on the classpath and make java-tests-compile depend on java-compile
- copy non-java files from the source path to the classes dir to remove the need for the sourcepath on the classpath (which would break the war) for finding .xsl etc.
[YAM] Yammering <HC>
- contents get wrapped in paragraphs, which results in empty tags in the html. solution: move contents under Unit instead of Paragraph
[CoW] wysywig editing
- changes to the HtmlToYamConverter <VT>
  - html generation
    - div shouldn't be in a p (contents)
    - classes:
      - identify the title-derived headers and put class = cow-title-heading
      - footnote sub and links class = cow-footnote
      - auto-generated section anchors class = cow-section-anchor
      - escapes class = cow-escape
      - add a class to citation URLs (cow-citation with id = to the cite keys)
  - when doing FCK edit run YF.generate with no include processing; document
  - XSL changes
    - don't add backslashes in front of dashes unless they are at the start of a line preceded only by whitespace
    - put a space after dashes derived from li
    - don't put any blank lines before the title
    - don't copy the title as a first level heading
    - don't add a blank line at the start of tables
    - table rows must be separated by "---"
    - don't add a space before table column separators
    - when processing "h1" etc., look for a span of class "cow-sec-number" within the heading. if it exists, discard that span and put "%1" etc as at present. if it doesn't exist, put "%1*" etc.
    - when there's a cow-contents div, replace it with %contents
    - don't generate a --- after the last table row
    - ordered lists should start in "#" not "o"
    - don't escape e.g. the % in %include; in fact don't do escaping at all for now and we'll see how much of a pain this proves to be (maybe none)
    - process STRONG like B
  - note: to generate diffs from some sample .yams try "ant -Djava.test.case=gate.yam.YamTest java-test" and look at the ...-fault file in comparison with the .yam in test/resources/yam
[YAM] Yammering <HC>
- added lists of links and includes to YamParseTree (to support YamFile dependency tracking)
[YAM, CoW] dependency regeneration <AR>
- gate.yam.depend.Dependencies API:
- testing cow hookup / fix bug with file endings (Deps was expecting link file suffixes to be in the wrong form, e.g. .yam or no suffix or something. Links should be .html.
- See also the manual.
  - includes(YamFile, List<String>): update the dependency graph with the fact that YF includes a list of files
  - linkedBy(YF, List<String>): ditto for linking to
  - setStorageDir(File): where to save its serialised form
  - needs to use Concurrent data structures
- YamFile: add new API to support dependency regeneration and to move some YAM-specific stuff out of WikiController
  - has a Dependencies member (set by clients eg. WikiController, one per Wiki; may be null)
  - calls Dependencies methods at each generate, delete, etc.
  - new methods:
    - getOutputLocations: get a list of outputs (tex, pdf, html)
    - (existing) generate - ensure only single generate of any particular YAM output
    - generateDependencies: regenerate all dependent out-of-date files
    - delete: delete .yam and outputs
    - create: create empty.yam and generate
    - rename: rename .yam and outputs
    - wasModified: update depedencies for a modification (edit)
- WikiController:
  - refactor so that YamFile does create/rename/delete etc.
  - later on: gets upload, delete, SVN up etc. also send events to Dependencies
  - serialises Dependencies into ~/.cow (need to add config stuff now)
- gate.yam.depend.Dependencies API:
  - Dependencies' main role is to let YamFile know which .yams need regenerating whenever YamFile gets a change event (wasModified, create, delete, rename). This probably means that it needs methods for all these events that return a list of out-dated (dirty) files.
    - The call sequence then goes like this:
      - when WikiController is asked to save an edit it calls YamFile.wasModified
      - YF calls Dependencies.wasModified(this.file), which returns a list of all .yams which include YF (that being the set which now need regenerating due to the edit on YF)
      - YF then does the regeneration of all the dirty dependencies
    - Therefore Dependencies needs new methods for each event, taking a File in each case.
    - I haven't got my head properly around the code in Dependencies now, but the above implies that the indices (map keys) need to be the ids (canoncical paths) of files that are linked to or includes (i.e. the indices need to answer the question "which are all the files that link to this id?")
  - Why do these methods take File instead of YamFile? Because we also need to keep dependency data for non-YAM files that are uploaded or deleted. (Actually in the case of inclusion only YamFiles are relevant, but for linking any file or directory can be relevant.) A YAM which links to a non-YAM file needs to be regenerated if that file is created (uploaded), deleted or renamed, for example.
  - In Dependencies.clear probably you only need to call clear() on the existing map, not create a new one? If you do have to create a new one, then don't bother clearing the old one first?
  - Dependencies and synchronisation
    - Webapps have to serve multiple concurrent requests from browsers. This means that any data which is shared across requests has to worry about concurrent modification. This applies to the static members of Dependencies, which are shared across the whole application.
    - One way to deal with this is to make all the methods of the class that manipulate shared data synchronised (though some attention has to be paid to the semantics of static sync methods vs. those of instance methods). Another is to have syncronized blocks within methods that reference the particular member objects that relevant during that block of code.
    - This stuff is pretty difficult to get right, so ideally we need a comprehensive test for this stuff. (In fact for the whole thing, as we don't have any multithreaded tests at present; probably there are libraries out there that can help with this type of testing.)
  - Because there are some pathological cases for regeneration (e.g. a file which includes all others will need regeneration whenever any change is made anywhere) we will probably need to implement some kind of queue for regeneration requests.
- still got the modify issue? When a file is created, modify is called. Unable to replicate. Possibly, user was creating via a create link, and then editing the created file - so create then modify do get called.
- change WC/create to only do one call to Deps for create. Originally, WC called create for the yam file, and create for the html file generated from the yam. But Deps only stores dependencies for the yam. Cannot replicate: must have been changed at some point. <AR>
- write the WikiController and Wiki code to start hooking up the new Dependencies stuff and give to Angus <HC>
- generate and regenerate
  - issues to deal with later:
    - Quartz job to serialise Dependencies? Once per minute? <AR>
[CoW, YAM] bugs
- bug 1819990
  - fails to find body end tag in gate-server/html/demos/movies.html
    - cannot replicate
    - adopt ucu/MungeBanner to replace the page parsing stuff in WikiPage or wherever it is
    - look at using tag soup as a neko fallback
  - /home/hamish/public_html/gau-stats-before-move.html leads to out of memory error, probably because of some fault in HTML parsing
    - cannot replicate
  - getBody index out of bounds exception on http://localhost:8080/cow/wiki/show/2/house/loft-2005/arthur-jan05.html
    - Cannot replicate - this is an empty html file, which now gives an erroneous yam file containing a single line break
- bug 1819991 (fixed)
  - when there are no wiki objects, WC.show throws an NPE -- http://localhost:8080/cow/wiki/area/1
- bug 1819994 (won't fix)
  - there's probably a bug in ant target ordering that means that when ant doc has never been called a webtest-workstation fails; also have to run ant webtest at least once before can do ant run-dev after ant clean
- bug 1819993 (moved to backlog)
  - there is no way to cancel a create, i.e. if you click on a create link, the relevant file or directory is created and there's no way to back out
- bug 1838822 <JS>: fixed
  - create new wiki (admin controller)
    - the path should be validated: allow any letter, number, dash, dot, or underscore
    - when a path is entered that doesn't point to valid SVN sandbox there's an exception; there should be a message "${path} is not a valid Subversion sandbox" and the create new wiki page redisplayed. the Wiki DB record should not be created
- bug 1819995: moved to backlog
  - image: height should come last so can spec just width and also position? or will there be some other general mech for positioning?
- bug 1838827 <JS> fixed
  - "help" and "login" links should not be displayed on login page
  - the "Admin" link should not be displayed on /admin pages (e.g. /admin/list)
- bug 1819992 <JS> fixed
  - on windows it is possible to create a non-canonical path (without the drive letter)
- bug 1838819 <JS>
  - the "new page name" box should have a tooltip saying "Enter a new page name (e.g. 'new-page.html') or a new directory name (e.g. 'newdir'). For other page types use 'upload'. Note that names can't contain spaces, apostrophes etc." This text should be in a message bundle (not hard-coded).
[CoW] misc
- DataSource doesn't get the DataArea stuff; load app.properties and get the version from there
- change the "preview" flag to "p" to match the others
- moved YamHelp to YamService
[CoW] testing
- added a webtest-quick that runs against an existing server
  - if upgrading the webtest plugin note that there are patches against the plugin directory for:
    - copying the resources and making the links non-absolute (from jpv)
    - look for a webtest.rungrails property and not to run the server if it isn't true
    - some small style changes
[CoW] config, DBs and other user data
- added gate.cow configuration in Config and replaced DataArea etc. system properties stuff
[CoW] gate.ac.uk/wiki <HC, AR>
- JSPWiki2YAM: change o to #
- JSPWiki2YAM: try converting existing wiki and test <AR>
  - Problems:
    - A single underscore in jspw is a valid non-formatting character, but starts an italic in yam. The code that escapes special characters in the translation has been turned off with if(true) return, deliberately. Fix in JSPWikiToYamConverter. Fixed.
    - Headings in jspw do not have to be followed by a blank line, but do in yam - undocumented syntax? Blank line needs adding in translation. Fixed.
    - Headings containing a $ get treated as if they contain a capture group, leading to an error. Code uses regexes via replaceAll, when should probably use Strings via replace. Fixed.
    - Need to escape * and = in jspw in some or all cases. Fixed.
    - Titles: JSPWiki has no notion of title within its syntax, although all pages do have a title of the page name placed at the top (but this is not defined in syntax). However, users sometimes put in a title of their own at level 1. And sometimes they don't. YAM assumes the first para is the title. If the user has placed a heading with a level as the first para, then this gives e.g. %2* Some Heading. If the user has placed no heading, and assumes the JSPW page title will act as one, then they get their first content para as a title in YAM. Current planned fix: blanket title of the page name. Fixed.
    - Relative links have "VIEW" prepended, and are replaced with "create" links. What is adding the "VIEW"? Fixed.
    - Valentin.html; JapeHelp.html etc: < and > not translating. angle brackets are perhaps assumed to be elements in jspw, and not rendered. users therefore have to use &lt etc. Fixed.
    - GateWiki.html link to GateDemoMovies not working: case-misnamed in JSPWiki working copy and SVN. Was called GATEDemoMovies, but renamed and not checked in. Fixed.
    - teletype turned on part way through SourceForge.html: failing to place the url in bracketed form. The person writing the JSPWiki page intended the URL to be an unlinked, teletyped string. YAM will assume it is for linking, and also try to teletype it. Tricky to sort out in code, fixed by editing the offending wiki page. So it might happen again on some future page... Fixed.
    - GateWiki.html: faulty (and unneccesary?) link to an admin page. JSPWikiMarkupParser makes a mess of this link, escaping the element < and > surrounds. Remove such admin links. Fixed.
    - Links containing commas don't convert. GLEAM.html: faulty "AJAX and Web Services" link; Eclipse plugin external link broken on Eclipse.html page. Escape commas and spaces in bracketed URLs. Fixed.
    - encoding problem in MultiSourceIE.html ("Who's who"); SemanticWebTerms ("document's author"); InformationExtractionPapers (lots) etc etc. There is a problem with JSPWiki txt files encoding, with some characters being encoded and displayed as control characters. This is most likely because someone copied and pasted out of a pdf document or something similar. We can't recover the original characters. The most faithful conversion is with input encoded as ISO-8859-1 and output as UTF-8, which gives the same incorrect encodings in JSPWiki and yam.
    - attachments: e.g. to GateDemoMovies.html. Need to get attachments from similarly named directory, copy to output, and add to a link section at the bottom of the page. Done.
- grails 1.0
  - upgrade svnkit.jar (1.1.2 in grails RC3)
- edit link not showing at top level of wiki areas? (fixed in PageController)
- notes on old edit flow (pre-PageController)
  - views:
    - layouts/wiki.gsp (used by show): <g:if test="${editAllowed && editLink}"> <span class="pageLink"> <g:link action="edit" id="${editLink}">Edit this page</g:link> </span> </g:if>
    - views/wiki/edit.gsp <g:form controller="wiki" action="preview" id="${previewLink}" method="post"> <span class="button"><g:actionSubmit value="Preview" /></span> <textarea style="width:100%;" class="pageText" id="editorarea" name="editedtext" rows="25" cols="80">${pageText}</textarea> </g:form>
    - views/wiki/show.gsp preview stuff <g:if test="${preview}"> <g:form controller="wiki" action="save" id="${saveLink}" method="post"> <input type="hidden" name="editedTextPath" value="${editedTextPath}" /> <input type="hidden" name="saveLink" value="${saveLink}" /> <span class="button"><g:actionSubmit action="save" value="Save this edit" /></span> </g:form> </g:if>
  - WikiController
    - show...edit...preview...show...save
    - show puts editAllowed and editLink, the latter points to the id and request path, and is used to point to the edit action by wiki.gsp
    - edit puts previewLink (${wiki.id}${wikiPage.requestPath}?preview=true) and pageText and edit.gsp uses the link to point to preview from an edit form
    - preview regenerates the HTML, and puts: editedTextPath:tempFilePath, saveLink:"${wiki.id}${requestPath}" then renders the show.gsp view with a form forwarding to save with the file path
    - save overwrites the wiki files from the temp files, puts a message in flash scope and redirects to show
  - links
    - show html: http://localhost:8080/cow/wiki/show/1/doc/index.html
      - Edit: http://localhost:8080/cow/wiki/edit/1/doc/index.html?f=1
        
        edit: http://localhost:8080/cow/wiki/edit/1/doc/about-backlog.html?f=1
        
        "save": /cow/wiki/preview/1/doc/about-backlog.html?p=1&f=1
        
        preview: http://localhost:8080/cow/wiki/preview/1/doc/about-backlog.html?p=1&f=1
        
        Save this edit: /cow/wiki/save/1/doc/about-backlog.html
      - (with form): http://localhost:8080/cow/wiki/edit/1/doc/index.html
        
        edit: http://localhost:8080/cow/wiki/edit/1/doc/about-backlog.html
        
        Preview: /cow/wiki/preview/1/doc/about-backlog.html?p=1
        
        preview: http://localhost:8080/cow/wiki/preview/1/doc/about-backlog.html?p=1
        
        Save this edit: /cow/wiki/save/1/doc/about-backlog.html
      - Directory: http://localhost:8080/cow/wiki/directory/1/doc/?d=1
    - show dir: http://localhost:8080/cow/wiki/directory/1/doc/?d=1
      - New Page: http://localhost:8080/cow/wiki/newpage/1/doc/
        
        newpage: http://localhost:8080/cow/wiki/directory/1/doc/?d=1&n=1
        
        New page name: /cow/wiki/create/1/doc/
        
        show: http://localhost:8080/cow/wiki/show/1/doc/?_action_Create=Create&newPageName=squillions.html
      - Upload: http://localhost:8080/cow/wiki/preupload/1/doc/
        
        upload: http://localhost:8080/cow/wiki/directory/1/doc/?d=1&n=2
        
        File to upload: /cow/wiki/upload/1/doc/
        
        show: http://localhost:8080/cow/wiki/show/1/doc/?_action_Upload=Upload
[CoW, YAM] bugs
- bug 1819990
  - fixed in PageController and FileUtils
  - relative links that go outside of the current wiki area generate an exception (because it tries to convert a non-int for the wiki id)
  - coded, but needs tests: doesn't pick up charset information from headers of static pages (will perhaps need to find the encoding from the doc before processing into the dom).
[CoW] PageController <HC>
- webflow exception serving binaries: moving it back into the controller instead of the gsp and using the 1.0.1 snapshot removed the problem
- can't edit directory index pages implicitly
- rich yam edits remove the title: change to xsl translator
- html edits lose the header
[YAM, CoW] dependency regeneration <AR, HC>
- generate and regenerate
  - to begin with just execute the regenerate queue requests immediately
    - server mode edits: ~/.cowrc.d/staging/$session.user ?
      - first move all the saving stuff into YamService
      - in workstation mode save just overwrites the sandbox file and does generate
      - in server mode, needs to use a staging area, and when saving update (if conflict redirect back to edit with a flash message), log, overwrite, commit, unlock
    - issues to deal with later:
      - priority queue for regenerate requests? <HC>
      - Quartz job done, needs test<AR>
      - YamFile.needsRegeneration will become redundant? <HC>
[CoW] additional wiki functions (e.g. wysiwyg edit, delete, rename) <HC>
- wysywig editing
  - disable uploads, pictures etc. in fckeditor
[CoW] additional wiki functions (e.g. wysiwyg edit, delete, rename) <AR>
- changes to the HtmlToYamConverter
  - XSL/converter changes <AR>
    - linebreaks
      - it adds spaces / new lines before the title
      - Also too many added after titles, and in other places
      - Possible problem with newlines and lists
      - it loses newlines within URL texts (this is probably the fault of the parser, and maybe can't be fixed easily)
[gate.ac.uk] JSPWiki2YAM: try converting existing wiki and test <AR>
- Problems:
  - TaoProjectWiki.html contains leading and unmatched [[ for some reason? These have been placed there by the author. They are ignored by JSPWiki, but picked up by yam. solution: edit the JSPWiki page.
  - = is being incorrectly escaped in verbatim blocks
  - Several config and system pages created, that we probably don't need e.g. VariableTag.html, SystemInfo.html
  - TaoProjectWiki.html is linked from GateWiki.html. It should, however, be restricted to some users only. JSPWIki gives a "forbidden" message.
  - Although attachments are now added, links to attachments in the text of the page (as opposed to the list at the end) may need to be adjusted - check.
  - grep for broken links (localhost/create) and check them.
- Load into a runnig CoW
- dir listings: hide .* and *.yam by default, and have a config option or a "show hidden" link to show them

[CoW] PageController <HC>
- couldn't duplicate the gatewiki conversion probs that Angus couldn't find... my error presumably
- regen: just execute the YS regenerate queue requests immediately
[CoW] security <HC>
- tmp file paths should include some prefix (to prevent an attack that downloads some known resident of /tmp) or put them into ~/.cowrc.d
[CoW] gate.ac.uk/wiki <AR>
- XSL/converter changes: deal with bare predicates
- Check that dependencies is working properly when using PageController (doesn't seem to queue anything? or poss I got the example wrong; it now gets called from YamService.generate)
[CoW]
- upgraded to grails 1.0.1 (6715) from 6688 snapshot (seems to create more copies of services so had to change YamServiceTests)
- added a default index page in area 2 when created via DataArea
[CoW] tests
- fixed the PageTest stuff to use a cow-local .cowrc etc. by setting user.home to ./webtest/user-home
- changed webtest-quick to webtest-page and made it run irrespective of if grails is running
- added webtest-page to cruise
- put "1 2 3" etc. in doc/test-resources
- added cow/dev-user-home and made run-dev use that
[CoW] PageController etc. <HC>
- enumerate the redirects and GSP links for PC/PS and create a taglib
- layouts/page.gsp replaces wiki.gsp
- create URI is now not special; a 404 that's not a bad wiki area is trapped by PC.show and redirected to the new page flow
[CoW] Build
- move the various resources into src/java/gate, grails-app/utils/gate/cow and test/unit/gate (these will be copied into the ~/.grails classes directory)
- updating config to be like latest grails apps: delete the grails.war.dependencies in Config (hard to maintain, doesn't contribute much)
- Malformed refresh string (found ';' but no 'url/='): 0;/cow/page/show;jsessionid=ihit6iwer6ey/1/doc/
[CoW] replacing Acegi with JSecurity, cleaning up
- branched to jsec-gatewiki
- delete (or move to doc/old for future ref)
  - delete WikiController, AdminController, views; generate-all Wiki
  - delete acegi, related controllers and views
  - delete related webtests
  - delete web-app/*.gsp
  - moved to doc/old: plugins/grails-acegi-0.2.zip plugins/acegi-0.2 webtest/tests/RequestmapTest.groovy webtest/tests/RoleTest.groovy webtest/tests/Login.groovy webtest/tests/UserTest.groovy webtest/tests/WikiTest.groovy webtest/tests/RegisterTest.groovy web-app/plugins/acegi-0.2 web-app/workstation-mode.gsp grails-app/views/wiki grails-app/views/register grails-app/views/admin grails-app/views/user grails-app/views/requestmap grails-app/views/role grails-app/views/layouts/wiki.gsp grails-app/controllers/RoleController.groovy grails-app/controllers/WikiController.groovy grails-app/controllers/AcegiHelper.groovy grails-app/controllers/UserController.groovy grails-app/controllers/AdminController.groovy grails-app/controllers/CaptchaController.groovy grails-app/controllers/RequestmapController.groovy grails-app/controllers/RegisterController.groovy grails-app/controllers/WikiPage.groovy grails-app/conf/AcegiConfig.groovy grails-app/domain/Requestmap.groovy grails-app/domain/Role.groovy grails-app/domain/User.groovy
- Dependencies: generate-all from Wiki puts the following into create.gsp (which is invalid): <g:select optionKey="id" from="${Linked by.list()}" name="dependencies.id" value="${wiki?.dependencies?.id}" ></g:select>
  - Remove the getDependencies (and wikiURI) methods from Wiki
- build: put grails lib on build classpaths and removed redundant libraries from lib (all that are in grails/lib in fact); for some reason the webtests couldn't then see svnkit, so I left that one in lib
[CoW] cleaning up, new security model: jsec-gatewiki branch
- use pageURI to create links in PC and PS, and alter the taglib to be sensitive to implicitID
- add in the 1 and 2 mappings and debug
- a little more PageTest... note that it does a hacky svn revert afterwards
- merged branch back into trunk
[CoW] YAM <HC>
- change YamCommand to set createPage to ""
[CoW] PageController <HC>
- renamed CowLinkTagLib to PageLinkTagLib
- errors: UrlMappings: "500"(view:'/error'), and make that cow style

[CoW] security <AR>
- new version of jsec and debug - done
  - permissions checks in the template GSPs and/or the taglib [document how this now works] - done, see docs.
- presumably we need to encodeAsHtml whatever the user puts into the form-based page edit if we want to prevent them embedding javascript? or will yam2html sufficiently mangle it anyhow? Done - encodeAsHtml added to edit.gsp
- check all GSP and controller rendering to ensure that encodeAsURL and encodeAsHTML are used whenever there is (or could be) user-supplied data - Done.

[CoW] <TH>
- SVN integration: evaluate 3rd party repository browsing webapps
- check/update the bug tracker and other bugs below

[CoW] <TH>
- changed the help link to point at the user manual

[CoW] security <AR>
- A user with ReadAndWrite permissions on a wiki can edit the actual wiki object itself. e.g. create a user with ReadAndWrite on a new area (4 in this case) and then try: http://localhost:8080/wiki/show/4
  - build into controllers: safeguards against deleting controllers that are used in role

Old rejected stuff

File system watching to do updates
- Watch the registered sandboxes using a low-priority thread that doesn't hog machine resources. The point is to spot YAM files that have changed more recently than the .html generated from them and queue them for regeneration.
- Consume out-of-date YAM files from the queue and translate them using gate.yam.Yam
Making YamFile a spring bean: it is hard to do conditional construction of these, and the use case of substituting a different implementation in XML config doesn't apply here.
Using Tomcat or Jetty virtual directories for serving registered sandboxes. It will need some investigation in Jetty code, and also, the separate impementation for different Web container. Additionally, some implemenations will be essential in order to serve different directories in the same context, while 'context' in a Web container's vocabulary assume 'different application'.
[CLONE]:
- Use a listener on the ontology to delete labels at appropriate times (using new ontology API): this is not necessary after removing LabelIndex.
- RDF labels should not be used for private synonyms. They will screw up NE tagging (discuss with Valy and Kali).
[YAM] regen the parser html / add the javacc2html jar
- neither javacc2html nor javatohtml (which also covers .jj and .jjt files) supports java 5 :-(
[CoW] help area, URLs
- change conf/UrlMappings so that the main content area (id 2) doesn't have the /wiki/ and id/ bits
  - /action/... -> wiki 2, the main content (new)
  - /wiki/action/1/... -> help (as now)
  - /wiki/action/3...n/... -> other areas (as now)
  - /doc/now.html -> wiki/show/2/doc/now.html
  - perhaps: "/$path**?" { controller "wiki"; action "show" ... }
- the problem is that it is hard to avoid mapping /js, /css etc., and also the g:link tags in the GSPs start behaving strangely... probably not worth the effort!
an edit request that misses the trailing / redirects to show; it should redirect to edit, i.e. the page model slashURI should point to edit instead of show when built for triage. BUT: when would such a request be made?

PageController bugs
- newpage flow bug: show404s on non-html/non-dir files (e.g. a non-existent .jpg) create a new page "can't create" error
  - Improved error message to suggest trying upload <IR>
- edit bug: with rich text editor can't cancel (poss only on a new page when in w/s mode)
  - can't reproduce this bug in either server or w/s mode <IR>
SVN auth
- workstation mode shouldn't require svn authentication (why is it doing commits? if there is a good reason then let it I guess... in which case it does need to do authentication)
- server mode needs to do svn authentication
  - infrastructure in place to associate SVN authentication information with each wiki in the db, nicer admin interface still required.
[CoW] PageController <HC>
- use JSecUtils to get username
- finish newpageFlow
  - the exit states have to redirect conditionally depending on state returned by npF.create, to allow proper redirecting at the end of the flow
  - use new messages template in all views/page GSPs (passing pageScope)
  - put some text in new docs, from messages
- the default main index page didn't link to the help area correctly; added a feature to backlog to be able to link between areas
- added a .yam to the default main area index
- added grails-app/i18n/cow.properties for messages and put page controller messages there
- added a validate step in new page flow
[CoW] misc <HC>
- added link to WikiController from Admin page, and "browse" links in the areas list
- changed main.gsp layout to mainly mirror page.gsp; added cow favicon
- adjusted the admin URL mapping so that the Admin link doesn't get ** on the end
- added tabindex to input pages
- better logging
[CoW] tests <HC>
- added selenium tests
- fixed bug with non-registration of new sandboxes in WikiController
- fixed but in URL mappings that was sending /cow//plugins... requests to PageController
[CoW]
- upload flow <TH, IR>
  - prob with jsec and uploads...
    - tried workaround at http://jira.codehaus.org/browse/GRAILS-2774 but no luck
    - tried grails 1.1
      - need -Dgrails.plugins.dir=plugins when running grails; got mysterious url mappings error and gave up...
  - did gwt upload... <TH>
- upgraded webtest plugin to fix issue with GWT <IR>
- auth&auth
- create a selenium test for the upload stuff
- auth&auth tests
  - document the redirect to workaround the jsec/npe problems in the user manual (hoping for a fix in grails 1.1...)
- AbstractTranslator.processUrls should use encodeAsURL. Attempted - but java.io.URLEncoder.encode used instead. But, tests later found to fail so commented out. actually not needed

[CoW] security <AR>
- do we need to encode the output of the PageLinkTagLib and other tag libraries? probably would be good to encode the query part of the output in PageLinkTagLib. PageLinkTagLib query encoding done. All other taglibs checked and ok.

[YAM] dependencies <AR>
- check that quartz is still working, fix as necessary. Checked, working.
- fix Dependencies for new pages etc. Done, see svn logs for description.

[CoW] gate.ac.uk/cow
- parser for gate.ac.uk pages <TH>

[YAM] latex output <HC>
- fixed basic errors in tmp/yam/index.tex (cd ../../; ant yamjar); ../../bin/yam2html -llatex index.yam && tkdiff index.tex desired-index.tex
[YAM] lists bugs <HC>
- test/resources/yam/yam-lists-bug.yam: an ordered list within a nested unordered list which is at the current nesting level gets pushed after the rest of the unordered list
- fixed, needed to attach to next level up in the active lists stack, not the top-level parent
- tested using: ant -Djava.test.case=gate.yam.translate.YamTranslatorTest yamparser java-test java-test-reports
- test/resources/yam/yam-lists-bug2.yam: the final list element gets detached for some reason
  - now attaches at the first parent list that is indented at equal or less than
- for all this see cow/tmp in revision 957
- fixed table plus bulleted list bug (first list item always missed); added yam-tables-bug test
[CoW] infrastructure
- ian fixed logging work "ant java-test", e.g. from AbstractTranslatorTest; need to run grails first and is configured via Config.groovy

[CoW] security <AR>
- a new JsecRolePermissionRel is created even if an equal one exists. This means that half a dozen more are added each bootstrap. need to amend method that creates them in SecurityService.

[CoW] security <AR> - solved by above
- Security objects do not always persist when using HSQL. For example, if you start from an empty database, then bootstrap, an admin role will be created. Shut down cleanly, bootstrap again, and sometimes, the admin role has to be re-created because it was not persisted. This may be a problem with JsecRolePermissionRels, duplicates of defaults being created many times if you bootstrap repeatedly. Answer: check to see if this is a known HSQL problem, perhaps fixed in a later grails, or maybe move to a different database?

[CoW] dependencies <AR>
- Check that pages queued for regeneration are being regenerated at some point
- Dependency regeneration.
  - If you delete database, Dependencies are not being regenerated. When you create a new Dependencies for an existing wiki, it needs to find all the existing dependencies between pages and record them. Unfortunately, Dependencies is not in the default package, and does not know about things like Wikis and YamService, and so has no access to the regeneration code (which is in YamService). Need to move something.
    - perhaps define a dependencies service, which has a get method that first does an exists, and if it does not, regenerates the wiki
    - could also put the current YamService.regenerate in here

[CoW] security <AR>
- JsecRolePermissionRels are being created at Bootstrap, even if one exists. Querying with criteria, prototypes, and HQL, all fail to find existing JsecRolePermissionRels and therefore exclude them from being constructed again.

[YAM] fixed width <AR, MG>
- change fixed-width text to ^...^ to reduce the clashes with equals signs in normal text (NB changes might also be needed in JSPWikiToYamConverter, to remove escaping of =)
  - rewrite yam test files
  - scripts for site conversion (MG)
- change yam documentation re fixed width and check it ok in html

[CoW] security <AR>
- Default roles are created at bootstrap if they do not already exist. This means that if an administrator decides legitimately to remove certain roles or permissions, they will be recreated next bootstrap. Perhaps only create default objects if no other objects of this type exist?
- SF 2556528. Clicking on "new page" from a user with write permission produces and unauthorized error. Editing existing pages is permitted. another way to test: open any html page, then change the name in the browseraddress bar. If the current user is not admin, then an unauthorized error comes up.
- Selenium test added for the above (previous test was for admin creating a new page in workstation mode, which is different)

[YAM] Dependency regeneration <AR>
- integration tests: DependenciesServicesTests

[CoW] selenium tests <AR>
- Server user-management test suite repeaaaatedly failed at "assertTextPresent, Edit wiki page: index.html", firefox 3.05, Linux. Could not be replicated in the Selenium IDE, could not be replicated on a similar linux / firefox setup.
- Duplicate: for some users, Selenium tests fail when run from ant, even though they pass when run through the IDE, and even though wait conditions seem ok. The tests fail in the same way if you delete .cowrc.d For example, when Angus runs on his desktop:
  - server
    - IDE - succeeds when entire suite run without any breakpoints at full speed
    - ant fails at "assertTextPresent Edit wiki page: index.html" even though there is a waitForPageToLoad and a waitForLocation just before the step
  - workstation
    - IDE - succeeds when entire suite run without any breakpoints at full speed
    - ant fails in upload at "assertTextPresent Upload successful" even though there is a waitForPageToLoad and a waitForLocation just before the step

[CoW] selenium tests <AR>
- Remove assumptions on user from user-writing selenium tests

[CoW] selenium tests <AR>
- Dependency regeneration: add selenium tests for interface aspects
  - regeneration button
  - regeneration on updating a Wiki
  - setting regeneration time

[CoW] security <AR>
- if you try to create a new user with password "showerhead" you get a red box with nothing in it (i.e. the message string is empty)

TH
- simple sventon integration via /admin
- notes in the developer guide
- gettign it running without jetty (i.e. via grails builtin jetty):
  - war in cow/wars
  - there now exists a URL mapping to prevent /sventon being interpreted as a directory in main
  - there's a hook commented out in scripts/_Events.groovy that may or may not be running it properly - generated 404s - so needs fixing
  - HC couldn't duplicate TH's result in getting jetty to run sventon and configuring a repo (on java 1.6.0, tried file and http urls, just got blank pages)
- find a way to get the selenium upload test working when running in production (e.g. via: ant -Dgate.cow.grails.command="prod run-war" grails)
- add a simple cancel button to the upload dialog
- uploading over already existing files should not work unless the user has selected "overwrite existing files" in a checkbox in the upload dialogue (this was present in the webflow version I think)
- it would be good if the redirect at the end of a successful upload went to the directory view, not to index.html when it exists
- can you change the look&feel to be more like the rest of cow? e.g. use a tooltip for the "Choose a file or ZIP archive to upload in the current directory." message, stop using pop-up windows, instead put messages in _messages.gsp style, don't change the spacing of the top-bar when a user clicks upload

[CoW, YAM] bugs (http://sourceforge.net/tracker/?group_id=208243&atid=1004913)
- bug not on tracker/Fixed
  will work with clean build? main index template doesn't exist in production deploy (do we need production deploy now grails run-war exists? check issue about application context path)
- bug not on tracker/Fixed
  YAM regeneration doesn't work for new pages <AR,HC>
- bug not on tracker/Can't replicated
  check that these old xsl bugs have been cleared up, and that it is preserving everything except whitespace
  - loss of predicate and underlining in gatewiki.html:

      =svn co %(https://gatewiki.svn.sourceforge.net/svnroot/gatewiki/trunk,
      https://gatewiki.svn.sourceforge.net/svnroot/gatewiki/trunk) gatewiki=
      becomes =svn co
      https://gatewiki.svn.sourceforge.net/svnroot/gatewiki/trunk gatewiki= 

      - _Set your project JDK from Platform settings._
      becomes
      - Set your project JDK from Platform settings.

bug 1838819 Closed/Fixed
- create new page needs some validation and help text
  - when create is pressed with nothing entered from the "new page" directory view an error message should be displayed and the "new page" view redisplayed
  - when create is pressed with nothing entered from the "new page" directory view with a page name containing spaces the "new page" view should be redisplayed with a message saying page names can't contain spaces; also, the validation should disallow other inappropriate characters: probably just allow any letter, number, dash, dot, or underscore
  - create new page for an existing page should message "File ${new-name} exists" and redisplay the page
  - create new page shouldn't forward to edit but to show (i.e. the result should be an empty page in show view, which can then be edited)
  - validation using command object: class NewPageCommand { // the property name has to be the same as the form field to be // validated String newPageName static constraints = { newPageName(blank:false,minSize:1,matches:"[a-zA-Z0-9._-]+") } } ... def create = { NewpageCommand cmd -> ... if (cmd.hasErrors()) { flash.message = "Invalid page name" ...
bug 1943771 Closed/Can't replicated
- when checking for existence of a relative URI anchor parts should first be deleted, e.g. gatewiki.html#section-3.3.
bug 2526041 Closed/Can't replicated
- create a directory with spaces in the name (in server mode)
- then create a file there and try to edit
- svn exception due to url encoding of the spaces
- also: no reporting of the exception!
bug 2526442 Closed/Fixed
- when in server mode, working with a wiki area that is not in ...sandboxes (e.g. /home/hamish/tmp) creating a directory and then creating a file within it fails
  - this is probably because of this code in PageService.toStaging, which assumes that all sandboxes live in the user.home svn rep: SVNURL pageDirSvnURL = SVNURL.fromFile(new File("${conf.svnrep}/trunk/${pageWikiDir}"))
bug 2537529 Open
- the shutdown-prod ant target needs to stop the jetty instance spawned by grails
bug (not on sourceforge)
- it would appear that you can't upload into the help wiki even if logged in as an admin user
- I haven't tried editing pages so I don't know if that works or not

[CoW] semantics integration
- GateService from grails-semantics-plugin <OW, DD, HC, VT>
  - wraps QuestIO / OWLIM / ORDI
  - translates QuestIO or Sparql queries
  - send them to the ontology service
  - creates XHTML and sends it back to the client
- first tasks for Olaf:
  - using the GATE ontology API
    - java or groovy to take an ontology URL (file: to start with) and:
      - lazily initialise the GATE library (i.e. check for previous initialisation, if none then initialise)
      - check a pool of ontology LRs for one with the requested URL
        
        if present, return it
        
        else, create a GATE ontology LR for the URL
    - unit tests to verify this code and to do example queries using SERQL
  - a Grails service to live in the "semantics plugin"
    - provide a GateController with a serql action
    - this action takes a query (probably in the query part of the URL) and returns an (X)HTML fragment representing the results

[CoW] selenium tests
- When running selenium tests, we sometimes get:
  - org.springframework.scheduling.quartz.JobMethodInvocationFailedException:
    - Invocation of method 'execute' on target class [class DependenciesJob]
    - failed; nested exception is java.lang.NoClassDefFoundError:
    - DependenciesJob$executeclosure1
  - but not when running cow normally. Maybe it is something to do with the timing of compiling quartz jobs? or related to the "too many open files" problem that the selenium tests often have when there is no server running (i.e. when using "ant test-selenium"). the problem generally seems to be when the upload test is running
[CoW] remembered passwords
- it seems that at least in Firefox usernames and passwords are being remembered across different instances of CoW running on different URLs. Is this a Cow or browser bug?

disallow changing of the path when updating wikis in WikiController (or update the SandboxManager entries)
add -l pretty to yam2html and test on some in-use files
the login link appears when running in workstation mode - it shouldn't
checked that upload works in production mode on jetty (the current SAM wiki problem is therefore Tomcat-specific)

when you check in CoW stuff that breaks the server start (e.g. code that doesn't compile in Bootstrap) Hudson reports this as test failures in the Selenium tests (which is good) but the output from those failures doesn't show the compile errors (which is bad). is there a way to get this output? a separate "run-server" test?
fix deprecation warnings in gate.versioning
get svnkit to default to LF only for all text files, inc. yam, code, etc., in the sandboxes
make logfile Rolling

why does the search form in cowpage.gsp cause the user name link in server mode to miss the space between first and second name? fix cowpage.gsp...
stop the cowpage.gsp layout displaying the top search box in advanced.gsp
- /admin/solr: include the advanced search form via iframe
  - maybe this just gets moved to /search/advanced?
We need eventually to have a separate index for each wiki area (i.e. for each SVN sandbox file tree), because some users will not have permission to search all the trees. As a simplifying assumption for the first version we'll only support workstation mode, and search only the main wiki area.
- how to search different trees separately with Solr?
tell solr to put it's data directory in ../nutch+solr/solr-data, not cow/solr

is default role given to new users, or just new registrants?

sprint 30/03/09 to 10/04/09
- [AR] change the directory in the permission model to be a regular expression. Then we can have directory permissions such as "/sam(?!/internal).*"
  - tests
- [AR] document directory permissions
  - what empty strings mean for patterns
  - how include and exclude patterns interact
  - reference for regexp syntax
  - what to do about slashes: the required dir will not have a leading slash
- [AR] move the svn up from cron to quartz
  - if the repository is locked, skip this one
  - if we get conflicts (should never happen!)... log it as an error
  - provide similar facility as for the regen stuff (choose a time interval, maybe a "do it now" button, ideally capture the update messages...)
  - selenium tests
  - integration tests: see comments in SvnServiceTests.groovy
  - set logging appropriately
  - document
- [AR, HC] versioning
  - remove the "0.1" from the data directory and model it as domain class CowDataVersion
  - write generic upgrade code in Bootstrap
- [MG] convert the GATE team wiki, start running gate.ac.uk experiments
  - ask HC re. create-custom-cow-site.sh script, site update protocol etc.
- [NA] search
  - add directory-level security to solr
  - set up indexing from the admin pages
  - search via nutch+solr; admin interface for running nutch over sandboxes (with solr indexing); user interface for searching
    - /search/: accept posts from the search box on the main page view
  - quartz job to collect edited/newly updated docs and index them
  - support for upto 20 cores (20 sandboxes)
  - separate logging for solr (as solr uses java logging)
  - some documentation in gatewiki.html (move the nutch+solr.html stuff there for developers; add some user stuff when it is working properly)

[HC] update create-custom-cow-site.sh so that it
- has modes or usage methods for "new site, update site"
- addresses the discussion in gatewiki.html#runtime-dependencies

Sprint ended 8th may:

IR, MG [6, 3]
- think about the following Hudson issues:
  - a job to do basic testing against the production servers and send mail (and SMS?!) when they break
- finish sventon integration sufficient for gate.ac.uk
  - move sventon config to cowrc.d/sventon-config [Done]
  - configuring SVNREPs other than the default
    - [Done] CoW now configures sventon with one "repository" for each wiki area, named "wiki-N", pointing at the root of whatever repository the top sandbox directory was checked out from (so doesn't do externals but normal repos will work).
  - base-config.properties assumes a file: URL... shouldn't?
    - No longer relevant given the new config mechanism - now it just pulls the repos root out of the svn working copy.
HC [2 days]
- a daemon script that can be linked from /etc/init.d which does a run-prod
- [MG] versioning
  - finish upgrade code
    - write Bootstrap.upgrade03(List wikis) as commented
    - amend gate.cow.data in Config.groovy
  - tag cow and bump version to 0.3-snapshot (data version 0.3) and upgrade the various installations
- document tables / lists newlines thing
NA, HC [NA 5, AR 0.2 days]
- /search/advanced: maybe do the same as for advanced? [Done]
  - removed this page and added other indexing related advanced features on the administerIndex.gsp page. There isn't much on the advanced search page anyway. Almost everything there is already used in our search functionality.
- a page to list what is in the indexing queue [Done]
- Implement a queue for indexing of wiki areas [Done]
- move crawl-urlfilter.txt to urlfilter-default.txt with only static content in it. create crawl-urlfilter.txt at runtime for filtering. [Done]
installation loose ends arising from the sprint
- sam repository: turn off cron and use quartz
- team wiki: deploy and go live
- gate.ac.uk: think further if it makes sense on gatewiki
- gate-top index and install on gate-compute2 [Done]

0.3
- upgrade sam wiki [MG, IR]
- tag the gatewiki repository as 0.3 [MG]
- fix dependencies - dependent files are not being committed after changes [MG]
- a daemon script that can be linked from /etc/init.d which does a run-prod [MG]
- gate.ac.uk trialing [IR, MG, HC]
- Hudson testing against the production instances - heartbeat runs on most servers[MG]
- document tables / lists newlines thing [HC]

breadcrumbs: see http://www.useit.com/alertbox/breadcrumbs.html; a simple design where the directories above the current page are listed along with the top-level is probably enough, e.g. for the page doc/yam/summary.html we would have "You're here: Home > Help > Doc > Yam" as breadcrumbs. see also Darryl Pentz discussion in Jan on the mailing list. [NA]
changed the behaviour of PageController.show and HtmlTranslator so:
- /page/show/... is never thought to be an non-existant page during translation (so is not coloured red)
- non-existant pages have a m[issing]=1 parameter added to the url
- only non-existant pages with the missing param are forwarded to create new page
- i.e. it will now only offer to create missing pages when they were found to be non-existant within the current area during translation.
evaluated the grails nav plugin:
- the Navigation Plugin gives you site navigation menus by convention. You just define a property in your controllers to determine what actions are shown and in what groups. http://www.grails.org/Navigation+Plugin
- this is nice, but from the mailing list it seems not without issues, and we would have to do some work to figure out how to add it into layouts, how to have several instances for side bar and so on, and also how to populate it from YAM lists, which are the obvious basis for navigation in CoW. so: I guess we won't use this for now
implemented Navigation for Specific Directory Trees:
- in PC, where it said pageBody = FileUtils.getBody(pageFile), changed this to also do PageService.getNavigationModel(pageFileDir, wiki.path)
  - for each layout file in conf.navigation.files
    - find the nearest one to current dir, up the tree to wiki top
    - get its body
    - set as var in the model to pass to the layout
layouts for whole wiki-areas [NA]
- add a "areaLayoutName" to Wiki, which would reference a layout GSP in the site-specific plugin; PageLinkTabLib would then use this for the layout in that wiki area
- modify PageLinkTaglib.layoutLink
- the check to see which area layouts are present should be done in Bootstrap in fact, and store the list of them in a config var, just like setGuestLayoutPath() does when called from Bootstrap now

non-native pages like javadocs: [NA, HC]
- any directory that now contains a file called .cow:raw-html will now render any non-YAM HTML files directly (see gatewiki.html#raw-html). this means you can serve e.g. javadocs. BUT it is a potentially nasty security hole and is turned off by default, so we need to add a better mechanism:
  - add a list of path patterns to each wiki area, which would match either files or directories (AR might know the GORM invocations? a new table RawHtmlPatterns?)
  - do an Ant API path match in PageController.show, according to config settings (IR would know the Ant API call?)
  - when a non-native HTML (see the developer guide discussion path matches serve it raw as with the .cow:raw-html mechanism
  - check this works for javadocs and for other gate.ac.uk pages (tao etc.)
    - finish the documentation in the user guide at gatewiki.html#raw-html
  - do we need a link back to the Wiki area from pages served raw? this could be nice, but a) isn't typical, e.g. for javadocs, and b) would need some munging of the page, e.g. by insertion (maybe of iframe?) just after the body tag or similar. so perhaps not worth it? [didn't do this]

finish sventon integration [IR]
- proper proxying of sventon requests through a CoW controller (/admin/sventon)
  - sventon pages contain all sorts of absolute path links: <a href="/sventon/repos/...">, <script src="/sventon/js/sventon.js" />, <div id="/sventon/ajax/..."> (where /sventon is the context path where sventon is deployed)
  - need to fix these up to point through cow, i.e. /sventon/repos -> /admin/sventon/repos (for root-level CoW) or /sam/sventon/ajax -> /sam/admin/sventon/ajax (for Cow deployed under /sam)
  - sventon pages include a <base> element, need to fix that up too. This will break any relative links in the CoW top banner (if there are any).
  - /ajax and /js need to be proxied raw, /anythingelse needs a CoW banner
  - thankfully, sventon.js uses a global JavaScript variable for the contextPath, so don't need to massage the JavaScript code (phew).
  - when fetching the proxied content from sventon, add ;jsessionid=<session-id-from-cow> to the URLs, to keep sventon and cow sessions in sync. Any jsessionid in URLs generated by sventon need to be stripped during the HTML massaging process.
- once proxying works, we can link from pages direct to their history.
- simpler idea:
  - remove the localhost filter, allowing direct access to /sventon, but...
  - add a filter that checks for an object in the session before permitting access.
  - this object is put in the session by CoW, posting to a URL that is still restricted to localhost only.

grails 1.1.1 [IR]
MG recommendation: The one thing I would change is to set the DTD to HTML 4.01 and not XHTML. The file isn't valid XHTML and the server is pushing the file out as text/html so it makes no sense to use the XHTML DTD. Also strict browsers will start to behave strangely if the XHTML isn't valid and may not display anything at all.

conflicts no longer being reported properly in edit flow
- fixed - in the case of repository changes to a file, PageService was using the repository version of the file in its entirety, rather than doing any merging.
when you do an edit and the svn auth fails, there is an exception thrown (org.tmatesoft.svn.core.SVNException: svn: CowAuthenticationManager only provides a single authentication token) but it isn't reported to the user
- fixed - any exceptions that occur in the edit flow (and are picked up by an on(Exception).to "errorExit") are reported to the user on the redirected show page. SVN auth errors present a custom error message, other exceptions just print a generic message, but special cases for other exception types are easy to add.

jape yamalyser
- fix the jasper [IR, HC]
  - fixed - I've repackaged GATE's copy of jasper-compiler-jdt.jar into a different package name using JarJar links so it no longer conflicts with the (slightly older) version provided in grails/lib.
- spring xgapp pool [IR]

Add a text box for svn log message (in server mode).
Added plugins facility to YAM and did Google and Twitter plugins.
Renamed web-app/images to web-app/__images. Documented remaining top-level namespace conflicts in main area wiki.

Implemented NA's solution for raw HTML in iFrame; managed to reuse the show action to do the heavy lifting, and added a print facility.
Turned off the dev mode logo to make debugging easier.
Generalised PageLinkTagLib.createPageURIBase(attrs) to allow for multiple query elements (e.g. ?type=dir&print=1)
Failed miserably to write any tests for the new stuff.
Implemented a single "Edit" button and allow user prefs re. what type of editor for yam htmls (default to form-based)
- If there is a logged in user, store preference per user in the DB
- If there is no logged in user (workstation mode or pages with anon write permission), store preference in a browser cookie.
Sventon history view now uses an iframe surrounded by the usual CoW header, breadcrumb trail and a link back to "show" the page whose history you are viewing.
issue about what to do with unsuccessful edits
- edit cancellations now svn revert and delete the staging copy, so the next edit will start from clean.

doing some tests for the FckEditor (should be ok now we're on selenium not canoo?). there was a bug report that said all the list nesting disappeared after an edit - might have been operator error of some sort, but would be good to check. also might be good to upgrade to latest FckEditor if there's a newer release
- The loss of nesting is due to the fact that the HTML-to-yam XSL gets confused by the fact that we now nest the nested <ul> inside the parent <li>, and it adds a blank (well, whitespace-only) line before the first item of the nested list. This is now fixed in HtmlToYamConverter.
when creating a user with a bad password field I got this error:
- Password must contain at least two digits
- Server Error :-(
- Error Details
- Message: Cannot invoke method hasProperty() on null object Caused by: Error processing GroovyPageView: Error executing tag <g:form>: java.lang.NullPointerException: Cannot invoke method hasProperty() on null object
- Fixed by passing the actual UserCommand object in the model for the render call, rather than passing a map.
jape yamalyser [HC, NA]
- do wrapping
- recognise inter-unit gaps and create Edit annotations with "numBlankLines" attr
- do edits
Solr search issues [NA]
- when you say "index" from the admin pages, probably it should delete the previous index? or at least offer you the option
- when you fail to delete the previous index before re-indexing, results from outside of the current sandbox (e.g. you previously indexed some other sandbox then moved to a new one) will be presented as file:// with the full path; this is a security hole and should never happen
- searching for a quoted string e.g. "software architecture for language engineering" doesn't work; is this supported by the soly QL? perhaps put a summary of that QL at the bottom of the search results page?
fix run-prod [IR]
- ant run-prod was giving NPE in eventConfigureJetty - fixed locally, GRAILS-5055 in JIRA - now getting a different NPE (GRAILS-4673) which will be fixed when we upgrade to grails 1.2. In the meantime we work around this by treating DataSource.groovy as an external configuration file (via grails.config.locations) loaded from the config directory, rather than as a standard compiled class in grails-app/conf.
find a better way to avoid showing the search box in the admin area than checking for null id
- Now conditions on webRequest.controllerName to only show the box for PageController and SearchController requests.
ehcache: "jsecurity seems to create a dir called /tmp/jsecurity-ehcache/. prob is that when you run several cows as different people it creates perms probs" possible solution: http://www.jsecurity.org/node/1046
- Now generates ehcache config at BootStrap time pointing to a temp dir under cowrc.d
Layout issues
- improve the float layout, e.g. to deal with top-bar links scrolling to under the logo (try yui frames thing from GIA book?)
- ditch the jquery menus and have static html there.
GAU
- g8rs.net
  - run fail2ban
  - ge1/2 cross-snapshots (inc. /etc and /home) into /home/backup
  - stop running on 8080
externals in gau-main-tree for sale, gate etc.
browser compatibility testing [IR]
start pushing TAO into the g8rs main wiki area [IR]
- add a secure rsync from hudson to push the TAO build (minus .svns) into ge1:/data/herd/g8rs-data/gau-main-tree/sale/tao (make sure that there's a .cow:no-iframe file in the directory that Tex4HT creates)
- ref in the tex4ht css in the gau cowguest.gsp.
allow ordering of search results by recency [NA]
move .grails to cow/dot-grails to avoid conflicts on server start
dependencies issue [IR]
- the serialised dependencies files should include the run mode in the name (dev, prod, test), or else they may be saved relative to one wiki in e.g. dev mode then restored relative to another in production mode?
Sventon has a line of links that let you look up the tree to parent directories. Changed so that only those levels that the current session has permission on will be clickable [IR]
turned off regen of sandboxes from bootstrap (make it a config option) [HC]
removed the border style from YAM table elements
start pushing TAO and javadoc into the g8rs main wiki area [IR]
- make sale/tao, javadoc dirs and download/snapshots on g8rs non-writeable (so people don't try and edit non-svn content there) [note: I've added a topdoc/systems-admin section under GE1 stuff for rules like this that we need to remember for the migration]
possible issues to check up on [HC, IR]
- will it manage to serve GB movies?! otherwise some cludge with apache serving gate-big? Just tried the big demo movie and it seemed to work for me. Not sure how it'll cope under load though. (IR)
- am I right to remember that when checking out to staging it first does a depth 0 co then an up on the edited file? if not we may end up with very big staging areas? That is correct (IR)
edit of /credits.html (which is a raw HTML file) doesn't work (some markup problem seems to prevent the execution of the FCK javascript) [IR]
- fixed the FckEditorTagLib to use encodeAsJavaScript on all values that go into JS string literals, which has fixed this.
solr [NA]
- try searching on http://gate.ac.uk/... too many open files bug
- when matching the path prefix on the file:// uri for search result hits, if the prefix doesn't match then log an error and throw away that hit (the error text: "failed to match hit prefix; this probably means the sandbox was moved on disk and the index needs regenerating")
- please add a field that indexes file names (e.g. business.html)
upload [IR]
- upload of .tgz fails without message
- upload of .zip of a directory claims to work but creates a zero-length file for directories
frame the mailing list archives (sf, gmane, ...) on new-trunk/gau-main-tree/mail/archives.html (not a yam page, just html) [NA]
user guide to logging in, editing etc., revise gatewiki.html [HC]
g8rs.net [HC, IR]
- CSS the twitter div a little to make it different (diff colour? make the URLs within it smaller to hide their ugliness?) [IR]
contact page [NA]
- gate.cow.ContactController with a single default action which
  - displays a captcha (see RegisterController and also Config.groovy for the captcha setup) and allows sending of an email to gate.cow.
tell grails to also run on https and get jetty to redirect logins there (done, though we'll not use this in production) [IR/MG]
buy a ssl certificate for *.gate.ac.uk [HC]
secure login
- 443 handling via apache VH [IR]
  - Infrastructure done, using a temporary certificate until we get the real one.
analyse the security holes found by ratproxy [MG]
javascript in gau site plugin to replace bitly links in the twitter with "Read more..." [MG]
delete flow [IR]
- directory-level view
  - change to 3-column table
    - file type icon
    - file name
    - select checkbox
make a page containing a table with two rows, one of which is empty. Save or preview results in the entire page content being dropped without warning. (i.e. the error reporting seems broken; faults in table syntax generate yam errors; need to look into this)
- The underlying parser error for this has now been fixed but the principle still stands that we should handle yam parsing errors more cleanly. [IR]
in the edit flow, the log message is not stored in the flow under some circumstances (probably from preview to continue editing) [IR]
I get a fair amount of flicker between menu items; is it possible to get the selection mechanism to work a little slower or something? (fixed, IR)
the iframe/frame combination on the /mail page is messy... let's just link to the pages and delete the frame stuff [HC]
lose all the XML-style tag closing stuff in YAM output, we're going with HTML not XMTML [HC]
get the hudson jobs to also do doc and to push the results (and the test results - see the links from cow/doc/index.html) [IR]
titles including markup incorrectly include the markup in the html title (e.g. %br puts a <br/> into the title) [MG]
when editing a page with no title the leading blank lines get removed [MG]
when previewing an edit the links are wrong (they point to "/page/edit..." locations); can we use a BASE directive to solve this? [MG]
change "switch to rich editor" to "switch to WSYWIG editor" [MG]
an empty .html causes NPE when served [MG]
finish the documentation on Firefox profiles, specifically how you create them to remember certificates [MG]
get edits of raw html to also include the headers [HC]
make use of recaptcha an option, and get audio to work this way (user credentials to go into .cowrc.groovy on ge1) [MG]
added a configurable /robots.txt [HC]
directory listings should not include directories for which the user doesn't have read permission [HC]
fix the sandbox corruption problem [IR]
- implement checkCancelled() in the SVN event handler; SVN operations originating with an edit pre-empt background quartz updates.
- at shutdown, wait for in-progress edit-based svn operations to complete and cancel background ones.
- log shutdowns
- operations on the staging area should use a diff sandbox object
rationalise the tabbing orders [MG]
logging [IR]
- turn off logging for cow db realm
- use a fifo into a log rotator for the stdout log
- add timestamps (though not in seconds!)
Set up mod_rewrite rules on g8rs.net to handle JSPWiki URLs
- /wiki/Wiki.jsp?page=TaoAnything goes to the JSPWiki at services.g.a.u
- /wiki/Wiki.jsp?page=AnythingElse goes to the new wiki sale/internal/wiki/AnythingElse.html
- can add special cases for other page titles if necessary
g8rs.net API docs: get hudson to keep the gapi and javadoc, then change the link from cow/doc/index.html to hudson [MG]
run jmeter on the open site; it manages at least 50 requests a second for the home page and all associated files and it was perfectly happy serving up 10 simultaneous requests for the main ANNIE demo movie (all 87.3 MB of it), I didn't try more than ten requests as JMeter was struggling with out of memory errors [MG]
fix conflict between Quartz versions [has been done during 1.1.1 upgrade]
- Grails provides Quartz 1.6.x
- the Quartz plugin uses 1.5.2
- these are incompatiable
  - currently fixed by placing 1.6.1 in the quartz plugin directory
  - THIS WAS A NASTY HACK!
fixing the selenium tests under hudson: seems worth trying to rewrite the tests that are unreliable, with more pauses and/or a diff set of selenium actions to do roughly the same things [MG]
double the amount of RAM for run-prod to increase the headroom (not done earlier to try and make sure it is headroom) [IR]
fix the solr index (chat with HC and IR about plans for this) [NA]
- is the fact that search by the anonymous user returns no results (e.g. for "gate") because of the indexing fault or some other reason? [NA]
change gate.cow.robots for gate.ac.uk [HC]
HC said "strangely, google search over the GMANE list archive returns less hits than the GMANE search (try e.g. "wyner"). perhaps add a GMANE search YAM plugin like the google one? see the GMANE search pages' "short url" link for what the request should look like" ... but mark investigated (below) and it seemed a) google had caught up and b) if we stop it being filtered it will be ok... [MG]
- for the "wyner" search
  - google (with filtering of similar pages turned off) returns 10 items: 6183, 6181, 6186, 6189, 6176, 6153, 6185, 6177, 4956, 5036
  - gmane search returns 11 items: 5036, 5045, 5810, 5879, 6153, 6176, 6186, 6212, 4956, 6189, 6183
  - only 7 pages are common to both lists: 6183, 6186, 6189, 6176, 6153, 4956, 5036
  - 3 pages only appear in the google search: 6181, 6185, 6177
  - 4 pages only appeat in the gmane search: 5045, 5810, 5879, 6212
  - so there are at least 14 pages which are (possibly) relevant to the query
merge in stuff from gate-server/html that has been added since the branch to gau-main-tree (e.g. the projects dir has had new stuff...) [VT] [done - nothing needed]
webflows [IR]
- selenium test for delete flow