Sprint Closeout and Startup -- 2010-05-05
=========================================

:Who: Dave, Robert, Chad

:Note: sprint #5 will continue until the Friday before the meeting in May, i.e. 2010-05-21.


Good, Bad, Ugly
---------------

- Dave - many issues related to external contrstraints on time

- Chad - OK so far

- Roger - seems to be a lot of overhead associated with the process, but generally a worthwhile process.  Discipline is good - forces focus on issues that are designed rather than ad-doc

- Chad - Trac tool overly complicated, confusing interface.  Prefers pivotaltracker.com which has a much nicer interface. 

- Robert - sprint went really well this time.  Need to update trac more often.

- Roger - specifically for this sprint - was a bit more difficult due to unknowns (e.g. timing for Dryad, Giri, etc)

- Robert - need to find a mechanism for better engagement of others - Bruce, Giri, Jim, Ryan

- Roger - completely dependent on Giri for DAAC install so coordination critical



Goals for new Sprint
--------------------

Overall goal is implementing Member Node synchronization:  Usecase #06


Member Nodes - get(), getSystemMetadata(), listObjects()
- Dryad MN operational
- ORNL/DAAC MN operational
- Metacat server running


Coordinating Nodes - get(), search()
- need examples from Giri and Bruce
- need updated Mercury build - jar and war files
- enable mercury indexing
- complete search mechanism 
- CN buildout -> cn_service accessible -> integration testing


Coordinating Node - support UC06
- Metacat create() 
- Task queue
- Polling of listObjects
- component to retrieve content given direction from task queue and store in metacat
- converter from metacat to mercury converter depends on some infrastructure such as a message queue


Stories
-------
New:

- (Story, 5pt) Setup message queue for MN synchronize process and possibly for triggering the metdata conversion process (i.e. feeding into Mercury indexing)

  - evaluate options for queue - activemq, qpid, ...
  - Define the installation process for selected library to incorporate in cn_service
  - define messages
  - implement tests for messages


- (story, 3pt) Design and implement the scheduling component (the scheduler) that add messages to a queue for later processing 

  - based on quartz
  - configuration options (scheduling etc)
  - JMS listener will run something that will poll listObjects() (see Story of polling listObject())


- (story, 5pt) Design and implement a component that responds to tasks from the scheduler (above) for polling listObjects()

  - able to use the ITK client for this (handling the listObjects call)
  - maintain state of last time listobjects was called, serialize objects
  - update UC06 text to flesh out design


- (story, 8pt) Design and implement the component that will retrieve content from MN and store in Metacata

  - use client from the java itk library (https://trac.dataone.org/browser/software/cicore/trunk/itk/d1-java)
  -retrieve systemMetadata from MN based on ID read from messaging queue
  -parse systemMetadata to discover scimeta or data
  -if scimeta then store sysmeta and scimeta in metacat
  -otherwise only store sysmeta


- (story, 20pt) Setup a few VMs for integration testing of CNs

  - automatic build of CNs based on SVN
  - testing of repplication between CNs
  - at least one MN for testing UC06 interactions


- Dave - integration test client essential for this sprint

  - update the stories and tasks associated with ITK client implementation (especially python client)


- (Story - pull from project backlog into sprint #5) Update the CN buildout to use the latest available versions of Metacat, Mercury, and CN_service


- (story, 3pts) Refactor Metacat to make the CRUD service a singleton (2 hrs)

  - remove the servlet context passed into CRUD service
  - fix MC REST service client to use mime-multipart encoding

    - fix MC REST service to do API calls instead of servlet calls



#586 Some work remains - about 1 hr, move to next sprint

#79 - reassign to Waltz.  flesh out details.  Move to next sprint.  Lower priority - implement after UC06

#269 - All tasks move across to next sprint
  - Dave review documentation for Mercury
  
#288 - needs integration test to run it

#346 - completed - close

#565 - reassign to Chad

#262 - Assign to Dave, and adjust tasks (remove 263) 268 - not really part of 262 but required for testing of listObject() - part of general MN testing

#348 - in progress, move forward

#350 - move forward

#352 - increase time and move forward

#591 - move forward - lower priority

#582 - need to prepare some stuff for the May meeting

#536 - move forward. #554 needs to be updated

#605 - move forward

#452 - completed.  close ticket.  

#543 - done

#447 - move forward if necessary

#602 - move forward

#561, 562, 563 - remove.  Duplicates of other ORNL/DAAC related tasks / stories

#593, #594, #595

#337 - move forward, low priority

#331 - move to this sprint

#353 - move into this sprint

#601 - Move to next sprint, add a couple of stories for testing this, but don't refactor the code (i.e. document the outcome and make recommendations)
  - recommend a test of this in the test harness