CoreDev Sprint 2012.37-Block.5.3
================================



Roger
-----

chris B.
--------
- local admin work at UT
- Test puppet nodes, 
    - remove the need to use apt-get. everything controlled by puppet
    - Requirement: need to be able to roll updates from one CN to the next (offset)
    - cn-buildout scripts will need to be refactored

Chris
-----
Ben
---

- Stage setup for 1.0.4
- Documents not yet fully replicated  (about half since early Satuday morning)
    -- still only 86k/113k on ORC
- Working with SAEON getting metcat issues sorted
    --Metacat-Metacat data replication, only about 20 objects in question. Not a high priority and would rather see them become a MN.
- Working on some Morpho bugs
        --Released v1.10.0 on 9/10/2012
- Morpho work involves a fair bit of libclient refactoring work
    -- refine design proposal for Thurs, meet with Matt at NCEAS to hash out plan forward. Likely less libclient refactoring than we have been discussing  so that we can actually move forward.
    --identify concrete goals and discrete components to develop in Morpho.
- new tag of Metacat to fix a minor bug
- will work on the Morpho design/code
- Working in stage, found the reason we're missing the /portal connection on certain CNs
- Worked with Matt on Morpho impl
- Working on cn-buildout currently


Rob
---

- Working on libclient design issues that are being driven by Morpho (e.g. datastore)
    - rewrite of D1Object looks inevitable, may need to deprecate this particular class
    - need to push libclient to releases.dataone.org, and then make changes from that point on
    - may need td to continue support of D1Object until other MN admins can make the transition
    - for instance, MNode is tied to the v1 params, etc
- Continuing to work with Ryan on Dryad MN
  -- Some issues  with mutability of content that need to be sorted out (metadata only)
  -- Friday: hasn't heard back from Ryan yet, sent him a few more bugs,, dealing with malformed system metadata
  -- still having issues with system metadata, trying to manually understand the issues since the tester wasn't clear on the issue
- Incorporating ideas for libclient based on Jing's work
 - What about the scenario where systemmetadata needs to be changed after it's uploaded to an MN and sync'd to CNs.  There's no MN.updateSystemMetadata()


Robert
------

- stage update running on 1.0.4
- SSL did stop at one point.
- replication between CNs proceeding, but still slow
    - ended but is missing 27K documents
- indexing is producing lots of logging, will reduce that in the code
- no exceptions apparent so far
- some discrepancy between UCSB and (UNM, ORC) for listObjects
  -- fixed by restarting tc
- Discrepancy on UNM between identifierList and objectCount
    - seem to be related to integration test pids, will remove these for now to emulate the production environment more closely
- will upgrade java on production, will require reboots on CNs
    - we'll need sysadmin support for this
    - upgrade UNM and ORC first, then alter RR, then upgrade UCSB
- Log aggregation testing went well
- Ensuring that sync is working corrently in stage - will turn sync on in stage now that replication is working
- If there's no new content, will update the last harvest time on one instance to exercise the sync tasks to be sure d1_sync is working as advertised
- 115807 objects in stage in the identifier tables
    - Ben is seeing some discrepancies on the order of 125 pids
    - May be test docs?
    - ben will add an ORE to the CN, to make sure it replicates




Skye
----

- Testing 1.0.4 issues, 
- Need to finalize testing the index back-off strategy
-- New mockup of member node list: https://cn-dev-unm-1.test.dataone.org/d1Search/nodeTable.html
- Requires support on the Drupal side for implementation
- Will require a few data services to be implemented on the CNs for support -> likely to be part of 1.1 release
- Dashboard notes at: http://epad.dataone.org/DashboardNodes
- Discussion on issue of OpenLayers use within ONEMercury.
  - Update OpenLayers vs. remove OpenLayers
- Can be available for usability groups during AHM
- Will work on CN indexing acting as an admin to get around security issues
- working on serving up the .deb files for SOLR since the launchpad PPA location has been wiped
    - could also use the tools at launchpad to generate the debs
    - put them into ubuntu-extras at first
- Testing the backoff strategy for indexing in stage - working well so far
- Will work on minor CSS change
- Fixed the map initialization problem in ONEMerc - could be pushed into 1.0.4
- Looked at upgrading openlayers, would only be a minor upgrade
- working on deb packaging
- securing the search index, currently connecting as the public user, and can't index private data
  - parser API doesn't use any libclient, changing this
  - looking at an admin ssolr endpoint, wouldn't go through security servlet, would only be exposed locally
  - best is to migrate to using libclient

Matt
----

- Starting to get cranked up on R work next week


Dave
----

- Preparations for AHM
    - difficult process of organizing AHM schedule
    - re-aquianting with othe working group efforts
    - looking at next prioritites for future development


1.0.4 Deploy
------------

Need to test:
- Startup log aggregation
- Index back-off
- Make sure that UCSB and UNM consistent (use the replicate button)

- Discussion on the release
- Backups are a time issue
    - need to create SQL backups, /var/metacat backups
    - If /var dies, these do no good, bututif individual .db files are corrupt, this may help
    - can push the backup scripts
Jing
----