Member Node Wranglers Fridays at 10:00 am AK 11:00 AM PDT 12:00 MDT 1:00pm CDT 2:00pm EDT) https://www1.gotomeeting.com/join/430697153 13 December 2013 Attending: Laura, Bruce, John, Chris, Rebecca Regrets: Dave, Amber Agenda: 1. High profile issues (or current items of interest) CN synch issue - hope to see resolution next week (week of 16 Dec); Robert and Skye busy writing code, hope to be ready for testing middle of next week; it's not a lot of code but it is very detailed; if we are ready to go then, we'll be monitoring testing over the break - if not, it will be after the beginning of the year Security breach on one of the virtual machines at UCSB; moving to key based authentication for the future (lesson learned: don't use the same username/password combo everywhere). This issue should be cleared up by Monday, then return to CN stuff. A somewhat difficult question: when do we expect a synch issue like this to recur? I.E. how common is this type of event? (ex post facto statistics say ~ once every 5 years ...) Can we use this episode to update the Project risk register? Is likelihood still low? Can we confirm that the impact was as assessed? (i.e. we did not lose data, did not lose user service, impact was some internal effort and some delay for some development - right? YES, need to update risk register. Is the CN synch issue related to firewall rules or Hazelcast? Replication among 3 CNs to replicate science metadata - config in metacat didn't get changed in metacat on all CNs; issue compounded by network split in Hazelcast. result: science metadata not replicated across all CNs so it is out of synch. ORC is the only CN right now; other two will need to be synched with ORC (union) and then "put back into production". Access policies and replication policies seem to be the most important things impacted (it's inconsistent across CNs) Risk register: risk - likelihood - impact (high/low) for each risk factor Hazelcast has bitten us twice now; as a 3rd party product, we want to look at another solution that works better without the potential problems Hazelcast pro (availability); others' pro (consistency) Add a Mitigation strategy: [May need better rewording by someone directly familiar, such as Chris] 1) DataONE is writing a auditing script(s) to help confirm merging and intergity of system metadata about CN's 2) DataONE is writing a monitoring harness to identify more quickly if/when this re-occurs. 3) Revisit examiniation of possible hazelcast equivalents Another issue: when we take a CN down to update, need to say the other (authoritative) CN is read-only temporarily while the update is happening. New MNs announcements? - holding off until after CN issues resolved What makes a MN officially in production? For example, GOA has data out there but we show them as still in testing in redmine. Internal communication regarding the steps to put a MN in production needs to be improved (next thing after the CN fix). Need a step where we evaluate the metadata itself. IDCC workshop 2/27/14? - email out to us 12/12/13 to identify questions to be answered in preparation for the workshop; we will use Matt's previous workshop's information, need to sort out who will be going to the workshop (Bruce, Amber, Dave (unconfirmed), Chris, maybe another developer?) Note to coredev asking if any of them want to come. Isis (developer at UNM) would like to get together to talk about dashboard next week. 2. Status of MNs (leftover from 12/6 until updated) * Y5Q2 * holding off on announcing NPN and SEAD until the CN issues are resolved * Gulf of Alaska DP (3531) - The GoA PIs et al. met before Thanksgiving, which is what we were waiting for before we could announce. Matt? * FRIM (4083) - Jing Tao D1 contact, Omar Ali is tech MN POC, currently installing latest version of Metacat, Jing sent an email to Omar. I haven't seen a reply yet. * LTER-Europe (3232) - Matt? * Dryad (3118) - From 11/8/13 (ish): Todd Vision said the metadata issues have been resolved and he has requested a timeline from Ryan - I'll share with team when I get it (RK) * From 11/14/13 MNF, Ryan said there are two issues, one of which should be resolved by code that he was to review Thurs afternoon, and another related to the fact that data in their development system isn't clean (i.e. may be incomplete, * EDAC (3221) - just a few more tickets to address; Rob and Soren working these. * Kansas (3188) - as of 12/12/13, exceptions are resolved and KUBI is ready to go to production. * U Illinois Chicago (3213) - Bob says "We still have to finish up installing the prerequisites for Metacat (Java). The cert is installed and functional. I'll be in touch with Chris to schedule our next install session." Laura to follow-up with Bob/Chris. * PPBio (3748) - Matt said last week: LLNC server migration is complete. May need support now for further configuration. Laura to be in touch with Debora. * SAEON (3205) - moving metacat to more permanent server - Matt? * NKN (3238) - Luke says "we are on track here in Idaho with our attempts to stand up the MN stack. We've been sidetracked with other priorities, but are committed to continuing forward." At 12 Dec MNF, Luke said they likely won't be moving on it until after the holidays. * Y5Q3 * USGS ScienceBase (4002) - Laura to check in with Mike Frame. * DFC (3532) - Bruce has emailed Lisa - update? (also invite to IDCC workshop) * Taiwan TERN (3211) - to be discussed at October ILTER meeting - ask Matt next week, Carol didn't get a chance to talk to anyone when she was there in Oct. * Y5Q4 * EU BON (3964) - Their December meeting has been pushed out to 29-31 January. * MPC (3708) - decided to go with GMN with Dublin Core for metadata; they think they will pursue OAI-PMH in the future for other purposes. All DC terms are contained in EML, Dave recommends create simple EML documents, so D1 doesn't have to do anything special to use; allows richer metadata if they choose. * Backlog * Contacting those who contacted us first; have already received word back from Movebank (Sarah Davidson) (moved to deferred) and met with Mike Huhns from USCarolina (planning); EPA AED meeting 7 January to discuss broader EPA involvement; Montana State University deferred; holding off on the two Brazilian entities to see what happens with PPBio (hope for success story there to encourage others' involvement); no response yet from other potential MNs contacted * iRODS is blocker for iEcolab and iPlant * DSpace is blocker for UTK * After holidays plan to get back in touch with Alison re: Australian involvement - make that next week. - Laura * Holding off on others in Backlog until there is some movement on current PY efforts. * GLEON - CUASHI for streaming data, metacat for archive, interested in being part of DataONE; they want to demonstrate to their membership, but want it in production to do that 3. Old action items 4. Not-high profile issues 5. Around the room