Member Node Wranglers Fridays at 10:30 am Alaska 11:30 am Pacific 12:30 noon Mountain 1:30 pm Central 2:30 pm Eastern GTM info: * https://www1.gotomeeting.com/join/698027049 5 September 2014 Attending: Bruce (phone and epad), Laura, Skye, Chris, Mark, Robert, Ben, Rebecca (late but no charge this time) :) Amber (late) Regrets: Dave, Agenda: We've restructured the agenda to enable developers to come and know where things stand in their and others' MNs before moving on to more detailed discussions. 1. Status of upcoming MNs * Y1Q1 * ORNL MNs (RGD (4248) and EDORA (4247)) - Chris has been working with Ranjeet about formatID changes that need to be made on their end; CN-side changes to accomodate RGD/EDORA will be reflected in CCI 1.4.0 due out next week (week of 8 Sept) * MPC (3708) - Some details regarding system metadata, resource maps, and science metadata structure to be sorted out. Useful discussion about this Tuesday with MPC. We owe Wendy response on scimeta question (Bruce). ?? how to do Resource Maps if data is NOT in DataONE ?? dealing with those questions (not using CN resolvable URLs, etc.). * ORE verify ticket: https://redmine.dataone.org/issues/5942 * formatId verify ticket: https://redmine.dataone.org/issues/5937 * we need a discussion about GMN and how the PASTA GMN is working (will work) with LTER and how MPC's GMN - Laura/Mark to set up in a little while (let Mark get up to speed on GMN) - maybe f2f at AHM? Yep. * DFC (3532) - Chris has created certs for them for dev/sandbox/stage * UIC (3213) - latest from Bob is that they're without a sysadmin again, he'll have to rethink priorities for next couple months, and Laura it to ping him again in two weeks. Put Bob/Rebecca in contact re: $$ - sent Bob an email nudging him to think about what can realistically be accomplished in next two months and get with RK. * PPBio (3748) - MNDD/logo submitted, Listed on public website as upcoming - awaiting feedback from Debora * NKN (3238) - no change from last time: running in sandbox, loaded test data * IARC (4700) - Jim at MNF 9/4/14, * Y1Q2 * GBIF (4730) - From Tim: * We are writing up an MOU with DataONE now, and are committing to standing up a DataONE MN within around 12 months or so. Some of the D1 common Java libraries are a little messy (not thread safe etc) so I have started from scratch. I’ve done the XML bindings, the security layer and audit log handling and have a basic framework for a pluggable backend but it is far from complete. I’m currently working on https://github.com/timrobertson100/dataone but I have a big set of changes locally that I have not yet pushed there, so it is not worth looking right now. It’s a side project for me right now though, so it’s not progressing quickly. * We need to get clarification from Tim re: time frame for development; ask Dave if he can inquire * MoC on hold until after Phase 1 closeout and Phase 2 initiation efforts are done --- Discuss the MoC request from GBIF. Mark, Laura, and Rebecca to work on this MOU/MoC. Done! Announcement went out today from GBIF and DataONE will do one next week * Sierra Nevada Global Change Observatory (4296) - Antonio visited Mark 8/14; Antonio going home 9/7, will talk with his folks when he gets home * GLEON (3422) - Nothing new. Laura helping Corinna with the MNDD. (Very) rough draft out to Corinna this afternoon. * USCarolina (3689) - Just sent an email to Pat asking if he'd like to have a "sit down" - hope to see where he is and ensure he has what he needs to move forward. * MsTMIP at ORNL (6162) - We are starting conversations about MsTMIP. Funding from DataONE runs out 31 October; they're going to use a flavor of ISO, need to find out which, get examples, etc. * Y1Q3 * SAEON (3205) - new POC is Mike Metcalfe; email from Wim Hugo indicated they're upgraded their metacat on a different VM and they have some questions re: authentication, etc. - Jing would be a good POC for SAEON (talk with Dave) 2. High profile issues (or current items of interest) * DataONE and XSEDE: latest info from Line: 1. The draft letter to become a (level 3) SP has been circulated to relevant people within DataONE 2. The collaboration was discussed at the Leadership team teleconference – level 3 will be a starting point 3. The letter is currently being reviewed by the DataONE leadership team Current status: the letter went to LT a couple weeks ago, Rebecca will send on to Bill cy Amber/Dave) There is an appendix - is this information required as part of the SP agreement? In review with Dave. What's the next step after we become an XSEDE SP? We live as an XSEDE SP for a while, then we look at the possibility of an XSEDE MN? (nothing beyond us becoming a SP) before the AHM. * Issue discovered during LTER's transition from metacat to PASTA GMN - metacat EML for both sysmeta and science metadata is parsed out and stored in a database, but when it is reconstituted from the CN, it does NOT look the same as the original. Mark, Roger, Chris, Ben (?) looking into this from two perspectives: * what to do for the LTER metacat data which we want to incorporate into the GMN LTER MN to retain "authoritativeness" or ownership of the data, and * what causes this in the first place, as this is potentially a Big Deal. * metacat versions prior to 1.9 would shred the EML and store in RDB, 1.9 forward does not do that * IF... this issue is LTER only, a simple way forward would be to harvest (meta)data from the CNs and dump it in the GMN ... maybe... Roger is looking at this right now * follow-up: after investigating, Roger briefly responded to me saying "both getting the objects from the MN and from the CN is problematic and the fixes are out of my hands. I'll just wait until someone fixes the issues or determines how they want to deal with things." He gave some detailed information about what he found to Dave/Mark/Ben/Chris, so hopefully we can address this issue soon. * It is an issue, perhaps beyond LTER, but even if it is only LTER it is a Big Deal. Hold off for Mark. * email to Matt et al. re: putting this on the CCIT work plan * as of 9/5/14 * Mark has created a ticket for this issue - https://redmine.dataone.org/issues/3557 - current metacat objects can be hosted in the new GMN; however there are inconsistencies (sysmeta can vary between CN and original, scimeta can vary as well) * Plan a meeting week of 8 Sept or 15 Sept with Jing, Roger, Mark, Chris, Laura, ??? to discuss * Mark did sync the PASTA GMN in stage, but there are some oddities 1.5 Current MNs Dryad listObjects issue, see https://redmine.dataone.org/issues/6010 Ryan says they haven't been able to address this issue last 2 weeks 5. Around the room Ben - in Brazil last week, biodiversity MN, very early stages. SiBBr (http://www.sibbr.gov.br/) on a related note, UTK/USGS and DataONE also working with ICMBio (http://www.icmbio.gov.br/portal/), Pedro Correa (visiting Knoxville/OR for a year) Chris - EDAC (EPSCoR Tier 4 MN) (Hays) needs to sort our their DNS issue, then Chris can generate certs Mark - see above Rebecca - question on item from LT meeting * Sep 5: Certificate management issues encountered with Member Node deployment and Investigator Tool support (notably ONE-R). This is prompting a re-evaluation of the authentication mechanisms in use by DataONE infrastructure and the possibility of adding support for OAuth in parallel with the existing client side certificate mechanisms. * Chris: CILogon and X509 great from functional perspective but user perspective - not so great; looking at adding on another layer of authentication to improve user experience; OAuth 2.0 as Robert - nope Skye - looking at moving CCI 1.4.0 to production next week 3. Old action items MN Documentation - MN Deployment and MN Ongoing Operations - Laura to do this -- working on identifying needs and best methods to address those needs, including ask.dataone.org, documentation in mule1 or on dataone.org, etc. Laura to ask Rob about order of testing (when and where) - document this, also what does "good to go" mean? 4. Not-high profile issues Process for end-of-implemention (when/how to indicate a MN is "upcoming" on the dashboard, etc.) - Laura and Amber - we had previously decided that when a MN goes into stage testing, it is appropriate to show them as an upcoming MN. However, in redmine we don't currenty have a status indicating what stage a MN is working in. We have a "testing" status, so we had thought we'd use that - when a MN changes status to "testing", we can show it as upcoming. We still need to explicitly define the process and who does what when, and try it out on the next MN(s) in the queue. Amber: maybe list them on web as "upcoming" when we go to staging -- but we need a redmine state to show that staging has started. Dave to look and see if can notify Amber and Laura (and Bruce?) when there are changes to the stage and production node lists. Need a redmine state to show that staging has started. (Was under Current MNs related to SANParks, CDL, etc.) LT to address Memorandum of Understanding (re: operations and service expectations) between DataONE and MNs - is there anything I (LM) can be doing to move this along, draft something up, pull out the work previously done, etc.?? (Rebecca, Mark, & Laura have been tasked with this and will begin work week of 8/11) -- or maybe not. Wait a bit until Phase 1-Phase 2 transition over? Tickler (things to revisit periodically) * DUG discussion re: increased MN involvement with DataONE - distribution list to Rebecca 8/1/14 - when is a good time to get this out to the field? Action: document still needs to be cleaned, but not clear which edits are to be accepted. Bruce to work with Laura to get a finalized version of this document ready for Bill to send out. Target sending this out after PEP is completed. Defer action on this until end of August and revisit this in terms of available cycles. Put this as an action at LT for AHM if it hasn't been dealt with before then. Purpose of MN Description Document (past and future) Intent is to describe the (potential) MN, identify the types/quantity/formats of data they hold, - perhaps we need a "friendlier" format, perhaps an interview process; Workflow: should this information (MNDD) be collected at the beginning of the process, or is the way we've been doing it lately (after the fact) a new way of doing business?? Also consider if this information gathering (form or interview) is the best use of resources for those potential MNs who may or may not become a MN if implemented as a first step. Possibly change the workflow? Is a pdf the best way to view the information? Next steps: Laura to come up with alternative(s) to current MNDD - content is good, but format/mechanism needs some work. Another thing: Laura and Amber to look at workflow for last stages of implementation, test with EDAC too late for that, need to pick another one. Also -- how does the MN DD relate to: the node document: https://cn.dataone.org/cn/v1/node - developers have/create this information (node registration, see updateNodeCapabilities) and redmine?? <--- work with redmine as mostly-authoritative source Could a spreadsheet be a viable solution? Maybe. A database would work. In any case, some information is appropriately "private" - how would we handle that? Revisit the default "only results with data" checkbox to unchecked; plan is to move the checkbox from search page to results page but remain checked by default, initial draft in development environment.