#persist

EarthCube Brokering and DataONE

Participants:  
EarthCube Brokering:  Jay Pearlman,  Mattia Santoro, Enrico Boldrini
DataONE:  Dave Vieglais, Chris Jones, Bob Cook, and Yaxing Wei

Not able to particpate:  Matt Jones and Stefano Nativi 
Tardy:  SiriJodha Singh Khalsa,

Background

Brokering:  Stefano Nativi's talk on Brokering at AGU:
http://www.agu.org/focus_group/essi/documents/Union_2012_Nativi.pdf

DataONE:  Bill Michener's talk on DataONE (July 2012 to the NSF Science Advisory Board)
http://www.ornl.gov/filedownload?ftp=e;dir=uP12mVATj9OB


Agenda
1.  Introduction:  Goal is to initiate discussions about possible CI collaborations between DataONE and EarthCube Brokering
     - Discovery & Access Brokering is being used for the GEO System of Systems, in addition to EarthCube
2.  description of DataONE CI (Dave Vieglais, Matt Jones) 15 minutes
  - DataONE architecture docs: http://mule1.dataone.org/ArchitectureDocs-current/index.html
  - service oriented architecture, with distributed content (Member Nodes)
  - standard APIs that provide communication between Member Nodes
  - Coordinating Nodes (three currently):  keep track of content among Member Nodes, along with replication
  - security and authentication
  - metadata harvested and indexed; used by search interface (ONEMercury) and API
  - able to harvest a number of different scienc metadata formats; DataONE has mapping from each standard into the DataONE required metadata
  - investigator tool kit-- common tools that have been adapted for the DataONE API
          - Python library and command line tool
          - R plugin:  R code can be used directly with DataONE API
  - currently have ~6 Member Nodes and have another dozen in development 
  - provenance tracking, search capabilities are being enhanced
  - DataONE does not have ability to search / access outside of the current Member Nodes
  - new Member Nodes are identified through DataONE User Group and also self-identified
  
  DataONE is one of the DataNET projects; other DataNET projects have come on-line in the past year or so
      DataONE (https://dataone.org)
      DataNET Federation Consortium:  Reagan Moore, http://datafed.org/
      Data Conservancy:  Sayeed Choudhury at Johns Hopkins, http://dataconservancy.org/
      Terra Populus (Steve Ruggles at Minnesota, http://www.terrapop.org/)
      SEAD (Margaret Hedstrom, Beth Plale) http://sead-data.net/
      
3..  description of EarthCube Brokering (Stefano Nativi / Mattia) (10 minutes)

• http://www.eurogeoss.eu
• Interoperability arrangements via a service bus
  • metadata models, data models, encoding formats, vocabs
  • protocols: discovery, access, viz, etc
  • sets of heterogeneous CIs, each using its own service bus
• Each system implements it's own service bus
• Developed 3rd party components that implement each service bus in the brokering infrastructure framework
  • either on the client or on the service provider side
• SOA - Brokering approach
  • Users
  • Producers
  • Registry
  • Client apps need to know about diverse interfaces
• service consumer <-- broker (+registry) --> service provider
• Complexity moved from client&producer sides to middleware&cyber-infrastructure
• Allows Network of Networks
  • no need for federal specs
  • low entry barrier
  • multidisciplinary
• New capabilities
  • semantic discovery, data access, qa/qc, etc. other services
• Facilitates sustainability
• Brokering Framework
  • Discovery Broker (CSW/OpenSearch/OAI-PMH, etc) + Web 2.0 Resources
  • Semantic Broker (augments Discovery Broker through OpenSearch SPARQL)
  • Access Broker
    • problems: different services, formats, CRS, resolution, etc.
    • broker invokes transformation services to enable interoperability at the data level
• GEO Web Portal
  • provides access to catalogs
    
    
• current brokering focus is on geospatial data 
  
  

4.  discussion on collaboration between EarthCube Brokering and DataONE CI

• long-term data preservation (DataONE)
• benefit would be to make the DataONE data (heterogeneous ecological / field data) accessible through brokering
• access broker could obtain a data object (e.g., geoTIFF) and modify (subset, mosaic, reproject, reformat, change resolution, etc.) based on the user request
• Peformance and scalability. Brokering framework is cloud-based (e.g. Amazon cloud)
• Access-control for the brokering services. not available at this moment. brokering is now focusing on open and freely available data
• How the identification is achieved across distributed systems? Brokering framework relies on data providers for data preservation, identification, and other data management acitivies. Brokering framework provides a nice complementary layer for existing systems like DataONE CI.
  
  

next steps:

• Brokering needs to test DataONE API and some data resources.
  • CSW interface or similar discovery interface
  • Brokering is Java-based. DataONE provides Java library. DataONE API provides lower-level capabilities than ONE Mercury user search interface (e.g. identifier retrieval, resolving identifier across member nodes, ...)
  • 
    
• API to access DataONE Member Nodes:
  • http://mule1.dataone.org/ArchitectureDocs-current/apis/MN_APIs.html
• API to access DataONE Coordinating Nodes
  • http://mule1.dataone.org/ArchitectureDocs-current/apis/CN_APIs.html
• To understand DataONE messaging and object types:
  • http://mule1.dataone.org/ArchitectureDocs-current/apis/Types.html
• Java common library (type definitions etc): https://repository.dataone.org/software/cicore/trunk/d1_common_java/
• Java client library: https://repository.dataone.org/software/cicore/trunk/d1_libclient_java/
• Questions: log into IRC:  irc.ecoinformatics.org, channel #dataone
• DataONE discovery & access demonstration science scenarios (DataONE ProvWG is working on several scientific workflows focusing on climate modeling inter-comparison by involving discovery&access&publish capabilities provided by DataONE)
  
  
  
  

action:
contact infor for everyone