Notes for Sprint 2012-17 (2012-04-22 - 2012-05-05)
==================================================
Sprint Goals:
- Deploy production CNs and at least one production MN (as data source) plus replication target MNs
- Release versions of ITK components: CLI, R-plugin, ONE-Mercury
- Revise documentation, prepare for publish
Standup Notes 2012-04-23
------------------------
Outage Notice: There is a scheduled 24 hour power outage at UNM, 2012-05-25 17:30 - 2012-05-26 17:30. Routers and firewall will be offline and thus UNM network will be inaccessible from off campus. VM hosts etc remain online, but inaccessible.
UPDATE: Scheduled outage has been cancelled in 2012, perhaps due to outrage.
Process to Release
~~~~~~~~~~~~~~~~~~
Update Sandbox Environment
- version 1.0 common, libclient
- latest revisions of other components
Push updates to Stage (?)
Setup production environment
- Renew DataONE wildcard certificate (expires 2012-07-17)
- Verify operation of production virtual machines
- Install and configure CN software
- Install MN software (replication targets)
- Verify operation of production environment
- Configure production MN sources
Build version 1.0 release of software
- CNs
- MNs
- ITK Components
Backlog notes
- 2411 update comments and more to backlog
- Need a remove() / delete() operation that removes an object from a MN (2603)
- rename delete() to archive()
- create new method delete()
- Should this also be a CN method? Callable by authorized MN, audit process. Operation could be handled by update system metadata operation?
- quick way to remove offending objects is to set access policy to basically nobody.
2601 - bagit + ORE
Need a public document for how to become a member node
- SOLR Schema field names
- Review consistency of REST endpoints,
- How to better describe the API endpoints to be clear on interface definitions implemented by client vs server (i.e. the session parameter)
- Session should be indicated as optional in docs, falling back to public.
Design planning for CLI
-----------------------
% dataone
DataONE Command Line Interface
> package create pkg-abc
> set format-id eml://ecoinformatics.org/eml-2.1.0
> package scimeta add knb-lter-gce.297.17
> set format-id text/plain
> package scidata add INV-GCEM-0705a1
> package scidata add INV-GCEM-0705a2
> package scidata add INV-GCEM-0705a3
> package save
> package done/end/leave
%
create - upload to dataone
get <pid> - get from dataone and load into memory
update <newpid> - update object in dataone
delete - delete object from dataone
new <pid> - create new package in memory
open <filename> - open new package from disk
save <filename> - save new package to disk
clear - clear package in memory and start over
add <pid> [filename] - add the content of pid to package (eventually creating the object on a MN if filename provided)
remove <pid> - (implies removing all associated linkages too)
link <scimetapid> <scidatapid> -- adds and makes "documents" linkages (implies add)
unlink <scimetapid> <scidatapid> -- removes linkage
show - display package in memory (display everything that is known about the package)
show <pid> - show contents of file (if text)(hex dump if binary?)
done - leave package mode
Refactoring existing commands
-----------------------------
Get rid of:
meta
name
print
delete
Replace:
leave with done
scimeta with add
scidata with add
dataone package create <pkgpid> <scimetapid> <scidatapid> [...]
dataone package delete <pkgpid>
dataone package update <pkgpid> <newpkgid> <scimetapid> <scidatapid> [...]
dataone package show <pkgpid>
dataone -e 'package new pkgid; package add <pid> ; ...'
or
dataone -e 'package; new pkgid; add <pid>; ...'
Authorization
-------------
1. Identity mapping is stored on the CNs.
- mapping is performed by users
- verification of identity is performed by account administrators
- legacy identities need to be mapped as well, and these will not likely be in CILogon
2. Identity mapping defines equivalence of identities.
Transitivity: Given Subjects A, B, and C, with A mapped as equivalent to B, and B mapped as equivalent to C, by simple inference, A should be considered equivalent to C.
Given a certificate with Subject = A, and SubjectInfo containing Persons A and B with Person A having equivalentIdentity entries of B, and person B having equivalentIdentity entries of C, the list of subjects considered equivalent is A + B + C.
(See algorithm below)
Is this scalable? What happens when a person has a couple dozen identities?
Limiting hops from subject of certificate likely to introduce permission interpretation issues.
3. Identity information is transmitted as properties of the client certificate used when making the secure connection to a service.
- ambiguity when interpreting SubjectInfo which contains a list of Person and list of Group. Each Person contains a list of Subject that specifies equivalentIdentity. Does presence of a Person in a certificate SubjectInfo indicate equivalence to the Subject of the certificate or is it necessary to specify this in the equivalentIdentity Subject list? Better (more efficient) ways to represent this?
4. Interpreting Access to an object
Access control is applied on a per object basis, with rules encoded in the SystemMetadata.accessPolicy, which is composed of a list of one or more AccessRule entries, each of which has one Subject and one or more Permission entries.
An object with no accessPolicy is not accessible by any subject other than the rightsHolder Subject specified in the systemMetadata for the object.
5. Permissions are hierarchical. READ < WRITE < CHANGEPERMISSION. Thus a Subject with a CHANGEPERMISSION entry has WRITE and READ access.
6. Special Subjects "public", "authenticatedUser", and "verifiedUser" are defined.
7. Identity management - how does an identity and/or identity mapping become deprecated or revoked?
8. There is no mechanism to indicate if an identity mapping has been verified.
Actions:
- Need tool (java and python) that given session, returns list of equivalent identities
- Need tool (java and python) that given list of equivalent identities and an AccessPolicy, returns whether the list has permission to read, write, changepermission (rnahf: restored from earlier: seems to have been accidentally erased)
- Design mechanisms for revoking identity. Is it sufficient to remove the verified flag from a Person, and then only allow equivalence mapping between verified identities.
- Future Option: add the list of equivalentIdentities to the session certificate
For recursive algorithm:
https://repository.dataone.org/software/cicore/trunk/d1_common_java/src/main/java/org/dataone/service/types/v1/util/AuthUtils.java
test cases:
https://repository.dataone.org/software/cicore/trunk/d1_common_java/src/test/java/org/dataone/service/types/v1/util/AuthUtilsTestCase.java
Roger's algorithm for obtaining list of Subjects that are equivalent from a session:
getequivalentIdentities( session )
subjects = ["public"]
if session == null
return subjects
sessionSubject = normalize(session.DN)
subjects.append( sessionSubject )
subjects.append( "authenticatedUser" )
if session.subjectInfo == null
return subjects
for person in session.subjectInfo.person
if (person.subject == sessionSubject) and person.verified
subjects.append( "verifiedUser" )
for group in person.isMemberOf
subjects.append( group )
for subject in person.equivalentIdentity
subjects.append( subject )
for person2 in session.subjectInfo.person
if person2.verified and person2.subject not in subjects
subjects.append( person2.subject )
for group in person2.isMemberOf
subjects.append( group )
return subjects
- Start with empty list of subjects
- Add the symbolic subject, "public"
- If the connection was not made with a certificate, stop here.
- Get the DN from the Subject and serialize it to a standardized string. This
string is called Subject below.
- Add Subject
- Add the symbolic subject, "authenticatedUser"
- If the certificate does not have a SubjectInfo extension, stop here.
- Find the Person that has a subject that matches the Subject.
- If the Person has the verified flag set, add the symbolic subject,
"verifiedUser"
- Iterate over the Person's isMemberOf and add those subjects (which are groups)
- Iterate over the Person's equivalentIdentity and for each of them:
- Add its subject
- Find the corresponding Person
- If the Person has the verified flag set and if "verifiedUser" has not
already been added, add it.
- Iterate over that Person's isMemberOf and add those subjects