API Design

Service Type Discussion

Services maintain types, similar to mime-types, with a 'type/sub-type' format. A Connector may have multiple service types represented, each with a sub-type reflecting that Connector (ie, contact/twitter, link/twitter, message/twitter), and a Collection may have multiple sub-types, with the primary type representing that Collection (ie, contact/twitter, contact/google, contact/facebook).

Connector Level

Basic Connector Architecture Notes

Connectors are supported by data stores that are historical and archival. This means that they include all records as atomic and immutable entries, with 'edits' and 'deletes' being stored as additional entries that reference the target entity in the store, rather than modifying or deleting it in place. Additionally, the data in the store is preserved in a non-compressed and easily accessible format, such that the data will remain accessible via multiple means as technologies evolve over the years.

Records in a collection should always support the following fields on each record:

• Record ID
• Timestamp
• Source
• Reference ID (only if 'edit' or 'delete')
• Document Body

Basic Connector API Methods

Within the flow of the Locker, Connectors will primarily serve Collections as data sources, and the exposed methods reflect that pattern.

• /getTypes: Returns the list of Service Types currently represented in the Connector.
• /getAll: Retrieve all records, in chronological order.
• /getID: Retrieve a specific record, and any records that reference it.
• /getSince: Retrieve all records since either a Timestamp or ID argument.
• /getCurrent: Retrieve a consolidated set of records with all edits or deletes applied to the set.

Collection Level

Basic Collection Architecture Notes

Collections maintain two data stores - a historical and archival store, similar to Connectors, and a dynamic document store, that provides a more flexible and query-able index of the Collection. The historic document store is the record of the objects that the Collection has composed from other data sources (Connectors or apps), with Reference IDs pointing to source data for each object. The dynamic document store is the primary resource addressed by the data methods of the API.

Basic Collection API Methods

Collections are the primary data resource for apps, and the exposed methods reflect that. GET methods are currently exposed, and POST methods will be updated shortly. GET methods will be provided for all retrieval endpoints, and all edits and deletes will be done via POST endpoints (to more clearly reflect the fact that 'edits' and 'deletes' create records of those types that reference internal records, rather than modifying data in place).

Core Locker Data Access API

/list?{params}
	
	params:
		type=x
		subtype=x
		
	type spec:
		the name of a type; incompatible with a subtype param in the same query
		returns the subtypes available in that type
		
	subtype spec:
		the name of a subtype; incompatible with a type param in the same query
		returns the types that are available for that subtype
		

/fields?{params}

	params:
		type/subtype
		
	type/subtype spec:
		either type or full type/subtype name
		returns the list of indexed fields available on records within that type or type/subtype

/via?{params} # RENAME

	params:
		id
		
	id spec:
		id of a record
		returns a list of all objects composed from that record (traversing the via chain upward)

/get{type}?{params}
	
	params:
		id=x
		time=[]
		subtypes=[]
		terms=[]
		limit=x
		offset=x
		
	id spec:
		optional
		id of a record, if known
		
	time spec:
		optional
		array of one or two time values 
		if only one value, it is a simple filter; if two values, it is a range filter
	
	subtypes spec:
		optional
		array of subtypes (implicit OR)
		filters to just the subtypes passed in the array
	
	terms spec:
		required
		array of json-style key-value pairs
		allows dot-notation for addressing nested document fields
		all terms are evaluated with an implicit boolean AND; boolean OR evaluations must be explicitly declared
		OPERATORS:
			+		greater than; post-pended on a value (ie, 25+ is equivalent to 'greater than 25')
			-		less than; post-pended on a value (is 25- is equivalent to 'less than 25')
			+.		greater than or equal to; post-pended on a value (ie, 25+. is equivalent to 'greater than or equal to 25')
			-.		less than or equal to; post-pended on a value (ie, 25-. is equivalent to 'less than or equal to 25')
			OR		boolean OR
			!=		not equal to
			COUNT	returns the count of the nested query (ie, COUNT(name:'Thomas' , age:25+.) would return the number of people whose name is Thomas and who are 25 or older)
		
	limit spec:
		optional
		defines a max number of results to return for a query
		
	offset spec:
		optional
		defines an offset into the query results; can be paired with limit to provide paging

Sample Method Calls

1. get all emails to rand.fitzpatrick@gmail.com from jason@singly.com or jasoncavnar@gmail.com with the tag 'singly'

/getMessages?subtypes=[IMAP]&terms=[to:'rand.fitzpatrick@gmail.com', (from:'jason@singly.com' OR from:'jasoncavnar@gmail.com'), tag:'singly']

2. get all photos tagged Jason Cavnar or Jeremie Miller which are labeled with the event name 'Hashable Party'

/getPhotos?terms=[(tagged:'Jason Cavnar' OR tagged:'Jeremie Miller'), event:'Hashable Party']

3. get all contacts who are 21 or older in the state of California who have 'node.js' in their bio

/getContacts?terms=[age:21+., address.state:'CA', bio:'node.js']&limit=30