JSON-RPC in a nutshell

JSON-RPC is a well documented standard for webservices. It is like XML-RPC but instead of XML, the data exchange format is JSON (Java Script Object Notation).

  • all requests are HTTP POST requests
  • all requests look like this:
{
  "method": "methodName",
  "params": ["an", "array", "with", "some", "parameters"],
  "id": "my mandatory request reference id",
  "jsonrpc": "2.0"
}
  • without the id, the server gets notified but sends nothing back (only HTTP 200 OK but no data)
  • all responses from the server return HTTP 200 OK - even when an error occured
  • all responses return the same id as in the request
  • because of this id, asynchronous communication is possible
  • in case of a success, the response looks like this:
{
  "result": {
    "message": "you sent some parameters"
  },
  "id": "my mandatory request reference id",
  "jsonrpc": "2.0"
}
  • in case of an error, the response looks like this:
{
  "error": {
    "code": 12345,
    "message": "you sent me some wrong parameters"
  },
  "id": "my mandatory request reference id",
  "jsonrpc": "2.0"
}

In our examples below we use "id" : "1" but of course, you should use a better id (e.g. the permId or something else which is unique)

openBIS endpoints

We try to use openBIS v3 as often as possible. If no other information is given, all POST requests in our examples request the application server (AS) using the URL below. The typical port for the AS is 8443 (Change your hostname and your port number accordingly)

POST https://openbis-hostname:8443/openbis/openbis/rmi-application-server-v3.json

Not all methods have yet been implemented in the new v3 API. We sometimes need one of the following

POST https://openbis-hostname:8443/openbis/openbis/rmi-general-information-v1.json

The v1 JSCON API has many more endpoints which are described here

For requests regarding datasets on the datastore server (DSS), we have again different enpoints. The DSS usually runs on a different port, typically 8444. If we need datastore specific information (e.g. list of files) we need to use this endpoint:

POST https://openbis-dss-hostname:8444/datastore_server/rmi-dss-api-v1.json

You might have more than one DSS.

To play with the openBIS API (or any other webservice) I recommend using the Chrome plugin Postman. There is a (newer, shinier) Chrome App with the same name but I found its layout and readability not being as good.

Log in and out

Log into openBIS - get a token

Before you can request anythin from openBIS, you need to obtain a session token. Any subsequent request need this token.

POST https://openbis-hostname:8443/openbis/openbis/rmi-application-server-v3.json

{
  "method": "login",
  "params": [
    "my_username",
    "my_password"
  ],
  "id": "1",
  "jsonrpc": "2.0"
}

You will get a response like this:

{
  "result": "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6",
  "id": "1",
  "jsonrpc": "2.0"
}

The string of the result is your session token. It contains your username, a dash '-' and an arbitary hexcode.
This token is always the first parameter value of any openBIS request.

Logout - destroy a session token

To end a session and destroy a session token, we use the logout method:

POST https://openbis-hostname:8443/openbis/openbis/rmi-application-server-v3.json

{
  "method": "logout",
  "params": [
    "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6"
  ],
  "id": "1",
  "jsonrpc": "2.0"
}

The response is as shown below. Because the result is null anyway, you could omit the id in the request.

{
  "result": null,
  "id": "1",
  "jsonrpc": "2.0"
}

check if session token is valid

You can check whether your session is still valid by using the v1 API:

POST http://openbis-hostname:8443/openbis/openbis/rmi-general-information-v1.json

{
  "method": "isSessionActive",
  "params": [
    "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6"
  ],
  "id": "1",
  "jsonrpc": "2.0"
}

The 'result' of the response should contain true if your session is still valid, false otherwise.

If your session token is invalid, any request will result in an error:

{
  "error": {
    "code": 0,
    "message": "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6' is invalid: user is not logged in.",
    "data": {
      "@id": 1,
      "exceptionTypeName": "ch.systemsx.cisd.common.exceptions.InvalidSessionException",
      "message": "Session token 'my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6' is invalid: user is not logged in."
    }
  },
  "id": "1",
  "jsonrpc": "2.0"
}

Fetching meta-information

There are a number of methods to get information about our openBIS server. All information can be fetched in a similar way, by just changing the method name:

kind of information

methodName

spaces

searchSpaces

sample types

searchSampleTypes

dataset types

searchDataSetTypes

experiment types

searchExperimentTypes

material types

searchMaterialTypes

vocabulary

searchVocabularyTerms

file types

 

If we have no special search criteria (just list them all), the typical request then looks like this:

POST https://localhost:8443/openbis/openbis/rmi-application-server-v3.json

{
   	"method": "methodName",
   	"params": [ 
		"my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6",
        {
            /* search criteria */
        }, 
        {
            /* fetch options */
        } 
    ],
    "id": "1",
    "jsonrpc": "2.0"
}

working with Samples

The following methods requests data about a specific sample. Every sample is identified by an identifier and a permId.

get sample by identifier

To fetch a sample, we use the getSamples method and providing an identifier attribute together with a corresponding @type

POST https://openbis-hostname:8443/openbis/openbis/rmi-application-server-v3.json

{
  "method": "getSamples",
  "params": [
    "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6",
    [
      {
        "identifier": "/TEST/TEST-SAMPLE-2",
        "@type": "as.dto.sample.id.SampleIdentifier"
      }
    ],
    { /* fetch options, see below */ }
  ],
  "id": "1",
  "jsonrpc": "2.0"
}

get sample by permId

Fetching a sample by permId is done exactly the same way. Instead of an identifier we provide a permId attribute and a different @type:

POST https://openbis-hostname:8443/openbis/openbis/rmi-application-server-v3.json

{
  "method": "getSamples",
  "params": [
    "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6",
    [
      {
        "permId": "20130415091923485-402",
        "@type": "as.dto.sample.id.SamplePermId"
      }
    ],
    { /* fetch options, see below */ }
  ],
  "id": "1",
  "jsonrpc": "2.0"
}

create a sample

A sample needs a name (code), must be of a certain type (typeId), and must belong to a space (spaceID). It might contain specific properties as well as parents (parentIds), children (childIds) and many other connections.

POST https://openbis-hostname:8443/openbis/openbis/rmi-application-server-v3.json

{
  "method":"createSamples",
  "params":[
   "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6",
   [
     {
       "properties":{},
       "typeId":{
         "permId":"SAMPLE_TYPE",
         "@type":"as.dto.entitytype.id.EntityTypePermId"
       },
       "code":"THIS_SAMPLE_HAS_NO_NAME",
       "spaceId":{
         "permId":"MY_USERNAME",
         "@type":"as.dto.space.id.SpacePermId"
       },
       "tagIds":[
         {
           "code":"THIS_IS_A_TAG_NAME",
           "@type":"as.dto.tag.id.TagCode"
         }
       ],
       "@type":"as.dto.sample.create.SampleCreation",
       "experimentId":null,
       "containerId":null,
       "componentIds":null,
       "parentIds":null,
       "childIds":null,
       "attachments":null,
       "creationId":null,
       "autoGeneratedCode":null
     }
   ]
 ],
 "id":"1",
 "jsonrpc":"2.0"
}

If you want to specify one or more parents for that new sample, you need to specify it like this:

       "parentIds": [{
       		"permId" : "20160706001644827-208",
            "@type":"as.dto.sample.id.SamplePermId"
        }],

Fetch options

When you get a sample or any other openBIS entity, you often want to get some additional information along with the result, such as:

  • its properties
  • its parents
  • its datasets

These informations can be obtained with additional fetch options, which can be nested, too:

POST https://openbis-hostname:8443/openbis/openbis/rmi-application-server-v3.json

{
  "type": {
    "@type": "as.dto.sample.fetchoptions.SampleTypeFetchOptions"
  },
  "properties": {
    "@type": "as.dto.property.fetchoptions.PropertyFetchOptions"
  },
  "parents": {
    "@type": "as.dto.sample.fetchoptions.SampleFetchOptions",
    "properties": {
      "@type": "as.dto.property.fetchoptions.PropertyFetchOptions"
    }
  },
  "dataSets": {
    "@type": "as.dto.dataset.fetchoptions.DataSetFetchOptions",
    "properties": {
      "@type": "as.dto.property.fetchoptions.PropertyFetchOptions"
    }
  }
}

The fetch options are the third (mandatory) parameter in your 'params' array. It may be an empty dictionary.
Here is an overview over all the possible fetch options and its type. Parents and childrens are usually of the same @type as the requested entity. This means, the parent of a sample is a parent too, the children of a
dataSet are dataSets too and so on.

name of entity

@type

Remarks

type

as.dto.sample.fetchoptions.SampleTypeFetchOptions

 

properties

as.dto.property.fetchoptions.PropertyFetchOptions

 

parents

as.dto.sample.fetchoptions.SampleFetchOptions

identical to the requested entity

childrens

as.dto.sample.fetchoptions.SampleFetchOptions

identical to the requested entity

container

as.dto.sample.fetchoptions.SampleFetchOptions

identical to the requested entity

components

as.dto.sample.fetchoptions.SampleFetchOptions

 

dataSets

a:qs.dto.dataset.fetchoptions.DataSetFetchOptions

 

experiment

as.dto.experiment.fetchoptions.ExperimentFetchOptions

 

space

as.dto.space.fetchoptions.SpaceFetchOptions

 

project

as.dto.project.fetchoptions.ProjectFetchOptions

 

materialProperties

as.dto.material.fetchoptions.MaterialFetchOptions

 

history

as.dto.history.fetchoptions.HistoryEntryFetchOptions

 

tags

as.dto.tag.fetchoptions.TagFetchOptions

 

registrator

as.dto.person.fetchoptions.PersonFetchOptions

 

modifier

as.dto.person.fetchoptions.PersonFetchOptions

 

attachments

as.dto.attachment.fetchoptions.AttachmentFetchOptions

 

content

as.dto.common.fetchoptions.EmptyFetchOptions

 

get Datasets

Datasets are our links to the actual data, which is hosted on the datastore servers. Like samples,
datasets can have parents, children
Unlike samples, datasets only have a permId but not identifier. Its parents and containers
are datasets too, which means they are of the same @type.

POST https://openbis-hostname:8443/openbis/openbis/rmi-application-server-v3.json

{
  "method": "getDataSets",
  "params": [
    "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6",
    [
      {
        "permId": "20130412142942295-198",
        "@type": "as.dto.dataset.id.DataSetPermId"
      }
    ],
    {
      "parents": {
        "@type": "as.dto.dataset.fetchoptions.DataSetFetchOptions"
      },
      "containers": {
        "@type": "as.dto.dataset.fetchoptions.DataSetFetchOptions"
      },
      "physicalData": {
        "@type": "as.dto.dataset.fetchoptions.PhysicalDataFetchOptions"
      },
      "linkedData": {
        "@type": "as.dto.dataset.fetchoptions.LinkedDataFetchOptions"
      },
      "dataStore": {
        "@type": "as.dto.datastore.fetchoptions.DataStoreFetchOptions"
      },
      "sample": {
        "@type": "as.dto.sample.fetchoptions.SampleFetchOptions"
      },
      "properties": {
        "@type": "as.dto.property.fetchoptions.PropertyFetchOptions"
      },
      "@type": "as.dto.dataset.fetchoptions.DataSetFetchOptions"
    }
  ],
  "id": "1",
  "jsonrpc": "2.0"
}

get Dataset files

Finally we want to fetch the original files themselves. Our application server only hosts the metadata, not the original files. But it knows whom to ask. In the above example, we get back a lot of metadata, including 'dataStore'. This metadata object contains information about on which dataStore server our data is actually stored:

POST https://openbis-hostname:8443/openbis/openbis/rmi-application-server-v3.json

"dataStore": {
  "@type": "as.dto.datastore.DataStore",
  "@id": 11,
  "fetchOptions": {
    "@type": "as.dto.datastore.fetchoptions.DataStoreFetchOptions",
    "@id": 12,
    "cacheMode": "NO_CACHE"
  },
  "code": "DSS1",
  "downloadUrl": "https://openbis-hostname:8444",
  "remoteUrl": "https://openbis-hostname:8444",
  "registrationDate": 1365753969172,
  "modificationDate": 1464467813297
}

To actually fetch the list of files, we use the v1 API on the specified datastore server:

POST https://openbis-hostname:8444/datastore_server/rmi-dss-api-v1.json

{
  "method": "listFilesForDataSet",
  "params": [
    "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6",
    "20130412142942295-198",
    "/",
    true
  ],
  "id": "1",
  "jsonrpc": "2.0"
}

The argument list of 'params' is as follows:

  • our session token
  • the **permId of our dataset
  • the folder to start with, usually '/' which means the root directory, '/start/here' if your starting point is somewhere else
  • true = recursively look for all files in all subfolders; false = just get the list of the specified folder

This is the result we get back from the above query:

{
  "result": [
    {
      "@type": "FileInfoDssDTO",
      "@id": 1,
      "pathInDataSet": "thumbnails_256x256.h5ar",
      "pathInListing": "thumbnails_256x256.h5ar",
      "isDirectory": true,
      "fileSize": "-1"
    },
    {
      "@type": "FileInfoDssDTO",
      "@id": 2,
      "pathInDataSet": "thumbnails_256x256.h5ar/wA10_d1-1_cCy5.png",
      "pathInListing": "thumbnails_256x256.h5ar/wA10_d1-1_cCy5.png",
      "isDirectory": false,
      "crc32Checksum": -1396446681,
      "fileSize": "1505"
    },
    ... (many more files)...
  ],
  "id": "1",
  "jsonrpc": "2.0"
}

the 'isDirectory' flag indicates if whether the entry is a folder or not. Together with the earlier
information about our datastore server ('downloadUrl'), we are now able to actually download a file:

https://openbis-hostname:8444/datastore_server/thumbnails_256x256.h5ar/wA10_d1-1_cCy5.png

upload data

Uploading data is done in three steps:

  • get the DSS URL
  • upload the file(s) to the session workspace
  • register the file(s) in openBIS

1. get available DSS

Before we can upload any data, we need to ask the application server (AS) which DSS are available. In many cases, we only have one DSS, but in some cases we can have more than one, so we have to choose. Fetching the list of available DSS done like this:

POST https://openbis-hostname:8443/openbis/openbis/rmi-general-information-v1.json

{
  "method": "listDataStores",
  "params": [
    "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6"
  ],
  "id": "1",
  "jsonrpc": "2.0"
}

which gives you following result:

{
  "result": [
    {
      "@type": "DataStore",
      "@id": 1,
      "code": "DSS2",
      "downloadUrl": "https://openbis-hostname:8445/datastore_server",
      "hostUrl": "https://openbis-hostname:8445"
    },
    {
      "@type": "DataStore",
      "@id": 2,
      "code": "DSS1",
      "downloadUrl": "https://openbis-hostname:8444/datastore_server",
      "hostUrl": "https://openbis-hostname:8444"
    }
  ],
  "id": "1",
  "jsonrpc": "2.0"
}

In our example we get two different datastores, and the attributes of our interest are downloadUrl which we need for the upload process and code which we need to initiate the registration process.

2. upload a file to the DSS

Now we have a choice of a DSS, we can start uploading a file. Our endpoint for uploading files:

downloadUrl/session_workspace_file_upload

In the URL we place the additional attributes:

  • filename
  • id
  • startByte
  • endByte
  • sessionID

all of the above attributes are mandatory. If you don't know the file size, you can just set startByte=0 and endByte=0.

Example:

POST https://openbis-hostname:8445/datastore_server/session_workspace_file_upload?filename=my_filename&id=1&startByte=0&endByte=0&sessionID=my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6

You need to include Content-Type: multipart/form-data in your HTTP header in order to upload the file.

After successful upload, the response includes a status message, the filename and the size in bytes:

{
  "id": 1,
  "status": "ok",
  "endByte": 0,    
  "startByte": 0,
  "filename": "my_filename",
  "size": 1280
}

3. start data registration process

Once the file is uploaded to the user session workspace, we can initiate the data registration process. Every data file needs to be coupled to a dataset object which in turn needs to be connected to a sample. Without this important registration step, our file remains in the session workspace and gets deleted as soon as the session expires.

The registration process can be started by posting a request to our application server (AS):

{
  "method": "createReportFromAggregationService",
  "params": [
    sessionToken,
    "DSS2",
    "my_dropbox_plugin",
    additional_parameters
  ]
}

This will start the dropbox my_dropbox_plugin (a Jython script which lives inside your DSS plugins folder) to look for files in our user session workspace and register them in openBIS. DSS2 is the code of the datastore server where we uploaded our file. The additional_parameters are sent to the process method of our my_dropbox_plugin. These parameters typically include a sampleIdentifier, because all datasets need to be connected to a sample.

Example:

POST https://openbis-hostname:8443/openbis/openbis/rmi-query-v1.json

{
  "method": "createReportFromAggregationService",
  "params": [
    "my_username-160601233044713x0F476AEA0FE875E4E87CB16C6D6902D6",
    "DSS2",
    "jupyter-uploader-api",
    {
      "sample": {
        "identifier": "/FOO/BAR"
      },
      "container": {
        "name": "Analysis name",
        "description": "This is a description of my analysis"
      },
      "dataSets" : [
        {
          "dataSetType" : "JUPYTER_RESULT",
          "sessionWorkspaceFolder" : "results/",
          "fileNames" : ["results/blublu"]
        }
      ]
    }
  ],
  "id": "1",
  "jsonrpc": "2.0"
}
  • No labels