# openBIS setup guide

This document guides you through the basics of installing and configuring openBIS. Its purpose it to allow newcomers to install openBIS and play with it, so exhaustive details are left out on purpose. The complete openBIS administration manual is maintained here Installation and Administrator Guide of the openBIS Server.

## Prerequisites

The successful installation of openBIS requires a machine with:

• Linux or Mac OS X operating system
• PostgreSQL 11. You are required to have database superuser privileges (by default, the `postgres` user has those).
• Java 11.

Please, make sure all the prerequisites are met before continuing to the next chapters of this document.

## Deployment overview

The basic openBIS deployment consists of two servers, the Application Server (AS) and the Data Store Server (DSS). Generally speaking, the AS manages the metadata and links to the data while the DSS manages the data itself. The AS employs a PostgreSQL server to persist users, authorization information, various entities and their metadata, as well as index information about all datasets. The DSS operates on a managed part of the file system (data store), where the data is kept. One AS can manage multiple DSS instances, which can make sense e.g. if the data are acquired in multiple, distributed laboratories each having a local DSS.

In this document we will describe an installation procedure with a single DSS, where all three servers (AS, DSS and PostgreSQL) run on the same computer. In general, each of them could be deployed on a dedicated machine, which would improve the overall performance of the system.

Note, that regardless of the particular deployment scenario, the servers must be started in the following order

1. PostgreSQL
2. openBIS Application Server
3. Data Store Server

This is required since the AS needs a running PostgreSQL server to start and the DSS needs an AS to boot up. These dependencies are depicted on the deployment overview diagram with arrows pointing from the dependent towards the dependee.

To avoid inconsistent states of the system, shutdowns must be executed backwards - starting from DSS and proceeding with AS and PostgreSQL respectively.

## Installation

It is considered a good practice to install openBIS under a dedicated operating system user, but it is not mandatory to do so. Throughout the document we assume that openBIS will be installed and maintained by a user named `openbis`. The default installation path for the system will be the home folder of `openbis`. We also refer to the installation path as `OPENBIS_HOME`.

### PostgreSQL server configuration

Create a database user with the same name as the OS user (in our case `openbis`)

```$sudo -u postgres createuser openbis Shall the new role be a superuser? (y/n) n Shall the new role be allowed to create databases? (y/n) n Shall the new role be allowed to create more new roles? (y/n) n ``` The openBIS setup assumes that PostgreSQL is configured with the trust setting on localhost, which means that a user connecting from `127.0.0.1` can connect without providing a password. These settings are changed in the file `pg_hba.conf`, see http://www.postgresql.org/docs/9.3/static/auth-pg-hba-conf.html. If these settings are not correct, you will get an error of the form "Database role 'openbis' couldn't be created." If the PostgreSQL server is configured correctly, then the following sequence of commands must return without errors. ```> su - openbis > psql -U postgres -l ... > psql -l ... ``` ### Postgres on a remote server When the database is on a remote server, the pg_hba.conf and postgresql.conf files need to be configured as follows: pg_hba.conf ```# TYPE DATABASE USER ADDRESS METHOD # application access privilege host database_name all 0.0.0.0/0 md5 / trust host postgres all 0.0.0.0/0 md5 / trust``` So, the databse postgres need to be added to the file. Alternatively, instead of specifying each database and the postgres database, the database can be set to "all" as shown below: pg_hba.conf ```# TYPE DATABASE USER ADDRESS METHOD # application access privilege host all all 0.0.0.0/0 md5 / trust``` postgresql.conf `listen_addresses= '*' (or alternatively one or more IP addresses can be listed)` Also have look here how to configure openBIS to connect to the remote Postgres DB. ### Manual database creation If openBIS cannot be allowed to create its own database (because of missing privileges, for example), you can use the script bin/init_database.sh (in your openBIS installation directory) that will create SQL statements to initialize the database. These SQL statememts can be given to be executed by a database administrator. Example: ```$ bin/init_database.sh openbis_dev openbis

-- Execute the following SQL script as a database superuser
create user openbis;
create database openbis_dev with owner openbis encoding 'UTF8' template=template0 tablespace=pg_default;
alter database openbis_dev set default_with_oids = off;
alter database openbis_dev set join_collapse_limit = '32';
alter database openbis_dev set from_collapse_limit = '32';

\connect openbis_dev

begin;
create table database_version_logs(db_version varchar(4) not null, module_name varchar(250), run_status varchar(10), run_status_timestamp timestamp, module_code bytea, run_exception bytea);
insert into database_version_logs(db_version, run_status, run_status_timestamp) values ('0', 'SUCCESS', now());
alter database openbis_dev owner to openbis;
alter table database_version_logs owner to openbis;
commit;
```

1st argument to the script is the name of the database to be created.
2nd argument to the script is the name of the user openBIS will use to access the database we created.
Note: both the database name and user name need to be reflected in service.properties configuration files.

• Download the latest release of openBIS installer. It should be a TAR.GZ archive and its name should look like `openBIS-installation-<version>-r<revision>.tar.gz`, where `<version>` is the release number (e.g. S117).
```$ls -a openBIS-installation-XXX.tar.gz ``` Run the installation NOTE: openBIS will ask you to provide the install directory. For a new installation this directory is created by the openBIS installer, so it should not already exist. If the directory already exists the installation will fail. Extract the installation tarball ```> tar xzvf openBIS-installation-S117-r23517.tar.gz x openBIS-installation-S117-r23517/console.properties x openBIS-installation-S117-r23517/jul.config x openBIS-installation-S117-r23517/extract.sh x openBIS-installation-S117-r23517/run-console.sh x openBIS-installation-S117-r23517/run-ui.sh x openBIS-installation-S117-r23517/openBIS-installer.jar ``` #### Headless installation To install openBIS on a headless system you need to • Edit the file `console.properties` • Execute the script `run-console.sh` ```> cd openBIS-installation-S117-r23517 > ./run-console.sh ``` Congratulations ! If you have reached this far your openBIS system is installed and is ready to be started. Please do so by following the instructions in the next section. ## Starting/Stopping The starting scripts of openBIS assume that `java` is available on the system path, so you have to add it if this is not true. Launch the openBIS AS and DSS servers by executing `<OPENBIS_HOME>/bin/allup.sh`. ```$ ./bin/allup.sh
```

To verify AS and DSS function and there are no configurational issues, please check the log files of the two applications using the `bislog.sh` and `dsslog.sh` scripts. Additionally, the openBIS application should be accessible at `https://localhost:8443/openbis` and you should be able to login as the `admin` user with the password specified in the installation process.

You can shutdown both the servers by executing the `<OPENBIS_HOME>/bin/alldown.sh` script

```$./bin/alldown.sh ``` ## Configure openBIS to run on one standard port 443 We would like all users who work in the network to use the standard https port (443) to access openbis and the DSS (datastore server). We start openbis on port 8080 and DSS on port 8081. Those ports are not accessible from outside. Then we redirect all the requests to 443 port from outside to a right internal port depending on the URL suffix. We do this by switching on Apache forwarding (we use mod_proxy). See the details below. ### Apache configuration • we configure Apache instance (inside `/etc/httpd/conf.d/openbis.conf` file on Linux, see attached openbis.conf as an example) to: ```<VirtualHost *:443> ErrorLog /var/log/apache2/ssl_error_log TransferLog /var/log/apache2/ssl_access_log LogLevel info ServerName localhost SSLEngine on SSLProtocol all -SSLv2 SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW SSLCertificateFile /etc/pki/tls/apache.crt SSLCertificateKeyFile /etc/pki/tls/apache.key SSLProxyEngine on ProxyRequests Off ProxyPreserveHost On RewriteEngine On AllowEncodedSlashes On # # Rewrite url if there is a missing / at the end # # Remove the following line if using cifex! RewriteRule ^/$ /openbis/ [R,L]
RewriteRule ^/openbis$/openbis/ [R,L] RewriteRule ^/datastore_server$ /datastore_server/ [R,L]

#Replace localhost with your host name
ProxyPass /openbis/ http://localhost:8080/openbis/ timeout=600 Keepalive=Off
ProxyPassReverse /openbis/ http://localhost:8080/openbis/
ProxyPass /datastore_server/ http://localhost:8081/datastore_server/
ProxyPassReverse /datastore_server/ http://localhost:8081/datastore_server/

# Used by Screening instances
ProxyPass /datastore_server_screening http://localhost:8081/datastore_server_screening
ProxyPassReverse /datastore_server_screening http://localhost:8081/datastore_server_screening

ProxyPass /rmi-datastore-server-screening-api-v1 http://localhost:8081/rmi-datastore-server-screening-api-v1
ProxyPassReverse /rmi-datastore-server-screening-api-v1 http://localhost:8081/rmi-datastore-server-screening-api-v1
</VirtualHost>```

• we configure openbis and DSS servers not to use SSL (because Apache uses it already)
• install SSL module (mod_ssl) for Apache if it is not there: `yum install mod_ssl`.
• after changing the configuration you have to restart Apache. On Linux you call:
• ```/etc/init.d/httpd restart
```

make sure that Apache will be restarted automatically

```[root@~]# chkconfig --list | grep httpd
httpd           0:off   1:off   2:on    3:on    4:on    5:on    6:off```

### openBIS configuration

Create a file called `http.ini` in `/openBIS-server/jetty/start.d/`:

```# Module: http
--module=http
### HTTP Connector Configuration
## HTTP port to listen on
jetty.port=8080
## HTTP idle timeout in milliseconds
http.timeout=30000
## HTTP Socket.soLingerTime in seconds. (-1 to disable)
# http.soLingerTime=-1
## Parameters to control the number and priority of acceptors and selectors
# http.selectors=1
# http.acceptors=1
# http.selectorPriorityDelta=0
# http.acceptorPriorityDelta=0```

Rename the files `https.ini` and `ssl.ini`. These files are then then no longer used by jetty to serve at port 8443.

### Settings for the Data Store Server (DSS)

These changes are needed for all Jetty versions which are used in openBIS

• do the same in DSS service.properties (sprint/datastore_server/etc) for 'server-url' and 'download-url' properties, additionally add `use-ssl = false`
• Changes to DSS service.properties file:

```--- service.properties.ssl      Tue Mar  9 16:51:40 2010
+++ service.properties  Sat Mar 13 07:54:28 2010
@@ -8,3 +8,3 @@
# Port
-port = 8444
+port = 8081

@@ -43,4 +43,5 @@
# The URL of the openBIS server
-server-url = https://openbis-test.ethz.ch:8443
+server-url = http://openbis-test.ethz.ch:8080

+use-ssl = false

Prerequisites:

• `java``psql``pg_dump` are present and accessible from the shell

• ssh to the server
• copy openBIS installation tarball (a.k.a. "Installation and Upgrade Wizard") to `<OPENBIS_HOME>/servers` directory. You can download the latest version from here.
• run `<OPENBIS_HOME>/bin/upgrade.sh`

Sometimes changes of the configuration are needed when a server is upgraded. You can do it after the upgrade. If you want to do this before, you can copy all the configuration of the servers to a temporary directory, modify it there and "apply" it to the upgraded server using `backup-config.sh <dir>` and `restore-config-from-backup.sh <dir>` scripts.

### Upgrade installations with versions older than S118

The described upgrade steps are only valid for HCS installations prior S118 or for generic openBIS installations which have been created by the installation tarball. Generic openBIS installation assembled from two independent server archives (AS and DSS) are not be supported.

Prerequisites:

• `java``psql``pg_dump` are present and accessible from the shell

• ssh to the server
• copy openBIS installation tarball (to temporary directory (e.g. `/tmp`). You can download the latest version from here.
• extract the installation tarball

```> tar xzvf openBIS-installation-XXX.tar.gz
....
```
• edit the file `console.properties` and set the property `INSTALL_PATH`to <OPENBIS_HOME> e.g.

```#....
INSTALL_PATH=/home/openbis/
```
• execute `./run-console.sh`. The installation procedure will then upgrade the openBIS installation at `INSTALL_PATH`.

## Changing server configuration

The most important configuration files of openBIS AS and DSS can be found in `openBIS-server/jetty/etc` and `datastore_server/etc` folders.

Sometimes when many configuration files should be changed it can be convenient to copy all configuration files to one directory, modify them there and apply the changes. It can be done in the following way:

• Execute the following command to copy all configuration files to a folder of your choice

```$./bin/backup-config.sh <directory-path> ``` • Modify the files in <directory-path> • Apply the configuration changes by executing ```$ ./bin/restore-config-from-backup.sh <director-path>
```

### Configure JAVA options

The files `openbis.conf` and `datastore_server.conf` contain the JVM options for the respective server. For recommendation on how to configure them see
Recommended Java and Postgres memory settings.

By default, the AS and DSS configurations assume a 64bit JDK. If you are running 32bit JDK you must remove the `-d64` parameter from `openbis.conf` and `datastore_server.conf`.

### Configure authentication service

openBIS Application Server currently supports several authentication systems

• `file-authentication-service`: a self-contained system based on a UNIX-like passwd file.
• `dummy-authentication-service`: a "dummy" that allows everyone access to the system.
• LDAP
• Crowd system (see http://www.atlassian.com/software/crowd).

The latter two options will not be discussed in this document. For more information regarding LDAP or Crown integration check the Installation and Administrator Guide of the openBIS Server.

#### Dummy authentication service

If you only wish to play with openBIS and do not want to enforce security restrictions on the visiting users, you might want to choose the `dummy-authentication-service`.

To configure the dummy authentication scheme, edit `<OPENBIS_HOME>/server/openBIS-server/jetty/etc/service.properties` and set the value of the property `authentication-service` to `dummy-authentication-service`.

service.properties
```authentication-service = dummy-authentication-service
```

#### File authentication service

The default installation procedure preconfigures openBIS with `file-authentication-service`. It also creates the user acounts for `admin` and `etlserver`. If you wish to add more user accounts, you can do so with the help of the command line tool `<OPENBIS_HOME>/servers/openBIS-server/jetty/bin/passwd.sh` by executing the following :

```$passwd.sh add [-f <first name>] [-l <last name>] [-e <email>] admin Enter new password: ***** ``` All passwords for `file-authentication-service` are stored in the file `<OPENBIS_HOME>/servers/openBIS-server/jetty/etc/passwd`. ### Configure database options By default openBIS assumes that the database in accessible on localhost as a 'postgres' user with an empty password. To change that one has to edit `service.properties` and configure the usernames and passwords for an PostgreSQL superuser (in sample below `postgres`) and the user, which will take ownership of the openBIS database instance. Some of you have created this low-privileged user in #Create a database user for openBIS (optional). It is permissible, albeit not advisable to configure the same user (e.g. `postgres`) for `database.owner` and `database.admin`, providing the account is a PostgreSQL superuser. service.properties ```# Part of the name of the database. The full name of the database will be openbis_<kind>. database.kind=demo # ID of the user owning the data. In case of an empty string it is the same user who # started up openBIS Application Server. database.owner = openbisdbuser database.owner-password = ******* # ID of the user on database server with admin rights, like creation of tables. # Should be an empty string if default admin user should be used. In case of PostgreSQL # the default admin user is assumed to be postgres database.admin-user = postgres database.admin-password = ******* ``` Additionally, ensure the configured `database.kind` property corresponds to the variable `OPENBIS_DB` declared in the file `<OPENBIS_HOME>/bin/env`. In our example `database.kind` is configured as "demo", so the value of `OPENBIS_DB` should be "openbis_demo". ### Configure imaging database (HCS installations only) For HCS installations you have to make a similar configuration for an additional database (imaging) in the DSS properties in servers/core-plugins/screening/NUM/dss/data-sources/imaging-db/plugin.properties file. Please note that NUM is the highest number in the screening plugin. Currently this is 4. plugin.properties ``` version-holder-class = ch.systemsx.cisd.openbis.dss.etl.ImagingDatabaseVersionHolder databaseEngineCode = postgresql basicDatabaseName = imaging urlHostPart =${imaging-database.url-host-part:localhost}
databaseKind = ${imaging-database.kind:prod} scriptFolder =${screening-sql-root-folder:}sql/imaging
owner = ${imaging-database.owner:} password =${imaging-database.password:}
```

This defines the username/passwords of an administrator user to perform migrations for the DSS database and an "owner" i.e. normal account to read/write data. In the example above the database will be called `imaging_productive`. Make sure this name corresponds to the `IMAGING_DB` variable in `<OPENBIS_HOME>/bin/env`.

You can create the "owner" database user as described in #Create a database user for openBIS (optional)

Additionally openBIS Application Server configuration has to be changed (usually
in servers/core-plugins/screening/NUM/as/dss-data-sources/DSS1/plugin.properties file)

Please note that NUM corresponds to the highest number in the screening plugin.

```database-driver = org.postgresql.Driver
database-url = jdbc:postgresql://${imaging-database.url-host-part:localhost}/imaging_${imaging-database.kind:prod}
database-username = ${imaging-database.owner:} database-password =${imaging-database.password:}
```

## Appendix

Startup/shutdown scripts

Description

alldown.sh

stops AS and DSS

allup.sh

starts AS and DSS

bisup.sh

starts AS

bisdown.sh

stops AS

bislog.sh

shows AS log file

dssup.sh

starts DSS

dssdown.sh

stops DSS

dsslog.sh

shows DSS log file

Utility scripts

Description

backup-config.sh

Stores all configuration files in a specified directory

restore-config-from-backup.sh

Overrides servers' configurations with the configuration from a specified directory (which can be created using `backup-config.sh` and modified afterwards).

list-postgres-queries.sh

shows all current postgres queries (useful for debuging)

reset-display-settings.sh

resets all users display settings (edit the `TEMPLATE_USER` variable first)

Installation scripts

Description

env

Contains instance specific settings. This file is not updated from SVN.

upgrades openBIS to the new version

### List of important configuration files

Configuration file

Description

datastore_server.conf

Contains properties configuring the DSS process launch (e.g. JAVA options, number of log files).

dss-log.xml

DSS log configuration.

dss-service.properties

DSS application properties. This is the most important DSS configuration file.

jetty.xml

Jetty server configuration (see http://docs.codehaus.org/display/JETTY/Syntax+Reference)

jetty.properties

Supplementary Jetty properties

log.xml

AS log configuration.

openbis.conf

Contains properties configuring the AS process launch (e.g. JAVA options, number of log files).

service.properties

AS application properties. This is the most important AS configuration file.

web-client.properties

openBIS web UI configuration properties.

welcomePageSimple.html

Altering file will change the login page of the openBIS web interface. Administrators can use this to add banners (e.g. with important announcements) visible to all users loggin in openBIS.