Taming your SAP HANA Express (SE01E02). Hardening an SAP HANA Express 2.0 SP03 installation part 2. Securing the SAP HANA extended application services

This is the second blog of the season one of Taming your SAP HANA Express series, episode two.

In the SE01E01 we have seen how to implement a fully qualified XS Classic domain CA-signed SSL certificate.

This episode will tell the story of how to rename and secure the XSA domain.

It has two parts: rename and harden;

Let’s begin

You might not expect it but even if you rename the XSA domain from a short domain name like vhcalhxedb into the FQDN vhcalhxedb.sap.corp the new XSA domain will be issued with a new self-signed SSL certificate matching the new XSA domain name.
This is described in the following SAP note:

2243019 – Providing SSL certificates for domains defined in SAP HANA extended application services, advanced model

We will take care of it in the harden part of this blog.

A. Rename

The following blog describes quite accurately the steps required to rename the default XSA domain into a FQDN XSA domain. Please refer to it at all times!

The following statements were taken from my AWS host where the SAPCAL HXE appliance was deployed. You need to be the root user except if stated otherwise;

1. The rename sequence:

1a. Inspect the xscontroller.ini file communication section

First we need to edit the xscontroller.ini XSA engine configuration file the content of which can be viewed with cat /hana/shared/HXE/global/hdb/custom/config/xscontroller.ini.

The content of the file reveals the default short domain name and XSA API URL (under the communication section).

sid-hxe:/hana/shared/HXE/hdblcm # cat /hana/shared/HXE/global/hdb/custom/config/xscontroller.ini

[communication]

default_domain = vhcalhxedb

api_url = https://vhcalhxedb:39030

1b.  Amend the default domain and the api url to the required domain name

In my case I just added the .sap.corp domain suffix to the existing short domain name;

But of course you might want to have a truly bespoke XSA domain name like whatevername.dummy.nodomain

I am the vi editor (load, insert and ESC with :wq command); Remember if you want to discard any changes being made you may just issue the following vi escape sequence ESC :q!

sid-hxe:/hana/shared/HXE/hdblcm # vi /hana/shared/HXE/global/hdb/custom/config/xscontroller.ini

1c. Inspect the content of the xscontroller.ini file communication section once again

sid-hxe:/hana/shared/HXE/hdblcm # cat /hana/shared/HXE/global/hdb/custom/config/xscontroller.ini

[communication]

default_domain = vhcalhxedb.sap.corp

api_url = https://vhcalhxedb.sap.corp:39030

1d. In order to rename the XSA domain we will be using the hdblcm command line hana db lifecycle management utility as follows:

sid-hxe:/hana/shared/HXE/hdblcm #

You will need to create an xml formatted file, for instance pwd.xml, with all the passwords for the seamless execution.

The pwd.xml sits in the same directory and provides all the required passwords so we do not get prompted for passwords during the rename procedure:

But as you are using the SAPCAL image:

The first password is your operating system hxeadm user,

the second is the XSA Admin password and  then the system user password followed by the sapadm password and last but not least again the SYSTEMDB user password;

As a reminder you have set up all these passwords when you were deploying the SAPCAL image;

(For instance in my case I have made all the passwords equal the master password )

Make sure you have stopped HANA before going any further as follows

su -l hxeadm

hxeadm@sid-hxe:/usr/sap/HXE/HDB90> ./HDB stop

hxeadm@sid-hxe:exit

and you are back as a root user;

2. Let’s go and rename the XSA domain

2a. First attempt (unsuccessful)

sid-hxe:/hana/shared/HXE/hdblcm # cat pwd.xml | ./hdblcm –action=rename_system –nostart –skip_hostagent_calls –certificates_hostmap=vhcalhxedb=vhcalhxedb.sap.corp –xs_domain_name=vhcalhxedb.sap.corp –read_password_from_stdin=xml -b

SAP HANA Lifecycle Management – SAP HANA Express Edition 2.00.031.00.1528768600

****************************************************************

Local Host Name: vhcalhxedb

System Properties:

HXE /hana/shared/HXE HDB_ALONE

HDB90

version: 2.00.031.00.1528768600

host: vhcalhxedb (Database Worker (worker), XS Advanced Runtime Worker (xs_worker))

edition: SAP HANA Express Edition

plugins: afl,epmmds

Start reading from input channel…

… Done.

Running in batch mode

Cannot resolve host name ‘vhcalhxedb.sap.corp’

Host name vhcalhxedb.sap.corp is not accessible.

Log file written to ‘/var/tmp/hdb_HXE_hdblcm_rename_system_2018-12-24_13.17.38/hdblcm.log’ on host ‘sid-hxe’.

Whoops! Obviously the new domain name needs to added to the /etc/hosts for the internal IP resolution.

Add the vhcalhxedb.sap.corp IP route to the /etc/hosts file….

sid-hxe:/hana/shared/HXE/hdblcm # cat /etc/hosts

# Syntax:

#

# IP-Address  Full-Qualified-Hostname  Short-Hostname

#

127.0.0.1 localhost

10.0.0.11 sid-hxe.dummy.nodomain sid-hxe

10.0.0.11 vhcalhxedb.sap.corp vhcalhxedb

2b. Second attempt (successful)

sid-hxe:/hana/shared/HXE/hdblcm # cat pwd.xml | ./hdblcm –action=rename_system –nostart –skip_hostagent_calls –certificates_hostmap=vhcalhxedb=vhcalhxedb.sap.corp –xs_domain_name=vhcalhxedb.sap.corp –read_password_from_stdin=xml -b

SAP HANA Lifecycle Management – SAP HANA Express Edition 2.00.031.00.1528768600

****************************************************************

Local Host Name: vhcalhxedb

System Properties:

HXE /hana/shared/HXE HDB_ALONE

HDB90

version: 2.00.031.00.1528768600

host: vhcalhxedb (Database Worker (worker), XS Advanced Runtime Worker (xs_worker))

edition: SAP HANA Express Edition

plugins: afl,epmmds

Start reading from input channel…

… Done.

Summary before execution:

=========================

SAP HANA Express Edition

Rename Parameters

Installation Path: /hana/shared

Source System ID: HXE

Target System ID: HXE

Target Instance Number: 90

Skip all SAP Host Agent calls: Yes

Remote Execution: ssh

Do not start SAP HANA Express Edition System but start service (sapstartsrv) instead: Yes

Host Name: vhcalhxedb

System Usage: production

Certificate Host Names: vhcalhxedb -> vhcalhxedb.sap.corp

XS Advanced App Working Path: /hana/shared/HXE/xs/app_working

XS Advanced Domain Name (see SAP Note 2245631): vhcalhxedb.sap.corp

Note: SAP HANA Express Edition System will be stopped

Renaming System…

SAP HANA Lifecycle Management – Database Rename  2.3.48

Renaming instance…

Stopping system…

All server processes stopped on host ‘vhcalhxedb’ (worker, xs_worker).

Stopping sapstartsrv service…

Removing sapservices entry…

Updating system configuration files…

Renaming instance…

Creating sapservices entry…

Performing rename of XS…

xsa-rename: Renaming XSA SSFS DAT file

xsa-rename: Renaming XSA SSFS KEY file

Starting service (sapstartsrv)…

Updating Component List…

Updating SAP HANA Express Edition Instance Integration on Local Host…

SAP HANA Express Edition System renamed

You can send feedback to SAP with this form: https://vhcalhxedb:1129/lmsl/HDBLCM/HXE/feedback/feedback.html

Log file written to ‘/var/tmp/hdb_HXE_hdblcm_rename_system_2018-12-24_13.19.54/hdblcm.log’ on host ‘sid-hxe’.

You can display the log file as follows:

sid-hxe:/hana/shared/HXE/hdblcm #

cat /var/tmp/hdb_HXE_hdblcm_rename_system_2018-12-24_13.19.54/hdblcm.log

2c. Log as the non-root hxeadm user in order to start the HANA database

su -l hxeadm

hxeadm@sid-hxe:/usr/sap/HXE/HDB90> ./HDB start

StartService

Impromptu CCC initialization by ‘rscpCInit’.

See SAP note 1266393.

OK

OK

Starting instance using: /usr/sap/HXE/SYS/exe/hdb/sapcontrol -prot NI_HTTP -nr 90 -function StartWait 2700 2

Start

OK

StartWait

OK

hxeadm@sid-hxe:/usr/sap/HXE/HDB90>

If you want to check on the HANA DB startup progress progress you may tail this log file:

hxeadm@sid-hxe:/usr/sap/HXE/HDB90> tail -f /usr/sap/HXE/HDB90/vhcalhxedb/trace/xscontroller_0.log

or alternatively you can use the ./HDB info command

3 Caveats:

After the XSA rename procedure I have encountered the following problem when trying to use the new XSA domain URL, namely the redirect_uri has an invalid domain.

For instance I got the following redirect URL when trying to connect to the HANA Cockpit (port 51039):

https://vhcalhxedb.sap.corp:39032/uaa-security/oauth/authorize?response_type=code&redirect_uri=https%3A%2F%2Fvhcalhxedb%3A51039%2Flogin%2Fcallback&client_id=sb-xsa-cockpit!i1&state=2620251216420

Why?  Simply because the new self-signed certificate has not been yet loaded in my browser’s cache? I was not entirely sure.

Regardless, as I was about sign the XSA domain with a trusted CA (Certificate Authority) I decided not to bother that much at this stage.

B. Harden

As aforementioned the following SAP note explains the situation with the XSA self-signed certificate.

2479411 – Untrusted/unsecure error when using HANA Cockpit 2.0 when XSA uses self-signed certificate.

The following applies if you have installed the HANA cockpit 2.0 that uses the XSA. This is what you get with the SAP HANA Express SAPCAL image.

When launching the cockpit via the web browser, you can see the untrusted/unsecure error because the XSA is using a self signed certificate (located in the file default.root.crt.pem).

This certificate is located in the directory [/hana/shared/<SID>/xs/controller_data/controller/ssl-pub/router], but there is no [obvious] option to generate the CSR file, so the certificate could be eventually signed by your CA.

Solution:

The following SAP note describes the required steps to remedy this situation. However it omits to say how to generate the CSR for the XSA self-signed certificate

And where is the private key ? Let’s see ?

Trying to find the private key…..

grep -r –exclude-dir=log –exclude-dir=ssh –exclude=*history -I -l -e ‘—–BEGIN PRIVATE*’ -e ‘—–BEGIN RSA*’ -e ‘—–BEGIN EC*’ /hana/shared/HXE/HDB90/ 2> /dev/null

but you will not be able to find any trace of the private key…..it has been wiped out during the rename procedure

Given the fact we neither have the CSR nor the private key; thus this is going to be a rather challenging task. And indeed it is.

I am about to describe how to circumvent the fact that the HANA XSA Cockpit does not offer any option to generate the CSR for its self-signed certificate and that the private key is not available.

For the sake of transparency instead of using the OPENSSL utility I used Portecle a fantastic graphic SSL utility.

SSL toolbox:

SSL shopper

Portecle

Let’s go:

In order to create the CSR and then the full certificate chain and then get the public/private key par I used a very handy Portecle, locally on my Mac. What I am about to show requires good level of understanding of the SSL security topics, certificates manipulation etc. Portecle has an excellent How-to section in you want to up skill yourself in the SSL security topics.

The steps.

As aforementioned almost everything an be done on a laptop with Portecle:

Create a new keystore

create new key store of the following format:

next from Tools menu generate a key pair

Generate key pair:

and generate a new certificate:

key pair entry alias: we will keep it the same as the CN.

and the certificate is in our newly created keystore:

we can double click on the certificate alias name to reveal its content:

and also inspect its PEM encoding:

Let’s save it into a file (we will; be asked to enter a password; use a password that you can easily remember)

So we have created a new certificate; next we need to sign it;

Back to square one we need a CSR for our certificate.

How to generate a CSR for a keystore key pair entry?

That’s the password we have used while saving the key pair

We are just about to generate the CSR:

that’s done

we can even test that both our certificate and its CSR are valid ones:

The certificate decoder

copy and paste the content of the vhcalhxedb_sap_corp.pem file

The CSR decoder

copy and paste the content of the

SAP Global PKI Certificate Web Enrollment

There two CA reply choices or formats: x.509 and PKCS#7

we will need the PKCS#7 format rather as it includes the certificate chain

Next in order to make the CA signing process complete we will import the CA reply into the certificate so it becomes signed and trusted.

From now on the CA signing procedure of our certificate is complete:

From this moment we can export it from the keystore as follows:

but we will rather need the PEM encoding:

And we can use the following openssl command to convert the traditional private key into the required pkcs8 private key format

$ openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in vhcalhxedb_sap_corp_SAPNetCA_G.pem -out pkcs8.key

Enter pass phrase for vhcalhxedb_sap_corp_SAPNetCA_G.pem:

$

Finally let’s use the certificate matcher to makes sure we got our private key right:

Certificate Key Matcher

The epilog

From now on we can transport both the signed certificate (full chain) in the PEM format and the private key in the PKCS#8 format into HANA host and reset the XSA certificate as follows:

hxeadm@sid-hxe:/usr/sap/HXE/home/xsa-sll> XSA set-certificate -c cert.pem -k pkcs8.key

Picked up _JAVA_OPTIONS: -Xss2m

USERNAME: XSA_ADMIN

PASSWORD>

Deploying router certificate for default domain. Public key ‘/usr/sap/HXE/home/xsa-sll/cert.pem’, private key ‘/usr/sap/HXE/home/xsa-sll/pkcs8.key’

Stopping XSA …

Starting XSA …

OK

and do not forget to wipe out the private key you have transported to your HANA Express!

and voila here is the outcome

in Chrome

or in FireFox

or in Safari