Setting up workspace for Janssen development
This is a step-by-step guide for developers and contributors to setup Janssen development environment on a personal workstation. Once setup, it will enable building and testing Janssen project components locally.
Here we are handcrafting the Janssen setup so we can understand each component and setting that is necessary to run Janssen server and tests. You can also leverage IDE based development environment setup.
For the purpose of this guide, we are following steps and commands required on Ubuntu OS (version 18 or above). For all other OS platforms, like Windows, Mac, same steps and commands with platform specific changes can easily be derived.
- Pre-Requisites
- Get Code
- Setup Data Store
- Setup Configuration Files
- Setup SSL
- Build and Deploy
- Run Tests
Pre-requisites
Download and Install required software
JDK
For development as well as at runtime, Janssen requires any JDK with version 8 or above. Janssen in production environment uses Amazon Corretto (11.0.8) which is an OpenJDK distribution. You can download it from here.
Git
Janssen code is hosted on Github. You can use any Git client to interact with repositories.
Maven
Maven is a build tool used by Janssen. You can download it from here
MySQL
Janssen uses persistance storage to hold configuration and transactional data. Janssen supports variety of persistance technologies including LDAP, RDBMS and cloud storage. For this guide, we are going to use MySQL relational database as our persistance store. You can download and install MySQL from here
Jetty server
Janssen uses Jetty as web application server. You can download and install Jetty 9 from here.
Setup environment variables
JAVA_HOME
: This should be set so that it points to your JDK installation directoryJETTY_HOME
: This is the directory where you have installed or unpacked your jetty distribution. This directory should containstart.jar
so that$JETTY_HOME/start.jar
is accessible.JETTY_BASE
: Set this variable by creating an empty directory where you intend to deploy Janssen auth server web application-
Host name: For Janssen modules to work correctly, you need to assign a host name to your local machine’s IP.
localhost
is not supported. To do this, we need to make changes tohosts
file. On Ubuntu and other Linux destributions, this file is/etc/hosts
, while for Windows same file can be found atC:\Windows\System32\drivers\etc\hosts
. Make an entry similar to below inhosts
file:127.0.0.1 test.local.jans.io
Here,
test.local.jans.io
can be any name of your choice. We will refer totest.local.jans.io
as our host name for rest of this guide. -
Before we start, we need to initialize few Jetty modules as shown below:
cd $JETTY_BASE java -jar $JETTY_HOME/start.jar --create-startd java -jar $JETTY_HOME/start.jar --add-to-start=server,annotations,resources,http-forwarded,threadpool,console-capture,jsp,websocket,https,ssl
Get code
-
JanssenProject code is hosted on Github. You can find it here. Project has multiple repositories for various modules. This guide will take us through setup of Janssen Auth server module. For this, you need to clone repositories listed below:
-
jans_setup (local clone location will be referred to as
setup-code-dir
going forward) -
jans-auth-sever (local clone location will be referred to as
auth-server-code-dir
going forward)
Compile and package code locally as
cd auth-server-code-dir mvn -DskipTests package
Above should compile code successfully and also create files in
target
folder underauth-server-code-dir
. Some of these files will be helpful to us in next steps. -
Setup data store
Janssen uses persistance storage to hold configuration and transactional data. Janssen supports variety of persistance mechanisms including LDAP, RDBMS and cloud storage. For this guide, we are going to use MySQL relational database as a persistance store.
As a first step, let’s create schema and users.
-
Log into MySQL using
root
user (or any other user with sufficient privileges)sudo mysql root
-
Create new database(schema) for Janssen
mysql> CREATE DATABASE jansdb;
-
Create new db user
CREATE USER 'jans'@'localhost' IDENTIFIED BY 'PassOfYourChoice';
-
Grant privileges to new user on
jans
schemaGRANT ALL PRIVILEGES ON jansdb.* TO 'jans'@'localhost';
-
Exit MySQL login
Next, we will load basic configuration data into MySQL. This data is required by Janssen modules at the time of start up. We will use a helper script that will create required schema, tables, users, permissions and also insert basic configuration data in required tables.
-
Get MySQL database data load script file from here. This script is a data dump which can directly be loaded in your local MySQL database. But before we load it, we need to replace generic host name in the script with the one that we have set for our local environment, which is
test.local.jans.io
. To do that, open script in a text editor and- replace string
https://testmysql.dd.jans.io
withhttps://test.local.jans.io:8443
- replace string
testmysql.dd.jans.io
withtest.local.jans.io
- replace string
-
Script is now ready to be executed.
-
Import data load script into your local MySQL
sudo mysql -u root -p <root-password> < jansdb_dump.sql
Setup Configuration Files
Janssen stores configuration required at the boot time in the file system. It is stored
at /etc/jans/conf
. We need to create this directory on our local file system.
sudo mkdir /etc/jans
sudo mkdir /etc/jans/conf
mkdir $JETTY_BASE/custom
mkdir $JETTY_BASE/custom/pages
Now, we need to copy teplates of these configuration files from our code base.
sudo cp <auth-server-code-dir>/server/target/conf/* /etc/jans/conf/
Among copied files, there are two files that are notable:
jans.properties
: Holds details like type of persistance to use, localtion of certificates etc.jans-sql.properties
: Since we are using MySQL RDBMS persistence store, details in this file will be used to connect MySQL.
Edit values in both these files as recommended below.
sudo vim /etc/jans/conf/jans.properties
- edit
persistence.type: ldap
topersistence.type: sql
- edit
certsDir
tocertsDir=/etc/jans/conf
todo: change this to /etc/certs
- edit value of
jansAuth_ConfigurationEntryDN
tojansAuth_ConfigurationEntryDN=ou=jans-auth,ou=configuration,o=jans
so that it matchesjans-auth
entry injansAppConf
table in DBtodo: add more entries here for each module added in jans installation like scim etc
sudo vim /etc/jans/conf/jans-sql.properties
- Set
db.schema.name
to name of schema you created while importing data load script. - Set
connection.uri
tojdbc:mysql://localhost:3306/jansdb
- Set
auth.userName
to the new user that we created above, i.ejans
- Set
auth.userPassword
to passwod that you set while creating new user, i.ePassOfYourChoice
- Set
password.encryption.method
to method you have selected to encrypt the password foruserPassword
property. If you are using plain text password for your local setup, comment out this property.
Properties of jans-sql.properies
listed above are most likely to be customised as per your local setup.
Other properties from this file can be set to standard values as given below.
connection.driver-property.serverTimezone=UTC
# Prefix connection.driver-property.key=value will be coverterd to key=value JDBC driver properties
#connection.driver-property.driverProperty=driverPropertyValue
#connection.driver-property.useServerPrepStmts=false
connection.driver-property.cachePrepStmts=false
connection.driver-property.cacheResultSetMetadata=true
connection.driver-property.metadataCacheSize=500
#connection.driver-property.prepStmtCacheSize=500
#connection.driver-property.prepStmtCacheSqlLimit=1024
# Password hash method
password.encryption.method=SSHA-256
# Connection pool size
connection.pool.max-total=40
connection.pool.max-idle=15
connection.pool.min-idle=5
# Max time needed to create connection pool in milliseconds
connection.pool.create-max-wait-time-millis=20000
# Max wait 20 seconds
connection.pool.max-wait-time-millis=20000
# Allow to evict connection in pool after 30 minutes
connection.pool.min-evictable-idle-time-millis=1800000
binaryAttributes=objectGUID
certificateAttributes=userCertificate
Setup SSL
Janssen uses secure socket layer (SSL) to secure HTTP communication. To enable same of our local setup, we need to configure self signed ceritificates.
- We will use Java keytool to generate key pair as given below.
keytool -genkeypair -alias jetty -keyalg EC -groupname secp256r1 -keypass secret -validity 3700 -storetype JKS -keystore /tmp/keystore.test.local.jans.io.jks -storepass secret
What is your first and last name?
[Unknown]: test.local.jans.io
What is the name of your organizational unit?
[Unknown]: test.local.jans.io
What is the name of your organization?
[Unknown]: local.jans.io
What is the name of your City or Locality?
[Unknown]: <short name of your city>
What is the name of your State or Province?
[Unknown]: <short name of your state>
What is the two-letter country code for this unit?
[Unknown]: <short name of your country>
Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /tmp/keystore.test.local.jans.io.jks -destkeystore /tmp/keystore.test.local.jans.io.jks -deststoretype pkcs12".
Above command will create a .jks
file under /tmp/
.
Next, we will make changes in Jetty configuration to use the keystore.
-
copy your keystore file which you generated above to
$JETTY_BASE/etc/
cp /tmp/keystore.test.local.jans.io.jks $JETTY_BASE/etc/
-
Now edit
ssl.ini
and change value of below propertiesvim $JETTY_BASE/start.d/ssl.ini
Uncomment and apply appropriate values to properties below according to inputs that you gave during creation of keystore.
jetty.sslContext.keyStorePath=etc/keystore.test.local.jans.io.jks jetty.sslContext.keyStorePassword=secret jetty.sslContext.trustStorePath=etc/keystore.test.local.jans.io.jks jetty.sslContext.trustStorePassword=secret jetty.sslContext.keyManagerPassword=secret
-
Add more keys to keystore. These keys are required for running tests. When asked, the password for source keystore is
secret
.keytool -importkeystore -srckeystore <my.code.base>/jans-auth-server/server/profiles/default/client_keystore.jks -destkeystore /tmp/keystore.test.local.jans.io.jks
-
Generate JWT
- download
jans-auth-client-1.0.0-SNAPSHOT-jar-with-dependencies.jar
fromhttps://maven.jans.io/maven/io/jans/jans-auth-client/1.0.0-SNAPSHOT/
- change to download location
cd <download-location>
- now run
java -Dlog4j.defaultInitOverride=true -cp jans-auth-client-1.0.0-SNAPSHOT-jar-with-dependencies.jar io.jans.as.client.util.KeyGenerator -keystore '/tmp/keystore.test.local.jans.io.jks' -keypasswd secret -sig_keys RS256 RS384 RS512 ES256 ES384 ES512 -enc_keys RS256 RS384 RS512 ES256 ES384 ES512 -dnname 'CN=Jans Auth CA Certificates' -expiration 365 > /tmp/keys_client_keystore.json
This command adds additional keys in
keystore.test.local.jans.io.jks
and creates a JSON file with web keys. We will use web keys files later to update db entries. - download
-
Provide keystore to Janssen and your code base from where the tests are going to get executed
- copy keystore file to profiles in
client
,server
modules in code baseTODO: see if this step is required as we are now adding certs in cacerts
- Client:
mkdir <auth-server-code-dir>/client/profiles/test.local.jans.io cp /tmp/keystore.test.local.jans.io.jks <auth-server-code-dir>/client/profiles/test.local.jans.io/
- Server:
mkdir <auth-server-code-dir>/server/profiles/test.local.jans.io cp /tmp/keystore.test.local.jans.io.jks <auth-server-code-dir>/server/profiles/test.local.jans.io/
-
Copy keystore to Janssen
sudo mkdir /etc/certs sudo cp /tmp/keystore.test.local.jans.io.jks /etc/certs/jans-auth-keys.jks sudo chown <your-user>:<your-user> /etc/certs/jans-auth-keys.jks
- copy keystore file to profiles in
-
Update your Java cacerts
-
extract certificate
openssl s_client -connect test.local.jans.io:8443 2>&1 |sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > /tmp/httpd.crt
-
Update cacerts of your JRE. For example if JRE being used my maven is
/usr/lib/jvm/java-11-openjdk-amd64
keytool -import -alias jetty -keystore /usr/lib/jvm/java-11-openjdk-amd64/lib/security/cacerts -file /tmp/httpd.crt
-
setup java web keys
Update keystore secret in database config
- update value of keystore secret in DB under entry
select JansConfDyn gluudbtest.jansAppConf where Doc_id="jans-auth"
Update JSON Web keys in database config
- open
/tmp/keys_client_keystore.json
and copy content into db field
SELECT jansConfWebKeys FROM gluudbtest.jansAppConf where doc_id = "jans-auth";
Build and Deploy
cd auth-server-code-dir
mvn -DskipTests install
This will create a .war
file which we will use to deploy.
-
Move and rename war
mv server/target/jans-auth-server.war $JETTY_BASE/webapps/jans-auth.war
-
Get template xml files from setup repo
cp <setup-code-dir>/templates/jetty/jans-auth.xml $JETTY_BASE/webapps/ cp <setup-code-dir>/templates/jetty/jans-auth_web_resources.xml $JETTY_BASE/webapps/
-
Now we can run Janssen auth server:
$ java -Djetty.home=$JETTY_HOME -Djetty.base=$JETTY_BASE -Dlog.base=/home/dhaval/temp/jetty-logs -Djans.base=/etc/jans -Dserver.base=$JETTY_BASE -jar $JETTY_HOME/start.jar
Run Tests
Janssen integration tests need a Janssen server to execute successfully. Now that you have a Janssen instance running on your local machine, you can use it to run tests. We need to give our local workspace all the essential information about target Janssen server. This is configured in form of profile
. Steps below will help us create profile in our local code workspace (auth-server-code-dir
).
Create profile for client
module
-
Create profile directory for client module
mkdir auth-server-code-dir/client/profiles/test.local.jans.io
-
copy contents of default profile to new profile directory
cp <auth-server-code-dir>/client/profiles/default/* <auth-server-code-dir>/client/profiles/test.local.jans.io/
-
Customize the values as per your setup
cd <auth-server-code-dir>/client/profiles/test.local.jans.io/
- In file
config-oxauth-test-data.properties
update values oftest.server.name
andswd.resource
properties with your host name. i.etest.local.jans.io
and update path inclientKeyStoreFile
property withprofiles/test.local.jans.io/client_keystore.jks
- In file
Create profile for server
module
-
create profile directory for server module
mkdir <auth-server-code-dir>/server/profiles/test.local.jans.io
-
copy contents of default profile to new profile directory
cp <auth-server-code-dir>/server/profiles/default/* <auth-server-code-dir>/server/profiles/test.local.jans.io/
-
Customize the values as per your setup
cd <auth-server-code-dir>/server/profiles/test.local.jans.io/
- Edit config-oxauth.properties
- Update values of
server.name
,config.jans-auth.issuer
,config.jans-auth.contextPath
properties to your host name - comment out the properties for LDAP
- Update values of
- Edit config-oxauth-test.properties
-
By default, this file contains properties applicable to LDAP backend. But since we are using MySQL as backend, we will remove all the current properties in this file and put properties listed below in it:
server.name=test.local.jans.io config.oxauth.issuer=http://localhost:80 config.oxauth.contextPath=http://localhost:80 config.persistence.type=sql #sql config.sql.db.schema.name=<your schema> config.sql.connection.uri=jdbc:mysql://<your host>:3306/<your schema> config.sql.auth.userName=<db username> config.sql.auth.userPassword=<db plain text password> config.sql.connection.driver-property.serverTimezone=UTC config.sql.password.encryption.method=<comment out this property> config.sql.auth.userName=<db username> config.sql.auth.userPassword=<db password> # Password hash method config.sql.password.encryption.method=SSHA-256 # Connection pool size config.sql.connection.pool.max-total=20 config.sql.connection.pool.max-idle=10 config.sql.connection.pool.min-idle=5 # Max time needed to create connection pool in milliseconds config.sql.connection.pool.create-max-wait-time-millis=20000 # Max wait 20 seconds config.sql.connection.pool.max-wait-time-millis=20000 # Allow to evict connection in pool after 30 minutes config.sql.connection.pool.min-evictable-idle-time-millis=1800000
-
-
Edit config-oxauth-test-data.properties and update values of below properties:
test.server.name=test.local.jans.io test.server.url=https://test.local.jans.io:8443/jans-auth clientKeyStoreFile=profiles/test.local.jans.io/keystore.test.local.jans.io.jks clientKeyStoreSecret=secret
- Edit config-oxauth.properties
-
Now you can run test cases for each module with
mvn -Dcfg=test.dd.jans.io -fae -Dcvss-score=9 -Dfindbugs.skip=true -Dlog4j.default.log.level=TRACE -Ddependency.check=false clean test