-
Notifications
You must be signed in to change notification settings - Fork 9
KBPlus AWS Install Guide
WORK IN PROGRESS!!!!
This document details how to install and configure a new clean KBPlus instance on AWS infrastructure. It's very much a living document and work in progress - Your input is most welcome! The instructions are generic, but illustrated with examples from the installation of the k-int test system. Please make sure the commands are appropriate to your local environment.
Create AWS and RDS instances (You'll want an elastic IP too) - Take care to make them in the same security group! This guide builds on the default AWS ubuntu 1310 server image. In development of this guide, a medium instance (Ubuntu 1310 server edition) for the web server and a micro RDS instance have been used.
This procedure was last tested on 08-01-2014 and used to create a working app with shib authentication. The app is hosted at https://kbplus.k-int.com/kbplus. Instance started on demand, please holler if you want access. The doc has been tested with kbplus 4.0 beta 1
The following packages are recommended. Ideally apt packaged forms should be used over manual installs to provide easier updating when critical security fixes and other updates are needed. It is necessary to upgrade components in KB+ from time to time, and hard coded dependencies are brittle and will cause build breaks. Avoid broken builds by setting up infrastructure using modern server management idioms rather than legacy practices.
apt-get install apache2 libapache2-mod-shib2 tomcat7-user libshibsp6 libtcnative-1
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer
adduser kbplus
You will need a certificate for your domain!
KB+ can operate in localauth mode. This is particularly useful for testing as it works on development machines and non-shib hosts. Set localauth=true in the demo-config.groovy file and the system will authenticate using a traditional auth form. The accounts listed in the config file will be created (admin/admin) you are advised to change the password if running on a public facing system.
Copy the certificates supplied by your CA to
/etc/ssl/certs/
In our examples, these will be
/etc/ssl/certs/k-int.com.crt
/etc/ssl/certs/k-int.com.key
/etc/ssl/ca.crt
These files will be referenced in the next steps using the SSLCertificateKeyFile and SSLCACertificateFile directives in the apache virtual hosts config
a2enmod proxy_http proxy ssl shib2 proxy_ajp
Set up virtual host file for the new domain. In this case, we're setting up https://kbplus.k-int.com so we're editing /etc/apache2/sites-available/kbplus.k-int.com-ssl.conf
<VirtualHost *:443>
ServerAdmin webmaster@k-int.com
ServerName kbplus.k-int.com
DocumentRoot /var/www/kbplus
<Directory /var/www/kbplus>
AllowOverride None
Order allow,deny
allow from all
</Directory>
ErrorLog /var/log/apache2/error.log
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
CustomLog /var/log/apache2/ssl_access.log combined
SSLEngine on
SSLCertificateFile /etc/ssl/certs/k-int.com.crt
SSLCertificateKeyFile /etc/ssl/certs/k-int.com.key
SSLCACertificateFile /etc/ssl/ca.crt
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory /usr/lib/cgi-bin>
SSLOptions +StdEnvVars
</Directory>
BrowserMatch "MSIE [2-6]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
# MSIE 7 and newer should be able to use keepalive
BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
</VirtualHost>
Create the directory for your index.html, in this case, /var/www/kbplus and set up an index.html so you can test the https config.
Copy your real cert files as sent by a CA to /etc/ssl/certs/ This is normally .pem, <site.key> and ca.pem
Enable the new site, in the config above
a2ensite kbplus.k-int.com-ssl
Restart apache
apachectl restart
After this, if all is sorted, you should be able to visit https://kbplus.k-int.com (Or whatever domain you configured) and see your index.html page.
You will need to edit /etc/shibboleth/shibboleth2.xml and set the entityId (And also add the APJ prefix onto the application element)- For our test case, this is 'https://kbplus.k-int.com/shibboleth', you should use whatver URI you have registered as an SP. You will also need to configure SSO according to local policy and configuration. in UKFAM, we add
<SSO discoveryProtocol="SAMLDS" discoveryURL="https://wayf.ukfederation.org.uk/DS">
SAML2 SAML1
</SSO>
We also need to add application metadata (Commented out in default shibboleth2.xml file). Here is the stanza for the uk federation. YMMV.
<MetadataProvider type="XML" uri="http://metadata.ukfederation.org.uk/ukfederation-metadata.xml"
backingFilePath="/etc/shibboleth/ukfederation-metadata.xml" reloadInterval="14400">
<MetadataFilter type="RequireValidUntil" maxValidityInterval="2592000"/>
<MetadataFilter type="Signature" certificate="ukfederation.pem"/>
</MetadataProvider>
For this to work, you will need the uk federation metadata file in /etc/shibboleth/ukfederation.pem
Finally, make sure that the prefix attribute is set on the application element
<ApplicationDefaults entityID="https://kbplus.k-int.com/shibboleth"
REMOTE_USER="eppn persistent-id targeted-id"
attributePrefix="AJP_">
Your shib service will need a self signed cert (Or other cert, determined by local policy). Most shib instructions say this is generated at install time, but for the ubuntu package at least, you need to run shib-keygen.
Add the following stanza to your http virtual host config file (/etc/apache2/sites-available/kbplus.k-int.com.conf in this example)
ProxyPass /kbplus ajp://localhost:8009/kbplus
<Location /kbplus>
AuthType shibboleth
ShibRequestSetting requireSession 1
require valid-user
</Location>
ProxyIOBufferSize 65536
# This is for testing - not needed in production
<Directory /var/www/kbplus/shibboleth>
AllowOverride None
AuthType shibboleth
ShibRequestSetting requireSession 1
require valid-user
</Directory>
Create the directory (/var/www/kbplus/shibboleth in this case) and add an index.html in that directory. Then restart shibd and apache
service shibd restart
apachectl restart
Once this is done, you should be able to visit
https://your-server/Shibboleth.sso/Metadata
To download the shib config file you will need to supply to the federation when registering a SP for this app.
#Tomcat app config
Our install sets up the kbplus tomcat as a user level process. Create a tomcat install using the tomcat7-instance-create script. This script cleanly separates tomcat libraries from configuration for easy maintenance and upgrade (Hosting).
sudo su - kbplus
tomcat7-instance-create tomcat
Edit tomcat/conf/server.xml to uncomment the APR loader (~line 26)
<!--APR library loader. Documentation at /docs/apr.html -->
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
Comment out port 8080 connector:
<!--
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
URIEncoding="UTF-8"
redirectPort="8443" />
-->
Also uncomment the AJP Module running on port 8009, disable tomcatAuthentication, enable UTF-8, and allow access from localhost=127.0.0.1 apache only, stanza should be
<!-- Define an AJP 1.3 Connector on port 8009 -->
<Connector port="8009"
protocol="AJP/1.3"
redirectPort="8443"
tomcatAuthentication="false"
packetSize="65536"
URIEncoding="UTF-8"
address="127.0.0.1"
/>
Then edit the grails config for the app in this deployment
mkdir ~/.grails
Edit ~/.grails/demo-config.groovy Example file follows
Finally, download the kbplus-4.0-beta war and copy it to tomcat/webapps
datasource.url='jdbc:mysql://AWSHOSTNAME.rds.amazonaws.com/kbplus?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8'
dataSource.username='kbplus'
dataSource.password='knowplus'
aggr.es.cluster='elasticsearch'
publicationService.baseurl='http://knowplus.edina.ac.uk:2012/kbplus/api'
// no slash at end of SystemBaseURL
SystemBaseURL="http://localhost:8080/demo"
docstore='http://deprecated/deprecated'
kbplusSystemId='AWSTestIntsance'
ZenDeskBaseURL='https://projectname.zendesk.com'
ZenDeskLoginEmail='Zen.Desk@Host.Name'
ZenDeskLoginPass='Zen.Desk.Password'
KBPlusMaster=true
juspThreadPoolSize=10
sysusers=[
[ name:'admin', pass:'change_admin_pass', display:'Admin', roles:['ROLE_ADMIN', 'ROLE_USER' ] ]
]
JuspApiUrl='https://www.jusp.mimas.ac.uk/'
grails.plugins.springsecurity.shibboleth.active = true
ldap.active=false
ldap.authorities.retrieveGroupRoles=false
grails.plugins.springsecurity.shibboleth.roles.attribute='ROLE_USER'
grails.plugins.springsecurity.shibboleth.principalUsername.attribute='persistent-id'
grails.plugins.springsecurity.shibboleth.username.attribute='persistent-id'
kbplus.authmethod='shib'
doDocstoreMigration=true
Edit $TOMCAT_HOME/bin/setenv.sh and make sure that JAVA_OPTS is set to at least
JAVA_OPTS="-Djava.awt.headless=true -Xmx2G -Xms1G -server -server -XX:MaxPermSize=1024m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled"
Hat-Tip to colleagues at http://www.abes.fr/ for spotting this omission, and their patience in working through the issues.
KB+ uses elasticsearch to provide faceted full text search over titles, packages and other objects in the system.
KB+ requires elasticsearch 1.x, e.g. 1.7.5.
Either use the apt repository to download and install elasticsearch, or download manually. Also configure boottime startup using update-rc.d:
- Either Repository (use 1.x repository) or
- Manual download and manual installation
The cluster name should tally with aggr.es.cluster in the demo-config.groovy file (See Tomcat App Config below). By default, this is set to elasticsearch also, so the defaults should work fine for you.
After elastic search has been installed, the es_mappings need to be copied. From KBPlus repo, copy conf_hints/es_mappings/ contents to elasticsearch conf/mappings/kbplus (eg to /etc/elasticsearch/mappings/kbplus/ when using default paths) .
Elasticsearch seems to work best with the following config in /etc/elasticsearch/elasticsearch.yml. YMMV. After editing, don't forget to "service elasticsearch restart"
cluster:
name: "elasticsearch"
node:
name: "Xavier ES"
master: true
data: true
http:
port: 9200
enabled: true
ToDo - TBH if you've got this far you can probably figure this out, but will add more here soon.
You probably want to take note of the
doDocstoreMigration=true
Flag in the demo-config.groovy file. This makes sure that any existing documents held in the JCR jackrabbit store are migrated to blobs in the database,
- Official SP installation notes on shibboleth wiki
- UK Federation SP Setup Docs
- Notes on registering a Service Provider in the UK federation
- Shib notes from usc
And
- Another SP install guide - clean and readable
- [Configuring apache with mod proxy and shib auth ] (https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPJavaInstall)
- [Relevant email from Shib list] (https://groups.google.com/forum/#!topic/shibboleth-users/K90-JyuvSaI)
Important note...
If you're terminating ssh at the LB and passing HTTP to the backend you may need to use ShibURLScheme as in
ShibURLScheme https
<Directory /var/www/kbplus/shibboleth>
AllowOverride None
AuthType shibboleth
ShibRequestSetting requireSession 1
require valid-user
</Directory>
To get the correctly forwarded auth challenge. Also you must set server name (Or ShibURLScheme will point to https://x.y.z:80 which won't work). To do this, the following stanza at the top of the vhosts file worked for us:
ServerName https://kbplus.k-int.com
UseCanonicalName On