-
Notifications
You must be signed in to change notification settings - Fork 0
Building TopStack
TopStack provides Amazon compatible cloud services for your OpenStack private cloud. Management of a cloud and cloud resources is provided through both a programmatic interface and the StackStudio user interface. The goal of TopStack is to produce an ubiquitous Open Source cloud computing platform that will meet the needs of public and private cloud providers regardless of size. Information and documentation on TopStack can be found at the TopStack.org site.
The following instruction provide the steps necessary to build and install TopStack from source.
While TopStack will run and deploy on a multitude of platforms, the instructions provided on this page are for a specific OS and version. We suggest using a 64 bit instance of Ubuntu 12.04 or greater and please note that Ubuntu must be a 64 bit version. The 32 bit version will not work as a 64 bit version of Ubuntu is required due to the use of the Chef Server which only runs on a 64 bit instance.
If you're in a bind to get servers up and running quickly, try using Virtual Box. Virtual Box is a free, open source offering from Oracle, that will allow you to create multiple OS isntances on your system. When coupled with Vagrant, it's quite easy to get an Ubuntu instance up and running fairly quickly.
Quick OS Startup Steps:
- Install Virtual Box
- Follow instructions on the Vagrant Getting Started page using the 64 bit Ubuntu instance (http://files.vagrantup.com/precise64.box).
Note that if using the Vagrant getting started page to fire up your Ubuntu instance for the purpose of installing TopStack, be 100% sure to use the 'http://files.vagrantup.com/precise64.box' box (in place of the 32 bit box in the getting started instructions) as TopStack will not be able to fully deploy without a 64 bit instance.
TopStack requires some infrastructure to be in place in order to run. In addition to a public or private cloud, TopStack services require a starter image to be installed into the cloud. In addition, TopStack makes heavy use of Chef, so an available Chef server is required. The following steps describe installation of these items in detail.
TopStack uses Chef for configuration of launched instances. Any Chef 10 or Chef 11 server will work; the node on which TopStack is installed will need to have credentials to manage chef, since nodes will be added dynamically.
Steps to install and configure Chef are described on [Chef for Topstack](Chef for Topstack).
TopStack runs on modern JEE version; you'll need a container with Servlet 3.0 or greater. We test with Tomcat 7.
There are additional steps to configure Tomcat to run TopStack; those steps are described here: TopStack Deployment Instance Setup. Those steps require some files from TopStack, so you'll need to build as described below.
Install MySQL from the web or use a package from a repository. MySQL 5.5 or greater is required.
We use the excellent protocol buffers library from Google for internal messaging. To build TopStack projects, you must download and install the protobuf compiler.
apt-get install protobuf-compiler
will probably work (on Ubuntu), or you can download and install from source: protobuf-2.4.1.tar.bz2
We're currently using protobuf 2.4.1.
When properly the installed, the output of "ant compile.proto" on the ToughResources project should look like so:
compile.proto:
[apply] Applied protoc to 1 files and 0 directories.
The protocol buffers messages are shuttled around within TopStack over a ZeroMQ transport. We require 3.2.2 or greater ZeroMQ library.
Install the shared library:
# If on Linux, these are prerequisites (sample for Ubuntu)
sudo apt-get install autoconf automake pkg-config libtool
wget http://download.zeromq.org/zeromq-3.2.2.tar.gz
tar -zxf zeromq-3*
cd zeromq-3*
./autogen.sh && ./configure --prefix=/usr && make && sudo make install
Install the Java bindings:
# If on Linux, these are prerequisites (sample for Ubuntu)
sudo apt-get install git autoconf automake pkg-config libtool openjdk-6-jdk
git clone git://github.com/zeromq/jzmq.git
cd jzmq
./autogen.sh && ./configure --prefix=/usr && make && sudo make install
Ensure that tomcat has the zmq jar in it's shared library path. It must be loaded outside the war, so that the native code is only loaded once.
cp src/zmq.jar ${TOMCAT_HOME}/lib
If you choose to build or run unit tests using Eclipse, You'll also need to ensure that Eclipse has access to ZeroMQ. You can add a "-Djava.library.path=/usr/local/lib" to the default VM to ensure all projects have access to the ZeroMQ library.
If this is a brand new environment, we recommend installing OpenStack, Grizzly or later. TopStack will work with OpenStack-derived clouds such as HP or Rackspace, but additional configuration may be required.
All of the TopStack services and shared libraries are included in this project as submodules. Check out all the following projects (either by git submodule update from this project, or by cloning directly.)
- TopStackResources
- TopStackCore
- TopStackScheduler
- TopStackDNS53
- dasein-cloud-openstack (Our fork, you can use official fork, potentially)
- TopStackCompute
- TopStackInternal
- TopStackAutoScale
- TopStackLoadBalancer
- TopStackMetricSearch
- TopStackMonitorCommon
- TopStackMonitor
- TopStackMessageAgent
- TopStackRoot
- TopStackChefRepo
Java Tools
You'll need the standard Java build tools; a JDK (1.6+), Apache Ant, Apache Maven.
You will probably want to create a build.properties file in your home directory. That will allow you to customize settings for your environment. See the following file for settings:
ToughResources/build.properties.sample
We have some dependencies that come from Maven, but a few are checked in as jars.
The build is still Ant, with some ant tasks to download maven dependencies. The first build will pull down all maven-based dependencies, which will take a while, but subsequent builds should be speedy.
TopStack works from a base image, which must be deployed to your cloud. See the steps for Building a Base Image.
TopStack requires information about the cloud you're using, in order to connect and to spin up the correct VM types. The root project ToughResources contains a sample file, cloud-config.xml.sample, which contains the settings for an OpenStack Grizzly cloud. Copy this file and make changes to match your configuration. You can set a variable in the build.properties (described above) to allow the build and deploy process to find your cloud config:
external.cloud.config=<path-to-cloud-config>/cloud-config.xml
There are specific properties that required by TopStack unit and integration tests, such as credentials to use and where TopStack should be deployed.
Multiple environments can be configured; we use different environments to test against different OpenStack versions. If no environment is specified, an environment of "dev" is assumed. The minimal setup is create a "dev.properties"; a sample file is included in the ToughResources project, env/dev.properties.sample. Copy that file to a new location and specify the location in the build.properties:
external.env.dir=<path-to-env-dir>
The TopStack database must be seeded with initial data. The initial schema is defined in TopStackResources/resources/configDB.sql. The following steps show how to do this:
# Assuming Ubuntu 12.04; on other platforms, install MySQL 5.5
apt-get install mysql-server
mysql
mysql> create database topstack;
mysql> exit;
mysql < {TOPSTACK}/TopStackResources/resources/configDB.SQL
With all the configuration performed about, you can now run builds! The primary build is through ant, though dependencies are fetch using maven. For each project, run:
ant
The default target is usually appropriate for all projects. However, the "dasein-cloud-openstack" project is maven only, so it should be installed with:
mvn install -DskipTests
With a Tomcat instance properly configured (See Instance Setup), you can deploy the binaries you just built to have a running TopStack.
NOTE: You should deploy TopStackInternal first, and then create an account as detailed below before continuing to deploy the rest of the services.
You will need to create an account in the database in order to use TopStack services. You can do this with a curl command like:
curl 'http://(deployment host):8080/ASInternal?Action=CreateAccount&UserName=(user name for DB)&AccessKey=(your access key)&SecretKey=(your secret key)&APITenant=(your tenant id)&CloudName=(cloud zone, such as nova)&Email=sample@example.com'