Installing Docker

Install Docker on both machines

sudo apt-get install -y docker.io

systemctl start docker

Managing Containers

To stop a container: sudo docker stop <contianer_id/name>
To remove a container: sudo docker rm <contianer_id/name>
To restart a container: sudo docker restart <contianer_id/name>
To access a container bash: sudo docker exec -it <contianer_id/name> bash

Setting up MySQL using Docker on Machine 1

Start a docker container mapping a location (e.g. /home/autolab/JavaAutolab/sqldata) and specifying the root password for MySQL: sudo docker run --name autolab-db --net=host -v /home/autolab/JavaAutolab/sqldata:/var/lib/mysql -v /etc/localtime:/etc/localtime:ro -e MYSQL_ROOT_PASSWORD=your_password_here -d mysql:latest
sudo docker exec -it autolab-db bash to enter bash of mysql
mysql -u root -p and enter password.
Enter create database Autolab in the mysql console.
Exit from container.

Setting up GitLab using Docker on Machine 1

Change SSH port of the machine to 2222: sudo vi /etc/ssh/sshd_config

# What ports, IPs and protocols we listen for

Port 22

Change this to 2222 Listen to the new port by restarting SSH - /etc/init.d/ssh restart Download and start GitLab Docker image. It will be started on port 80.

sudo docker run --detach --hostname gitlab.example.com --publish 443:443 --publish 80:80 --publish 22:22 --name gitlab --restart always --volume /srv/gitlab/config:/etc/gitlab --volume /srv/gitlab/logs:/var/log/gitlab --volume /srv/gitlab/data:/var/opt/gitlab gitlab/gitlab-ce:latest Visit localhost in the browser. Reset 'root' password of GitLab as follows: http://docs.gitlab.com/ce/security/reset_root_password.html In GitLab user as root with an empty repository as test. Create a GitLab user as lab_author. Create GitLab user for students as 2013B2A7866G and other ID numbers. To redirect HTTP requests to HTTPS replace the gitlab.rb,localhost.key,localhost.crt files in the main directory in the container using -
sudo docker cp ./gitlab.rb gitlab:/etc/gitlab/gitlab.rb
sudo docker exec -it gitlab mkdir /etc/gitlab/ssl
sudo docker cp ./localhost.key gitlab/etc/gitlab/ssl/
sudo docker cp ./localhost.crt gitlab/etc/gitlab/ssl/

Deploying Load Balancer on Machine 1

Clone this repo: https://github.com/prasadtalasila/JavaAutolab
Change the directory to load_balancer and build load balancer image - sudo docker build -t load_balancer .
To start the container, specify the full path of the directory to store the submissions. It will be mapped to /load_balancer/submissions of the container. Example: sudo docker run --name load_balancer --net=host -v /home/autolab/JavaAutolab/load_balancer/submissions:/load_balancer/submissions -v /etc/localtime:/etc/localtime:ro -d -i -t load_balancer
To access the bash shell of the container sudo docker exec -it load_balancer bash
Generate SSH key for GitLab: ssh-keygen -t rsa -C "admin@example.com" This command will prompt you for a location and filename to store the key pair and for a password. When prompted for the location and filename, just press enter to use the default.
Add SSH Key to GitLab root user by doing the following: cat ~/.ssh/id_rsa.pub Copy-paste the key to the 'My SSH Keys' section under the 'SSH' tab in root profile at GitLab (Profile Settings-->SSH Keys). Please copy the complete key starting with ssh-rsa and ending with your username and host.
Define parameters in nodes_data_config.json: cd load_balancer vi nodes_data_config.json
Clone dummy repo for ssh init - git clone git@<gitlab_ip_without_port>:root/test.git. Make sure the GitLab IP in the clone command and the config file is identical. (eg. localhost at both places). This is extremely important.
Run the server: nohup nodejs load_balancer.js &
Press enter and exit.

Deploying Main Server on Machine 1

Change the directory to main_server and build main server image - sudo docker build -t main_server ..
sudo docker run --net=host --name main_server -v /etc/localtime:/etc/localtime:ro -d -i -t main_server will give a container id.
To access the bash shell of the container sudo docker exec -it main_server bash
Define parameters in conf.json and labs.json: cd main_server vi conf.json vi labs.json
Run the server: nohup nodejs main_server.js &
Press enter and exit

Deploying Execution Node on Machine 2

Clone this repo: https://github.com/prasadtalasila/JavaAutolab
Build execution node image - Change the directory to execution_node and build main server image - sudo docker build -t execution .
sudo docker run --net=host -v /etc/localtime:/etc/localtime:ro -d -i -t execution will give a container id.
To access the bash shell of the container sudo docker exec -it <container id> bash
Generate SSH key: ssh-keygen -t rsa -C "admin@example.com" This command will prompt you for a location and filename to store the key pair and for a password. When prompted for the location and filename, just press enter to use the default.
Add SSH Key to GitLab root user - cat ~/.ssh/id_rsa.pub Copy-paste the key to the 'My SSH Keys' section under the 'SSH' tab in root profile at GitLab (Profile Settings-->SSH Keys). Please copy the complete key starting with ssh-rsa and ending with your username and host.
Clone dummy repo for ssh init - git clone git@<gitlab_ip_without_port>:root/test.git
Define parameters in conf.json and scores.json: cd execution_nodes vi conf.json vi scores.json
Before proceeding further, make sure the load balancer is running.
Run the server: nohup nodejs execute_node.js &
Press enter and exit
Now add the container Port and IP in nodes_data_conf.json file in the load balancer.
Repeat for upto 5 containers.

Checking Deployment:

GitLab should be accessible on http://<machine_1_ip>
Visit https://<machine_1_ip>:<main_server_port>/status to see the status of all microservices.

Firewall Rules:

Machine 1

Allow incoming SSH connections on port 2222.
Allow incoming SSH connections on port 22.
Allow all incoming requests on port 80.
Allow all incoming requests on port 443.
Allow all incoming requests on port <main_server_port>.
All other ports should be accessible only via machine 2.

Machine 2

Allow incoming SSH connections on port 22.
All other ports should be fully accessible only via machine 1.

References

https://hub.docker.com/_/mysql/
http://docs.gitlab.com/omnibus/docker/
https://docs.docker.com/engine/installation/linux/ubuntulinux/
https://github.com/wsargent/docker-cheat-sheet
http://stackoverflow.com/questions/19456129/how-to-change-url-of-a-working-gitlab-install/28005168#28005168

Generating HTTPS keys and certificates using openssl:

$ openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem
$ openssl rsa -in key.pem -out key.pem