Installation instructions on Azure cloud

Installing Docker

  • Install Docker on both machines

sudo apt-get install -y docker.io

systemctl start docker

Managing Containers

  • To stop a container: sudo docker stop <contianer_id/name>
  • To remove a container: sudo docker rm <contianer_id/name>
  • To restart a container: sudo docker restart <contianer_id/name>
  • To access a container bash: sudo docker exec -it <contianer_id/name> bash

Setting up MySQL using Docker on Machine 1

  • Start a docker container mapping a location (e.g. /home/autolab/JavaAutolab/sqldata) and specifying the root password for MySQL: sudo docker run --name autolab-db --net=host -v /home/autolab/JavaAutolab/sqldata:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=your_password_here -d mysql:latest
  • sudo docker exec -it autolab-db bash to enter bash of mysql
  • mysql -u root -p and enter password.
  • Enter create database Autolab in the mysql console.
  • Exit from container.

Setting up GitLab using Docker on Machine 1

  • Change SSH port of the machine to 2222: sudo vi /etc/ssh/sshd_config

# What ports, IPs and protocols we listen for

Port 22

Change this to 2222 Listen to the new port by restarting SSH - /etc/init.d/ssh restart Download and start GitLab Docker image. It will be started on port 80.

sudo docker run --detach --hostname gitlab.example.com --publish 443:443 --publish 80:80 --publish 22:22 --name gitlab --restart always --volume /srv/gitlab/config:/etc/gitlab --volume /srv/gitlab/logs:/var/log/gitlab --volume /srv/gitlab/data:/var/opt/gitlab gitlab/gitlab-ce:latest Visit localhost in the browser. Reset 'root' password of GitLab as follows: http://docs.gitlab.com/ce/security/reset_root_password.html In GitLab user as root with an empty repository as test. Create a GitLab user as lab_author. Create GitLab user for students as 2013B2A7866G and other ID numbers. To redirect HTTP requests to HTTPS replace the gitlab.rb,localhost.key,localhost.crt files in the main directory in the container using -
sudo docker cp ./gitlab.rb gitlab:/etc/gitlab/gitlab.rb
sudo docker exec -it gitlab mkdir /etc/gitlab/ssl
sudo docker cp ./localhost.key gitlab/etc/gitlab/ssl/
sudo docker cp ./localhost.crt gitlab/etc/gitlab/ssl/

Deploying Load Balancer on Machine 1

  • Clone this repo: https://github.com/prasadtalasila/JavaAutolab
  • Change the directory to load_balancer and build load balancer image - sudo docker build -t load_balancer .
  • To start the container, specify the full path of the directory to store the submissions. It will be mapped to /load_balancer/submissions of the container. Example: sudo docker run --name load_balancer --net=host -v /home/autolab/JavaAutolab/load_balancer/submissions:/load_balancer/submissions -d -i -t load_balancer
  • To access the bash shell of the container sudo docker exec -it load_balancer bash
  • Generate SSH key for GitLab: ssh-keygen -t rsa -C "admin@example.com" This command will prompt you for a location and filename to store the key pair and for a password. When prompted for the location and filename, just press enter to use the default.
  • Add SSH Key to GitLab root user by doing the following: cat ~/.ssh/id_rsa.pub Copy-paste the key to the 'My SSH Keys' section under the 'SSH' tab in root profile at GitLab (Profile Settings-->SSH Keys). Please copy the complete key starting with ssh-rsa and ending with your username and host.
  • Define parameters in nodes_data_config.json: cd load_balancer vi nodes_data_config.json
  • Clone dummy repo for ssh init - git clone git@<gitlab_ip_without_port>:root/test.git. Make sure the GitLab IP in the clone command and the config file is identical. (eg. localhost at both places). This is extremely important.
  • Run the server: nohup nodejs load_balancer.js &
  • Press enter and exit.

Deploying Main Server on Machine 1

  • Change the directory to main_server and build main server image - sudo docker build -t main_server ..
  • sudo docker run --net=host --name main_server -d -i -t main_server will give a container id.
  • To access the bash shell of the container sudo docker exec -it main_server bash
  • Define parameters in conf.json and labs.json: cd main_server vi conf.json vi labs.json
  • Run the server: nohup nodejs main_server.js &
  • Press enter and exit

Deploying Execution Node on Machine 2

  • Clone this repo: https://github.com/prasadtalasila/JavaAutolab
  • Build execution node image - Change the directory to execution_node and build main server image - sudo docker build -t execution .
  • sudo docker run --net=host -d -i -t execution will give a container id.
  • To access the bash shell of the container sudo docker exec -it <container id> bash
  • Generate SSH key: ssh-keygen -t rsa -C "admin@example.com" This command will prompt you for a location and filename to store the key pair and for a password. When prompted for the location and filename, just press enter to use the default.
  • Add SSH Key to GitLab root user - cat ~/.ssh/id_rsa.pub Copy-paste the key to the 'My SSH Keys' section under the 'SSH' tab in root profile at GitLab (Profile Settings-->SSH Keys). Please copy the complete key starting with ssh-rsa and ending with your username and host.
  • Clone dummy repo for ssh init - git clone git@<gitlab_ip_without_port>:root/test.git
  • Define parameters in conf.json and scores.json: cd execution_nodes vi conf.json vi scores.json
  • Before proceeding further, make sure the load balancer is running.
  • Run the server: nohup nodejs execute_node.js &
  • Press enter and exit
  • Now add the container Port and IP in nodes_data_conf.json file in the load balancer.
  • Repeat for upto 5 containers.

Checking Deployment:

  • GitLab should be accessible on http://<machine_1_ip>
  • Visit https://<machine_1_ip>:<main_server_port>/status to see the status of all microservices.

References

  • https://hub.docker.com/_/mysql/
  • http://docs.gitlab.com/omnibus/docker/
  • https://docs.docker.com/engine/installation/linux/ubuntulinux/

Generating HTTPS keys and certificates using openssl:

  • $ openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem
  • $ openssl rsa -in key.pem -out key.pem

Security Rules for Main Server VM:

Inbound Allowed

default-allow-ssh

Source: Any

Protocol: TCP

Port: *

Destination: Any

Port: 22

ssh

Source: Any

Protocol: TCP

Port: *

Destination: Any

Port: 2222

student_main_server

Source: Any

Protocol: Any

Port: *

Destination: Any

Port: 9000 (or the main server host port)

gitlab_http

Source: Any

Protocol: Any

Port: *

Destination: Any

Port: 80

gitlab_https

Source: Any

Protocol: Any

Port: *

Destination: Any

Port: 443

Outbound

None

Security Rules for Execution Node VM:

Inbound Allowed

default-allow-ssh

Source: Any

Protocol: TCP

Port: *

Destination: Any

Port: 22

Outbound

None