Skip to content

Commit

Permalink
Documentation revisionx
Browse files Browse the repository at this point in the history
  • Loading branch information
@xybu
xybu committed Jan 11, 2015
1 parent e86ae42 commit 3936e36
Show file tree
Hide file tree
Showing 2 changed files with 132 additions and 116 deletions.
20 changes: 11 additions & 9 deletions ga-docs/docker.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,9 @@ GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1"
Setup
=====

(1) Create a non-root user to avoid sudo
(1) Grant docker access permission to non-root users to avoid sudo

```bash
# create a user called "slave"
# keep password empty so it cannot be logged in via SSH
sudo useradd -m slave

# add current user and www-data to docker group
sudo gpasswd -a ${USER} docker
sudo gpasswd -a www-data docker
Expand Down Expand Up @@ -60,20 +56,26 @@ CONTAINER_ID=`sudo docker create -t -i ubuntu bash`
sudo docker start -a -i $CONTAINER_ID

# The following commands run inside docker container
apt-get update
apt-get install -y build-essential cmake automake checkinstall gcc gdb software-properties-common
apt-get update && apt-get upgrade
apt-get install -y build-essential cmake automake checkinstall gcc gdb software-properties-common binutils bison m4 cproto python3.4 python2.7 libcurl3 python3-pip
apt-get autoclean

# create a user called "slave" to avoid default root permission
sudo useradd -m slave
# (END)

# Do the following in a bash process OUTSIDE the docker
sudo docker commit $CONTAINER_ID docker_username/image_name:tag
sudo docker push docker_username/image_name
# Use your own credential in the above two commands.
# For C development, I have created docker image "xybu/c_dev:jan_15"
# For C development, I have created docker image "xybu/cdev:v1"

```

Integrate with gitlab-ag
========================


By default, `ga-hook/delegates/ga-grader_queue.py` enables Docker integration and assumes
the image `xybu/cdev:v1` which is configured as step 2 above specifies. If you want to
disable Docker (not recommended) or use another image or change virtualization solution
you will need to modify `ga-grader_queue.py` on your own.
228 changes: 121 additions & 107 deletions ga-docs/installation.md
View file
Original file line number Diff line number Diff line change
@@ -1,107 +1,121 @@
# Installation

The installation guide assumes GitLab has been set up on a Ubuntu-based server.
For other systems, the shell commands may need to change.

## Prerequisites

gitlab-ag requires `git` and `curl` packages installed on the machine, and needs
`python-daemonize` package for Python3k. Besides, since it needs to run as a
standalone website, `PHP`(>5.4) with PDO and SQLite support and a web server
like `Nginx` (recommended) are needed. PHP extensions like `APCu` and `opcache`
are recommended. Ubuntu has built-in support for Python.
Here is a guide for [setting up a Ubuntu server from scratch](http://xybu.me/setting-up-a-ubuntu-server/).

```bash
sudo apt-get install git curl python3-pip
sudo pip3 install daemonize
```

[Docker](http://docker.com/) is recommended as a container to securely run
grading programs. Refer to `docker.md` for installation.

## Install gitlab-ag

(1) Grab the source code

First download the source code to a dir which will be the parent dir of gitlab-ag
web root.

```bash
git clone https://github.com/xybu/gitlab-ag.git
```
or
```bash
wget https://codeload.github.com/xybu/gitlab-ag/zip/master
unzip master.zip
```

Rename the generated directory if necessary and enter it. We assume the
default name `gitlab-ag`.

(2) Change credentials

This step can be skipped with little harm to security.

Open `ga-include/ga-session.php` and change the constant `SESSION_SALT` to
a more complex string.

(3) Change file permissions

The following directories need `RWX` permission on web server worker user:

* ga-data
* ga-hook/logs
* ga-hook/pushes
* ga-hook/queue
* ga-hook/fails

The following files need `RX` permission on web server worker user:

* ga-hook/delegates/ga-get_repo.py
* ga-hook/delegates/ga-grader_queue.py

Other files should have `RX` (for `.php` code) or `R` permission for web server
worker user.

(4) Set-up web server

gitlab-ag requires two virtual sites to be added to your web server.

An example config file for Nginx, named `nginx.conf.example`, is given for reference.

First create a site whose root dir points to `gitlab-ag`. The port can be picked
up arbitrarily as long as not used. This site is the admin panel of gitlab-ag and
should be visible to public (or at least yourself). For security, be sure to have
this virtual website deny accesses to anywhere except for `ga-assets`, `index.php`,
and `admincp.php`.

Second create a site whose root dir points to `gitlab-ag/ga-hook`. The port should
be secret, and the site should be accessible only from GitLab host and gitlab-ag
machine (if they are on the same machine, deny all accesses from other than
`127.0.0.1`). Set up URL rewrite rule so that `webhook/123` can be redirected to `ga-webhook.php?key=123`. For Nginx, put the following line in a `server` block:

```
rewrite ^/(webhook|syshook|callback)/(.*)$ /ga-$1.php?key=$2? last;
```

For extra security, make 100% sure that the hook site cannot be accessed by
public users. And make sure no user except for web server worker on the machine can access `ga-data`. (For example, for Nginx, set the owner and group of `ga-data` to `www-data` and
permission bits to `0700`).

(5) Run the installer

In your browser, visit `http://url_to_gitlab_ag/`. For example, if the site with root
`gitlab-ag` listens to port `8080` and the machine domain is `example.com`, then open
`http://example.com:8080/`. An installer webpage will show up. Fill in the form carefully
and hit `Hook with GitLab` button, upon success gitlab-ag will add its system hook url
to GitLab. And then click `Install` button to its right and the config file
`ga-data/ga-config.php` will be generated. To rerun the installer, you need to delete this
file and visit the gitlab-ag url.


## Setup GitLab

There is no particular action to do on GitLab side. However, if there are already projects
in GitLab and you want gitlab-ag to monitor them also, you may need to add webhooks manually.
New projects will be added gitlab-ag hook automatically.
# Installation

The installation guide assumes GitLab has been set up on a Ubuntu-based server.
For other systems, the shell commands may need to change.

## Prerequisites

gitlab-ag requires `git` and `curl` packages installed on the machine, and needs
`python-daemonize` package for Python3k. Besides, since it needs to run as a
standalone website, `PHP`(>5.4) with PDO and SQLite support and a web server
like `Nginx` (recommended) are needed. PHP extensions like `APCu` and `opcache`
are recommended. Ubuntu has built-in support for Python.
Here is a guide for [setting up a Ubuntu server from scratch](http://xybu.me/setting-up-a-ubuntu-server/).

```bash
sudo apt-get install git curl python3-pip
sudo pip3 install daemonize
```

[Docker](http://docker.com/) is recommended as a container to securely run
grading programs. Refer to `docker.md` for installation.

## Install gitlab-ag

(1) Grab the source code

First download the source code to a dir which will be the parent dir of gitlab-ag
web root.

```bash
git clone https://github.com/xybu/gitlab-ag.git
```
or
```bash
wget https://codeload.github.com/xybu/gitlab-ag/zip/master
unzip master.zip
```

Rename the generated directory if necessary and enter it. We assume the
default name `gitlab-ag`.

(2) Change credentials

This step can be skipped with little harm to security.

Open `ga-include/ga-session.php` and change the constant `SESSION_SALT` to
a more complex string.

(3) Change file permissions

The following directories need `RWX` permission on web server worker user:

* ga-data
* ga-hook/logs
* ga-hook/pushes
* ga-hook/queue
* ga-hook/fails

The following files need `RX` permission on web server worker user:

* ga-hook/delegates/ga-get_repo.py
* ga-hook/delegates/ga-grader_queue.py

Other files should have `RX` (for `.php` code) or `R` permission for web server
worker user.

(4) Set-up web server

gitlab-ag requires two virtual sites to be added to your web server.

An example config file for Nginx, named `nginx.conf.example`, is given for reference.

First create a site whose root dir points to `gitlab-ag`. The port can be picked
up arbitrarily as long as not used. This site is the admin panel of gitlab-ag and
should be visible to public (or at least yourself). For security, be sure to have
this virtual website deny accesses to anywhere except for `ga-assets`, `index.php`,
and `admincp.php`.

Second create a site whose root dir points to `gitlab-ag/ga-hook`. The port should
be secret, and the site should be accessible only from GitLab host and gitlab-ag
machine (if they are on the same machine, deny all accesses from other than
`127.0.0.1`). Set up URL rewrite rule so that `webhook/123` can be redirected to `ga-webhook.php?key=123`. For Nginx, put the following line in a `server` block:

```
rewrite ^/(webhook|syshook|callback)/(.*)$ /ga-$1.php?key=$2? last;
```

For extra security, make 100% sure that the hook site cannot be accessed by
public users. And make sure no user except for web server worker on the machine can access `ga-data`. (For example, for Nginx, set the owner and group of `ga-data` to `www-data` and
permission bits to `0700`).

(5) Run the installer

In your browser, visit `http://url_to_gitlab_ag/`. For example, if the site with root
`gitlab-ag` listens to port `8080` and the machine domain is `example.com`, then open
`http://example.com:8080/`. An installer webpage will show up. Fill in the form carefully
and hit `Hook with GitLab` button, upon success gitlab-ag will add its system hook url
to GitLab. And then click `Install` button to its right and the config file
`ga-data/ga-config.php` will be generated. To rerun the installer, you need to delete this
file and visit the gitlab-ag url.

(6) Setup crontab

```bash
sudo crontab -e

# Add the following lines to the end
# For GitLab 7.6.2 and below, restart GitLab at 4:30am every day to deal with zombie processes
30 4 * * * gitlab-ctl restart > /var/log/gitlab-restart.log
# Because ga-grader_queue does not start with system, delete the pidfile if it exists
@reboot rm -f /PATH_TO_GITLAB/ga-data/ga-grader_queue.pid

Now update crond service
sudo update-rc.d cron defaults
```

## Setup GitLab

There is no particular action to do on GitLab side. However, if there are already projects
in GitLab and you want gitlab-ag to monitor them also, you may need to add webhooks manually.
New projects will be added gitlab-ag hook automatically.

0 comments on commit 3936e36

Please sign in to comment.