-
Notifications
You must be signed in to change notification settings - Fork 182
Set Up and Run SmokeDetector
To set up SmokeDetector (SD), please execute the following commands:
git clone https://github.com/Charcoal-SE/SmokeDetector.git
cd SmokeDetector
git checkout deploy
sudo pip3 install -r requirements.txt --upgrade
pip3 install --user -r user_requirements.txt --upgrade
Next, copy config.sample
to a new file called config
, and edit the values required. If you are going to be running this SmokeDetector instance as a normal SD instance with metasmoke, see the "config" file with MS below.
You should now be able to run SmokeDetector.
You will need a keybase account. You will also need to have the "smoke detector runner" role on metasmoke (MS). Contact an MS admin to request the role. When given the role, you should separately be added to the "charcoal" channel on keybase.
In the files section for the keybase "charcoal" channel, there is a "config" file. You should use that config file as a basis for the "config" file you use in your "SmokeDetector" directory.
- Set
location
to the name you choose for your SD instance. This should be your username, followed by a slash and an identifier that's unique among all your Smokey instances. This is the name that your instance will display in messages in chat and what will be displayed on MS for status. - Set
metasmoke_key
- On metasmoke in the dropdown for your username, select "My SmokeDetectors".
- Click on "create a new key" and enter the name you used for
location
above. - Once you complete the dialog, your SD instance will be listed and you can copy the "Key" to
metasmoke_key
.
- Save the "config" file, as those are the only changes you need.
If a file "rooms_custom.yml" exists in the SD directory, then it will be used instead of the "rooms.yml" file. If you are not running a full SD instance, which is reporting into Charcoal HQ and all other rooms, you will need to create a "rooms_custom.yml" which describes the rooms in which you want SD to be reporting and accepting commands. For more information about the format used in "rooms.yml" and "rooms_custom.yml", please see: Rooms configuration.
At this point, you should be able to run SmokeDetector using the instructions below.
Using a AWS EC2 instance can be a quick way to get an SD instance up and running. Based on reports from @iBug in chat, a t2.micro instance, what's available from the AWS Free Tier for 12 months, doesn't have enough CPU credits, at 6/hour, for the instance to be the active SD instance full-time. Brief testing with a t3.micro, 12 CPU credits/hour, indicated that it's not sufficient to run SD full time without additional CPU charges. It's expected that a t3.small, 24 CPU credits/hour, should be sufficient for a full-time active SD instance. However, a t2.micro instance does appear to be enough for SD to run as a standby instance to fill in on an irregular basis. An additional note is that SD on the AWS Linux AMI described below did report some out of memory errors when running on a t3.micro (1GB) without any swap configured. On the other hand, SD functions on a AWS Lightsail instance with 512MB RAM and 2GB swap configured, but that lightsail instance doesn't have enough CPU credits to operate SD for more than a few hours.
To set-up a AWS EC2 instance for SD you can do the following:
-
Create the EC2 instance (launch a AWS instance tutorial)
- Create an AWS account (address, verifiable phone number, and credit card required) and Sign-in to the AWS console.
- Click "Launch Instance ->"
- Click "Select" for "Amazon Linux 2 AMI (HVM), SSD Volume Type - ami-0277b52859bac6f4b" (the first one in the Amazon Machine Image (AMI) list)
- You can pick any flavor of Linux you're familiar with, just remember to substitute installation commands for the correct package manager. For example, the above version of Linux uses
yum
. Other versions may useapt-get
,rpm
, orpacman
etc. In addition, other versions of Linux may already have versions of some of the packages installed in the commands below, or may be missing yet other packages. The commands listed below were only tested in the AMI mentioned above.
- You can pick any flavor of Linux you're familiar with, just remember to substitute installation commands for the correct package manager. For example, the above version of Linux uses
- Click "Review and Launch"
- Click "Launch"
- Create a public key pair, or use one you've previously created.
- If creating a new public key pair, download the private key to somewhere on your system from which you can run
ssh
.
- If creating a new public key pair, download the private key to somewhere on your system from which you can run
- Click "View Instances"
- Select the new instance
- Click "Connect"
- Use ssh to connect to EC2 instance:
- Copy-&-paste the example ssh line in the popup into a command line running in the directory where you stored the .pem file with the private key (created or selected earlier).
-
Execute the following commands. You can copy-&-paste all of them at one time. The commands above the blank line get your EC2 instance to where SmokeDetector can be installed (only tested in the AMI mentioned above). The commands below the blank line clone SmokeDetector from GitHub and install its Python dependencies (the commands below the blank line are a duplicate of those at the top of this page, but are included below to make copy-&-pasting easier):
sudo yum -y update sudo fallocate -l 2G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile sudo sh -c 'echo "/swapfile swap swap defaults 0 0" >> /etc/fstab' sudo yum -y install gcc sudo yum -y install git sudo yum -y install python37 sudo yum -y install python3-devel sudo python3 -m pip install pip --upgrade git clone https://github.com/Charcoal-SE/SmokeDetector.git cd SmokeDetector git config user.email "smokey@erwaysoftware.com" git config user.name "SmokeDetector" git checkout deploy sudo /usr/local/bin/pip3 install -r requirements.txt --upgrade pip3 install --user -r user_requirements.txt --upgrade
-
Create the "config" file
- See The "config" file with MS above if you are going to use this SD instance with metasmoke.
- If you are not going to run this SD instance with metasmoke, then copy
config.sample
to a new file calledconfig
, and edit the values required.
To run, use python3 nocrash.py
, or python3 nocrash.py standby
(preferably in a daemon-able mode, like a screen
session).
You can also use python3 ws.py
, but then SmokeDetector will be shut down after 6 hours;
when running from nocrash.py
, ws.py
will automatically be restarted.
(This is to be sure that closed WebSockets, if any, are reopened.)
nocrash.py
is the controlling Python code for SmokeDetector. It runs ws.py
, which is the SD instance, in a subprocess and restarts the SD instance when it stops. ws.py
may stop as the result of various chat !!/
commands, or do to errors. The syntax for nocrash.py
is:
python3 nocrash.py [standby] [--loglevel=(debug|info|warning|error)] [no_se_activity_scan] [no_deletion_watcher] [no_edit_watcher]
-
standby
— Starts the SmokeDetector instance in standby mode. -
no_se_activity_scan
— Causes the SmokeDetector instance not to watch the SE WebSocket for new posts, even when the instance is active. This gives you an SD instance which will listen for commands in chat, but won't be scanning for new posts, other than ones you!!/scan
or!!/report
. If you are not testing something which is directly involved with scanning posts from the SE WebSocket (i.e. you don't need to scan a large number of posts, or test/use the SE WebSocket and per-site post fetching queues), then usingno_se_activity_scan
will probably be quite beneficial to you, as it will allow you to focus on what you are testing without seeing large amounts of output from scanning posts. This can also be used at times when the SE WebSocket is having problems. -
no_deletion_watcher
— Disables SmokeDetector'sDeletionWatcher
. The deletion watcher uses the SE WebSocket to watch for the deletion of posts which have been reported by SD. Using this in combination withno_se_activity_scan
andno_edit_watcher
will result in SD not opening the SE WebSocket, which can be helpful if the SE WebSocket is having problems, testing, etc. Using all those options will result in an SD instance which is monitoring chat for commands, which will allow performing chat commands without SD scanning SE sites based on any WebSocket reports. -
no_edit_watcher
— Disables SmokeDetector'sEditWatcher
. The edit watcher uses the SE WebSocket to watch for edits on posts which have been scanned by SD. Using this in combination withno_se_activity_scan
andno_deletion_watcher
will result in SD not opening the SE WebSocket, which can be helpful if the SE WebSocket is having problems, testing, etc. Using all those options will result in an SD instance which is monitoring chat for commands, which will allow performing chat commands without SD scanning SE sites based on any WebSocket reports. -
--loglevel=(debug|info|warning|error)
— Set the level of log output to send to the console. By default, there's quite a bit of information logged to the console while scanning.
Portions of this page were taken from README.md.
Smokey is made with <3 by the Charcoal Team and the other awesome contributors from Charcoal HQ.