Infrastructure

Galaxy Infrastructure

aka, what does what?

Resources are generally divided in to two domains, usegalaxy.org resources for stuff related to running Test/Main, and galaxyproject.org resources for all the other services.

Note: there is a separate playbook, infrastructure-playbook that is responsible for managing most of the non-Galaxy infrastructure, and even some of the low level stuff on Galaxy VMs and nodes. In specific, checking the inventory files of the various Ansible "environments" should give you some insight on what does what, if it's not covered in this document.

usegalaxy.org resources

The following hosts run services for usegalaxy.org and test.galaxyproject.org.

TACC Hosts

These hosts live under the .tacc.utexas.edu domain (but most have aliases under one or more of our domains). Unless needed for the task you are performing, you should use your own TACC account. See Getting Set Up At TACC.

The admin login (for those with access) for all hosts in this section is root.

host	login	purpose	aliases
galaxy04	g2main (update main)	CVMFS release manager (stratum 0) server for the main.galaxyproject.org CVMFS repo	cvmfs0-tacc0.galaxyproject.org main-installer.galaxyproject.org
galaxy07	g2test gxsrc	Galaxy Test web, jobs, and database server, data manager server	test.galaxyproject.org test-datamanager.galaxyproject.org
galaxy08		RabbitMQ and Slurm controller
galaxy-web-03 galaxy-web-04	g2main gxsrc	Galaxy Main job handlers, also handle Pulsar staging for XSEDE resources with uWSGI and the nginx upload module	galaxy-web-03.galaxyproject.org galaxy-web-04.galaxyproject.org
galaxy-web-05 galaxy-web-06	g2main gxsrc	Galaxy Main web and FTP servers	usegalaxy.org (DNS round robin to both VMs) galaxy-web-05.galaxyproject.org galaxy-web-06.galaxyproject.org
galaxy-db-02	g2main	Galaxy Main database server
roundup49-roundup64	g2test g2main	Galaxy cluster nodes

• The per-host Galaxy Main aliases can be useful for testing the state of each VM independently
• With the exception of roundup* (which are bare metal nodes), the VMs run on TACC's VMWare cluster
• VMs are configured with varying amounts of memory/CPU by purpose

CVMFS Hosts

These hosts live under the .galaxyproject.org domain. CVMFS contents are also reachable via http://datacache.galaxyproject.org/.

host	login	purpose	virtualization host	aliases
cvmfs0-tacc0	g2main (update main) root (admin)	CVMFS release manager (stratum 0) server for the main.galaxyproject.org CVMFS repo	TACC VMware cluster	main-installer.galaxyproject.org alias of galaxy04
cvmfs0-psu0	g2test (update data) root (admin)	CVMFS release manager (stratum 0) server for the data.galaxyproject.org CVMFS repo	westvleteren
cvmfs0-psu1	g2test (update test)	CVMFS release manager (stratum 0) server for the test.galaxyproject.org CVMFS repo	smart3	test-installer.galaxyproject.org
cvmfs1-psu0	g2test (snapshot test/data) g2main (snapshot main) root (admin)	CVMFS replica server (stratum 1) server for all *.galaxyproject.org CVMFS repos	thumper
cvmfs1-tacc0	g2test (snapshot test/data) g2main (snapshot main) centos (admin)	CVMFS replica server (stratum 1) server for all *.galaxyproject.org CVMFS repos	Jetstream @ TACC
cvmfs1-iu0	g2test (snapshot test/data) g2main (snapshot main) centos (admin)	CVMFS replica server (stratum 1) server for all *.galaxyproject.org CVMFS repos	Jetstream @ IU

Distributed Computing Hosts

host	login	purpose	notes
jetstream2.galaxyproject.org	rocky	Pulsar Slurm controller	Jetstream2
jetstream-iu0.galaxyproject.org	centos	Pulsar Slurm controller	Jetstream @ IU
jetstream-tacc0.galaxyproject.org	centos	Pulsar Slurm controller	Jetstream @ TACC
jeltz.galaxyproject.org	root	datacache	Guest of smart1
login2.stampede2.tacc.utexas.edu	xcgalaxy	Pulsar Test/Main megablast/lastz and manual submissions	TACC Stampede2, login from galaxy04 only
vm027.bridges.psc.edu	xcgalaxy	Pulsar Large memory tools (Trinity, Unicycler, etc.)	PSC Bridges

galaxyproject.org resources

These hosts live under the .galaxyproject.org domain.

Virtualization hosts

All virtualization hosts run SmartOS unless otherwise noted.

host	guests	notes
smart1	nat1 jeltz eddie sentry	Hosts NAT for private network hosts, datacache server, Test Toolshed, Sentry
smart2	radegast	Hosts Main Toolshed
smart3	duvel stats cvmfs0-psu1	Hosts InfluxDB/Grafana, Galaxy Test CVMFS repo
smart5	leeroy jenkins0	Hosts Jenkins server/worker
thumper.p	cvmfs1-psu0	Hosts CVMFS stratum 1
westvleteren.p	cvmfs0-psu0 westvleteren	Backups CVMFS data repo
orval.p	orval	depot.galaxyproject.org

Service Hosts

Our standalone AWS account (infrastructure@...) is being phased out in favor of the Penn State Cloud Program due to contract advantages and simplified billing. At this time, you must have a Penn State Access Account (i.e. you must be a PSU employee) to gain access to the AWS Dashboard for these services.

host	login	purpose	virtualization host	notes
newtrail	root (admin)	LDAP Kerberos KDC DNS galaxyproject.org email	EC2
artifact	root (admin)	LDAP Kerberos KDC DNS galaxyproject.org email Website proxies	EC2	Proxies galaxyproject.org, training.galaxyproject.org
duvel	root (admin)	LDAP Kerberos KDC DNS	smart3
radegast	shed gxsrc root (admin)	Toolshed	smart2	toolshed.g2.bx.psu.edu
eddie	shed gxsrc root (admin)	Test Toolshed	smart1	testtoolshed.g2.bx.psu.edu
orval	sites depot bioarchive singularity root (admin)	depot	orval.p	depot.galaxyproject.org
stats	root (admin)	InfluxDB/Grafana	smart3	stats.galaxyproject.org
sentry	root (admin)	Sentry	smart1	sentry.galaxyproject.org
rodenbach	root (admin)	DHCP TFTP PSU local network services	metal
leeroy	jenkins root (admin)	Jenkins server	smart5	jenkins.galaxyproject.org
jenkins0	jenkins root (admin)	Jenkins worker	smart5

Non-host Cloud Services

name	login	purpose	provider
wheels	galaxy-wheels IAM user	wheels.galaxyproject.org	S3/CloudFront
docs	galaxy-docs IAM user	docs.galaxyproject.org	S3/CloudFront
galaxyproject.org	galaxy-hub IAM user	galaxyproject.org	S3/EC2 (artifact)
training		training.galaxyproject.org	S3/GitHub Pages (live) S3/EC2 (artifact) (archive)
biostar.usegalaxy.org		biostar.usegalaxy.org archive	S3/EC2 (artifact)
status	infrastructure@galaxyproject.org	Galaxy services status	statuspage.io