-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
salt-api can't bear a lot of concurrency when using tornado backend? #55953
Comments
another thing is about salt-master, we just do this with development environment,the salt-master of product environment is more cpus and ram than dev env. |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue. |
Thank you for updating this issue. It is no longer marked as stale. |
@liu624862560 this sounds like your master is overloaded by a bunch of minion returns coming at once. There are a number of options in this case.
What way looks better for you? I also want to hear someone else @saltstack/team-core. |
BTW, how it works if you're doing the same request not using salt-api? |
Description of Issue
here is question, we want use salt-api to send some shell command to minions, we use tornado_rest salt-api, we have one master and about 800 minions, while we use requests and multiprocessing/threading to make requests to salt-api, when we set about 100 minions and do test.ping function, we get about ten returners like [{}], and if we set about 400 minions, we get many [{}],we use request like this
and we just want get results synchronously, don't want use async/returners/event,so is there something I don't know with config? can salt-api do as we want? and anyone has some ideas, please leave you message, thanks for all of you.
Setup
salt-api config
Steps to Reproduce Issue
some logs like this:
it looks like something wrong with SaltReqTimeoutError, we don't have much error like this with cherrypy_rest salt-api,cherrypy_rest is good but not fast we when have many minions to control at once
Versions Report
Salt Version:
Salt: 2018.3.3
Dependency Versions:
cffi: 1.6.0
cherrypy: unknown
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.7.2
libgit2: 0.24.6
libnacl: Not Installed
M2Crypto: 0.21.1
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.6
mysql-python: 1.2.5
pycparser: 2.14
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: 0.24.2
Python: 2.7.5 (default, Aug 4 2017, 00:39:18)
python-gnupg: Not Installed
PyYAML: 3.10
PyZMQ: 15.3.0
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4
System Versions:
dist: centos 7.3.1611 Core
locale: UTF-8
machine: x86_64
release: 3.10.0-693.5.2.el7.x86_64
system: Linux
version: CentOS Linux 7.3.1611 Core
The text was updated successfully, but these errors were encountered: