Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jobserver #5912

Open
wants to merge 89 commits into
base: next
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
55b2416
init jobserveR
kanwarujjaval Jan 14, 2025
e9bb5b8
lint and deps
kanwarujjaval Jan 14, 2025
db5fd7d
Merge branch 'next' of github.com:Countly/countly-server into jobserver
kanwarujjaval Jan 14, 2025
2e2e2f4
deepscan errors
kanwarujjaval Jan 14, 2025
8918bca
test config
kanwarujjaval Jan 14, 2025
3010da0
handle promise/callback/fn function for job run
kanwarujjaval Jan 14, 2025
807434e
Job runner improvements
kanwarujjaval Jan 14, 2025
05d3d4a
redundant log here, job baseclass already does this
kanwarujjaval Jan 14, 2025
3485290
hard fix the pulse version, expecting changes in the development
kanwarujjaval Jan 14, 2025
bdf6252
define node engine in packagejson
kanwarujjaval Jan 14, 2025
d1eb6a5
events dont exist :)
kanwarujjaval Jan 14, 2025
ed21279
remove obsolete
kanwarujjaval Jan 14, 2025
3d4201d
support enable/disable of jobs externally
kanwarujjaval Jan 14, 2025
24125d8
self contained better job scheduling
kanwarujjaval Jan 14, 2025
967fa24
Expanded job baseclass with documented example
kanwarujjaval Jan 14, 2025
7b4dc94
JobServer with oplog watcher to communicate with main countly process
kanwarujjaval Jan 14, 2025
372ee07
progress job
kanwarujjaval Jan 14, 2025
480cbc4
codacy fix
kanwarujjaval Jan 15, 2025
8179924
Merge branch 'next' of github.com:Countly/countly-server into jobserver
kanwarujjaval Jan 15, 2025
ef82822
complex job operations from oplog
kanwarujjaval Jan 15, 2025
cc12690
refactor for code cleanup because why not
kanwarujjaval Jan 15, 2025
7b21626
README update
kanwarujjaval Jan 15, 2025
984016c
fix docs
kanwarujjaval Jan 15, 2025
5acfa2f
remove new logger file :D
kanwarujjaval Jan 15, 2025
baf399f
progress with bookmark data
kanwarujjaval Jan 15, 2025
3c38282
so many tiny docs and logs
kanwarujjaval Jan 15, 2025
372ddd6
codacy fix
kanwarujjaval Jan 15, 2025
a3acc44
add design pattern conventions
kanwarujjaval Jan 16, 2025
e103818
wip mark in readme
kanwarujjaval Jan 16, 2025
4445d5a
Add simple example
kanwarujjaval Jan 16, 2025
97ed753
sane defaults for Job
kanwarujjaval Jan 16, 2025
b6a5421
sane defaults for global
kanwarujjaval Jan 16, 2025
46c77aa
Prevent job scheduler collision
kanwarujjaval Jan 16, 2025
36da81c
Merge branch 'next' of github.com:Countly/countly-server into jobserver
kanwarujjaval Jan 20, 2025
67f2588
migrate crashes jobs
kanwarujjaval Jan 23, 2025
d6eb77f
migrate hooks jobs
kanwarujjaval Jan 23, 2025
4a45e76
migrate logger jobs
kanwarujjaval Jan 23, 2025
b91fe81
migrate reports jobs
kanwarujjaval Jan 23, 2025
84fba28
migrate stats jobs
kanwarujjaval Jan 23, 2025
774822c
migrate views jobs
kanwarujjaval Jan 23, 2025
254a52c
disable api.js jobs
kanwarujjaval Jan 23, 2025
dbb0981
migrate core jobs
kanwarujjaval Jan 23, 2025
b87ae76
prevent double loading of jobs in old and new
kanwarujjaval Jan 23, 2025
b7edf02
fixes for runnow and priority/retry configs
kanwarujjaval Jan 23, 2025
2f33110
scan new job every 30 seconds
kanwarujjaval Jan 23, 2025
70f5b6a
refactoring and backward compatiblity injections in jobserver
kanwarujjaval Jan 23, 2025
b25f4db
basic api for jobs
kanwarujjaval Jan 23, 2025
ceaf8b9
init jobs fe
kanwarujjaval Jan 23, 2025
0edd2a7
refactor job view
kanwarujjaval Jan 23, 2025
3d70ec6
lint fix
kanwarujjaval Jan 25, 2025
880dc46
wip: fixes for frontend
kanwarujjaval Jan 25, 2025
b317dca
simplify server connections
kanwarujjaval Jan 25, 2025
290b4db
remove hooks from multi process and job scheduling
kanwarujjaval Jan 28, 2025
6be5c9b
remove job handler from fetch.js
kanwarujjaval Jan 28, 2025
9d13e0c
remove cluster from cache
kanwarujjaval Jan 28, 2025
6454c86
remove cluster from batcher
kanwarujjaval Jan 28, 2025
4a2d445
remove ipchandler from logger
kanwarujjaval Jan 28, 2025
3f27384
remove job handler from requestprocessor
kanwarujjaval Jan 28, 2025
c0ad9e9
remove cluster from tracker
kanwarujjaval Jan 28, 2025
bb7c836
cleaner import in jobserver
kanwarujjaval Jan 28, 2025
ee53851
much slimmer api.js
kanwarujjaval Jan 28, 2025
078b0c7
Merge branch 'next' of github.com:Countly/countly-server into jobserver
kanwarujjaval Jan 28, 2025
1c9abc0
remove cluster from pluginmanager
kanwarujjaval Jan 28, 2025
5e54286
missing dependency for jobs ui
kanwarujjaval Jan 29, 2025
7460d6e
resolve conflicts
kanwarujjaval Jan 29, 2025
79886ad
remove push from default json for temporarily
kanwarujjaval Jan 29, 2025
9015238
disable old jobs tests
kanwarujjaval Jan 29, 2025
2a882ac
update alerts to new job server
kanwarujjaval Jan 30, 2025
4b83b75
clean
kanwarujjaval Jan 30, 2025
a66d851
add dev deps
kanwarujjaval Jan 31, 2025
8f38f9c
fix alertProcessor job schedule
kanwarujjaval Jan 31, 2025
9038da4
remove tests for old jobs
kanwarujjaval Jan 31, 2025
a291aae
remove cluster from system-utility
kanwarujjaval Jan 31, 2025
a3ef55d
small updates system-utility
kanwarujjaval Jan 31, 2025
6910c61
update Alerts to check hourly,daily,monthly queries more robustly
kanwarujjaval Feb 1, 2025
0db8ab6
allow default disabled state in jobs and Manual only jobs
kanwarujjaval Feb 1, 2025
32bdb3b
migrate AppExpire but disabled by default
kanwarujjaval Feb 1, 2025
65b35ba
remove clear job job
kanwarujjaval Feb 1, 2025
774273f
remove clues to the old jobs
kanwarujjaval Feb 1, 2025
d08297c
literally 1 jsdoc added
kanwarujjaval Feb 1, 2025
771d3cf
some cleaning up
kanwarujjaval Feb 2, 2025
a6d2134
fixing cross countly comm
kanwarujjaval Feb 2, 2025
908609c
remove obsolete migration jobs
kanwarujjaval Feb 2, 2025
2e2b37b
fix per job logger
kanwarujjaval Feb 2, 2025
9763887
improvement to single run job
kanwarujjaval Feb 2, 2025
c04f249
try to handle funnel and flow jobs :)
kanwarujjaval Feb 2, 2025
3c93269
close server before flushall, and keepalive between node and nginx
kanwarujjaval Feb 2, 2025
200cb3c
fix logger job and simplify
kanwarujjaval Feb 2, 2025
2d53330
config tweak for jobs
kanwarujjaval Feb 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
basic api for jobs
  • Loading branch information
kanwarujjaval committed Jan 23, 2025
commit b25f4db7d2a03de8e83e845983fec05baa362910
12 changes: 10 additions & 2 deletions api/utils/requestProcessor.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,14 @@ const validateUserForDataWriteAPI = validateUserForWrite;
const validateUserForGlobalAdmin = validateGlobalAdmin;
const validateUserForMgmtReadAPI = validateUser;
const request = require('countly-request')(plugins.getConfig("security"));
const Handle = require('../../api/parts/jobs/index.js');

try {
require('../../jobServer/api');
log.i('Job api loaded');
}
catch (ex) {
log.e('Job api not available');
}

var loaded_configs_time = 0;

Expand Down Expand Up @@ -2732,7 +2739,8 @@ const processRequest = (params) => {
*
*/
validateUserForGlobalAdmin(params, async() => {
await Handle.suspendJob(params);
// await Handle.suspendJob(params);
// TODO: UPDATE TO NEW JOB PROCESSOR
});
break;
}
Expand Down
Loading