…/timing-logs

…test/timing-logs

…test/timing-logs

* install zip & unzip

* fetch depth

…into test/timing-logs

* Debug (#1232)

* Revert "Debug (#1232)" (#1240)

This reverts commit 49a5c9e.

* Fix phase locking (#1230) (#1241)

* add logs for 2 phase lock

* add update log

* add root in log

* add update log

* use update lock

* add log in update repo

* use save

* use exec

* log rows affected

* use no key update

* use repo update

* cleanup

* add defer

* add defer

* increase timeout by 5 mins

* Case 3 and case 4 (#1198)

* Send WM commit status to conductor

* Rename file

* Modify log type

* Add state for stopping blobber from committing WM

* Send file meta root to rpc server

* Export function to use it in conductor client

* Modify field type

* Add code to get file meta root

* Modify file meta root retrieval

* Add field

* Apply rename for conductor test based on conductor state

* Add comment

* Log error

---------

Co-authored-by: Ebrahim Gomaa <ebrahim.gomaa.hg@gmail.com>

* Feature/ Challenge based on rounds (#1226)

* Fix

* Updated gosdk

* Fix

* Fix

* Fix

* Logging

* Debug

* Fix

* Fix

* Debug

* Debug

* Debug

* Fix unit tests

* Logigng

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Fix

* Fix

* Updated gosdk

* Fix

* Fix

* log config too

* Fix phase locking (#1230)

* add logs for 2 phase lock

* add update log

* add root in log

* add update log

* use update lock

* add log in update repo

* use save

* use exec

* log rows affected

* use no key update

* use repo update

* cleanup

* add defer

* add defer

* Fix

* Added logging

* Debug

* Debug

* Added reference to logs

* Fix

* Fix

* Updated gosdk

* Use slice instead of ll

* Validator last 5 transactions

* Fix

* Fix

* Updated gosdk

* Resolved comments

* Updated gosdk

---------

Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com>

* Increase limit on number of open challenges per fetching (#1249)

* Fix

* Updated gosdk

* Fix

* Fix

* Fix

* Logging

* Debug

* Fix

* Fix

* Debug

* Debug

* Debug

* Fix unit tests

* Logigng

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Fix

* Fix

* Updated gosdk

* Fix

* Fix

* log config too

* Fix phase locking (#1230)

* add logs for 2 phase lock

* add update log

* add root in log

* add update log

* use update lock

* add log in update repo

* use save

* use exec

* log rows affected

* use no key update

* use repo update

* cleanup

* add defer

* add defer

* Fix

* Added logging

* Debug

* Debug

* Added reference to logs

* Fix

* Fix

* Updated gosdk

* Use slice instead of ll

* Validator last 5 transactions

* Fix

* Fix

* Updated gosdk

* Resolved comments

* Updated gosdk

* Fix

* Increase time limit

* Increase time limit

* Fix

* Fix

* Fix

* Fix

* Debug

* Debug

* Debug

* Fix

* Fix

* Fix

* Cleanup logging

---------

Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com>

* Refactor commit (#1239)

* add szwg to commit

* lock for query

* aggregate queries

* check len

* rmv debug

* only update file ref

* move update fields to ref

* fix numBlock download stats

* rmv log

* rmv commented collector code

* Error on renaming dir (#1250)

* throw err when renaming dir

* fix unit test

* Fix replace blobber (#1251)

* Async processing (#1225)

* fix pre download

* rmv hash

* add async processing

* use range

* fix unit test

* fix delete change

* add logs

* fix unit test

* fix delete cmd

* save file ref

* add conn timing logs

* set default to 32MB

* fix timing log

* add inner lock

* fix test

* add ctx to cancel go routine

* parallel write to file

* fix connection

* revert storage changes

* empty commit

* Fix phase locking (#1230)

* add logs for 2 phase lock

* add update log

* add root in log

* add update log

* use update lock

* add log in update repo

* use save

* use exec

* log rows affected

* use no key update

* use repo update

* cleanup

* add defer

* add defer

* rmv allocObj from connObj

* Download block limit (#1254)

* add limit as 500 to numBlocks

* add daily and req limit

* fix test

* fix test

* Fix download stats (#1256)

* fix download stats

* fix typo

* Fix empty alloc cleanup (#1252)

* Fix replace blobber

* Fix empty alloc cleanup

* Debug Rounds fetch (#1255)

* Fix

* Updated gosdk

* Fix

* Fix

* Fix

* Logging

* Debug

* Fix

* Fix

* Debug

* Debug

* Debug

* Fix unit tests

* Logigng

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Updated gosdk

* Fix

* Fix

* Updated gosdk

* Fix

* Fix

* log config too

* Fix phase locking (#1230)

* add logs for 2 phase lock

* add update log

* add root in log

* add update log

* use update lock

* add log in update repo

* use save

* use exec

* log rows affected

* use no key update

* use repo update

* cleanup

* add defer

* add defer

* Fix

* Added logging

* Debug

* Debug

* Added reference to logs

* Fix

* Fix

* Updated gosdk

* Use slice instead of ll

* Validator last 5 transactions

* Fix

* Fix

* Updated gosdk

* Resolved comments

* Updated gosdk

* Fix

* Increase time limit

* Increase time limit

* Fix

* Fix

* Fix

* Fix

* Debug

* Debug

* Debug

* Fix

* Fix

* Fix

* Cleanup logging

* Fix

* Debug

* Fix

---------

Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com>

* remove prefix (#1257)

* remove .gpg file if already exist

* introduce flock to apt commands

* check on docker context existance and install git

* change buildx to action

* refine buildx build name based on runner name

* create docker context based on runner name

* increase blobber and validator job timeout

* Refine docker buildx creation and usage

* Create Unique buildx per job per runner

* remove qemu and upgrade go action version to v4

* changed runners labels build-&-publish-docker-image.yml

* added server cleanup along with blobber build

* ADD disk cleanup for conductor test

* fix cleanup script

* fix cleanup script

* added sudo to perform cleanup

* bypassing cleanup failure

* bypassing cleanup failure for validators

* Fix Save change (#1263)

* add and update change

* fix empty allocation id in change

* add rename dir

* Fix move change when srcPath is same as destPath (#1270)

* fix move change when dest is same as src

* add fileID for new ref

* fix unit test

* Fix challenge timing issue (#1277)

* Fix challenge timing issue

* Fix challenge timing issue

* Fix challenge timing issue

* bypass cleanup if it is already running (#1280)

* Revert "Fix challenge timing issue (#1277)" (#1283)

* Limit Max File Size on blobber (#1274)

* Added update in config

* 5TB config

* Reverted max_file_size config on blobber and fetching from 0chain

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix unit tests

---------

Co-authored-by: dabasov <yuderbasov@gmail.com>

* add l2 caching for alloc (#1258)

* add l2 caching for alloc

* rmv return from commit repo

* use update

* fix update

* add log

* rmv update object

* add log for allocation

* move lock to middleware

* fix save alloc update

* empty commit

* fix blobber size update

* fix unit test

* cleanup

* rmv commit in initMap

* fix unit test

* fix renamefile mock db

* add commit method to enhancedDB

---------

Co-authored-by: Yury <yuderbasov@gmail.com>

* add list query param (#1273)

* Optimize GetRefs, correct and add indexes on ref table (#1284)

* Correct index on ref table, add path index for get refs optimization

* Fix gorm tags on ref table

* Fix typo

* fix goose migration tags

* add pg_trgm extension

* sharder keep list (#1266)

* changed get round

* updated version

* Fix/finalize (#1285)

* Debug finalize

* Debug finalize

* Debug finalize

* Debug finalize

* Debug finalize

* Fix finalize

* Debug

* Fix

* FIX

* Fixed

* Fixed

* fix list file (#1288)

* Cleanup worker for challenge tables (#1278)

* add cleanup worker for challenge tables

* move cleanupGap to config

---------

Co-authored-by: Yury <yuderbasov@gmail.com>

* add err check and increase wait time (#1289)

* Client stats and blacklist (#1286)

* add client stats and blacklist

* add client stats check

* fix goose migration

* fix migration

* fix typo

---------

Co-authored-by: Yury <yuderbasov@gmail.com>

* Fix nested rename dir (#1295)

* add log for rootRef

* add objTreePath

* fix rename nested dir

* use save (#1297)

* fix retry redeem wm check (#1299)

* Use rename in place of copy buffer (#1298)

* use rename than copy buffer

* fix storage tests

* fix update latest wm (#1301)

* Fix blobber stats (#1305)

* fix blobber stats

* add stats worker

* rmv listAlloc

* build stats on start

* Feat/remove native sc calls (#1306)

* removed native MakeSCRestAPICall

* removed native MakeSCRestAPICall

---------

Co-authored-by: Jayash Satolia <73050737+Jayashsatolia403@users.noreply.github.com>

* WM cleanup (#1296)

* add cleanup worker for wm

* add hdd tablespace

* add log for hdd path

* update default hdd path

* update owner

* add init db script

* we are already creating with tablespace in initdb.sh

* update initdb script.sh

* update sql for write_markers_archive table

---------

Co-authored-by: Manohar Reddy <b.manu199@gmail.com>
Co-authored-by: Yury <yuderbasov@gmail.com>

* Fix/max file size updateWorker and added config for storagesc update interval (#1308)

* Added worker for max file size

* Added config for update worker

* Debug

* Added logs for update worker

* Added fix for worker timings =

* Debug

* Debug

* Debug

* Debug

* Debug

* Removed logging

* Removed logging

* Refactor =

* Refactor

* fix tablespace permission issue in conductor test b0docker-compose.yml

* Fix/cond tablespace (#1309)

* updated docker-compose wrt conductor test tablspace

* updated docker-compose wrt conductor test tablspace

* updated docker-compose wrt conductor test tablspace

* updated blobber env

* update gosdk version to latest of sprint-1.11 (#1310)

* fix blobber stats (#1314)

* race condition fix (#1316)

* Node recovery conductor tests (#1259)

* notify on validator tickert generatd

* fix race condition

* Added changes for fail upload commit (#1318)

* Feat/download verify ct (#1317)

* notify on validator tickert generatd

* fix race condition

* miss up download for CT

* debug logs

* Improve download performance (#1315)

* improve download file

* write data

* rmv content length

* fix unit test

* add new db snapshot migration (#1312)

Co-authored-by: Yury <yuderbasov@gmail.com>

* fix commit error (#1320)

* update gosdk (#1319)

Co-authored-by: Yaroslav Svitlytskyi <53532703+YarikRevich@users.noreply.github.com>

* fix where statement (#1321)

* Fix/loop break (#1322)

* updated gosdk

* add mutex

* updated gosdk

* merged changes

* merged changes

---------

Co-authored-by: Hitenjain14 <hitenjain146@gmail.com>

* Fix blobber stats panic (#1324)

* Fix blobber stats panic

* Fix

* Fix

* fix break in switch (#1326)

* change hasher to blake3 (#1325)

* change hasher to blake3

* add blake hash

* fix lint

* fix lint for hash write

* update gosdk

* Fix challenge worker (#1327)

* revert changes

* fix challenge worker

* fix lint

* add goto for accept case

---------

Co-authored-by: Yury <yuderbasov@gmail.com>

* Config watcher (#1329)

* add watch config

* add on config change

* Remove min lock demand from blobber (#1332)

* fix alloc lock (#1334)

* Feature: implement fork per run strategy (#1335)

* feature: added fork creation

* feature: improved Tenderly fork creation flow

* Merge pull request #1336

* rmv wm lock table

* fix unit tests

* rmv write_locks table

* rmv gorm tag

* empty commit

* Fix mem usage (#1337)

* updated gosdk to blobber

* change hasher to sha2 256

* fix unit test

* changed runner for lint tests.yml

---------

Co-authored-by: shahnawaz-creator <msac8225@gmail.com>
Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com>

* fix rb for failed marker (#1342)

* Fix lwm (#1338)

* add wm log

* fix log

* fix update allocation

* fix log

* lru of value

* fix alloc update

* Fix auth ticket (#1341)

* fix read auth ticket

* fix unit test

* convert to base64

* Fix expiration column name (#1343)

* Fix expiration column name

* Fix size param

* merged changes

* Revert "merged changes"

This reverts commit 39f717a.

* feature: added custom block number during Tenderly fork creation (#1345)

* fix update alloc (#1347)

* extended field for mimetype

* add aws secrets for blobbers (#1349)

* add aws secrets for blobbers

* add validator secrets

* typo

* update logs

---------

Co-authored-by: sanchit <sanchit4242.com>

* Fix root hash (#1346)

* add root logs

* check alloc root

* fix lint

* rmv wm cleanup

* fix lint

* fix encrypted key point (#1350)

* fix encrypted key point

* check root ref precommit

* update min-submit to 20% (#1348)

* update gosdk to v1.11.0 (#1351)

* fix wm retries (#1352)

---------

Co-authored-by: Jayash Satolia <73050737+Jayashsatolia403@users.noreply.github.com>
Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com>
Co-authored-by: Kishan Dhakan <kishandhakan17@gmail.com>
Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com>
Co-authored-by: Laxmi Prasad Oli <olilaxmiprasad@gmail.com>
Co-authored-by: Ebrahim Gomaa <ebrahim.gomaa.hg@gmail.com>
Co-authored-by: Amr Amin <amr_ashraf_mohamed_amin@yahoo.com>
Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com>
Co-authored-by: shahnawaz-creator <msac8225@gmail.com>
Co-authored-by: tapishsinha-rs <tapish.sinha@remotestate.com>
Co-authored-by: Dinmukhammed Kambarov <52813950+din-mukhammed@users.noreply.github.com>
Co-authored-by: Manohar Reddy <b.manu199@gmail.com>
Co-authored-by: Yaroslav Svitlytskyi <53532703+YarikRevich@users.noreply.github.com>
Co-authored-by: Hitenjain14 <hitenjain146@gmail.com>
Co-authored-by: Sanchit Sharma <47826073+Sanchit011@users.noreply.github.com>

* Hotfix update allocation worker

* Fix lint

* Fix

* Fixed updateAllocationInDB check as well

* check prev root and add constraint

* check if alloc is updated

* rmv check

---------

Co-authored-by: Jayash Satolia <jayash.satolia.min19@iitbhu.ac.in>

* fix challenge cancel commit

* change conn lifetime

…into test/timing-logs