Code

Fix#400: Re-merge YAML & CLI#403

Code

Merged

trizin merged 189 commits intomainoceanprotocol/pdr-backend:mainfrom yaml-cli2oceanprotocol/pdr-backend:yaml-cli2Copy head branch name to clipboardJan 16, 2024

+17,528-10,635

Commits

Commits on Nov 22, 2023

• First stab at porting various functions over to polars... lots to go

  idiom-bytes

  committed
  c5987e1

• TOHLCV df initialization and type checking added. 2/8 pdutil tests passing

  idiom-bytes

  committed
  c1fedf7

• black formatted

  idiom-bytes

  committed
  2a0a94b

• Fixing initialization and improving test. datetime is not generated if no timestamp is present

  idiom-bytes

  committed
  a579c4d

• Restructured pdutil a bit to reduce DRY and utilize schema more strictly.

  idiom-bytes

  committed
  c5efe9f

• test initializing the df and datetime

  idiom-bytes

  committed
  6c1f3ec

• improve init test to show exception without timestamp

  idiom-bytes

  committed
  b7a60b2

• fixing test_concat such that it verifies that schemas must match, and how transform handles datetime

  idiom-bytes

  committed
  754aaee

• saving parquet enforces datetime and transform. updated test_load_append and test_load_filtered.

  idiom-bytes

  committed
  fce6718

• black formatted

  idiom-bytes

  committed
  f9e6cb8

• data_eng tests are passing

  idiom-bytes

  committed
  d0dccaf

• initial data_eng tests are passing w/ black, mypy, and pylint.

  idiom-bytes

  committed
  026ad39

Commits on Nov 23, 2023

• _merge_parquet_dfs updated and create_xy test_1 is passing. all data_eng tests that are enabled are passing.

  idiom-bytes

  committed
  2006f1a

• 2exch_2coins_2signals is passing

  idiom-bytes

  committed
  933f1d8

• Added polars support for fill_nans, has_nans, and create_xy__handle_nan is passing.

  idiom-bytes

  committed
  6264c61

• Starting to deprecate references to pandas and csv in data_factory.

  idiom-bytes

  committed
  1b5b665

• Black formatted

  idiom-bytes

  committed
  a511bd5

• Deprecated csv logic in DataFactory and created tests around get_hist_df() to verify that its working as intended. I believe kraken data is returning null at the moment.

  idiom-bytes

  committed
  444862b

Commits on Nov 24, 2023

• All tests should be passing.

  idiom-bytes

  committed
  6666618

Commits on Nov 27, 2023

• Fix #370: YAML & CLI (#371)

  Show description for 4e06884
  

  trentmc

  and

  trizin

  committed
  4e06884

• Update CI to use pdr instead of scripts/ (#399)

  Show description for 80faf81
  

  trizin

  authored and

  trentmc

  committed
  80faf81

• Replace long try/except with _safe*() function; rename pdutil -> plutil; get linters to pass

  trentmc

  committed
  25bfa3e

• Update entrypoint script to use pdr cli (#406)

  trizin

  authored
  148cb94

• Add main.py back (#404)

  Show description for 6d7661a
  trizin

  authored
  6d7661a

• Merge from issue388-refactor-csvs-pandas, plus many changes

  trentmc

  committed
  43a43df

• Merge branch 'yaml-cli2' of https://github.com/oceanprotocol/pdr-backend into yaml-cli2

  trentmc

  committed
  fa3b672

• make black happy

  trentmc

  committed
  f7bbca4

• small bug fix

  trentmc

  committed
  27ac78e

• many bug fixes. Still >=1 left

  trentmc

  committed
  97370c7

• fix warning

  trentmc

  committed
  036ca3e

• Add support for polars where needed

  trentmc

  committed
  c96464d

• tweak docstring

  trentmc

  committed
  6b4383f

• Fix #408: test_sim_engine failing in yaml-cli2, bc hist_df is s not ms. Proper testing and documentation was added, as part of the fix

  trentmc

  committed
  6ff502d

• BaseContract tests that Web3PP type is input

  trentmc

  committed
  1b6556c

• goes with previous commit

  trentmc

  committed
  4c78f72

• tweak - lowercase

  trentmc

  committed
  24e4345

• Bug fix - fix failing tests

  trentmc

  committed
  9ca66e7

Commits on Nov 28, 2023

• Remove unwanted file

  trentmc

  committed
  e24cfcb

• (a) better organize ppss.yaml for usability (b) ensure user isn't annoyed by git with their copy of ppss.yaml being my_ppss.yaml

  trentmc

  committed
  9e9abc5

• add a more precise test for modeling

  trentmc

  committed
  7855bef

• make black happy

  trentmc

  committed
  8c7d0c4

• Small refactor: make transform_df() part of helper routine

  trentmc

  committed
  516d6e9

• Fix #414: Split data_factory into (1) CEX -> parquet -> df (2) df -> X,y for models

  trentmc

  committed
  1dc2836

• Fix #415: test_cli_do_dfbuyer.py is hanging #415

  trentmc

  committed
  4dc204c

• test create_xy() even more. Clarify the order of timestamps

  trentmc

  committed
  67c2b96

• Add a model-building test, using data shaped like data from test_model_data_factory

  trentmc

  committed
  5ba906c

Commits on Nov 29, 2023

• Fix #416: [YAML branch] No Feeds Found - data_pp.py changes pair standards

  trentmc

  committed
  06aa946

• For barge#391: update to *not* use barge's predictoor branch

  trentmc

  committed
  995df8b

• Merge branch 'main' into yaml-cli2

  trentmc

  committed
  97512a6

Commits on Nov 30, 2023

• Merge branch 'main' into yaml-cli2

  trentmc

  committed
  a1db778

• Update vps.md: nicer order of operations

  trentmc

  authored
  4d6e8ff

• For #417, #418 in yaml-cli2 branch. publisher TUSD -> USDT

  trentmc

  committed
  21a65d4

• Merge branch 'main' into yaml-cli2

  trentmc

  committed
  7fd76e3

• Merge branch 'main' into yaml-cli2

  trentmc

  committed
  50ab141

• Merge branch 'main' into yaml-cli2

  trentmc

  committed
  5a47226

• remove default_network from ppss.yaml (obsolete)

  trentmc

  committed
  49d546e

• Fix #427 - time now

  trentmc

  committed
  610062d

Commits on Dec 1, 2023

• Fix #428: test_get_hist_df - FileNotFoundError. Includes lots of extra robustness checking

  trentmc

  committed
  8156b30

• remove dependency that we don't need, which caused problems

  trentmc

  committed
  bedf654

• Merge branch 'main' into yaml-cli2

  trentmc

  committed
  151e79c

• Fix #421: Add cli + logic to calculate and plot traction metrics (PR #422)

  Show description for 851dab0
  idiom-bytes

  authored
  851dab0

• bug fix: YAML_FILE

  trentmc

  committed
  2d95c57

• fix breaking test; clean it up too

  trentmc

  committed
  ede6efe

• add barge-calls.md

  trentmc

  committed
  0e7b021

Commits on Dec 4, 2023

• Fix #433. Calculate metrics and draw plots for epoch-based stats (PR #434)

  Show description for 7dd9d0f
  idiom-bytes

  authored
  7dd9d0f

Commits on Dec 5, 2023

• git merge main

  trentmc

  committed
  de9b650

Commits on Dec 6, 2023

• Tweak barge-calls.md

  Show description for 1785713
  trentmc

  authored
  1785713

• Tweak barge-calls.md: more compactly show RPC_URL calc

  trentmc

  authored
  9905f60

• update stake_token

  trentmc

  committed
  af6ce32

• Merge branch 'yaml-cli2' of https://github.com/oceanprotocol/pdr-backend into yaml-cli2

  trentmc

  committed
  a3789f9

• bug fix

  trentmc

  committed
  59f3462

• Update release-process.md: bug fix

  trentmc

  authored
  206428a

• Tweak barge-calls.md

  trentmc

  authored
  8e571d6

• git merge main

  trentmc

  committed
  482e9fe

• Tune #405 (PR #406): Update entrypointsh script to use pdr CLI

  trentmc

  committed
  909a5ab

• Update vps.md: docker doesn't need to prompt to delete

  trentmc

  authored
  9168f78

• Update vps.md: add docker-stop instrs

  trentmc

  authored
  f5be9ac

• allow CLI to have NETWORK_OVERRIDE, for more flexiblity from barge

  trentmc

  committed
  ac7670e

• Merge branch 'yaml-cli2' of https://github.com/oceanprotocol/pdr-backend into yaml-cli2

  trentmc

  committed
  784ed60

• fix pylint issue

  trentmc

  committed
  ef2d6b2

• Update barge-calls.md: link to barge.md

  trentmc

  authored
  52c11f3

• Update release-process.md: fix typo

  trentmc

  authored
  8e4bb22

• touch

  trentmc

  committed
  823c02f

• Update vps.md: more instrs around waiting for barge to be ready

  trentmc

  authored
  c73502a

Commits on Dec 7, 2023

• add unit tests for cli_module

  trentmc

  committed
  d53fbb0

• Merge branch 'yaml-cli2' of https://github.com/oceanprotocol/pdr-backend into yaml-cli2

  trentmc

  committed
  699fb1c

• Towards #437: [YAML] Publisher error 'You must set RPC_URL environment variable'

  trentmc

  committed
  5362ac2

• Bug fixes

  trentmc

  committed
  93720d1

• refactor tweaks to predictoor and trader

  trentmc

  committed
  f66cee2

• Clean up some envvar stuff. Document ppss vars better.

  trentmc

  committed
  3988f29

• publish_assets.py now supports barge-pytest and barge-predictoor-bot

  trentmc

  committed
  8f2f2d4

• bug fix

  trentmc

  committed
  8addd57

• bug fix the previous 'bug fix'

  trentmc

  committed
  1dffc03

Commits on Dec 8, 2023

• Clean up how dfbuyer/predictoor/trader agents get feeds: web3_pp.query_feed_contracts() -> data_pp.filter_feeds(); no more filtering within subgraph querying; easier printing & logging. Add timefra…

  Show description for 720aa64
  trentmc

  committed
  720aa64

Commits on Dec 9, 2023

• fix breaking subgraph tests. Still breakage in trader & dfbuyer (that's next)

  trentmc

  committed
  75b943b

Commits on Dec 10, 2023

• Fix failing tests in tradder, dfbuyer. And greatly speed up the tests, via better mocking.

  trentmc

  committed
  5b19f0b

• Fix bugs for failing tests of https://github.com/oceanprotocol/pdr-backend/actions/runs/7156603163/job/19486494815

  trentmc

  committed
  711905e

• fix tmpdir bug

  trentmc

  committed
  f667f46

• Fix (hopefully) failing unit test - restricted region in querying binance api

  trentmc

  committed
  9f112fd

• consolidate gas_price setting, make it consistent; set gas_price to 0 for development/barge

  trentmc

  committed
  c9bbf41

• fix linter complaints

  trentmc

  committed
  94c74b3

• Fix remaining failing unit tests for predictoor_batcher

  trentmc

  committed
  ae9fe21

• Finish the consolidation of gas pricing. All tests pass

  trentmc

  committed
  d1eea21

Commits on Dec 11, 2023

• Merge branch 'main' into yaml-cli2

  trentmc

  committed
  96d157f

• Update vps.md: add debugging info

  Show description for 63d9de3
  trentmc

  authored
  63d9de3

• add to/from wei utility. Copied from ocean.py

  trentmc

  committed
  0053961

• tweak docs in conftest_ganache

  trentmc

  committed
  c118763

• tweaks from black for wei

  trentmc

  committed
  21b8b2f

• Make fixed_rate.py and its test easier to understand via better var naming & docs

  trentmc

  committed
  8f21c7b

• Make predictoor_contract.py easier to understandn via better var anming & docs

  trentmc

  committed
  21d0d89

• test fixed_rate calcBaseInGivenOutDT

  trentmc

  committed
  b0dc81d

• Refactor predictoor_contract: push utility methods out of the class, and into more appropriate utility modules. And, move to/from_wei() from wei.py to mathutil.py. Test it all.

  trentmc

  committed
  ffb693a

• Tweak docstrings for fixed_rate.py

  trentmc

  committed
  535de12

• Improve DX: show dev what the parameters are. Improve UX: print when done.

  trentmc

  committed
  3f6d60f

• Improve DX & UX for predictoor_contract

  trentmc

  committed
  f88a958

• Tweak UX (prints)

  trentmc

  committed
  bb5ab0f

• Update vps.md: export PATH

  trentmc

  authored
  1df1da3

• Logging for predictoor is way better: more calm yet more informative. Predictoors only do 1 feed now.

  trentmc

  committed
  5524b1f

• Merge branch 'yaml-cli2' of https://github.com/oceanprotocol/pdr-backend into yaml-cli2

  trentmc

  committed
  ef02b65

Commits on Dec 12, 2023

• TraderAgent -> BaseTraderAgent

  trentmc

  committed
  11b6bd0

• Rename parquet_dfs -> rawohlcv_dfs; hist_df -> mergedohlcv_df; update related

  trentmc

  committed
  4460259

• apply black to test_plutil.py

  trentmc

  committed
  568f93e

• apply black to test_model_data_factory.py

  trentmc

  committed
  055f0c7

• apply black to ohlcv_data_factory.py

  trentmc

  committed
  e564f75

• refactor test_ohlcv_data_factory: cleanup mocks; remove redundant test; pq_data_factory -> factory

  trentmc

  committed
  1559ec3

• Fix #443: [YAML] yaml timescale is 5m, yet predictoor logs s_per_epoch=3600 (1h)

  trentmc

  committed
  f11d881

• Update feed str() to give full address; and order to be similar to predict_feeds_strs. Show all info used in filtering feeds.

  trentmc

  committed
  833ec16

• Small bug fix: not printing properly

  trentmc

  committed
  d45cd34

• Tweak: logging in predictoor_contract.py

  trentmc

  committed
  e0de941

• Tweak: logging in trueval_agent_single.py

  trentmc

  committed
  c3fe7b9

• Two bug fixes: pass in web3_pp not web3_config to PredictoorContract constructor

  trentmc

  committed
  20171b6

• enhance typechecking

  trentmc

  committed
  a3948cc

• tweak payout.py: make args passed more obvious

  trentmc

  committed
  54a2c96

• fix broken unit test

  trentmc

  committed
  626e773

• make black happy

  trentmc

  committed
  c67fd2d

• fix breaking unit test

  trentmc

  committed
  9644408

• Tweak predictoor_contract DX & UX

  trentmc

  committed
  3e045c0

• Improve trueval: Have fewer layers of try/except, better DX via docstrings and more, better UX via logs

  trentmc

  committed
  9bfb404

• Rename TruevalAgentBase -> BaseTruevalAgent

  trentmc

  committed
  9d620cf

Commits on Dec 13, 2023

• (a) Fix #445: merge 3 trueval agent files into 1. (b) Fix #448 contract.py::get_address() doesn't handle 'sapphire-testnet' etc #448. (c) test_contract.py doesn't test across all networks we use, a…

  Show description for 0137395
  trentmc

  committed
  0137395

• Fix #450: test_contract_main[barge-pytest] fails

  trentmc

  committed
  9aa6c49

Commits on Dec 14, 2023

• renaming pq_data_factory to ohlcv_data_factory

  idiom-bytes

  committed
  addf921

• Removing all TODOs

  idiom-bytes

  committed
  6874ead

• Fix #452: Add clean code guidelines README

  trentmc

  committed
  422b12e

• Merge branch 'yaml-cli2' of https://github.com/oceanprotocol/pdr-backend into yaml-cli2

  trentmc

  committed
  8bca663

• removing dangling _ppss() inside predictoor_agent_runner.py

  idiom-bytes

  committed
  7e8a87e

• Fixing linter

  idiom-bytes

  committed
  e4fadc6

Commits on Dec 15, 2023

• Fix #454: Refactor: Rename MEXCOrder -> MexcOrder, ERC721Factory

  trentmc

  committed
  9f33dc2

• Fix #455: Cyclic import issue

  trentmc

  committed
  67a8cde

• Fix #454 redux: the first commit introduced a bug, this one fixes the bug

  trentmc

  committed
  8fe215d

• Fix #436 - Implement GQL data factory (PR #438)

  Show description for d3029a7
  

  idiom-bytes

  and

  trentmc

  authored
  d3029a7

Commits on Dec 17, 2023

• Fix #350: [Sim] Tweaks to plot title

  trentmc

  committed
  d759284

• Merge branch 'main' into yaml-cli2

  trentmc

  committed
  f70aece

• make black happy

  trentmc

  committed
  3c0b3e3

• Fix #446: [YAML] Rename/move files & dirs for proper separation among lake, AI models, analytics (#458)

  Show description for 974799a
  trentmc

  authored
  974799a

Commits on Dec 18, 2023

• Fix #459: In CI, polars error: col timestamp_right already exists (#460)

  Show description for bbfef0d
  trentmc

  authored
  bbfef0d

• Fix #397: Remove need to specify 'stake_token' in ppss.yaml (#461)

  trentmc

  authored
  a4c117b

Commits on Dec 19, 2023

• Docs fixes (#456)

  calina-c

  authored
  aa999b7

Commits on Dec 20, 2023

• Make Feeds objects instead of tuples. (#464)

  Show description for 80e9fb6
  calina-c

  authored
  80e9fb6

• Move and rename utils (#467)

  Show description for b5496c6
  calina-c

  authored
  b5496c6

Commits on Dec 21, 2023

• Objectify pairstr. (#470)

  Show description for 1dec231
  calina-c

  authored
  1dec231

Commits on Dec 29, 2023

• Towards #462: Separate lake and aimodel SS, lake command (#473)

  Show description for bd01d71
  calina-c

  authored
  bd01d71

Commits on Jan 4, 2024

• [Lake] integrate pdr_subscriptions into GQL Data Factory (#469)

  Show description for a0244c9
  idiom-bytes

  authored
  a0244c9

• Improve DRY (#475)

  Show description for 80725f8
  calina-c

  authored
  80725f8

Commits on Jan 5, 2024

• Add Code climate. (#484)

  calina-c

  authored
  1fb88ac

• Adds manual trigger to pytest workflow.

  calina-c

  committed
  243ae7e

Commits on Jan 9, 2024

• issue483: move the logic from subgraph_slot.py (#489)

  kdetry

  authored
  3a5c639

• Add some test coverage (#488)

  Show description for 56392e4
  calina-c

  authored
  56392e4

Commits on Jan 10, 2024

• git merge main

  trentmc

  committed
  27f94d2

• Fix #501: ModuleNotFoundError: No module named 'flask' (PR #504)

  trentmc

  authored
  cd69814

• Fix #509: Refactor test_update_rawohlcv_files (PR #508)

  trentmc

  authored
  77d7b4c

• Merge branch 'main' into yaml-cli2

  trizin

  authored
  ff66cbf

• Fix #505: polars.exceptions.ComputeError: datatypes of join keys don't match (PR #510)

  Show description for 69d3c8e
  trentmc

  authored
  69d3c8e

Commits on Jan 11, 2024

• Fix #517: aimodel_data_factory.py missing data: binance:BTC/USDT:None (PR #518)

  Show description for 4959b5d
  trentmc

  authored
  4959b5d

• Towards #494: Improve coverage 2 (#498)

  Show description for 474ec9c
  calina-c

  authored
  474ec9c

Commits on Jan 12, 2024

• Fix #519: aimodel_data_factory.py missing data col: binance:ETH/USDT:close (#524)

  Show description for 3f24fdd
  trentmc

  authored
  3f24fdd

• Replace dftool with pdr (#522)

  Show description for e21f9a2
  trizin

  authored
  e21f9a2

Commits on Jan 13, 2024

• Fix #525: Plots pop up unwanted in tests. (PR #528)

  Show description for 413d065
  calina-c

  authored
  413d065

• Issue 519 feed dependencies (#529)

  Show description for fdb4545
  calina-c

  authored
  fdb4545

• Update to #519: remove do_verify, it's redundant (#532)

  trentmc

  authored
  63891cd

• Fix #507: fix asyncio issues (PR #531)

  Show description for 3eae232
  calina-c

  authored
  3eae232

Commits on Jan 15, 2024

• #413 - YAML thorough system level tests (#527)

  Show description for 35a2799
  trizin

  authored
  35a2799

• Adds incremental waiting for subgraph tries. (#534)

  calina-c

  authored
  11f05d2

• Add publisher feeds filtering. (#533)

  Show description for 9402e5f
  calina-c

  authored
  9402e5f

• Pass the ppss.web3_pp instead of web3_config into WrappedToken class (#537)

  trizin

  authored
  995a103

Commits on Jan 16, 2024

• Fix #542: Add code climate usage to developer flow READMEs

  trentmc

  committed
  9422c82

• #538 - check network main subgraph query fails (#539)

  Show description for d14c702
  trizin

  authored
  d14c702

• #540 - YAML CLI topup and check network actions require address file (#541)

  Show description for 11b7244
  trizin

  authored
  11b7244

• Remove predictoor2 ref from pytest

  trizin

  authored
  a851f79