Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(core-forger): memory leak, log spam, cleanup #2341

Merged
merged 27 commits into from
Mar 31, 2019
Merged

refactor(core-forger): memory leak, log spam, cleanup #2341

merged 27 commits into from
Mar 31, 2019

Conversation

spkjp
Copy link
Contributor

@spkjp spkjp commented Mar 30, 2019

Proposed changes

Refactors the forger and addresses the following points:

  • during startup, don't block when no host is responsive
  • host selection now tries all configured hosts before throwing an error
  • host selection no longer uses recursion
  • no longer spams the log each possible event cycle when no host is available
  • uses a timeout to schedule the next __monitor() call instead of promise recursion
  • removes obsolete code

Resolves #2326

The memory leak was caused by two things:
(nodejs/node#6673)

  1. promise recursion in __monitor()
  2. promise recursion in __chooseHost()

The former is the lesser evil and only creeps up slowly over time, but once the forger relay becomes unresponsive the latter makes the memory leak really fast.

(It may be noted that the memory leak as described in #2326 seems to be almost non-present with Node.js 11 in the case of Core despite the provided script leaking in both versions)

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Refactoring (improve a current implementation without adding a new feature or fixing a bug)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Build (changes that affect the build system)
  • Docs (documentation only changes)
  • Test (adding missing tests or fixing existing tests)
  • Other... Please describe:

Checklist

  • I have read the CONTRIBUTING documentation
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

@codecov-io
Copy link

codecov-io commented Mar 30, 2019

Codecov Report

Merging #2341 into develop will increase coverage by 0.12%.
The diff coverage is 64.38%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2341      +/-   ##
===========================================
+ Coverage    66.01%   66.14%   +0.12%     
===========================================
  Files          397      398       +1     
  Lines         8578     8562      -16     
  Branches       414      412       -2     
===========================================
  Hits          5663     5663              
+ Misses        2870     2855      -15     
+ Partials        45       44       -1
Impacted Files Coverage Δ
packages/core-utils/src/httpie.ts 0% <ø> (ø) ⬆️
.../core-database-postgres/src/repositories/rounds.ts 75% <ø> (ø) ⬆️
...p2p/src/server/versions/internal/handlers/utils.ts 0% <ø> (ø) ⬆️
...es/core-api/src/versions/1/delegates/controller.ts 42.85% <0%> (+1.19%) ⬆️
packages/core-database/src/wallet-manager.ts 74.17% <100%> (ø) ⬆️
packages/core-database/src/database-service.ts 31.57% <33.33%> (-0.16%) ⬇️
packages/core-forger/src/manager.ts 69.41% <46.87%> (-0.59%) ⬇️
packages/core-forger/src/client.ts 86.84% <84.61%> (+11.33%) ⬆️
packages/core-forger/src/errors.ts 88.88% <88.88%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ee0398b...2d1e6ef. Read the comment docs.

packages/core-forger/src/manager.ts Outdated Show resolved Hide resolved
packages/core-forger/src/manager.ts Outdated Show resolved Hide resolved
packages/core-forger/src/client.ts Outdated Show resolved Hide resolved
packages/core-forger/src/client.ts Outdated Show resolved Hide resolved
packages/core-forger/src/client.ts Outdated Show resolved Hide resolved
packages/core-forger/src/client.ts Outdated Show resolved Hide resolved
@ArkEcosystem ArkEcosystem deleted a comment Mar 30, 2019
@alessiodf
Copy link
Contributor

alessiodf commented Mar 30, 2019

@supaiku0

(It may be noted that the memory leak as described in #2326 seems to be almost non-present with Node.js 11 despite the provided script saying otherwise)

I suggest you take a snapshot with heapdump and check it because promise recursion leaks are definitely still present in Node.js v11 and the script is accurate. Even if it doesn't leak as much, it's still misleading to say it's (almost) non-present and may give others looking at this PR the wrong impression that promise recursion is no longer a problem in Node.js v11, because it is:

Screenshot 2019-03-30 at 19 01 11

Screenshot 2019-03-30 at 19 01 15

@spkjp
Copy link
Contributor Author

spkjp commented Mar 30, 2019

I suggest you take a snapshot with heapdump and check it because promise recursion leaks are definitely still present in Node.js v11 and the script is accurate.

Yeah, you are right. The promise leak is still present in Node 11. But in the case of Core I can only reproduce the leak on Node 10. My wording was a bit misleading.

@ghost
Copy link

ghost commented Mar 31, 2019

The ci/circleci: test-node11-integration-1 job is failing as of 03b9b5dd5fba0861c6ecc4e0b789a130c1174125. Please review the logs for more information.

Once you've pushed the fixes, the build will automatically re-run. Thanks!

@faustbrian faustbrian merged commit 327b98c into ArkEcosystem:develop Mar 31, 2019
@kristjank
Copy link
Contributor

Referencing this: #2326

Copy link
Contributor

@kristjank kristjank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

remus-persona pushed a commit to PersonaIam/core that referenced this pull request Apr 25, 2019
* refactor: move core-api repositories into core-database-* (ArkEcosystem#2107)

* feat: implement AIP29 (ArkEcosystem#2122)

* refactor(core-webhooks): replace sqlite3 with lowdb (ArkEcosystem#2124)

* fix(core-elasticsearch): build failure (ArkEcosystem#2127)

* test(core-database): add tests to blocks & transactions business repositories (ArkEcosystem#2147)

* perf(crypto): integrate bcrypto (ArkEcosystem#2158)

* feat(core-logger-pino): initial implementation (ArkEcosystem#2134)

* feat(core-api): search delegates by usernames (ArkEcosystem#2143)

* feat(crypto): increase vendor field length to 255 bytes (ArkEcosystem#2159)

* refactor: replace micromatch with nanomatch and remove heavy deps (ArkEcosystem#2165)

* refactor: replace micromatch with the more lightweight nanomatch

* fix(core-http-utils): remove non-existent deps

* fix(core-vote-report): add vision dependency

* Update create.ts

* refactor(crypto): benchmarks (ArkEcosystem#2167)

* chore: move core-graphql to the deprecated folder (ArkEcosystem#2169)

* test: initial restructure & split of unit and integration tests (ArkEcosystem#2172)

* refactor: replace dayjs-ext with dato (ArkEcosystem#2168)

* refactor: peg block timestamp to slot (ArkEcosystem#2176)

* chore: circleci update coverage directory (ArkEcosystem#2177)

* chore: remove extraneous jest presets and config (ArkEcosystem#2182)

* refactor(core-tester-cli): rewrite from scratch (ArkEcosystem#2133)

* refactor(crypto): de/serialization (ArkEcosystem#2175)

* refactor: move transaction logic out of crypto (ArkEcosystem#2188)

* feat(core-tester-cli): test the vendor field (ArkEcosystem#2189)

* feat(core-tester-cli): add the ability to send multiple waves (ArkEcosystem#2191)

* feat(core): add support for reinstalling the current version (ArkEcosystem#2192)

* feat(core): add support for forced updates (ArkEcosystem#2190)

* style: apply prettier formatting

* test(core-container): remove old test

* feat(core-api): more params for delegates search endpoint (ArkEcosystem#2184)

* fix(core-tester-cli): set crypto network for debug commands (ArkEcosystem#2204)

* feat(core-api): add active delegates endpoint (ArkEcosystem#2205)

* refactor: move transaction type specific logic into core-transactions (ArkEcosystem#2201)

* fix: vote balance update (develop) (ArkEcosystem#2211)

* refactor(core-blockchain): remove old fast rebuild code (ArkEcosystem#2210)

* refactor(core-blockchain): remove old fast rebuild cold

* refactor: remove dead rebuild code

* style: apply prettier formatting

* refactor(core-blockchain): remove queue wrapper

* test(core-blockchain): remove queue wrapper tests

* fix(core-blockchain): use async queue methods

* fix(core-blockchain): use async queue methods

* test(core-blockchain): remove dead test

* fix(core-blockchain): add method that disappeared

* fix(core-blockchain): add drain call

* refactor: drop more unused code

* fix: dispatch missing event after block download finished

* refactor: remove obsolete config workaround

* refactor: drop even more obsolete rebuild code

* refactor: cleanup leftovers

* revert "refactor: remove obsolete config workaround"

* refactor: purely rely on in-memory wallets based on transactions (ArkEcosystem#2209)

* refactor: purely rely on in-memory wallets based on transactions

* refactor: purely rely on in-memory wallets based on transactions

* fix(core-database-postgres): compare vote balance objects

* test: use wallet manager instead of database

* refactor(core-database): use no longer used properties

* test(core-blockchain): expect call without arguments

* fix: await integrity verifier results

* test(core-blockchain): do not pass any arguments to buildWallets

* test(core-database): remove shutdown assertion

* test(core-api): wallet manager reference

* refactor(core-database-postgres): remove extraneous object.values call

* style(core-database-postgres): naming

* Update integrity-verifier.ts

* refactor(core-database-postgres): make __ methods private

* Update integrity-verifier.ts

* refactor(core-database-postgres): remove missed blocks from integrity verifier

* test: remove integration setup & tests from unit tests (ArkEcosystem#2194)

* fix(core-tester-cli): Fix the description of debug:serialize

The string provided as an input is supposed to be JSON that is to be
serialized.

* feat(core): add restart flags to update command (ArkEcosystem#2218)

* chore: circleci restore caching + re-organize jobs for unit / integration tests (ArkEcosystem#2222)

* chore: bump versions

* chore: yarn.lock

* fix: resolve core-tester-utils conflicts and various errors

* chore: use yarn setup on CircleCI

* chore: update CircleCI config

* fix(core-tester-cli): Don't hide errors from HTTP failures (ArkEcosystem#2223)

Before this patch we would just get the following error, leaving one to
wonder what went wrong:

  [1552298227459] ERROR (core-tester-cli/40747 on smle): Cannot destructure property `data` of 'undefined' or 'null'.

After this patch:

  [1552299329734] ERROR (core-tester-cli/3408 on smle): http://localhost:4003/api/v2/node/configuration: connect ECONNREFUSED 127.0.0.1:4003
  [1552299329735] ERROR (core-tester-cli/3408 on smle): Cannot destructure property `data` of 'undefined' or 'null'.

* chore: update dependencies and remove unused imports (ArkEcosystem#2212)

* refactor: replace axios with got (ArkEcosystem#2203)

* feat(core-tester-cli): add make:block command (ArkEcosystem#2221)

* fix(core-api): properly sort semver versions (ArkEcosystem#2229)

* refactor(core-forger): always use latest available height (ArkEcosystem#2231)

* fix: keep old vendor field padding (ArkEcosystem#2233)

* fix: properly cast Buffer to hex string (ArkEcosystem#2232)

* feat(crypto): Switch block id to full SHA256 (ArkEcosystem#2156)

After some predetermined block height, blocks' ids would be generated as
full SHA256, encoded as a hex string.

Both Block.getIdHex() and Block.getId() return the full SHA256 hex
string after the predetermined height. It does not make any sense to
represent 256-bit numbers in decimal

* fix(crypto): Only use toBytesHex() for old block ids

Convert Block's idHex from id using toBytesHex() only if dealing with
old block ids (before the switch to full sha256 block id).

* test(core-forger): Extend tests with a block with full sha256 id (ArkEcosystem#2235)

The new fixture was generated using:

  yarn --cwd packages/core-tester-cli tester make:block --log --network=testnet --previousBlock='{ "height": 1000000000, "id": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "idHex": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" }' --transactions=0

* refactor: move wallet class out of crypto (ArkEcosystem#2237)

* feat(core-database-postgres): transaction asset column (ArkEcosystem#2236)

* docs: less noise in the GitHub PR template (ArkEcosystem#2238)

* chore(core): Disable webhooks on testnet (ArkEcosystem#2239)

Webhooks are disabled on devnet and mainnet by default. Do the same with
testnet for consistency and because webhooks are broken and prevent
testnet from functioning:

$ yarn --cwd packages/core full:testnet
...
(node:44847) UnhandledPromiseRejectionWarning: TypeError: webhooks is not iterable
    at WebhookManager.getMatchingWebhooks (core/packages/core-webhooks/dist/manager.js:47:31)

* chore: add publish scripts for npm (ArkEcosystem#2240)

* chore: remove unused typedoc dependency (ArkEcosystem#2241)

* chore: setup linting for tests (ArkEcosystem#2242)

* chore: add update script for dependencies (ArkEcosystem#2244)

* chore: remove pretest and lint scripts from packages (ArkEcosystem#2243)

* fix(tests): Reference the home directory via HOME, not ~ (ArkEcosystem#2246)

"~" is expanded by the shell, if we use it in JavaScript it will end up
creating directories like "./packages/core-p2p/~/.core/"

Use HOME from the environment instead. If it is not set, then trying
"/.core/..." is fine.

* refactor: custom transaction types (ArkEcosystem#2247)

* fix(core-database): return array instead of joined string from applyOrder (ArkEcosystem#2245)

* chore: add publish script for the next channel (ArkEcosystem#2249)

* feat(core): support for the next channel (ArkEcosystem#2250)

* fix(core-webhooks): return an empty array if no webhooks exist for a given event (ArkEcosystem#2252)

* test(core-webhooks): unit test the database (ArkEcosystem#2253)

* docs: initial 2.3.0 changelog (ArkEcosystem#2255)

* chore: remove packages from codecov ignore list (ArkEcosystem#2256)

* refactor: remove missed blocks and productivity (ArkEcosystem#2257)

* fix(core-utils): handle NSect edge case (ArkEcosystem#2259)

If the search range is narrowed too much then the condition
"if (low + this.nAry >= high) {" would catch it and return. But it could
happen that we still have neighboring elements and that condition is not
true.

Check whether the probe declared that the highest match is at some
height and if we have probed the next height - the fact that the next
height was not declared a match, implicitly means that it is not a
match. And if we have two neighboring heights, the lower one a match and
the higher one not, then we don't need to search further and have the
final result.

* fix(core-blockchain): handle unhandled event (ArkEcosystem#2260)

* feat(core-logger-pino): set different log levels for console and file (ArkEcosystem#2261)

* fix: remove voter balances endpoint

* docs: fix broken references to plugins docs (ArkEcosystem#2264)

* chore: bump to 2.2.2

* chore: changelog

* chore: update readme files with all contributors and fix banners (ArkEcosystem#2266)

* chore: add missing oclif dependency (ArkEcosystem#2267)

* fix(core-snapshots): remove wallets, add asset handling (ArkEcosystem#2269)

* fix(core-logger-pino): crash on rotate, use compression (ArkEcosystem#2270)

* fix(core-p2p): pass all opts to httpie instead of just headers (ArkEcosystem#2271)

* chore: add missing dependencies (ArkEcosystem#2272)

* build(docker): remove core source and add channel support (ArkEcosystem#2274)

* build(docker): clean the yarn cache after the installation (ArkEcosystem#2275)

* misc(crypto): Update devnet blockid milestone (ArkEcosystem#2277)

Height 1895000 at Mon Mar 25 16:56:22 CET 2019

* Merge branch 'master' into develop (ArkEcosystem#2278)

* fix(core-logger-pino): prettify filestream (ArkEcosystem#2280)

* fix(core-database-postgres): asset migration on mainnet (ArkEcosystem#2281)

* chore: import lodash functions from function packages instead of the full lodash (ArkEcosystem#2285)

* release: 2.3.0-next.0 (ArkEcosystem#2286)

* feat(core-error-tracker-rollbar): initial implementation (ArkEcosystem#2287)

* feat(core-error-tracker-raygun): initial implementation (ArkEcosystem#2288)

* feat(core-error-tracker-airbrake): initial implementation (ArkEcosystem#2289)

* chore: move winston to maintained packages (ArkEcosystem#2291)

* chore: remove extraneous .gitattributes entries (ArkEcosystem#2292)

* refactor: drop the git commit hash identifier (ArkEcosystem#2293)

* chore(core-p2p): raise the minimum version to 2.3.0 (ArkEcosystem#2294)

* fix(core-database-postgres): support PG9.5 for asset column migration (ArkEcosystem#2295)

* release: 2.3.0-next.1 (ArkEcosystem#2296)

* fix(core-database): sort transactions by timestamp:desc by default (ArkEcosystem#2298)

* fix(core-snapshots): always load core-database-postgres (ArkEcosystem#2302)

* feat(core-snapshots): rollback by a given amount of blocks (ArkEcosystem#2300)

* refactor(core-snapshots): make core the only codec for reliability (ArkEcosystem#2301)

* refactor: replace any with proper types (ArkEcosystem#2276)

* fix(core-snapshots): serialize correct timestamp, dont corrupt buffers (ArkEcosystem#2304)

* fix(core-database): write block timestamp as transaction timestamp (ArkEcosystem#2305)

* fix(core-api): adjust schema validation of blocks show (ArkEcosystem#2306)

* refactor: use kebab case for aliases (ArkEcosystem#2283)

* feat(core-new-relic): initial implementation (ArkEcosystem#2290)

* perf(crypto): skip transaction verification overhead (ArkEcosystem#2307)

* fix(core): read the correct property for the new CLI version (ArkEcosystem#2309)

* fix(core-database-postgres): add EQ operator to block height (ArkEcosystem#2313)

* build(docker): create symbolic link for ark bin (ArkEcosystem#2312)

* fix(crypto): add signSignature to schema (ArkEcosystem#2315)

* fix(core-p2p): support common blocks with sha256 ids (ArkEcosystem#2317)

* fix(core-p2p): always refresh peer status (ArkEcosystem#2318)

* fix(core-database): map block to transaction in findById (ArkEcosystem#2316)

* feat(core-database-postgres): support IN operator for block id, height and generator (ArkEcosystem#2319)

* refactor(core-p2p): remove the remote interface (ArkEcosystem#2311)

* refactor(core): use nsfw for log watching instead of node-tail (ArkEcosystem#2310)

* refactor(core-snapshots): remove dead code (ArkEcosystem#2308)

* release: 2.3.0-next.2 (ArkEcosystem#2320)

* fix(crypto): check if block contains exceptions (ArkEcosystem#2323)

* fix(core-database-postgres): shutdown when integrity check fails (ArkEcosystem#2324)

* fix(core): preserve existing logs and attach the last line when the log is modified (ArkEcosystem#2322)

* perf(core-api): transaction transformer (ArkEcosystem#2327)

* fix(core-p2p): two-way ban (ArkEcosystem#2325)

* feat: add more block and transaction pool events (ArkEcosystem#2321)

* fix: accept already accepted peers (ArkEcosystem#2328)

* fix(core-database-postgres): run the asset migration before starting the record migration (ArkEcosystem#2332)

* chore: install node.js 11 now that it is stable (ArkEcosystem#2330)

* fix: use the maxTransactionsPerRequest configuration for API validation (ArkEcosystem#2331)

* fix(core-event-emitter): ensure the event listener won’t exceed the max listener count (ArkEcosystem#2329)

* fix(core-p2p): sort heights in getNetworkHeight as numbers (ArkEcosystem#2334)

* release: 2.3.0-next.3 (ArkEcosystem#2336)

* feat(core-database): add function to search a block by id or height (ArkEcosystem#2337)

* refactor(core-database-postgres): add findByHeight to avoid confusion with the misleading findByHeights (ArkEcosystem#2338)

* test(functional): broadcast and forge all transaction types (ArkEcosystem#2333)

* refactor(core-database): use a raw query to get a block by height (ArkEcosystem#2339)

* chore(ci): explicitly name integration test jobs (ArkEcosystem#2340)

* fix(core-database): search a block ID if there is no height match (ArkEcosystem#2342)

* feat(core-logger-signale): initial implementation (ArkEcosystem#2343)

* chore(ci): remove the postgres setup from unit test jobs (ArkEcosystem#2345)

* refactor(core-database): clearly separate manager and factory behaviour (ArkEcosystem#2346)

* refactor(core-forger): memory leak, log spam, cleanup (ArkEcosystem#2341)

* refactor: replace various any with types and simpler interface naming (ArkEcosystem#2348)

* refactor(core-transaction-pool): clearly separate manager and factory behaviour (ArkEcosystem#2347)

* refactor(core-event-emitter): expose all application events through an enum (ArkEcosystem#2349)

* refactor: reduce duplication in all logger packages (ArkEcosystem#2344)

* refactor: replace various any with types, property visibility and interface naming (ArkEcosystem#2352)

* fix(core-blockchain): initialize active delegates on startup (ArkEcosystem#2353)

* release: 2.3.0-next.4 (ArkEcosystem#2354)

* test: add TransactionFactory and RestClient test helpers (ArkEcosystem#2355)

* fix(crypto): use calculated id for exception check (ArkEcosystem#2356)

* refactor(core-api): remove active delegates endpoint (ArkEcosystem#2357)

* refactor(core-database): round logic (ArkEcosystem#2358)

* fix(core-p2p): frequent peer timeout errors (ArkEcosystem#2363)

* fix(core-p2p): remove ping timeout ban (ArkEcosystem#2366)

* release: 2.3.0-next.5 (ArkEcosystem#2367)

* docs: 2.3.0 changelog (ArkEcosystem#2372)

* docs: mention webhook recreation in changelog for 2.3.0

* fix: cope with dynamic round sizes (ArkEcosystem#2370)

* fix(core-logger-pino): initial rotation (ArkEcosystem#2380)

* fix(core-tester-cli): init config manager from network preset (ArkEcosystem#2384)

* refactor(core-container): pass suffix to container (ArkEcosystem#2387)

* feat(core-api): return slip44 and wif via node/configuration (ArkEcosystem#2388)

* fix(core): pass cli flags correctly (ArkEcosystem#2389)

* chore(crypto): bump vendorFieldLength and blockid milestone heights (ArkEcosystem#2395)

Due to 2.3 release re-schedule:

Height 8128000 at Thu Apr 25 12:05:22 UTC 2019 (assuming no block misses)
Height 8128000 at Thu Apr 25 12:37:13 UTC 2019 (assuming block misses as in the last 30000 blocks)

Height 8204000 at Thu May  2 12:58:42 UTC 2019 (assuming no block misses)
Height 8204000 at Thu May  2 13:45:25 UTC 2019 (assuming block misses as in the last 30000 blocks)

* feat(core-event-emitter): add off method (ArkEcosystem#2396)

* chore: use ARK as prefix for everything (ArkEcosystem#2397)

* fix(crypto): validate address is on network (ArkEcosystem#2394)

* fix(core-database): handle unknown transaction search (ArkEcosystem#2404)

* release: 2.3.0-next.6 (ArkEcosystem#2406)

* fix: prevent wallet managers from indexing/creating ghost wallets (ArkEcosystem#2405)

* fix(core-database): wrong delegate order after start up (ArkEcosystem#2431)

* docs: 2.3.0 changelog (ArkEcosystem#2435)

* docs: 2.3.0 changelog

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* release: 2.3.0 (ArkEcosystem#2452)

* fix(core-transaction-pool): refuse transactions from senders with pending second signature registrations (ArkEcosystem#2458)

* fix(crypto): deserialize type > 0 with vendor field instead of skipping it (ArkEcosystem#2459)

* release: 2.3.1 (ArkEcosystem#2460)

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* Persona Core

* fix(core-snapshots): use correct genesis block instead of hardcoding devnet (ArkEcosystem#2462)

* fix(core): don't pass suffix flag to bip38 and bip39 command (ArkEcosystem#2464)

* release: 2.3.12 (ArkEcosystem#2465)

* chore: fix absolute banner urls (ArkEcosystem#2469)

* docs(template): add release section to pull request template (ArkEcosystem#2472)

* Persona Core

* Persona Core

* fix(crypto): add exceptions for transactions with invalid recipients (ArkEcosystem#2471)

* refactor(core): remove support for old release channels (ArkEcosystem#2476)

* release: 2.3.14 (ArkEcosystem#2477)

* release: 2.3.15 (ArkEcosystem#2478)
@remus-persona remus-persona mentioned this pull request Apr 25, 2019
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants