Merge: Adds Protocol 20/Soroban support to the XDR and API. (#538)

* Correctly set `minAccountSequence` in `TransactionBuilder` for large values (#539)
* Add support of CAP40 ed25519SignedPayload signer for SetOptionOp (#542)
* Add TypeScript interfaces for Ed25519SignedPayload signers
* Coalesce all xLM, Xlm, etc. => 'XLM' native asset code (#546)
* make sure sodium is not an empty object in service workers (#567)
* Add futurenet passphrase for soroban. (#550)
* Include standalone network passphrase (#555)
* Soroban auth next updates (#570)
* Add strkey parsing for contracts, and Address helper for building ScAddresses (#572)
* Update xdr to include HashIdPreimageContractAuth.nonce (#577)
* Add Address.toBuffer method (#578)
* Soroban value overhaul (#582)
* Update generated XDR to latest version. (#587)
* Update XDR and contract invocations to conform to the latest version (Preview 9). (#601)
* added soroban transaction data as tx builder param (#604)
* Add Contract support for strkey-style contract IDs (#612)
* Add Contract.address() method (#614)
* Add wrappers to easily deal with the myriad of `ScVal` integer types. (#620)
* Upgrade `js-xdr` to v3.0.0 to support `bigint` encoding.
* Add wrappers to easily convert between `ScVal` and native types. (#630)
* Update XDR to support Preview 10 (#633)
* Fix TypeScript definition for `invokeHostFunction` options (#635)
* Add fully-qualified `Operation` type to XDR type definitions. (#591)
* Adds Operation.isValidAmount jsdoc (#609)
* Removes the dependency on `crc` by calculating checksum in strkey (#621)
* Improve type flexibility when building `ScVal`s from native types (#638)
* Fix Preview 10 contract invoke & release `v10.0.0-soroban.3` (#642)
* Add GitHub Action that compares before-and-after bundle sizes. (#644)
* Make opinionated judgements about string/symbol decoding (#645)
* Make Node 16 the minimum version. (#651)
* Add standalone/futurenet passphrases to jsdoc (#654)
* Clean up unnecessary dependency entries (#652)
* Add `SorobanDataBuilder` builder pattern to prepare transactions easier (#660)
* Adds a way to create a `TransactionBuilder` from an existing transaction. (#656)
* Add basic contract event parsing into a human-readable structure (#659)
* Drop support for deprecated hex contract IDs (#658)
* Add helpers method to build authorization entries. (#663)
* Fix TypeScript definition for new static `TransactionBuilder` constructor (#665)
* Add ability to clear out operations from the transaction builder (#670)
* Add an existing method to XdrLargeInt, jsdoc/types fixup (#668)
* Expand contract footprint with contract code ledger key. (#662)
* Decode base64-encoded strings in new SorobanDataBuilder (#673)
* Add ability to modify and retrieve the resource footprint (#680)
* Fixes error messages for required amount parameters (#679)
* Add `Asset.contractId()` helper to predict contract IDs (#684)
* Upgrade dependencies to their latest minor/patch version (#685)
* Improves `SorobanDataBuilder` construction and adds a getter (#686)
* Add JSON-ification of Soroban invocation trees. (#669)
* Add utilities for formatting token amounts (`formatTokenAmount` & `parseTokenAmount`) (#667)
* Merges the final Protocol 20 XDR (i.e. for testnet) into `soroban` (#688)
* Add support to convert strings to large integer and address `ScVal`s (#690)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Paul Bellamy <paul@stellar.org>
Co-authored-by: George <Shaptic@users.noreply.github.com>
Co-authored-by: Jun Luo <4catcode@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: OrbitLens <33724849+orbitlens@users.noreply.github.com>
Co-authored-by: Piyal Basu <pbasu235@gmail.com>
Co-authored-by: George Kudrayvtsev <george@stellar.org>
Co-authored-by: shawn <sreuland@users.noreply.github.com>
Co-authored-by: Silence <35656692+silence48@users.noreply.github.com>
Co-authored-by: Chad Ostrowski <221614+chadoh@users.noreply.github.com>
Co-authored-by: jeesun 지선 <jeesunikim@users.noreply.github.com>

.github/workflows/bundle_size.yml

@@ -0,0 +1,30 @@
+name: Bundle Size
+
+on:
+  pull_request:
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+
+      - name: Install Node.js 16
+        uses: actions/setup-node@v3
+        with:
+          node-version: 16
+
+      # Workaround for some `yarn` nonsense, see:
+      # https://github.com/yarnpkg/yarn/issues/6312#issuecomment-429685210
+      - name: Install Dependencies
+        run: yarn install --network-concurrency 1
+
+      - name: Build All
+        run: yarn build:prod
+
+      - name: Report Bundle Size
+        uses: preactjs/compressed-size-action@v2
+        with:
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          pattern: "dist/*.js"
+          compression: "none"

.github/workflows/gh_pages.yml

@@ -12,10 +12,10 @@ jobs:
         with:
           fetch-depth: 0
 
-      - name: Install Node (14.x)
+      - name: Install Node (16.x)
         uses: actions/setup-node@v3
         with:
-          node-version: '14.x'
+          node-version: 16
 
       - name: Install Dependencies
         run: yarn install

.github/workflows/npm_publish.yml

@@ -13,7 +13,7 @@ jobs:
       - name: Install Node
         uses: actions/setup-node@v3
         with:
-          node-version: '14.x'
+          node-version: 16
           registry-url: 'https://registry.npmjs.org'
 
       - name: Install Depencencies
@@ -22,7 +22,7 @@ jobs:
       - name: Test & Build
         run: yarn preversion
 
-      - name: Publish npm package
-        run: yarn publish
+      - name: Publish release npm package
+        run: yarn publish --tag beta
         env:
           NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.github/workflows/tests.yml

@@ -13,7 +13,7 @@ jobs:
       fail-fast: false
       max-parallel: 4
       matrix:
-        node-version: [14, 16, 18]
+        node-version: [16, 18, 20]
 
     steps:
       - name: Checkout
@@ -29,9 +29,6 @@ jobs:
       - name: Install Dependencies
         run: yarn install --network-concurrency 1
 
-      - name: Run Linter
-        run: yarn lint
-
       - name: Build All
         run: yarn build:prod
 
@@ -40,3 +37,6 @@ jobs:
 
       - name: Run Browser Tests
         run: yarn test:browser
+
+      - name: Run Linter Checks
+        run: yarn fmt && yarn lint && (git diff-index --quiet HEAD; git diff)

.gitignore

@@ -4,6 +4,7 @@
 /tmp/
 /dist/
 /config/dist/
+/lib/
 /coverage/
 /jsdoc/
 .DS_Store

CHANGELOG.md

@@ -4,6 +4,39 @@
 ## Unreleased
 
 
+## [`v10.0.0-beta.0`](https://github.com/stellar/js-stellar-base/compare/v9.0.0...v10.0.0-beta.0): Protocol 20
+
+### Breaking Changes
+ * **Node 16 is the new minimum version** to use the SDKs.
+ * The XDR has been massively overhauled to support [Soroban in Protocol 20](https://soroban.stellar.org/docs/category/fundamentals-and-concepts), which means new operations, data structures, and a transaction format as well as new overlay features ([#538](https://github.com/stellar/js-stellar-base/pull/538)).
+
+The core data structure of Soroban is a generic type called an `ScVal` (**s**mart **c**ontract **val**ue, which is a union of types that can basically represent anything [numbers, strings, arrays, maps, contract bytecode, etc.]). You can refer to the XDR for details, and you can utilize new APIs to make dealing with these complex values easier:
+ - `nativeToScVal` helps convert native types to their closest Soroban equivalent
+ - `scValToNative` helps find the closest native JavaScript type(s) corresponding to a smart contract value
+ - `scValToBigInt` helps convert numeric `ScVal`s into native `bigint`s
+ - `ScInt` and `XdrLargeInt` help convert to and from `bigint`s to other types and form sized integer types for smart contract usage
+
+### Added
+The following are new APIs to deal with new Soroban constructs:
+ - **`Address`, which helps manage "smart" addresses in the Soroban context.** Addresses there (used for auth and identity purposes) can either be contracts (strkey `C...`) or accounts (strkey `G...`). This abstraction helps manage them and distinguish between them easily.
+ - **`Contract`, which helps manage contract identifiers.** The primary purpose is to build invocations of its methods via the generic `call(...)`, but it also provides utilities for converting to an `Address` or calculating its minimum footprint for state expiration.
+ - **Three new operations** have been added related to Soroban transactions:
+   * `invokeHostFunction` for calling contract code
+   * `bumpFootprintExpiration` for extending the state lifetime of Soroban data
+   * `restoreFootprint` for restoring expired, off-chain state back onto the ledger
+ - The `TransactionBuilder` now takes a `sorobanData` parameter (and has a corresponding `.setSorobanData()` builder method) which primarily describes the storage footprint of a Soroban (that is, which parts of the ledger state [in the form of `xdr.LedgerKey`s] it plans to read and write as part of the transaction).
+   * To facilitate building this out, there's a new `SorobanDataBuilder` factory to set fields individually
+ - The `TransactionBuilder` now has a `cloneFrom(tx, opts)` constructor method to create an instance from an existing transaction, also allowing parameter overrides via `opts`.
+ - The following are convenience methods for building out certain types of smart contract-related structures:
+   * `buildInvocationTree` and `walkInvocationTree` are both ways to visualize invocation calling trees better
+   * `authorizeInvocation` helps multiple parties sign invocation calling trees
+   * `humanizeEvents` helps make diagnostic events more readable
+ - We've added a GHA to track bundle size changes as PRs are made. This protocol upgrade adds +18% to the final, minified bundle size which is significant but acceptable given the size of the upgrade.
+
+### Fixes
+* Improves the error messages when passing invalid amounts to deposit and withdraw operations ([#679](https://github.com/stellar/js-stellar-base/pull/679)).
+
+
 ## [v9.0.0](https://github.com/stellar/js-stellar-base/compare/v8.2.2..v9.0.0)
 
 This is a large update and the following changelog incorporates ALL changes across the `beta.N` versions of this upgrade.
@@ -15,7 +48,6 @@ The browser bundle size has decreased **significantly**:
   * `stellar-base.min.js` is **340 KiB**, down from **1.2 MiB** previously.
   * the new, unminified `stellar-base.js` is **895 KiB**.
 
-
 ### Breaking Changes
 
 - The build system has been completely overhauled to support Webpack 5 ([#584](https://github.com/stellar/js-stellar-base/pull/584), [#585](https://github.com/stellar/js-stellar-base/pull/585)).

Makefile

@@ -1,15 +1,20 @@
-XDR_BASE_URL_CURR=https://github.com/stellar/stellar-xdr/raw/curr
+XDR_BASE_URL_CURR=https://github.com/stellar/stellar-xdr/raw/9ac02641139e6717924fdad716f6e958d0168491
 XDR_BASE_LOCAL_CURR=xdr/curr
 XDR_FILES_CURR= \
 	Stellar-SCP.x \
 	Stellar-ledger-entries.x \
 	Stellar-ledger.x \
 	Stellar-overlay.x \
 	Stellar-transaction.x \
-	Stellar-types.x
+	Stellar-types.x \
+	Stellar-contract.x \
+	Stellar-contract-env-meta.x \
+	Stellar-contract-meta.x \
+	Stellar-contract-spec.x \
+	Stellar-contract-config-setting.x
 XDR_FILES_LOCAL_CURR=$(addprefix xdr/curr/,$(XDR_FILES_CURR))
 
-XDR_BASE_URL_NEXT=https://github.com/stellar/stellar-xdr/raw/next
+XDR_BASE_URL_NEXT=https://github.com/stellar/stellar-xdr/raw/440dc9512b6e72cf84965641c5eb495d6043ed73
 XDR_BASE_LOCAL_NEXT=xdr/next
 XDR_FILES_NEXT= \
 	Stellar-SCP.x \
@@ -20,7 +25,9 @@ XDR_FILES_NEXT= \
 	Stellar-types.x \
 	Stellar-contract.x \
 	Stellar-contract-env-meta.x \
-	Stellar-contract-spec.x
+	Stellar-contract-meta.x \
+	Stellar-contract-spec.x \
+	Stellar-contract-config-setting.x
 XDR_FILES_LOCAL_NEXT=$(addprefix xdr/next/,$(XDR_FILES_NEXT))
 
 XDRGEN_COMMIT=master
@@ -33,17 +40,17 @@ generate: src/generated/curr_generated.js types/curr.d.ts src/generated/next_gen
 src/generated/curr_generated.js: $(XDR_FILES_LOCAL_CURR)
 	mkdir -p $(dir $@)
 	> $@
-	docker run -it --rm -v $$PWD:/wd -w /wd ruby /bin/bash -c '\
-		gem install specific_install -v 0.3.7 && \
+	docker run -it --rm -v $$PWD:/wd -w /wd ruby:3.1 /bin/bash -c '\
+		gem install specific_install -v 0.3.8 && \
 		gem specific_install https://github.com/stellar/xdrgen.git -b $(XDRGEN_COMMIT) && \
 		xdrgen --language javascript --namespace curr --output src/generated $^ \
 		'
 
 src/generated/next_generated.js: $(XDR_FILES_LOCAL_NEXT)
 	mkdir -p $(dir $@)
 	> $@
-	docker run -it --rm -v $$PWD:/wd -w /wd ruby /bin/bash -c '\
-		gem install specific_install -v 0.3.7 && \
+	docker run -it --rm -v $$PWD:/wd -w /wd ruby:3.1 /bin/bash -c '\
+		gem install specific_install -v 0.3.8 && \
 		gem specific_install https://github.com/stellar/xdrgen.git -b $(XDRGEN_COMMIT) && \
 		xdrgen --language javascript --namespace next --output src/generated $^ \
 		'
@@ -53,7 +60,7 @@ types/curr.d.ts: src/generated/curr_generated.js
 		apk add --update git && \
 		git clone --depth 1 https://github.com/stellar/dts-xdr -b $(DTSXDR_COMMIT) --single-branch && \
 		cd /dts-xdr && \
-		yarn install && \
+		yarn install --network-concurrency 1 && \
 		OUT=/wd/$@ npx jscodeshift -t src/transform.js /wd/$< && \
 		cd /wd && \
 		yarn run prettier --write /wd/$@ \
@@ -64,7 +71,7 @@ types/next.d.ts: src/generated/next_generated.js
 		apk add --update git && \
 		git clone --depth 1 https://github.com/stellar/dts-xdr -b $(DTSXDR_COMMIT) --single-branch && \
 		cd /dts-xdr && \
-		yarn install && \
+		yarn install --network-concurrency 1 && \
 		OUT=/wd/$@ npx jscodeshift -t src/transform.js /wd/$< && \
 		cd /wd && \
 		yarn run prettier --write /wd/$@ \
@@ -83,6 +90,7 @@ $(XDR_FILES_LOCAL_NEXT):
 
 reset-xdr:
 	rm -f xdr/*/*.x
+	rm -f src/generated/*.js
 	rm -f types/curr.d.ts
 	rm -f types/next.d.ts
 	$(MAKE) generate

README.md

@@ -98,9 +98,9 @@ Make sure that you are using the latest version number. They can be found on the
 
 ### To develop and test js-stellar-base itself
 
-1. Install Node 14.x
+1. Install Node 16.x
 
-We support the oldest LTS release of Node, which is [currently 14.x](https://nodejs.org/en/about/releases/). Please likewise install and develop on Node 14 so you don't get surprised when your code works locally but breaks in CI.
+We support the oldest LTS release of Node, which is [currently 16.x](https://nodejs.org/en/about/releases/). Please likewise install and develop on Node 16 so you don't get surprised when your code works locally but breaks in CI.
 
 If you work on several projects that use different Node versions, you might find helpful to install a NodeJS version manager:
 

config/.eslintrc.js

@@ -1,6 +1,7 @@
 module.exports = {
   env: {
-    es6: true
+    es6: true,
+    es2020: true
   },
   extends: ['airbnb-base', 'prettier'],
   plugins: ['@babel', 'prettier', 'prefer-import'],

config/webpack.config.browser.js

@@ -14,7 +14,6 @@ const config = {
   },
   resolve: {
     fallback: {
-      crypto: require.resolve('crypto-browserify'),
       stream: require.resolve('stream-browserify'),
       buffer: require.resolve('buffer')
     },

package.json

@@ -1,9 +1,12 @@
 {
   "name": "stellar-base",
-  "version": "9.0.0",
+  "version": "10.0.0-beta.0",
   "description": "Low-level support library for the Stellar network.",
   "main": "./lib/index.js",
-  "browser": "./dist/stellar-base.min.js",
+  "browser": {
+    "main": "./dist/stellar-base.min.js",
+    "sodium-native": false
+  },
   "types": "./types/index.d.ts",
   "scripts": {
     "build": "yarn build:node && yarn build:browser",
@@ -13,14 +16,15 @@
     "build:browser:prod": "cross-env NODE_ENV=production yarn build:browser",
     "build:prod": "cross-env NODE_ENV=production yarn build",
     "test": "yarn build && yarn test:node && yarn test:browser",
-    "test:node": "nyc --nycrc-path ./config/.nycrc mocha",
+    "test:node": "yarn _nyc mocha",
     "test:browser": "karma start ./config/karma.conf.js",
     "docs": "jsdoc -c ./config/.jsdoc.json --verbose",
     "lint": "eslint -c ./config/.eslintrc.js src/ && dtslint --localTs node_modules/typescript/lib types/",
     "preversion": "yarn clean && yarn fmt && yarn lint && yarn build:prod && yarn test",
     "fmt": "prettier --config ./config/prettier.config.js --ignore-path ./config/.prettierignore --write './**/*.js'",
     "prepare": "yarn build:prod",
-    "clean": "rm -rf lib/ dist/ coverage/ .nyc_output/"
+    "clean": "rm -rf lib/ dist/ coverage/ .nyc_output/",
+    "_nyc": "nyc --nycrc-path ./config/.nycrc"
   },
   "mocha": {
     "require": [
@@ -52,9 +56,6 @@
       "yarn lint"
     ]
   },
-  "browser": {
-    "sodium-native": false
-  },
   "repository": {
     "type": "git",
     "url": "https://github.com/stellar/js-stellar-base.git"
@@ -69,60 +70,59 @@
   },
   "homepage": "https://github.com/stellar/js-stellar-base",
   "devDependencies": {
-    "@babel/cli": "^7.21.0",
-    "@babel/core": "^7.12.0",
-    "@babel/eslint-parser": "^7.21.3",
-    "@babel/eslint-plugin": "^7.19.1",
-    "@babel/preset-env": "^7.21.4",
-    "@babel/register": "^7.21.0",
-    "@definitelytyped/dtslint": "^0.0.163",
+    "@babel/cli": "^7.22.15",
+    "@babel/core": "^7.22.17",
+    "@babel/eslint-parser": "^7.22.15",
+    "@babel/eslint-plugin": "^7.22.10",
+    "@babel/preset-env": "^7.22.15",
+    "@babel/register": "^7.22.15",
+    "@definitelytyped/dtslint": "^0.0.177",
     "@istanbuljs/nyc-config-babel": "3.0.0",
-    "@types/node": "^20.1.4",
-    "@typescript-eslint/parser": "^5.59.6",
-    "babel-loader": "^9.1.2",
+    "@types/node": "^20.6.0",
+    "@typescript-eslint/parser": "^6.7.0",
+    "babel-loader": "^9.1.3",
     "babel-plugin-istanbul": "^6.1.1",
-    "buffer": "^6.0.3",
-    "chai": "^4.3.7",
+    "chai": "^4.3.8",
     "cross-env": "^7.0.3",
-    "eslint": "^8.37.0",
+    "eslint": "^8.49.0",
     "eslint-config-airbnb-base": "^15.0.0",
-    "eslint-config-prettier": "^8.8.0",
-    "eslint-plugin-import": "^2.25.2",
+    "eslint-config-prettier": "^9.0.0",
+    "eslint-plugin-import": "^2.28.1",
     "eslint-plugin-node": "^11.1.0",
     "eslint-plugin-prefer-import": "^0.0.1",
-    "eslint-plugin-prettier": "^4.2.1",
+    "eslint-plugin-prettier": "^5.0.0",
     "eslint-webpack-plugin": "^4.0.0",
     "ghooks": "^2.0.4",
     "husky": "^8.0.3",
     "jsdoc": "^4.0.2",
-    "karma": "^6.4.1",
+    "karma": "^6.4.2",
     "karma-chrome-launcher": "^3.1.0",
-    "karma-coverage": "^2.2.0",
+    "karma-coverage": "^2.2.1",
     "karma-firefox-launcher": "^2.1.1",
     "karma-mocha": "^2.0.0",
     "karma-sinon-chai": "^2.0.2",
     "karma-webpack": "^5.0.0",
-    "lint-staged": "^13.2.0",
+    "lint-staged": "^14.0.1",
     "minami": "^1.1.1",
     "mocha": "^10.2.0",
     "node-polyfill-webpack-plugin": "^2.0.1",
     "nyc": "^15.1.0",
-    "prettier": "^2.8.7",
+    "prettier": "^3.0.3",
     "randombytes": "^2.1.0",
     "sinon": "^15.0.3",
     "sinon-chai": "^3.7.0",
     "taffydb": "^2.7.3",
-    "terser-webpack-plugin": "^5.3.7",
+    "terser-webpack-plugin": "^5.3.8",
     "ts-node": "^10.9.1",
-    "typescript": "^5.0.3",
+    "typescript": "^5.2.2",
     "webpack": "^5.82.1",
     "webpack-cli": "^5.1.1"
   },
   "dependencies": {
     "base32.js": "^0.1.0",
-    "bignumber.js": "^9.1.1",
-    "crypto-browserify": "^3.12.0",
-    "js-xdr": "^2.0.0",
+    "bignumber.js": "^9.1.2",
+    "buffer": "^6.0.3",
+    "js-xdr": "^3.0.0",
     "sha.js": "^2.3.6",
     "tweetnacl": "^1.0.3"
   },