Skip to content

Conversation

@Algo-devops-service
Copy link
Contributor

@Algo-devops-service Algo-devops-service commented Jul 17, 2021

GitHub Logo

Highlights

This is a maintenance release focused around many test improvements and other optimizations. Additionally there are some added improvements, such as:

  • Local private network devMode for rapid transaction testing
  • Improved support for non-utf-8 data in asset parameter name, unit name, and url fields, through the addition of base64 fields
  • Support for CentOS 8 Stream in RPM
  • Changes to support future AVM 1.0

Changes

  1. REST API
    • New Feature - add handling for non-utf 8 asset names
    • Enhancements
      • Use byte format for asset b64 fields
      • Use dashes in v2 API.
    • Bug Fix - initialize past side effects with correct length during dryrun requests
  2. TEAL
    • New Features
      • Add Extract opcodes
      • Selector pseudo-op in support of ABI
      • vFuture: creator access and app_params_get
    • Enhancements
      • Infrastructure to check AVM stack manipulation opcodes better
      • Specs and unit test to avoid forgetting in the future
    • Bug Fixes
      • Addressing panic in assembler
      • Minor change to rename helper function in Extract ops
  3. Catchup
    • Enhancement - avoid requesting unsupported blocks by the ensure digest catchup path
    • Bug Fixes
      • Fixes to peer selector and TestBasicCatchpointCatchup
      • Fix potential nil dereferencing
  4. Ledger
    • Enhancements
      • Lruaccts write speedup
      • optimize txtail memory consumption
  5. Network
    • Enhancement - reduce unneeded contention around checking the peers connectivity
  6. Node
    • New Feature - add initial devMode support
    • Bug Fixes
      • Allow to parse ipv6 localhost "[::]:4601"
      • Fix regex for matching charset of valid DNS hosts, add test
  7. Logging
    • Enhancement - re-use host ID if present
  8. Tests
    • Enhancements
      • Add enums to player permutation tests
      • Add ledger coverage for onlineacct roundlru txtail
      • Implement hello circleci yaml
      • Improve TestMetricSegment test reliability
      • Restore TestPeriodicSync and TestBasicCatchpointCatchup tests.
      • Added benchmark to lruAccounts write function
      • Enable TestPartkeyOnlyRewards on macos
      • Add unit tests for ledger BlockQueue
      • Enable previously disabled TestConfigMigrate unit test
      • Enable unit tests on travis mac build
      • Improve lruaccts benchmark
      • Improve msgp codec testing
      • Improve testing of alloc bounded slices
      • Make TestPeersDownloadFailed and TestHistoricData predictable
      • Move extra page test to e2e_subs
      • Move travis_retry to skip rebuilding
      • Remove deprecated transaction pool e2e tests
      • Use periodicSyncLogger for cleaner test runs
    • Bug Fixes
      • Avoid division by zero during TestBasicCatchpointWriter
      • Fix 10 minute timeout in travis
      • Fix and enable TestNewAccountCanGoOnlineAndParticipate
      • Build: fix travis nightly build failure
      • Fix random failure in TestWebsocketNetworkPrioLimit
      • Fix random failures in TestPeriodicSync
      • Avoid running TestAgreementSynchronous10 on nightly build
      • Ensure deploy_linux_version generates a linux/amd64 binaries for algonet usage
      • Fix gotestsum install
      • Fix telemetry unit tests
      • Lookup in no-GOPATH locations for packages in gocodec testing
  9. Other
    • New Features
      • Add Fedora support into install_linux_deps.sh
      • Test conditional slack alert for build status
      • Use golangci-lint
    • Enhancements
      • Update Github engineering template
      • Replace yum-cron with dnf-automatic to support centos 8
      • Run misspell linter with -w flag
      • Specify truncated division is used in TEAL in documentation
      • Typos, grammar, inconsistencies
    • Bug Fixes
      • Add missing txna allowed args to doc and langspec
      • Fix darwin-arm64 builds
      • Fix docker repo update

Protocol Upgrade

This release does not contain a protocol upgrade.

Additional Resources

nicholasguo and others added 30 commits June 9, 2021 13:00
peerSelector.go: various bug fixes
- introduce peerSelectorPeer to wrap the network.Peer and add peerClass information, to be able to distinguish between peers of the same address but different classes.
- keep track of download failures to be able to increase the cost of each failure when failing more than succeeding. This is to evict the peer faster when constantly failing to download.
- initialize rankSum and rankSamples to initialRank of the class. Otherwise, the peer rank will have a very long warmup time before relfecting the correct rank.
- let resetRequestPenalty bound the rank within the class bounds. Otherwise, the penalty calculation pushes the rank out of the class bounds (bug).
- getNextPeer is local to the package
- getNextPeer, PeerDownloadDurationToRank and RankPeer use peerSelectorPeer instead of network.Peer
- refreshAvailablePeers distinguishes between peers with the same address but of different peer class
- findPeer returns the peer given the address and the peer class (instead of just the address)

catchpointCatchup_test.go:
- Remove comment about giving the second node all the stake, since it is not the case here.
- Use the round from the catchpoint instead of guessing the round as 36. In case the following catchpoint was obtained due to race conditions, checking for round 37 will be trivial, since it will also be obtained from the catchpoint.

catchpointService.go and service.go:
- Update the code to use peerSelectorPeer instead of network.Peer with peerSelector

peerSelector_test.go:
- Update the tests to use peerSelectorPeer instead of network.Peer with peerSelector
- Cleanup debugging printouts.
Add tests to increase coverage for onlineacct roundlru txtail
Fix go-algorand dev builds for darwin-arm64 machines (Apple's M1 CPUs).
testing: add unit tests for ledger BlockQueue
testing: enable unit tests on travis mac build
Add enums to player permutation tests:
This made it easier for me to understand the different 7 initial player states * 14 simulated events when reading through each permutation's expected actions. I hope the enum names I picked are accurate — would appreciate any feedback on better names.
testing: move extra page test to e2e_subs
mostly casing issues, ie, algorand > Algorand || sqlite > SQLite
tsachiherman and others added 22 commits July 12, 2021 12:52
This change is needed in order to support codec types such as the following:
```golang
//msgp:allocbound typeA 16
type typeA []typeB
```
since we want the codec object randomizer to recognize that `typeA` has a allocbound defined, and use that bound.
( this would be used in the feature/txnsync branch, but the change doesn't really related to any of the other changes in the feature/txnsync branch and would be a good change regardless )
Our logic should be as follows:

When Loading:

We first look inside the provided data-directory. If a config file is there, load it and return it
Otherwise, look in the global directory. If a config file is there, load it and return it.
When Saving:

If a data-directory was provided then save the config file there.
Otherwise, save the config file in the global directory
Optimized lruAccounts write function. gains: from ~260ns to ~200ns per write op
This adds a golangci-lint configuration file that runs an initial minimal set of linters: golint, govet, and misspell. This makes it easier to integrate with editors and IDEs and also opens the door to using from CI.
This PR improves the checkBoundsLimitingTag method of the message pack random generated object tester.
With this change, we're no longer ignoring data types that has no struct tags, but rather looking for corresponding msgp directives.

This is the same change applied to the feature/txnsync branch. This change is expected to be nop on the master branch, but would be required on the feature branch.
The waiting period wasn't long enough. Allow much longer wait period before giving up.

Also - omit unneeded log entries.
Use periodicSyncLogger in a test to avoid unnecessary log outputs.
Travis nightly build failed due to no-output for over 10m from build process.
Travis is currently unable to only notify on failures of a particular branch, however, it can notify on us on the build of a particular branch (both success and failures). Since we are also considering CircleCI, I think it would be best to just add the slack notification into the travis file now for the rel/nightly branch and notify the #devops-jenkins channel (where most of our jobs status activity goes) when it both succeeds/fails. The "successes" will also help us determine whether or not the test ran at all.
Adds extract opcodes that allow a substring to be extracted given a start index and a length (#2347).

The specs for the extract and extract3 opcodes are the same as the substring and substring3 opcodes, but takes a length rather than an end index. This also adds extract16bits, extract32bits, extract64bits which extracts 2, 4, and 8 byte strings respectively and converts them into integers.
Both CentOS 7 and CentOS 8 support dnf-automatic, so using that for cron update is preferred. This should enable using the same package on both systems.
This PR remove 3 e2e tests that are deprecated ( i.e. for over a year now ) :

TestClientPrioritizesPendingTransactions
TestTransactionPoolExponentialFees
TestTransactionPoolOrderingAndClearing
All the 3 tests were designed to test ordering, fees and prioritization of transactions in the transaction pool. These feature were completely removed when we released v1/v2, and we have no plans to bring these back.
…lect actual behavior (#2554)

Changes a helper function name from opExtractNBits to opExtractNBytes to reflect actual behavior.
…#2563)

In current logic, the agreement is sending a notification to the catchup whenever it sees a cert vote
threshold without a corresponding block. When this happens on a round that is no longer supported
by the current binary, it would lead to continues requests for the said block. This PR addresses the issue
by testing if the coming block is expected to be supported or not. In case we know that it won't be supported,
we avoid downloading it entirely.
#2562)

lookup in no-GOPATH locations for packages in gocodec testing; The existing code is trying to use the GOPATH as the base for the packages directories, however, this might not be the case when tests are being executed on travis.
The asset parameter fields name, unit name and url might contain non-utf 8 printable characters. This PR modifies the
REST API so that the existing field would only be returned in case the entire string is a utf-8 printable string. To complement that, a _b64 version of the field would be provided, allowing the client to support non-printable strings.
Use dashes to match the v2 API convention.
This PR changes the way we check our understanding of the stack during
assembly.  We try to give errors as we assemble, based on the types
that we believe are on the stack when we assemble an opcode that has
stack expectations.  Previously, we *only* check the very top of the
stack, since almost all opcodes only looked at or modified the top. In
addition, we were very conservative with opcodes that manipulated the
stack.  For example, once we assembled a `swap` we only knew that the
top of the stack was [any, any]. If the top was [int, bytes], we lost
track.  Now we know the stack after that is [bytes, int].

Several opcodes still need this enhancement. "dig" and "swap" are
done, and serve as examples.  The following should be done next:
`dup`, `dup2`, `cover`, `uncover`.

Slightly trickier, we can also improve `select`.  If both of the
"then" and "else" types are the same, we can know that `select` leaves
that type behind.

`setbit` can be improved to return the type of the thing it is modifying.

To enhance an opcode, write a typeOpcode function that determines,
based on the opcodes immediates and on the current state of the stack,
what checkStack should check confirm the prior stack state (for example,
dig n needs a stack of depth n+1), and to setup the stack
afterward (for example, we know the stack after `swap` in more detail
that simply any,any.  Register this typeOpcode function using
"stacky()" in the opcodes table.

These functions must be quite careful.  The typestack might not be long enough due to user error.  To report that nicely, we need to avoid panicing here, so checkStack can do its work. For example, `dig 4` needs to yield a StackValues of 5 StackAnys, even if the current typestack is only 3 deep.
Use byte format for asset b64 fields
@CLAassistant
Copy link

CLAassistant commented Jul 17, 2021

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
23 out of 24 committers have signed the CLA.

✅ winder
✅ algonautshant
✅ cce
✅ grakshith
✅ nicholasguoalgorand
✅ algorandskiy
✅ jannotti
✅ tsachiherman
✅ fabrice102
✅ algojohnlee
✅ figurestudios
✅ pzbitskiy
✅ algonathan
✅ Algo-devops-service
✅ AlgoStephenAkiki
✅ bricerisingalgorand
✅ brianolson
✅ algochoi
✅ onetechnical
✅ jdtzmn
✅ algobarb
✅ barnjamin
✅ algojack
❌ Jonathan Weiss


Jonathan Weiss seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codecov-commenter
Copy link

codecov-commenter commented Jul 17, 2021

Codecov Report

❗ No coverage uploaded for pull request base (rel/beta@dc829f6). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##             rel/beta    #2577   +/-   ##
===========================================
  Coverage            ?   46.99%           
===========================================
  Files               ?      348           
  Lines               ?    55715           
  Branches            ?        0           
===========================================
  Hits                ?    26185           
  Misses              ?    26593           
  Partials            ?     2937           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc829f6...f047ff2. Read the comment docs.

@onetechnical onetechnical marked this pull request as ready for review July 20, 2021 02:01
@algojohnlee algojohnlee merged commit 6cbee52 into algorand:rel/beta Jul 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.