Repository navigation
test: improve coverage across toolchain modules before 1.8.0 - #805
Open
bitflicker64 wants to merge 6 commits into
Open
bitflicker64 wants to merge 6 commits into
bitflicker64 wants to merge 6 commits into
Conversation
Register unit tests that already compiled but never ran (HugeClientBuilderTest, GremlinManagerTest), the Cypher and Adamic-Adar API tests and GraphModeTest, all of which pass against Server 1.8. Add unit tests for ServerException parsing of JSON, alternative-key, non-JSON and empty error bodies, for graph element property validation and for JsonUtil error paths. HugeClientFakeServerTest drives the client against a local fake server that returns 5xx HTML, non-JSON bodies, an unsupported API version, slow responses and dropped connections, and checks that legacy servers get requests without the graphspace prefix.
BackupFailureTest runs the backup manager against a local fake server. Server errors on shard queries are retried and then fail with the original ServerException, non-JSON shards fail backup, schema errors reach the caller, and invalid split sizes and label/type combinations are rejected. Add LocalDirectoryTest for plain and zip streams, append versus override, corrupt, empty and missing archives, and directory checks. Extend RetryManagerTest with retry success, exhaustion and default behavior, and add JsonGraphTest for the dump graph and formatters.
Add unit tests for DataTypeUtil conversion of numbers, booleans, UUIDs, text, dates with extra formats and time zones, JDBC temporal values and list/set values, with the matching invalid inputs. Cover the text, CSV and JSON line parsers, including quoted CSV fields and column count mismatches, and LoadMapping parsing of valid and invalid struct files. BatchInsertRetryTest runs BatchInsertTask against a local fake server: 5xx responses are retried until success or until the retry limit, while IllegalArgumentException and unacceptable messages fail immediately.
Add DateUtilsTest for time-zone aware parsing and invalid dates, and extend DataTypeUtilsTest with invalid numbers, text conversion, string dates in the default format, null values and unsupported list/set cardinalities.
Add backend tests for EntityUtil.merge, which applies entity updates according to @MergeProperty, and for HubbleUtil collection equality, date helpers, digests, the host pattern and the size and duration JSON serializers. Add frontend tests for storage and duration formatting, request payload sanitizing in removeNilKeys and convertStringToJSON.
Add hgtransport tests for URL prefixes, basic auth sources, explicit Authorization headers, user agent and content type, and for server errors, dropped connections, timeouts and request/response logging. Cover api.Response helpers, config validation in NewCommonClient and /versions against non-JSON bodies, JSON errors and dropped connections. The vertex package had no executing test: its only test is commented out. Add httptest coverage for vertex create, batch create and property updates, with and without a graphspace, plus error bodies and cancelled contexts.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #805 +/- ##
============================================
+ Coverage 44.74% 47.62% +2.87%
- Complexity 3323 4590 +1267
============================================
Files 377 602 +225
Lines 22486 29432 +6946
Branches 3006 3454 +448
============================================
+ Hits 10062 14017 +3955
- Misses 11352 14153 +2801
- Partials 1072 1262 +190 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose of the PR
More test coverage across the toolchain modules before the 1.8.0 release, as requested for the Java 17 migration. This PR only adds or registers tests. It changes no production code, dependencies, CI workflows or documentation.
Each module is a separate commit, so the PR can be reviewed one commit at a time.
Main Changes
hugegraph-client
HugeClientBuilderTestandGremlinManagerTest(unit),CypherApiTestandAdamicAdarAPITest(API) andGraphModeTest(functional). All pass against the locally built Server 1.8; the Server 1.7 baseline is exercised by CI only.SameNeighborsBatchApiTestis left to fix(client): Add SameNeighborsBatchApiTest to test suite #673.ServerExceptionTest: error bodies with the standard keys, the alternative keys, non-JSON bodies, empty bodies and empty causes.HugeClientFakeServerTest: the client against a localHttpServerthat returns 5xx HTML, non-JSON bodies or an unsupported API version, answers slowly or drops the connection. It also checks that requests to a legacy server leave out thegraphspaces/DEFAULTprefix.GraphElementTestandJsonUtilTest: property value validation and serialization error paths.hugegraph-tools (all registered in
FuncTestSuite)BackupFailureTest: backup against a fake server. A 5xx on the shard query is retried and then fails with the originalServerException; non-JSON shards and schema errors fail the backup; an invalid split size or label/type combination is rejected.LocalDirectoryTest: plain and zip streams, append and override, corrupt, empty and missing archives, directory checks.RetryManagerTest: retry success, exhaustion and the default of no retries.JsonGraphTest: the dump graph,JsonFormatterand formatter loading.hugegraph-loader (all registered in the
unitprofile)DataTypeUtilTest: conversion of numbers, booleans, UUIDs, text, dates (extra formats, time zones, timestamps, JDBC temporal types) and list/set values, each with invalid inputs.LineParserTest: text, CSV (quotes, escaped quotes) and JSON line parsing, missing and extra columns.LoadMappingTest: valid and invalid struct files (version, empty or duplicate structs, a missing input type, an unknown source type).BatchInsertRetryTest:BatchInsertTaskagainst a fake server. A 5xx is retried until it succeeds or the retry limit is reached.IllegalArgumentExceptionand the unacceptable messages fail at once.hugegraph-spark-connector
DateUtilsTestand moreDataTypeUtilsTestcases: invalid numbers, text conversion, string dates, null values, unsupported list/set cardinality.hugegraph-hubble
EntityUtilTestcovers@MergePropertyupdate semantics.HubbleUtilTestcovers collection equality, date helpers, digests, the host pattern and the size and duration serializers.removeNilKeys(algorithm request payloads) andconvertStringToJSON.hugegraph-client-go (
httptestonly, no live server)hgtransport: URL prefix, basic auth from config or URL, explicitAuthorization, user agent, content type, 5xx pass-through, dropped connections, timeouts, logger body handling.api.Responsehelpers,NewCommonClientconfig validation,/versionswith non-JSON, JSON error and dropped-connection responses.TestVertex, has its whole body commented out, so the vertex package showed 0.0% coverage with the CI test flags; it now shows 85.5%.Verifying these changes
install-candidate-sdk.shinto a separate Maven repository. Authentication was on, with the batch limits that CI sets.mvn test -Dtest=UnitTestSuitemvn test -Dtest=ApiTestSuitemvn test -Dtest=FuncTestSuiteHugeClientHttpsTest)HugeClientHttpsTestmvn test -Dtest=FuncTestSuite -Dtools.test.disposable=truemvn test -P unitmvn test -P file(unchanged)mvn testmvn test -P unit-testyarn lint,yarn i18n:check,yarn testgo test -race -count=1 ./...Notes:
HugeClientHttpsTestand the loader HTTPS tests fail for that reason in both runs. CI starts its own HTTPS server.hdfs,jdbcandkafkaprofiles (no HDFS, MySQL or Kafka started), the Hubble packaged API/browser test, and the opt-inHugeGraphIdentityIntegrationTest, which needs a server without authentication.Production issues found (tests not included)
These tests fail on the current code, so they are not part of this PR:
JsonLineParser.parsecatchesSerializeException, butJsonUtil.convertMapthrows the loader'sDeserializeException, so noReadExceptionis raised and--max-read-errorshas no effect. Reproduced withFileLoadTestresources (reload_json_failure_files): one valid edge line plus{"source_name":, with--max-read-errors 10. The load threwDeserializeException, and the valid edge was not loaded either.BatchInsertTaskretries onClientException, but the client now throws a rawIOExceptionfor timeouts and dropped connections. WithretryTimes=3, a fake server that answers after 2.5 s (client timeout 1 s) got one request beforeSocketTimeoutExceptionreached the caller. A dropped connection got two requests: the first try and one OkHttp retry, none from the loader. 5xx responses are retried correctly;BatchInsertRetryTestcovers that case.Long.MAX_VALUEwrap around.DataTypeUtil.parseLongusesLong.parseUnsignedLongfor non-negative input, so"9223372036854775808"becomes-9223372036854775808and"18446744073709551615"becomes-1instead of being rejected. The same helper parses numeric vertex IDs. The Spark connector'sDataTypeUtils.parseLonghas the same code (checked by reading, not run).DataTypeUtil.parseBlobhas its body commented out and returnsnullfor any input, for example"0x01"or"AQI=".client.Vertex.GetByID()and the other unassigned function fields cause a nil pointer panic instead of returning an error.DateUtils. It shares oneSafeDateFormatper pattern and callssetTimeZonebeforeparse, the code the loader fixed in fix(loader): fix thread-safety issue in DateUtil with concurrent date parsing #734. A port of the loader's concurrent test returned 1 wrong instant out of 1600. The connector currently always passes GMT+8, so the sink path does not hit it.Does this PR potentially affect the following parts?
Documentation Status
Doc - TODODoc - DoneDoc - No Need