Skip to content

test: improve coverage across toolchain modules before 1.8.0 - #805

Open
bitflicker64 wants to merge 6 commits into
apache:masterfrom
hugegraph:test/toolchain-coverage-1.8
Open

bitflicker64 wants to merge 6 commits into
apache:masterfrom
hugegraph:test/toolchain-coverage-1.8

Conversation

@bitflicker64

Copy link
Copy Markdown
Contributor

Purpose of the PR

More test coverage across the toolchain modules before the 1.8.0 release, as requested for the Java 17 migration. This PR only adds or registers tests. It changes no production code, dependencies, CI workflows or documentation.

Each module is a separate commit, so the PR can be reviewed one commit at a time.

Main Changes

hugegraph-client

  • Register tests that compiled but never ran: HugeClientBuilderTest and GremlinManagerTest (unit), CypherApiTest and AdamicAdarAPITest (API) and GraphModeTest (functional). All pass against the locally built Server 1.8; the Server 1.7 baseline is exercised by CI only. SameNeighborsBatchApiTest is left to fix(client): Add SameNeighborsBatchApiTest to test suite #673.
  • ServerExceptionTest: error bodies with the standard keys, the alternative keys, non-JSON bodies, empty bodies and empty causes.
  • HugeClientFakeServerTest: the client against a local HttpServer that returns 5xx HTML, non-JSON bodies or an unsupported API version, answers slowly or drops the connection. It also checks that requests to a legacy server leave out the graphspaces/DEFAULT prefix.
  • GraphElementTest and JsonUtilTest: property value validation and serialization error paths.

hugegraph-tools (all registered in FuncTestSuite)

  • BackupFailureTest: backup against a fake server. A 5xx on the shard query is retried and then fails with the original ServerException; non-JSON shards and schema errors fail the backup; an invalid split size or label/type combination is rejected.
  • LocalDirectoryTest: plain and zip streams, append and override, corrupt, empty and missing archives, directory checks.
  • RetryManagerTest: retry success, exhaustion and the default of no retries.
  • JsonGraphTest: the dump graph, JsonFormatter and formatter loading.

hugegraph-loader (all registered in the unit profile)

  • DataTypeUtilTest: conversion of numbers, booleans, UUIDs, text, dates (extra formats, time zones, timestamps, JDBC temporal types) and list/set values, each with invalid inputs.
  • LineParserTest: text, CSV (quotes, escaped quotes) and JSON line parsing, missing and extra columns.
  • LoadMappingTest: valid and invalid struct files (version, empty or duplicate structs, a missing input type, an unknown source type).
  • BatchInsertRetryTest: BatchInsertTask against a fake server. A 5xx is retried until it succeeds or the retry limit is reached. IllegalArgumentException and the unacceptable messages fail at once.

hugegraph-spark-connector

  • DateUtilsTest and more DataTypeUtilsTest cases: invalid numbers, text conversion, string dates, null values, unsupported list/set cardinality.

hugegraph-hubble

  • hubble-be: EntityUtilTest covers @MergeProperty update semantics. HubbleUtilTest covers collection equality, date helpers, digests, the host pattern and the size and duration serializers.
  • hubble-fe: Jest tests for storage and duration formatting, removeNilKeys (algorithm request payloads) and convertStringToJSON.

hugegraph-client-go (httptest only, no live server)

  • hgtransport: URL prefix, basic auth from config or URL, explicit Authorization, user agent, content type, 5xx pass-through, dropped connections, timeouts, logger body handling.
  • api.Response helpers, NewCommonClient config validation, /versions with non-JSON, JSON error and dropped-connection responses.
  • Vertex create, batch create and property update, with and without a graphspace. The package's only test, TestVertex, has its whole body commented out, so the vertex package showed 0.0% coverage with the CI test flags; it now shows 85.5%.

Verifying these changes

  • Trivial rework / code cleanup without any test coverage. (No Need)
  • Already covered by existing tests, such as (please modify tests here).
  • Need tests and can be verified as follows:
    • Environment: Toolchain at 039153c, Java 17.0.20, Maven 3.9.14, Node 18.20.8, Go 1.22.2. The Server 1.8 SDK and server were built from the CI-pinned commit 68855199 with install-candidate-sdk.sh into a separate Maven repository. Authentication was on, with the batch limits that CI sets.
    • Each new class was also run on its own twice, and a timeout check was confirmed to fail when its condition is removed.
Command Before After
client mvn test -Dtest=UnitTestSuite 77 passed 118 passed
client mvn test -Dtest=ApiTestSuite 376 run, 3 skipped 382 run, 3 skipped, 0 failed
client mvn test -Dtest=FuncTestSuite 94 run, 1 error (HugeClientHttpsTest) 104 run, 6 errors, all in HugeClientHttpsTest
tools mvn test -Dtest=FuncTestSuite -Dtools.test.disposable=true 30 passed 55 passed
loader mvn test -P unit 100 passed 134 passed
loader mvn test -P file (unchanged) 99 run, 2 errors, both HTTPS tests
spark-connector mvn test 59 passed 67 passed
hubble-be mvn test -P unit-test 614 passed 623 passed
hubble-fe yarn lint, yarn i18n:check, yarn test 162 suites, 1032 tests 165 suites, 1079 tests, lint clean
client-go go test -race -count=1 ./... all packages pass all packages pass

Notes:

  1. Port 8443 on the test host was held by an unrelated process, so the HTTPS server could not start. HugeClientHttpsTest and the loader HTTPS tests fail for that reason in both runs. CI starts its own HTTPS server.
  2. Not run: the loader hdfs, jdbc and kafka profiles (no HDFS, MySQL or Kafka started), the Hubble packaged API/browser test, and the opt-in HugeGraphIdentityIntegrationTest, which needs a server without authentication.

Production issues found (tests not included)

These tests fail on the current code, so they are not part of this PR:

  1. A malformed JSON line aborts the whole load. JsonLineParser.parse catches SerializeException, but JsonUtil.convertMap throws the loader's DeserializeException, so no ReadException is raised and --max-read-errors has no effect. Reproduced with FileLoadTest resources (reload_json_failure_files): one valid edge line plus {"source_name": , with --max-read-errors 10. The load threw DeserializeException, and the valid edge was not loaded either.
  2. Batch inserts never retry network failures. BatchInsertTask retries on ClientException, but the client now throws a raw IOException for timeouts and dropped connections. With retryTimes=3, a fake server that answers after 2.5 s (client timeout 1 s) got one request before SocketTimeoutException reached the caller. A dropped connection got two requests: the first try and one OkHttp retry, none from the loader. 5xx responses are retried correctly; BatchInsertRetryTest covers that case.
  3. LONG values above Long.MAX_VALUE wrap around. DataTypeUtil.parseLong uses Long.parseUnsignedLong for non-negative input, so "9223372036854775808" becomes -9223372036854775808 and "18446744073709551615" becomes -1 instead of being rejected. The same helper parses numeric vertex IDs. The Spark connector's DataTypeUtils.parseLong has the same code (checked by reading, not run).
  4. BLOB values become null. DataTypeUtil.parseBlob has its body commented out and returns null for any input, for example "0x01" or "AQI=".
  5. Unimplemented Go vertex methods panic. client.Vertex.GetByID() and the other unassigned function fields cause a nil pointer panic instead of returning an error.
  6. A latent race in the Spark connector DateUtils. It shares one SafeDateFormat per pattern and calls setTimeZone before parse, the code the loader fixed in fix(loader): fix thread-safety issue in DateUtil with concurrent date parsing #734. A port of the loader's concurrent test returned 1 wrong instant out of 1600. The connector currently always passes GMT+8, so the sink path does not hit it.

Does this PR potentially affect the following parts?

  • Nope
  • Dependencies (add/update license info)
  • Modify configurations
  • The public API
  • Other affects (typed here)

Documentation Status

  • Doc - TODO
  • Doc - Done
  • Doc - No Need

Register unit tests that already compiled but never ran
(HugeClientBuilderTest, GremlinManagerTest), the Cypher and Adamic-Adar
API tests and GraphModeTest, all of which pass against Server 1.8.

Add unit tests for ServerException parsing of JSON, alternative-key,
non-JSON and empty error bodies, for graph element property validation
and for JsonUtil error paths. HugeClientFakeServerTest drives the client
against a local fake server that returns 5xx HTML, non-JSON bodies, an
unsupported API version, slow responses and dropped connections, and
checks that legacy servers get requests without the graphspace prefix.
BackupFailureTest runs the backup manager against a local fake server.
Server errors on shard queries are retried and then fail with the
original ServerException, non-JSON shards fail backup, schema errors
reach the caller, and invalid split sizes and label/type combinations
are rejected.

Add LocalDirectoryTest for plain and zip streams, append versus
override, corrupt, empty and missing archives, and directory checks.
Extend RetryManagerTest with retry success, exhaustion and default
behavior, and add JsonGraphTest for the dump graph and formatters.
Add unit tests for DataTypeUtil conversion of numbers, booleans, UUIDs,
text, dates with extra formats and time zones, JDBC temporal values and
list/set values, with the matching invalid inputs. Cover the text, CSV
and JSON line parsers, including quoted CSV fields and column count
mismatches, and LoadMapping parsing of valid and invalid struct files.

BatchInsertRetryTest runs BatchInsertTask against a local fake server:
5xx responses are retried until success or until the retry limit, while
IllegalArgumentException and unacceptable messages fail immediately.
Add DateUtilsTest for time-zone aware parsing and invalid dates, and
extend DataTypeUtilsTest with invalid numbers, text conversion, string
dates in the default format, null values and unsupported list/set
cardinalities.
Add backend tests for EntityUtil.merge, which applies entity updates
according to @MergeProperty, and for HubbleUtil collection equality,
date helpers, digests, the host pattern and the size and duration JSON
serializers.

Add frontend tests for storage and duration formatting, request payload
sanitizing in removeNilKeys and convertStringToJSON.
Add hgtransport tests for URL prefixes, basic auth sources, explicit
Authorization headers, user agent and content type, and for server
errors, dropped connections, timeouts and request/response logging.
Cover api.Response helpers, config validation in NewCommonClient and
/versions against non-JSON bodies, JSON errors and dropped connections.

The vertex package had no executing test: its only test is commented
out. Add httptest coverage for vertex create, batch create and property
updates, with and without a graphspace, plus error bodies and
cancelled contexts.
@github-actions github-actions Bot added hubble hugegraph-hubble loader hugegraph-loader client hugegraph-client tools hugegraph-tools spark labels Oct 10, 2026
@codecov

codecov Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 47.62%. Comparing base (039153c) to head (1b96264).

Additional details and impacted files
@@             Coverage Diff              @@
##             master     #805      +/-   ##
============================================
+ Coverage     44.74%   47.62%   +2.87%     
- Complexity     3323     4590    +1267     
============================================
  Files           377      602     +225     
  Lines         22486    29432    +6946     
  Branches       3006     3454     +448     
============================================
+ Hits          10062    14017    +3955     
- Misses        11352    14153    +2801     
- Partials       1072     1262     +190     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client hugegraph-client hubble hugegraph-hubble loader hugegraph-loader spark tools hugegraph-tools

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant