Skip to content

Conversation

@tedyu
Copy link
Contributor

@tedyu tedyu commented Jan 3, 2023

What changes were proposed in this pull request?

The use of SortedMap in ErrorClassesJsonReader was mostly for making tests easier to write.

This PR replaces SortedMap with Map since SortedMap is slower compared to Map

Why are the changes needed?

SortedMap is slower compared to Map

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing test suite.

@github-actions github-actions bot added the CORE label Jan 3, 2023
@tedyu
Copy link
Contributor Author

tedyu commented Jan 3, 2023

cc @cloud-fan @srowen

@tedyu tedyu changed the title [CORE] Use Map in place of SortedMap for ErrorClassesJsonReader [SPARK-41853] [CORE] Use Map in place of SortedMap for ErrorClassesJsonReader Jan 3, 2023
.build()
private def readAsMap(url: URL): SortedMap[String, ErrorInfo] = {
val map = mapper.readValue(url, new TypeReference[SortedMap[String, ErrorInfo]]() {})
private def readAsMap(url: URL): Map[String, ErrorInfo] = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @MaxGekk FYI

Copy link
Contributor

@LuciferYang LuciferYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM

@srowen
Copy link
Member

srowen commented Jan 3, 2023

Merged to master

@srowen srowen closed this in ec59423 Jan 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants