Skip to content

Commit f344ab5

Browse files
fredbifredbi
committed
Remove dependency to mailru/easyjson by default
We want to remove this dependency by default, while still providing support to users who want to use easyjson. * fixes #68 This PR introduces an adapter to use ReadJSON and WriteJSON with different supporting libraries. By default only the standard library is enabled. A simple runtime registration allows the adapter to support easyjson. When import swag or swag/jsonutils or swag/yamlutils, easyjson is no longer a dependency: JSON serialization only requires the standard library. Supporting easyjson interfaces (e.g. for faster serialization) remains possible, by calling an independent module: swag/jsonutils/adapters/easyjson/json.Register(). 1. ordered map w/stdlib 2. registry with stdlib as default 3. fixed int vs float64 rendering 4. beefed up tests, including integration tests 5. honed doc 6. more tests TODO: * [x] hone test coverage * benchmarks * profiling Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
1 parent 6da37dd commit f344ab5

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

86 files changed

+7292
-748
lines changed

‎.codecov.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
ignore:
2+
- jsonutils/fixtures_test
3+
- jsonutils/adapters/ifaces/mocks

‎.github/workflows/go-test.yml‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ jobs:
4949
printf " \"${base_dir}\"" >> "$GITHUB_OUTPUT"
5050
all_mods+=("${base_dir}")
5151
((index++)) || true
52-
done < <(find . -name \*.mod | grep -v "\.git" | sort | uniq)
52+
done < <(find . -name go.mod | grep -v "\.git" | sort | uniq)
5353
printf "]" >> "$GITHUB_OUTPUT"
5454
5555
echo "::notice title=Modules found::${all_mods[@]}"
@@ -71,6 +71,7 @@ jobs:
7171
go-version: stable
7272
check-latest: true
7373
cache: true
74+
cache-dependency-path: '**/go.sum'
7475
- name: golangci-lint
7576
uses: golangci/golangci-lint-action@v8
7677
with:
@@ -105,15 +106,20 @@ jobs:
105106
go-version: '${{ matrix.go_version }}'
106107
check-latest: true
107108
cache: true
109+
cache-dependency-path: '**/go.sum'
108110

109111
- uses: actions/checkout@v5
110112
- name: Run unit tests on a single module in this repo
111113
shell: bash
112114
working-directory: '${{ matrix.module }}'
113-
env:
114-
# *.coverage.* pattern is automatically detected by codecov
115-
COVER_PROFILE: "${{ matrix.module }}.coverage.${{ matrix.os }}.${{ matrix.go_version }}.out"
116115
run: |
116+
# *.coverage.* pattern is automatically detected by codecov
117+
COVER_PROFILE="$(basename ${{ matrix.module }}).coverage.${{ matrix.os }}.${{ matrix.go_version }}.out"
118+
if [[ "${{ matrix.module }}" == "jsonutils/adapters/testintegration" ]] ; then
119+
# integration tests capture coverage elsewhere
120+
go test -v -race -coverprofile="${COVER_PROFILE}" -covermode=atomic -coverpkg=../../../... ./...
121+
exit 0
122+
fi
117123
go test -v -race -coverprofile="${COVER_PROFILE}" -covermode=atomic -coverpkg=$(go list)/... ./...
118124
119125
- name: Upload coverage to codecov

‎.golangci.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ linters:
1717
- gomoddirectives
1818
- gosmopolitan
1919
- inamedparam
20-
- intrange # disabled while < go1.22
20+
- intrange
2121
- ireturn
2222
- lll
2323
- musttag
@@ -29,6 +29,7 @@ linters:
2929
- recvcheck
3030
- testpackage
3131
- thelper
32+
- tagliatelle
3233
- tparallel
3334
- unparam
3435
- varnamelen

‎.mockery.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
all: false
2+
dir: '{{.InterfaceDir}}'
3+
filename: mocks_test.go
4+
force-file-write: true
5+
formatter: goimports
6+
include-auto-generated: false
7+
log-level: info
8+
structname: '{{.Mock}}{{.InterfaceName}}'
9+
pkgname: '{{.SrcPackageName}}'
10+
recursive: false
11+
require-template-schema-exists: true
12+
template: matryer
13+
template-schema: '{{.Template}}.schema.json'
14+
packages:
15+
github.com/go-openapi/swag/jsonutils/adapters/ifaces:
16+
config:
17+
dir: jsonutils/adapters/ifaces/mocks
18+
filename: mocks.go
19+
pkgname: 'mocks'
20+
force-file-write: true
21+
all: true
22+
github.com/go-openapi/swag/jsonutils/adapters/testintegration:
23+
config:
24+
inpackage: true
25+
dir: jsonutils/adapters/testintegration
26+
force-file-write: true
27+
all: true
28+
interfaces:
29+
EJMarshaler:
30+
EJUnmarshaler:

‎README.md‎

Lines changed: 78 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -9,91 +9,93 @@ Package `swag` contains a bunch of helper functions for go-openapi and go-swagge
99

1010
You may also use it standalone for your projects.
1111

12+
> **NOTE**
1213
> `swag` is one of the foundational building blocks of the go-openapi initiative.
13-
>
1414
> Most repositories in `github.com/go-openapi/...` depend on it in some way.
15-
> So does the CLI tool `github.com/go-swagger/go-swagger`,
16-
> and the code generated by this tool.
17-
18-
## Contents
19-
20-
`go-openapi/swag` now exposes a collection of relatively independent modules.
21-
22-
Here is what is inside:
23-
24-
* Module `cmdutils`
25-
26-
* [x] utilities to work with CLIs
27-
28-
* Module `conv`
15+
> And so does our CLI tool `github.com/go-swagger/go-swagger`,
16+
> as well as the code generated by this tool.
2917
30-
* [x] convert between values and pointers for any types
31-
* [x] convert from string to builtin types (wraps `strconv`)
32-
* [x] require `./typeutils` (test dependency)
18+
* [Contents](#contents)
19+
* [Dependencies](#dependencies)
20+
* [Release Notes](#release-notes)
21+
* [Note to contributors](#note-to-contributors)
22+
* [TODOs, suggestions and plans](#todos-suggestions-and-plans)
3323

34-
* Module `fileutils`
35-
36-
* [x] file upload type
37-
* [x] search in path (deprecated)
38-
39-
* Module `jsonname`
40-
41-
* [x] infer JSON names from go properties
42-
43-
* Module `jsonutils`
24+
## Contents
4425

45-
* [x] fast json concatenation
46-
* [x] read and write JSON from and to dynamic go data structures
47-
* [x] require `github.com/mailru/easyjson`
26+
`go-openapi/swag` exposes a collection of relatively independent modules.
4827

49-
* Module `loading`
28+
Moving forward, no additional feature will be added to the `swag` API directly at the root package level,
29+
which remains there for backward-compatibility purposes. All exported top-level features are now deprecated.
5030

51-
* [x] load from file or http
52-
* [x] require `./yamlutils`
31+
Child modules will continue to evolve or some new ones may be added in the future.
5332

54-
* Module `mangling`
33+
| Module | Content | Main features |
34+
|---------------|---------|---------------|
35+
| `cmdutils` | utilities to work with CLIs ||
36+
| `conv` | type conversion utilities | convert between values and pointers for any types<br />convert from string to builtin types (wraps `strconv`)<br />require `./typeutils` (test dependency)<br /> |
37+
| `fileutils` | file utilities | |
38+
| `jsonname` | JSON utilities | infer JSON names from `go` properties<br /> |
39+
| `jsonutils` | JSON utilities | fast json concatenation<br />read and write JSON from and to dynamic `go` data structures<br />~require `github.com/mailru/easyjson`~<br /> |
40+
| `loading` | file loading | load from file or http<br />require `./yamlutils`<br /> |
41+
| `mangling` | safe name generation | name mangling for `go`<br /> |
42+
| `netutils` | networking utilities | host, port from address<br /> |
43+
| `stringutils` | `string` utilities | search in slice (with case-insensitive)<br />split/join query parameters as arrays<br /> |
44+
| `typeutils` | `go` types utilities | check the zero value for any type<br />safe check for a nil value<br /> |
45+
| `yamlutils` | YAML utilities | converting YAML to JSON<br />loading YAML into a dynamic YAML document<br />maintaining the original order of keys in YAML objects<br />require `./jsonutils`<br />~require `github.com/mailru/easyjson`~<br />require `gopkg.in/yaml.v3`<br /> |
5546

56-
* [x] name mangling for go
47+
---
5748

58-
* Module `netutils`
49+
## Dependencies
5950

60-
* [x] host, port from address
51+
The root module `github.com/go-openapi/swag` at the repo level maintains a few
52+
dependencies outside of the standard library.
6153

62-
* Module `stringutils`
54+
* YAML utilities depend on `gopkg.in/yaml.v3`
55+
* JSON utilities depend on their registered adapter module:
56+
* by default, only the standard library is used
57+
* `github.com/mailru/easyjson` is now only a dependency for module
58+
`github.com/go-openapi/swag/jsonutils/adapters/easyjson/json`,
59+
for users willing to import that module.
6360

64-
* [x] search in slice (with case-insensitive)
65-
* [x] split/join query parameters as arrays
61+
## Release notes
6662

67-
* Module `typeutils`
63+
### v0.25.0 [draft, unreleased]
6864

69-
* [x] check the zero value for any type
65+
**New with this release**:
7066

71-
* Module `yamlutils`
67+
* requires `go1.24`, as iterators are being introduced
68+
* removes the dependency to `mailru/easyjson` by default
69+
* functionality remains the same, but performance may somewhat degrade for applications
70+
that relied on `easyjson`
71+
* users of the JSON or YAML utilities who want to use `easyjson` as their prefered JSON seriliazer library
72+
will be able to do so by registering this the corresponding JSON adapter at runtime. See below.
73+
* ordered keys in JSON and YAML objects: this feature used to rely solely on `easyjson`.
74+
With this release, an implementation relying on the standard `encoding/json` is provided.
7275

73-
* [x] converting YAML to JSON
74-
* [x] loading YAML into a dynamic YAML document
75-
* [x] require `./jsonutils`
76-
* [x] require `github.com/mailru/easyjson`
77-
* [x] require `gopkg.in/yaml.v3`
76+
**What coming next?**
7877

79-
---
78+
Moving forward, we want to :
79+
* provide an implementation of the JSON adapter based on `encoding/json/v2`, for `go1.25` builds.
80+
* provide similar implementations for `goccy/go-json` and `jsoniterator/go`, and perhaps some other
81+
similar libraries may be interesting too.
8082

81-
The root module `github.com/go-openapi/swag` at the repo level maintains a few
82-
dependencies outside of the standard library:
8383

84-
* YAML utilities depend on `gopkg.in/yaml.v3`
85-
* JSON utilities `github.com/mailru/easyjson`
84+
**How to explicitly register a dependency at runtime**?
8685

87-
This is not necessarily the case for all sub-modules.
86+
The following would maintain how JSON utilities proposed by `swag` used work, up to `v0.24.1`.
8887

89-
## Release notes
88+
```go
89+
import "github.com/go-openapi/swag/jsonutils/adapters/easyjson/json"
9090

91-
### v0.25.0 [draft, unreleased]
91+
func init() {
92+
json.Register()
93+
}
94+
```
9295

93-
* v0.25.0 will remove the dependency to `mailru/easyjson` by default.
94-
* users of JSON or YAML utility who want to use `easyjson` as their
95-
prefered JSON marshaler will be able to do so by registering it
96-
at runtime.
96+
Subsequent calls to `jsonutils.ReadJSON()` or `jsonutils.WriteJSON()` will switch to `easyjson`
97+
whenever the passed data structures implement the `easyjson.Unmarshaler` or `easyjson.Marshaler` respectively,
98+
or fallback to the standard library.
9799

98100
### v0.24.0
99101

@@ -121,14 +123,9 @@ With this release, we have largely modernized the API of `swag`:
121123

122124
---
123125

124-
Moving forward, no additional feature will be added to the `swag` API directly.
125-
126-
However, child modules will continue to evolve or some new ones may be added in the future.
127-
128-
129-
#### Note to contributors
126+
## Note to contributors
130127

131-
The mono-repo structure comes with some unavoidable extra pains...
128+
A mono-repo structure comes with some unavoidable extra pains...
132129

133130
* Testing
134131

@@ -153,19 +150,30 @@ The mono-repo structure comes with some unavoidable extra pains...
153150
> We'd like to adopt the rule that modules in this repo would only differ by a patch version
154151
> (e.g. `v0.24.5` vs `v0.24.3`), and we'll level all modules whenever a minor version is introduced.
155152
>
156-
> A script in `./hack` is provided to tag all modules in one go at the same level in one go.
153+
> A script in `./hack` is provided to tag all modules with the same version in one go.
154+
155+
* Continuous integration
156+
157+
> At this moment, all tests in all modules are systematically run over the full test matrix (3 platform x 2 go releases).
158+
> This generates quite a lot of jobs.
159+
>
160+
> We ought to reduce the number of jobs required to test a PR focused on only a few modules.
157161
158162
## Todos, suggestions and plans
159163

160164
All kinds of contributions are welcome.
161165

162166
A few ideas:
163167

164-
* [ ] Complete the split of dependencies to isolate easyjson from the rest
168+
* [x] Complete the split of dependencies to isolate easyjson from the rest
165169
* [ ] Improve mangling utilities (improve readability, support for capitalized words,
166170
better word substitution for non-letter symbols...)
167171
* [ ] Move back to this common shared pot a few of the technical features introduced by go-swagger independently
168172
(e.g. mangle go package names, search package with go modules support, ...)
169173
* [ ] Apply a similar mono-repo approach to go-openapi/strfmt which suffer from similar woes: bloated API,
170174
imposed dependency to some database driver.
175+
* [ ] Adapt `go-swagger` (incl. generated code) to the new `swag` API.
176+
* [ ] Improve CI to reduce needed tests
177+
* [ ] Replace dependency to `gopkg.in/yaml.v3` (`yamlutil`)
178+
* [ ] Factorize some tests, as there is a lot of redundant testing code in `jsonutils`
171179

‎cmdutils/go.mod‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
module github.com/go-openapi/swag/cmdutils
22

3-
go 1.20.0
3+
go 1.24.0

‎conv/go.mod‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,6 @@ require (
1212
gopkg.in/yaml.v3 v3.0.1 // indirect
1313
)
1414

15-
go 1.20.0
15+
go 1.24.0
1616

1717
replace github.com/go-openapi/swag/typeutils => ../typeutils

‎conv/go.sum‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,13 @@ github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfn
55
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
66
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
77
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
8+
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
89
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
910
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
1011
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
1112
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
1213
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
1314
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
15+
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
1416
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
1517
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

0 commit comments

Comments
 (0)