Skip to content

Commit

Permalink
chore(allsrv): add github.com/jsteenb2/allsrvc SDK module
Browse files Browse the repository at this point in the history 
We fixup our http client to make use of the
[github.com/jsteenb2/allsrvc](https://github.com/jsteenb2/allsrvc)
SDK. As you, we can clean up a good bit of duplication by utilizing
the SDK as a source of truth for the API types. We've broken up the SDK
from the service/server module. Effectively breaking one of the thorniest
problems large organizations with a large go ecosystem face.

When we leave the SDK inside the service module, its forces all the
depdencies of the service onto any SDK consumer. This creates a series
of problems.

1. The SDK creates a ton of bloat in the user's module.
2. The SDK undergoes a lot of version changes when coupled to the service module version.
3. Circular module dependencies are real, and can cause a LOT of pain.
    * Check out [perseus](https://github.com/CrowdStrike/perseus) to help visualize this!
4. If you do it this way, then other teams will also do it this way, putting tremendous pressure on your CI/build pipelines.

Instead of exporting an SDK from your service, opt for a separate module
for the SDK. This radically changes the game. You can use the SDK module
in the `Service` module to remain *DRY*. However, **DO NOT** import the
`Service` module into the SDK module!

Now that we have the tiny SDK module, we're able to obtain some important
data to help us track who is hitting our API. We now get access to the `Origin`
and `User-Agent` of the callee. Here is an example of a log that
adds the [version of the module](https://github.com/jsteenb2/allsrvc/blob/main/client.go#L21-L30)
as part of `User-Agent` and `Origin` headers when communicating with the server:

```json
{
  "time": "2024-07-06T20:46:58.614226-05:00",
  "level": "ERROR",
  "source": {
    "function": "github.com/jsteenb2/mess/allsrv.(*svcMWLogger).CreateFoo",
    "file": "github.com/jsteenb2/mess/allsrv/svc_mw_logging.go",
    "line": 32
  },
  "msg": "failed to create foo",
  "input_name": "name1",
  "input_note": "note1",
  "took_ms": "0s",
  "origin": "allsrvc",
  "user_agent": "allsrvc (github.com/jsteenb2/allsrvc) / v0.4.0",
  "trace_id": "b9106e52-907b-4bc4-af91-6596e98d3795",
  "err": "foo name1 exists",
  "err_fields": {
    "name": "name1",
    "existing_foo_id": "3a826632-ec30-4852-b4a6-e4a4497ddda8",
    "err_kind": "exists",
    "stack_trace": [
      "github.com/jsteenb2/mess/allsrv/svc.go:97[(*Service).CreateFoo]",
      "github.com/jsteenb2/mess/allsrv/db_inmem.go:20[(*InmemDB).CreateFoo]"
    ]
  }
}
```

With this information, we're in a good position to make proactive changes
to remove our own blockers. Excellent stuff!

Additionally, we've imported our SDK into the `Service` module to *DRY* up
the HTTP API contract. No need to duplicate these things as the server is
dependent on the http client's JSON API types. This is awesome, as we're
still able to keep things DRY, without all the downside of the SDK depending
on the Service (i.e. dependency bloat).

Lastly, we update the CLI to include basic auth. Try exercising the new updates.
Use the CLI to issue some CRUD commands against the server. Start the server
first with:

```shell
go run ./allsrv/cmd/allsrv | jq
```

Then you can install the CLI and make sure to add `$GOBIN` to your `$PATH`:

```shell
go install ./allsrv/cmd/allsrvc
```

Now issue a request to create a foo:

```shell
allsrvc create --name first --note "some note"
```

Now issue another create a foo with the same name:

```shell
allsrvc create --name first --note "some other note"
```

The previous command should fail. Check out the output from the `allsrvc`
CLI as well as the logs from the server. Enjoy those beautiful logs!

This marks the end of our time with the `allsrv` package!

Refs: [SDK module - github.com/jsteenb2/allsrvc](https://github.com/jsteenb2/allsrvc)
Refs: [Setting version in SDK via debug.BuildInfo](https://github.com/jsteenb2/allsrvc/blob/main/client.go#L21-L30)
Refs: [Perseus module tracker](https://github.com/CrowdStrike/perseus)
  • Loading branch information
@jsteenb2
jsteenb2 committed Jul 10, 2024
1 parent baf5cd9 commit 9d41492
Show file tree
Hide file tree
Showing 8 changed files with 323 additions and 417 deletions.
185 changes: 60 additions & 125 deletions allsrv/client_http.go
View file
Original file line number Diff line number Diff line change
@@ -1,182 +1,109 @@
package allsrv

import (
"bytes"
"context"
"encoding/json"
"io"
"net/http"
"time"

"github.com/jsteenb2/errors"

"github.com/jsteenb2/allsrvc"
)

type ClientHTTP struct {
addr string
c *http.Client
c *allsrvc.ClientHTTP
}

var _ SVC = (*ClientHTTP)(nil)

func NewClientHTTP(addr string, c *http.Client) *ClientHTTP {
func NewClientHTTP(addr, origin string, c *http.Client, opts ...func(*allsrvc.ClientHTTP)) *ClientHTTP {
return &ClientHTTP{
addr: addr,
c: c,
c: allsrvc.NewClientHTTP(addr, origin, c, opts...),
}
}

func (c *ClientHTTP) CreateFoo(ctx context.Context, f Foo) (Foo, error) {
req, err := jsonReq(ctx, "POST", c.fooPath(""), toReqCreateFooV1(f))
resp, err := c.c.CreateFoo(ctx, allsrvc.FooCreateAttrs{
Name: f.Name,
Note: f.Note,
})
if err != nil {
return Foo{}, InternalErr(err.Error())
}
return returnsFooReq(c.c, req)
newFoo, err := takeRespFoo(resp)
return newFoo, errors.Wrap(err)
}

func (c *ClientHTTP) ReadFoo(ctx context.Context, id string) (Foo, error) {
if id == "" {
return Foo{}, errIDRequired
}

req, err := http.NewRequestWithContext(ctx, "GET", c.fooPath(id), nil)
resp, err := c.c.ReadFoo(ctx, id)
if err != nil {
return Foo{}, InternalErr(err.Error())
if errors.Is(err, allsrvc.ErrIDRequired) {
return Foo{}, errIDRequired
}
}
return returnsFooReq(c.c, req)

newFoo, err := takeRespFoo(resp)
return newFoo, errors.Wrap(err)
}

func (c *ClientHTTP) UpdateFoo(ctx context.Context, f FooUpd) (Foo, error) {
req, err := jsonReq(ctx, "PATCH", c.fooPath(f.ID), toReqUpdateFooV1(f))
resp, err := c.c.UpdateFoo(ctx, f.ID, allsrvc.FooUpdAttrs{
Name: f.Name,
Note: f.Note,
})
if err != nil {
return Foo{}, InternalErr(err.Error())
}
return returnsFooReq(c.c, req)
newFoo, err := takeRespFoo(resp)
return newFoo, errors.Wrap(err)
}

func (c *ClientHTTP) DelFoo(ctx context.Context, id string) error {
if id == "" {
return errIDRequired
}

req, err := http.NewRequestWithContext(ctx, "DELETE", c.fooPath(id), nil)
resp, err := c.c.DelFoo(ctx, id)
if err != nil {
return InternalErr(err.Error())
}

_, err = doReq[any](c.c, req)
return err
}

func (c *ClientHTTP) fooPath(id string) string {
u := c.addr + "/v1/foos"
if id == "" {
return u
}
return u + "/" + id
}

func jsonReq(ctx context.Context, method, path string, v any) (*http.Request, error) {
var buf bytes.Buffer
if err := json.NewEncoder(&buf).Encode(v); err != nil {
return nil, InvalidErr("failed to marshal payload: " + err.Error())
}

req, err := http.NewRequestWithContext(ctx, method, path, &buf)
if err != nil {
return nil, err
if errors.Is(err, allsrvc.ErrIDRequired) {
return errIDRequired
}
}
req.Header.Set("Content-Type", "application/json")

return req, nil
return errors.Wrap(convertSDKErrors(resp.Errs))
}

func returnsFooReq(c *http.Client, req *http.Request) (Foo, error) {
data, err := doReq[ResourceFooAttrs](c, req)
if err != nil {
return Foo{}, err
func DataToFoo(data allsrvc.Data[allsrvc.ResourceFooAttrs]) Foo {
return Foo{
ID: data.ID,
Name: data.Attrs.Name,
Note: data.Attrs.Note,
CreatedAt: toTime(data.Attrs.CreatedAt),
UpdatedAt: toTime(data.Attrs.UpdatedAt),
}
return toFoo(data), nil
}

func doReq[Attr Attrs](c *http.Client, req *http.Request) (Data[Attr], error) {
resp, err := c.Do(req)
if err != nil {
return *new(Data[Attr]), InternalErr(err.Error())
}
defer func() {
io.Copy(io.Discard, resp.Body)
resp.Body.Close()
}()

if resp.Header.Get("Content-Type") != "application/json" {
b, err := io.ReadAll(io.LimitReader(resp.Body, 500<<10))
if err != nil {
return *new(Data[Attr]), InternalErr("failed to read response body: ", err.Error())
}
return *new(Data[Attr]), InternalErr("invalid content type received; content=" + string(b))
}
// TODO(berg): handle unexpected status code (502|503|etc)

var respBody RespBody[Attr]
err = json.NewDecoder(resp.Body).Decode(&respBody)
if err != nil {
return *new(Data[Attr]), InternalErr(err.Error())
}

var errs []error
for _, respErr := range respBody.Errs {
errs = append(errs, toErr(respErr))
}
if len(errs) == 1 {
return *new(Data[Attr]), errs[0]
}
if len(errs) > 1 {
return *new(Data[Attr]), errors.Join(errs)
func takeRespFoo(respBody allsrvc.RespBody[allsrvc.ResourceFooAttrs]) (Foo, error) {
if err := convertSDKErrors(respBody.Errs); err != nil {
return Foo{}, errors.Wrap(err)
}

if respBody.Data == nil {
return *new(Data[Attr]), nil
return Foo{}, nil
}

return *respBody.Data, nil
return DataToFoo(*respBody.Data), nil
}

func toReqCreateFooV1(f Foo) ReqCreateFooV1 {
return ReqCreateFooV1{
Data: Data[FooCreateAttrs]{
Type: "foo",
Attrs: FooCreateAttrs{
Name: f.Name,
Note: f.Note,
},
},
func convertSDKErrors(errs []allsrvc.RespErr) error {
// TODO(@berg): update this to slices pkg when 1.23 lands
switch out := toSlc(errs, toErr); {
case len(out) == 1:
return out[0]
case len(out) > 1:
return errors.Join(out)
default:
return nil
}
}

func toReqUpdateFooV1(f FooUpd) ReqUpdateFooV1 {
return ReqUpdateFooV1{
Data: Data[FooUpdAttrs]{
Type: "foo",
ID: f.ID,
Attrs: FooUpdAttrs{
Name: f.Name,
Note: f.Note,
},
},
}
}

func toFoo(d Data[ResourceFooAttrs]) Foo {
return Foo{
ID: d.ID,
Name: d.Attrs.Name,
Note: d.Attrs.Note,
CreatedAt: toTime(d.Attrs.CreatedAt),
UpdatedAt: toTime(d.Attrs.UpdatedAt),
}
}

func toErr(respErr RespErr) error {
func toErr(respErr allsrvc.RespErr) error {
errFn := InternalErr
switch respErr.Code {
case errCodeExist:
Expand All @@ -199,3 +126,11 @@ func toTime(in string) time.Time {
t, _ := time.Parse(time.RFC3339, in)
return t
}

func toSlc[In, Out any](in []In, to func(In) Out) []Out {
out := make([]Out, len(in))
for _, v := range in {
out = append(out, to(v))
}
return out
}
45 changes: 35 additions & 10 deletions allsrv/cmd/allsrvc/main.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,15 @@ package main

import (
"encoding/json"
"io"
"net/http"
"os"
"time"

"github.com/jsteenb2/errors"
"github.com/spf13/cobra"

"github.com/jsteenb2/allsrvc"
"github.com/jsteenb2/mess/allsrv"
)

Expand All @@ -23,16 +26,24 @@ func newCmd() *cobra.Command {
return c.cmd()
}

const name = "allsrvc"

type cli struct {
// base flags
addr string
pass string
user string

// foo flags
id string
name string
note string
}

func (c *cli) cmd() *cobra.Command {
cmd := cobra.Command{
Use: "allsrvc",
Use: name,
SilenceUsage: true,
}

cmd.AddCommand(
Expand Down Expand Up @@ -65,7 +76,7 @@ func (c *cli) cmdCreateFoo() *cobra.Command {
return json.NewEncoder(cmd.OutOrStderr()).Encode(f)
},
}
cmd.Flags().StringVar(&c.addr, "addr", "http://localhost:8091", "addr for foo svc")
c.registerCommonFlags(&cmd)
cmd.Flags().StringVar(&c.name, "name", "", "name of the new foo")
cmd.Flags().StringVar(&c.note, "note", "", "optional foo note")

Expand All @@ -85,11 +96,10 @@ func (c *cli) cmdReadFoo() *cobra.Command {
return err
}

return json.NewEncoder(cmd.OutOrStderr()).Encode(f)
return errors.Wrap(writeFoo(cmd.OutOrStdout(), f))
},
}
cmd.Flags().StringVar(&c.addr, "addr", "http://localhost:8091", "addr for foo svc")

c.registerCommonFlags(&cmd)
return &cmd
}

Expand All @@ -116,10 +126,10 @@ func (c *cli) cmdUpdateFoo() *cobra.Command {
return err
}

return json.NewEncoder(cmd.OutOrStderr()).Encode(f)
return errors.Wrap(writeFoo(cmd.OutOrStdout(), f))
},
}
cmd.Flags().StringVar(&c.addr, "addr", "http://localhost:8091", "addr for foo svc")
c.registerCommonFlags(&cmd)
cmd.Flags().StringVar(&c.id, "id", "", "id of the foo resource")
cmd.Flags().StringVar(&c.name, "name", "", "optional foo name")
cmd.Flags().StringVar(&c.note, "note", "", "optional foo note")
Expand All @@ -137,11 +147,26 @@ func (c *cli) cmdRmFoo() *cobra.Command {
return client.DelFoo(cmd.Context(), args[0])
},
}
cmd.Flags().StringVar(&c.addr, "addr", "http://localhost:8091", "addr for foo svc")

c.registerCommonFlags(&cmd)
return &cmd
}

func (c *cli) newClient() *allsrv.ClientHTTP {
return allsrv.NewClientHTTP(c.addr, &http.Client{Timeout: 5 * time.Second})
return allsrv.NewClientHTTP(
c.addr,
name,
&http.Client{Timeout: 5 * time.Second},
allsrvc.WithBasicAuth(c.user, c.pass),
)
}

func (c *cli) registerCommonFlags(cmd *cobra.Command) {
cmd.Flags().StringVar(&c.addr, "addr", "http://localhost:8091", "addr for foo svc")
cmd.Flags().StringVar(&c.user, "user", "admin", "user for basic auth")
cmd.Flags().StringVar(&c.pass, "password", "pass", "password for basic auth")
}

func writeFoo(w io.Writer, f allsrv.Foo) error {
err := json.NewEncoder(w).Encode(allsrv.FooToData(f))
return errors.Wrap(err)
}
5 changes: 3 additions & 2 deletions allsrv/cmd/allsrvc/main_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"net/http/httptest"
"testing"

"github.com/jsteenb2/allsrvc"
"github.com/jsteenb2/mess/allsrv"
"github.com/jsteenb2/mess/allsrv/allsrvtesting"
)
Expand Down Expand Up @@ -57,12 +58,12 @@ func (c *cmdCLI) expectFoo(ctx context.Context, op string, args ...string) (alls
return allsrv.Foo{}, err
}

var out allsrv.Foo
var out allsrvc.Data[allsrvc.ResourceFooAttrs]
if err := json.Unmarshal(b, &out); err != nil {
return allsrv.Foo{}, err
}

return out, nil
return allsrv.DataToFoo(out), nil
}

func (c *cmdCLI) execute(ctx context.Context, op string, args ...string) ([]byte, error) {
Expand Down
Loading

0 comments on commit 9d41492

Please sign in to comment.