Skip to content

Commit b95cd5b

Browse files
Introduce a beta playback command (stripe#486)
`stripe playback` is a prototype feature for the Stripe CLI. It is still in beta. This project is inspired by the [VCR](https://github.com/vcr/vcr) approach to testing. This aims to make VCR-like testing for Stripe integrations easier and more accessible across our SDK languages, and provide a foundation for new Stripe-specific testing approaches in the future. The `stripe playback` command starts Stripe API proxy server on your local machine. By pointing your API calls to this server when running tests, users can record all of their interactions with the Stripe API. These recordings are saved to a serialized format, and can be replayed on all subsequent runs of those same test. Playing back recordings yields faster, more consistent test suites, that don't need to talk to the `api.stripe.com`. This can be useful for CI environments. Since `playback` runs as a separate HTTP proxy server on your machine, rather than a native library like VCR, it's harder to do configuration from inside tests. As part of our work, we're looking into ways to make this easier, such as providing wrapper packages for each SDK language. More info on the current functionality is in `pkg/playback/README.md`.
1 parent ad8d81f commit b95cd5b

14 files changed

+1949
-2
lines changed

.vscode/launch.json

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,15 @@
1515
"env": {},
1616
"args": ["listen", "--log-level", "debug"]
1717
},
18+
{
19+
"name": "Launch (playback)",
20+
"type": "go",
21+
"request": "launch",
22+
"mode": "auto",
23+
"program": "${workspaceFolder}/cmd/stripe/main.go",
24+
"env": {},
25+
"args": ["playback"]
26+
},
1827
{
1928
"name": "Set config",
2029
"type": "go",

go.mod

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ require (
3333
github.com/spf13/jwalterweatherman v1.1.0 // indirect
3434
github.com/spf13/pflag v1.0.5
3535
github.com/spf13/viper v1.6.3
36-
github.com/stretchr/testify v1.4.0
36+
github.com/stretchr/testify v1.5.1
3737
github.com/thedevsaddam/gojsonq v2.3.0+incompatible
3838
github.com/tidwall/gjson v1.6.0
3939
github.com/tidwall/pretty v1.0.1
@@ -43,4 +43,5 @@ require (
4343
golang.org/x/sys v0.0.0-20200501145240-bc7a7d42d5c3
4444
gopkg.in/ini.v1 v1.55.0 // indirect
4545
gopkg.in/src-d/go-git.v4 v4.13.1
46+
gopkg.in/yaml.v2 v2.2.8
4647
)

go.sum

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,8 @@ github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXf
235235
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
236236
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
237237
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
238+
github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4=
239+
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
238240
github.com/subosito/gotenv v1.2.0 h1:Slr1R9HxAlEKefgq5jn9U+DnETlIUa6HfgEzj0g5d7s=
239241
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
240242
github.com/thedevsaddam/gojsonq v2.3.0+incompatible h1:i2lFTvGY4LvoZ2VUzedsFlRiyaWcJm3Uh6cQ9+HyQA8=

pkg/cmd/playback.go

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
package cmd
2+
3+
import (
4+
"fmt"
5+
"net/url"
6+
"os"
7+
"path/filepath"
8+
9+
"github.com/spf13/cobra"
10+
11+
"github.com/stripe/stripe-cli/pkg/playback"
12+
"github.com/stripe/stripe-cli/pkg/validators"
13+
)
14+
15+
const defaultPort = 13111
16+
const defaultWebhookPort = 13112
17+
const endpointsDocString = `
18+
--- Controlling the server ---
19+
You can configure the running server instance via HTTP GET endpoints (prefixed with "/playback/").
20+
21+
--- List of server control endpoints ---
22+
POST /playback/mode/[mode]
23+
Sets the server mode to one of ["auto", "record", "replay"]
24+
25+
POST /playback/cassette/setroot?dir=[path_to_directory]
26+
Set the root directory for reading/writing cassettes. All cassette paths are relative to this directory.
27+
28+
POST /playback/cassette/load?filepath=[filepath]
29+
Load the cassette file at the given filepath, relative to the cassette root directory.
30+
31+
POST /playback/cassette/eject
32+
Eject (unload) the current cassette and do any teardown. In record mode, this writes the recorded interactions to the cassette file.
33+
`
34+
35+
type playbackCmd struct {
36+
cmd *cobra.Command
37+
38+
mode string
39+
40+
apiBaseURL string
41+
filepath string
42+
cassetteDir string
43+
address string
44+
webhookURL string
45+
}
46+
47+
func newPlaybackCmd() *playbackCmd {
48+
pc := &playbackCmd{}
49+
50+
pc.cmd = &cobra.Command{
51+
Hidden: true,
52+
Use: "playback",
53+
Args: validators.NoArgs,
54+
Short: "Start a `playback` server",
55+
Long: `
56+
--- Overview ---
57+
The playback command starts a local proxy server that intercepts outgoing requests to the Stripe API.
58+
59+
It can also intercept incoming webhooks on /playback/webhooks.
60+
61+
There are three modes of operation:
62+
63+
"record": Any requests received are forwarded to the api.stripe.com, and the response is returned. All interactions
64+
are written to the loaded 'cassette' file for later playback in replay mode.
65+
66+
"replay": All received requests are terminated at the playback server, and responses are played back[1] from a cassette file. A existing cassette most be loaded.
67+
68+
"auto" (default): The server determines whether to run in "record" or "replay" mode on a per-cassette basis. If the cassette exists, operates in "replay" mode. If not, operates in "record" mode.
69+
70+
Currently, stripe playback only supports serving over HTTP.
71+
72+
[1]: requests are currently replayed sequentially in the same order they were recorded.
73+
` + endpointsDocString,
74+
Example: `stripe playback
75+
stripe playback --mode replay
76+
stripe playback --cassette "my_cassette.yaml"`,
77+
RunE: pc.runPlaybackCmd,
78+
}
79+
80+
pc.cmd.Flags().StringVar(&pc.mode, "mode", "auto", "Auto: record if cassette doesn't exist, replay if exists. Record: always record/re-record. Replay: always replay.")
81+
pc.cmd.Flags().StringVar(&pc.address, "address", fmt.Sprintf("localhost:%d", defaultPort), "Address to serve on")
82+
pc.cmd.Flags().StringVar(&pc.webhookURL, "forward-to", fmt.Sprintf("http://localhost:%d", defaultWebhookPort), "URL to forward webhooks to")
83+
pc.cmd.Flags().StringVar(&pc.filepath, "cassette", "default_cassette.yaml", "The cassette file to use")
84+
pc.cmd.Flags().StringVar(&pc.cassetteDir, "cassette-root-dir", "./", "Directory to store all cassettes in. Relative cassette paths are considered relative to this directory.")
85+
86+
// // Hidden configuration flags, useful for dev/debugging
87+
pc.cmd.Flags().StringVar(&pc.apiBaseURL, "api-base", "https://api.stripe.com", "The API base URL")
88+
pc.cmd.Flags().MarkHidden("api-base") // #nosec G104
89+
90+
return pc
91+
}
92+
93+
func (pc *playbackCmd) runPlaybackCmd(cmd *cobra.Command, args []string) error {
94+
fmt.Println()
95+
fmt.Println("Setting up playback server...")
96+
fmt.Println()
97+
98+
// --- Validate command-line args
99+
// Check that mode is valid
100+
if pc.mode != playback.Auto && pc.mode != playback.Record && pc.mode != playback.Replay {
101+
return fmt.Errorf(
102+
"\"%v\" is not a valid mode. It must be either \"%v\", \"%v\", or \"%v\"",
103+
pc.mode, playback.Auto, playback.Record, playback.Replay)
104+
}
105+
106+
// Check that cassette root directory is valid
107+
absoluteCassetteDir, err := filepath.Abs(pc.cassetteDir)
108+
if err != nil {
109+
return fmt.Errorf("Error with --cassette-root-dir: %w", err)
110+
}
111+
112+
cassetteDirInfo, err := os.Stat(absoluteCassetteDir)
113+
if err != nil {
114+
if os.IsNotExist(err) {
115+
return fmt.Errorf("the directory \"%v\" does not exist. Please create it, then re-run the command", absoluteCassetteDir)
116+
}
117+
return fmt.Errorf("Unexpected error when checking --cassette-root-dir: %w", err)
118+
}
119+
120+
if !cassetteDirInfo.Mode().IsDir() {
121+
return fmt.Errorf("The provided `--cassette-root-dir` option is not a valid directory: %v", absoluteCassetteDir)
122+
}
123+
124+
// Check webhook URL specifies a protocol
125+
parsedWhURL, err := url.Parse(pc.webhookURL)
126+
if err != nil {
127+
return fmt.Errorf("unable to parse \"%v\" as a URL. it should be a valid URL of the form [scheme]://[host]:[port]", pc.webhookURL)
128+
}
129+
130+
if parsedWhURL.Scheme != "http" && parsedWhURL.Scheme != "https" {
131+
return fmt.Errorf("unsupported protocol scheme \"%v\". must be \"http\" or \"https\"", parsedWhURL.Scheme)
132+
}
133+
134+
// --- Start up the playback HTTP server
135+
// TODO(DX-5702): `playback` should handle setup (and teardown) of `stripe listen` as well
136+
addressString := pc.address
137+
remoteURL := pc.apiBaseURL
138+
139+
httpWrapper, err := playback.NewServer(remoteURL, pc.webhookURL, absoluteCassetteDir, pc.mode, pc.filepath)
140+
if err != nil {
141+
return err
142+
}
143+
144+
server := httpWrapper.InitializeServer(addressString)
145+
go func() {
146+
err = server.ListenAndServe()
147+
fmt.Fprint(os.Stderr, err.Error())
148+
os.Exit(1)
149+
}()
150+
151+
// --- Print out post-startup summary on CLI
152+
fmt.Println()
153+
fmt.Println("------ Server Running ------")
154+
155+
switch pc.mode {
156+
case playback.Record:
157+
fmt.Printf("In \"record\" mode.\n")
158+
fmt.Println("Will always record interactions, and write (or overwrite) to the given cassette filepath.")
159+
fmt.Println()
160+
case playback.Replay:
161+
fmt.Printf("In \"replay\" mode.\n")
162+
fmt.Println("Will always replay from the given cassette. Will error if loaded cassette path doesn't exist.")
163+
fmt.Println()
164+
case playback.Auto:
165+
fmt.Printf("In \"auto\" mode.\n")
166+
fmt.Println("Can both record or replay, depending on the file passed in. If exists, replays. If not, records.")
167+
fmt.Println()
168+
}
169+
170+
fmt.Printf("Cassettes directory: \"%v\".\n", absoluteCassetteDir)
171+
fmt.Printf("Using cassette: \"%v\".\n", pc.filepath)
172+
fmt.Println()
173+
174+
fmt.Printf("Listening via HTTP on %v\n", addressString)
175+
fmt.Println()
176+
177+
fmt.Printf("Accepting webhooks on %v/%v\n", addressString, "playback/webhooks")
178+
fmt.Printf("Forwarding webhooks to %v\n", pc.webhookURL)
179+
fmt.Println("-----------------------------")
180+
fmt.Println()
181+
182+
select {}
183+
}

pkg/cmd/root.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,7 @@ func init() {
110110
rootCmd.AddCommand(newStatusCmd().cmd)
111111
rootCmd.AddCommand(newTriggerCmd().cmd)
112112
rootCmd.AddCommand(newVersionCmd().cmd)
113+
rootCmd.AddCommand(newPlaybackCmd().cmd)
113114

114115
addAllResourcesCmds(rootCmd)
115116

pkg/cmd/templates.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ func getUsageTemplate() string {
124124
{{rpad "payment_intents" 29}} Make requests (cancel, capture, confirm, etc) on payment intents
125125
{{rpad "..." 29}} %s
126126
127-
%s{{range $index, $cmd := .Commands}}{{if (not (index $.Annotations $cmd.Name))}}
127+
%s{{range $index, $cmd := .Commands}}{{if (not (or (index $.Annotations $cmd.Name) $cmd.Hidden))}}
128128
{{rpad $cmd.Name $cmd.NamePadding}} {{$cmd.Short}}{{end}}{{end}}{{else}}
129129
130130
%s{{range .Commands}}{{if (or .IsAvailableCommand (eq .Name "help"))}}

pkg/playback/README.md

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# Introduction
2+
`stripe playback` is a prototype feature for the Stripe CLI. It is still under development. This project is inspired by the [VCR](https://github.com/vcr/vcr) approach to testing popularized in Ruby.
3+
4+
For context, when using the Ruby VCR gem, you record any HTTP interactions made in your test suite (request & response). These recordings are stored in serialized format, and can be replayed in future tests to make them "fast, deterministic, and accurate".
5+
6+
While Ruby VCR is implemented as a Gem that you can directly use and configure in your test code - `stripe playback` runs as a separate HTTP proxy server on your machine. That means any configuration happens either at startup via the CLI, or via interprocess HTTP calls.
7+
8+
This WIP document aims to give a new user enough information to begin using `stripe playback`.
9+
# Usage
10+
11+
You start and configure the server via the `stripe playback` command. See `stripe playback --help` for a description of the flags, how to control the server once it is running (via HTTP endpoints).
12+
13+
`go run cmd/stripe/main.go playback --help`
14+
15+
When running in both record/replay mode, the server will print out interactions with it (exact formatting of output may have changed).
16+
17+
```
18+
### When in recordmode (playing responses from the remote API)
19+
--> POST to /v1/customers
20+
<-- 200 OK from HTTPS://API.STRIPE.COM
21+
22+
--> GET to /v1/customers
23+
<-- 200 OK from HTTPS://API.STRIPE.COM
24+
25+
--> GET to /v1/balance
26+
<-- 200 OK from HTTPS://API.STRIPE.COM
27+
28+
```
29+
```
30+
### When in replaymode (playing responses from CASSETTE)
31+
--> POST to /v1/customers
32+
<-- 200 OK from CASSETTE
33+
34+
--> GET to /v1/customers
35+
<-- 200 OK from CASSETTE
36+
37+
--> GET to /v1/balance
38+
<-- 200 OK from CASSETTE
39+
```
40+
41+
## Controlling the playback server
42+
Besides the command line flags at startup, there are also HTTP endpoints that allow you to control and modify the server's behavior while it is running.
43+
44+
`POST:` `/playback/mode/[mode]`: Sets the server mode to one of ["auto", "record", "replay"].
45+
46+
`POST:` `/playback/cassette/setroot?dir=[path_to_directory]`: Set the root directory for reading/writing cassettes. All cassette paths are relative to this directory.
47+
48+
`POST:` `/playback/cassette/load?filepath=[filepath]`: Load the cassette file at the given filepath, relative to the cassette root directory.
49+
50+
`POST:` `/playback/casette/eject`: Eject (unload) the current cassette and do any teardown. In `record` mode or `auto` mode when recording to a new file, this writes the recorded interactions to the cassette file. When replaying (whether in `replay` or `auto` modes) this is a no-op.
51+
52+
53+
## Example
54+
### In Window 1:
55+
56+
`go run cmd/stripe/main.go playback`
57+
(Start a recordmode HTTP server at localhost:13111, writing to the default cassette)
58+
59+
### In Window 2:
60+
61+
Record some test interactions using the stripe CLI, but proxy through the `stripe playback` server:
62+
63+
`stripe customers create --api-base="http://localhost:13111"`
64+
65+
`stripe customers list --api-base="http://localhost:13111"`
66+
67+
`stripe balance retrieve --api-base="http://localhost:13111"`
68+
69+
Stop recording:
70+
71+
`curl http://localhost:13111/playback/stop`
72+
73+
### In Window 1:
74+
Ctrl-C the record server to shut it down.
75+
76+
Then, start the replay server, which should read from the same default cassette.
77+
78+
`go run cmd/stripe/main.go playback --replaymode`
79+
80+
### In Window 2:
81+
82+
Replay the same sequence of interactions using the stripe CLI, and notice that we are now replaying from the cassette:
83+
84+
`stripe customers create --api-base="http://localhost:13111"`
85+
86+
`stripe customers list --api-base="http://localhost:13111"`
87+
88+
`stripe balance retrieve --api-base="http://localhost:13111"`
89+
90+
91+
## [WIP] Webhooks
92+
Webhooks are a WIP feature, but currently have basic functionality working. If you don't plan to make use of webhook recording/replaying, you can ignore this section.
93+
94+
Skeleton demo of functionality:
95+
96+
```
97+
# Terminal 1
98+
# Start the playback server using the default settings. (in record mode, and using the default ports)
99+
100+
> go run cmd/stripe/main.go playback
101+
102+
Seting up playback server...
103+
104+
/playback/mode/: Setting mode to record
105+
/playback/cassette/load: Loading cassette [default_cassette.yaml]
106+
107+
------ Server Running ------
108+
Recording...
109+
Using cassette: "default_cassette.yaml".
110+
111+
Listening via HTTP on localhost:13111
112+
Forwarding webhooks to http://localhost:13112
113+
-----------------------------
114+
115+
```
116+
117+
```
118+
# Terminal 2
119+
# Use stripe listen to forward webhooks to the playback server's webhook endpoint
120+
121+
> stripe listen --forward-to localhost:13111/playback/webhooks
122+
```
123+
124+
125+
```
126+
# Terminal 3
127+
# This effectively sets up a basic HTTP server on localhost:13112 that will echo out all requests
128+
# and always respond with a 200 status code.
129+
130+
> socat -v -s tcp-listen:13112,reuseaddr,fork "exec:printf \'HTTP/1.0 200 OK\r\n\r\n\'"
131+
```
132+
Finally, use the Stripe CLI to send requests and trigger webhooks to the playback server.
133+
```
134+
# Terminal 4
135+
136+
# Send a normal request
137+
stripe balance retrieve --api-base "http://localhost:13111"
138+
139+
# Trigger webhooks afterwards
140+
stripe trigger payment_intent.created
141+
142+
```
143+
You should see the `socat` server in Terminal 3 receive the forwarded webhooks. You should also see the `playback` server logging (and recording) all interactions (outbound API requests and inbound webhooks).
144+
145+
After all this, you can re-run the server in replay mode:
146+
147+
`go run cmd/stripe/main.go playback --replaymode`
148+
149+
Then, rerun the same commands in the same order in Terminal 4, and you should see **recorded** responses **and** webhooks being returned to your Stripe CLI client and to your `socat` server.
150+
151+
152+
# Testing
153+
Some of the tests require a `.env` file to be present at `/pkg/playback/.env` containing
154+
`STRIPE_SECRET_KEY="sk_test_..."`. To run the tests, create this file and define your own secret testmode key in it.

0 commit comments

Comments
 (0)