added raw urls support - on parsing and sending sending requests #42
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Since I liked your tool, I tried it and realized that it is not working correctly. Looking through the code, I saw that you use url.UrlParse to parse the URL then I checked the function that you are using to send requests.
Go is "pro-security" and functions like UrlParse will apply encodings, normalizations, etc, meaning that your payloads from midpaths and endpaths won't work.
On the other hand, Go's
http.Client
will also apply encodings, normalizations, in the idea to prevent vulnerabilities such as path traversal, etc.I researched, and there is no URL parser in go that can return raw urls/paths.
For this, I wrote a simple pkg that I published here: https://github.com/slicingmelon/go-rawurlparser.
I forked your tool, made the modifications to use my pkg to parse URLs, and then patched the request() func to be able to send all the payloads as they should be sent, similar to what
curl --path-as-is
does. I found that the only way to still usehttp.Request
and send raw URLs is to set the Opaque to the raw path.I've tested it, and it looks like it's working well now; I also intercepted the traffic with Burp, and all the payloads are sent as expected.
If for any reason you want back http.NewRequest instead of http.Request, you can achieve the same thing, too; just don't forget to set the Opaque.