Skip to content

Commit

Permalink
Merge branch 'burrowers:master' into master
Browse files Browse the repository at this point in the history
  • Loading branch information
@FrontMage
FrontMage authored May 16, 2024
2 parents 425899f + c41f026 commit 09822ab
Show file tree
Hide file tree
Showing 77 changed files with 7,310 additions and 1,258 deletions.
64 changes: 35 additions & 29 deletions .github/workflows/test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,59 +11,65 @@ on:
# as it runs many builds under the hood.
# The default -timeout=10m can be hit by the hosted runners.
#
# Also note that we don't use actions/cache for Go on purpose.
# Also note that we don't use Actions caching for Go on purpose.
# Caching GOMODCACHE wouldn't help much, as we have few deps.
# Caching GOCACHE would do more harm than good,
# as the tests redo most of their work if the garble version changes,
# and the majority of commits or PRs will do so.
# Moreover, GitHub saves and restores caches via compressed archives over the
# network, which means it can easily add one minute of overhead on its own for
# just a few hundred megabytes worth of files.

name: Test
jobs:
test:
strategy:
matrix:
go-version: [1.20.x]
# TODO: revert to macos-latest once we figure out https://github.com/burrowers/garble/issues/609.
os: [ubuntu-latest, macos-11, windows-latest]
go-version: [1.22.x]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/setup-go@v3
- uses: actions/checkout@v4
- uses: actions/setup-go@v4
with:
go-version: ${{ matrix.go-version }}
- uses: actions/checkout@v3
- name: Test
cache: false
# linux already runs the tests with -race below, plus extra checks,
# so skip the regular tests to prevent it from taking the longest.
- if: matrix.os != 'ubuntu-latest'
run: go test -timeout=15m ./...
# macos and windows failures with bincmp can be hard to reproduce locally,
# so upload the binaries as artifacts.
- uses: actions/upload-artifact@v3
if: failure()
with:
name: bincmp_output
path: bincmp_output/
- name: Test with -race
# macos and windows tend to be a bit slower,
# and it's rare that a race in garble would be OS-specific.
if: matrix.os == 'ubuntu-latest'
# macos and windows tend to be a bit slower,
# and it's rare that a race in garble would be OS-specific.
- if: matrix.os == 'ubuntu-latest'
run: go test -race -timeout=20m ./...

# Static checks from this point forward. Only run on one Go version and on
# Linux, since it's the fastest platform, and the tools behave the same.
# linux, since it's the fastest platform, and the tools behave the same.
- name: Test third-party project builds
if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.20.x'
if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.22.x'
run: |
go install
./scripts/check-third-party.sh
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.20.x'
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.22.x'
run: ./scripts/crlf-test.sh
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.20.x'
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.22.x'
run: diff <(echo -n) <(gofmt -d .)
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.20.x'
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.22.x'
run: go vet ./...
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.20.x'
- if: matrix.os == 'ubuntu-latest' && matrix.go-version == '1.22.x'
uses: dominikh/staticcheck-action@v1
with:
version: "2023.1.2"
version: "2023.1.6"
install-go: false

# We don't care about GOARCH=386 particularly,
# We don't care about GOARCH=386 particularly, hence -short,
# but it helps ensure we support 32-bit hosts and targets well.
# TODO: use CGO_ENABLED=1 once we figure out how to install gcc-multilib,
# and once gotooltest forwards the value of CGO_ENABLED to the scripts.
Expand All @@ -72,32 +78,32 @@ jobs:
env:
GOARCH: 386
steps:
- uses: actions/setup-go@v3
- uses: actions/checkout@v4
- uses: actions/setup-go@v4
with:
go-version: 1.20.x
- uses: actions/checkout@v3
- name: Test
run: go test -timeout=15m ./...
go-version: 1.22.x
cache: false
- run: go test -short ./...

test-gotip:
if: false # let tip for 1.23 settle first
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Go
env:
GO_COMMIT: 70f98a251efdbfd619c4ff466a43da299ad04752 # 2023-03-11
GO_COMMIT: 2184a394777ccc9ce9625932b2ad773e6e626be0 # 2023-12-21
run: |
cd $HOME
mkdir $HOME/gotip
cd $HOME/gotip
wget -O gotip.tar.gz https://go.googlesource.com/go/+archive/${GO_COMMIT}.tar.gz
tar -xf gotip.tar.gz
echo "devel go1.21-${GO_COMMIT}" >VERSION
echo "devel go1.22-${GO_COMMIT}" >VERSION
cd src
./make.bash
echo "GOROOT=$HOME/gotip" >>$GITHUB_ENV
echo "$HOME/gotip/bin" >>$GITHUB_PATH
- uses: actions/checkout@v3
- name: Test
run: go test -timeout=15m ./...
- run: go test -timeout=15m ./...
4 changes: 3 additions & 1 deletion AUTHORS
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,10 @@

Andrew LeFevre <jalefevre@liberty.edu>
Daniel Martí <mvdan@mvdan.cc>
Emmanuel Chee-zaram Okeke <ecokeke21@gmail.com>
Nicholas Jones <me@nicholasjon.es>
Zachary Wasserman <zachwass2000@gmail.com>
lu4p <lu4p@pm.me>
pagran <pagran@protonmail.com>
shellhazard <shellhazard@tutanota.com>
shellhazard <shellhazard@tutanota.com>
xuannv <xuan11290@gmail.com>
115 changes: 115 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,87 @@
# Changelog

## [v0.12.1] - 2024-02-18

This bugfix release fixes a regression in v0.12.0 that broke `x/sys/unix`.
See #830.

## [v0.12.0] - 2024-02-10

This release continues support for Go 1.21 and includes fixes for Go 1.22,
now that the final 1.22.0 release is out.

@lu4p improved the detection of types used with reflection to track `make` calls too,
fixing more `cannot use T1 as T2` errors when obfuscating types. See [#690].

@pagran added a trash block generator to the control flow obfuscator.
See [#825].

A number of bugfixes are also included:
* Avoid an error when building for `GOOS=ios` - [#816]
* Prevent the shuffle literal obfuscation from being optimized away - [#819]
* Support inline comments in assembly `#include` lines - [#812]

## [v0.11.0] - 2023-12-02

This release drops support for Go 1.20, continues support for Go 1.21,
and adds initial support for the upcoming Go 1.22.

@lu4p and @mvdan improved the code using SSA to detect which types are used with reflection,
which should fix a number of errors such as `cannot use T1 as T2` or `cannot convert T1 to T2`.
See: [#685], [#763], [#782], [#785], [#807].

@pagran added experimental support for control flow obfuscation,
which should provide stronger obfuscation of function bodies when enabled.
See the documentation at [docs/CONTROLFLOW.md](https://github.com/burrowers/garble/blob/master/docs/CONTROLFLOW.md).
See [#462].

A number of bugfixes are also included:

* Avoid panicking on a struct embedding a builtin alias - [#798]
* Strip struct field tags when hashing struct types for type identity - [#801]

## [v0.10.1] - 2023-06-25

This bugfix release continues support for Go 1.20 and the upcoming 1.21,
and features:

* Avoid obfuscating local types used for reflection, like in `go-spew` - #765

## [v0.10.0] - 2023-06-05

This release drops support for Go 1.19, continues support for Go 1.20,
and adds initial support for the upcoming Go 1.21.

@lu4p rewrote the code to detect whether `reflect` is used on each Go type,
which is used to decide which Go types should not be obfuscated to prevent breakage.
The old code analyzed syntax trees with type information, which is cheap but clumsy.
The new code uses SSA, which adds a bit of CPU cost to builds, but allows for a
more powerful analysis that is less likely to break on edge cases.
While this change does slow down builds slightly, we will start using SSA for more
features in the near term, such as control flow obfuscation. See [#732].

@pagran improved the patching of Go's linker to also obfuscate funcInfo.entryoff,
making it harder to relate a function's metadata with its body in the binary. See [#641].

@mvdan rewrote garble's caching to be more robust, avoiding errors such as
"cannot load garble export file". The new caching system is entirely separate
from Go's `GOCACHE`, being placed in `GARBLE_CACHE`, which defaults to a directory
such as `~/.cache/garble`. See [#708].

@DominicBreuker taught `-literals` to support obfuscating large string literals
by using the "simple" obfuscator on them, as it runs in linear time. See [#720].

@mvdan added support for `garble run`, the obfuscated version of `go run`,
to quickly test that a main program still works when obfuscated. See [#661].

A number of bugfixes are also included:

* Ensure that `sync/atomic` types are still aligned by the compiler - [#686]
* Print the chosen random seed when using `-seed=random` - [#696]
* Avoid errors in `git apply` if the system language isn't English - [#698]
* Avoid a panic when importing a missing package - [#694]
* Suggest a command when asking the user to rebuild garble - [#739]

## [v0.9.3] - 2023-02-12

This bugfix release continues support for Go 1.19 and 1.20, and features:
Expand Down Expand Up @@ -200,6 +282,39 @@ Known bugs:
* obfuscating the standard library with `GOPRIVATE=*` is not well supported yet
* `garble test` is temporarily disabled, as it is currently broken

[v0.12.1]: https://github.com/burrowers/garble/releases/tag/v0.12.1

[v0.12.0]: https://github.com/burrowers/garble/releases/tag/v0.12.0
[#690]: https://github.com/burrowers/garble/issues/690
[#812]: https://github.com/burrowers/garble/issues/812
[#816]: https://github.com/burrowers/garble/pull/816
[#819]: https://github.com/burrowers/garble/pull/819
[#825]: https://github.com/burrowers/garble/pull/825

[v0.11.0]: https://github.com/burrowers/garble/releases/tag/v0.11.0
[#462]: https://github.com/burrowers/garble/issues/462
[#685]: https://github.com/burrowers/garble/issues/685
[#763]: https://github.com/burrowers/garble/issues/763
[#782]: https://github.com/burrowers/garble/issues/782
[#785]: https://github.com/burrowers/garble/issues/785
[#798]: https://github.com/burrowers/garble/issues/798
[#801]: https://github.com/burrowers/garble/issues/801
[#807]: https://github.com/burrowers/garble/issues/807

[v0.10.1]: https://github.com/burrowers/garble/releases/tag/v0.10.1

[v0.10.0]: https://github.com/burrowers/garble/releases/tag/v0.10.0
[#641]: https://github.com/burrowers/garble/pull/641
[#661]: https://github.com/burrowers/garble/issues/661
[#686]: https://github.com/burrowers/garble/issues/686
[#694]: https://github.com/burrowers/garble/issues/694
[#696]: https://github.com/burrowers/garble/issues/696
[#698]: https://github.com/burrowers/garble/issues/698
[#708]: https://github.com/burrowers/garble/issues/708
[#720]: https://github.com/burrowers/garble/pull/720
[#732]: https://github.com/burrowers/garble/pull/732
[#739]: https://github.com/burrowers/garble/pull/739

[v0.9.3]: https://github.com/burrowers/garble/releases/tag/v0.9.3
[#672]: https://github.com/burrowers/garble/issues/672
[#675]: https://github.com/burrowers/garble/pull/675
Expand Down
56 changes: 51 additions & 5 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,11 @@ Obfuscate Go code by wrapping the Go toolchain. Requires Go 1.20 or later.
garble build [build flags] [packages]

The tool also supports `garble test` to run tests with obfuscated code,
`garble run` to obfuscate and execute simple programs,
and `garble reverse` to de-obfuscate text such as stack traces.
See `garble -h` for up to date usage information.
Run `garble -h` to see all available commands and flags.

You can also use `go install mvdan.cc/garble@master` to install the latest development version.

### Purpose

Expand Down Expand Up @@ -50,6 +53,29 @@ $ go1.17.1 download
$ PATH=$(go1.17.1 env GOROOT)/bin:${PATH} garble build
```

### Use cases

A common question is why a code obfuscator is needed for Go, a compiled language.
Go binaries include a surprising amount of information about the original source;
even with debug information and symbol tables stripped, many names and positions
remain in place for the sake of traces, reflection, and debugging.

Some use cases for Go require sharing a Go binary with the end user.
If the source code for the binary is private or requires a purchase,
its obfuscation can help discourage reverse engineering.

A similar use case is a Go library whose source is private or purchased.
Since Go libraries cannot be imported in binary form, and Go plugins
[have their shortcomings](https://github.com/golang/go/issues/19282),
sharing obfuscated source code becomes an option.
See [#369](https://github.com/burrowers/garble/issues/369).

Obfuscation can also help with aspects entirely unrelated to licensing.
For example, the `-tiny` flag can make binaries 15% smaller,
similar to the [common practice in Android](https://developer.android.com/build/shrink-code#obfuscate) to reduce app sizes.
Obfuscation has also helped some open source developers work around
anti-virus scans incorrectly treating Go binaries as malware.

### Literal obfuscation

Using the `-literals` flag causes literal expressions such as strings to be
Expand Down Expand Up @@ -78,6 +104,10 @@ exit the entire program without printing a stack trace, and source code
positions and many names are removed.
Similarly, `garble reverse` is generally not useful in this mode.

### Control flow obfuscation

See: [CONTROLFLOW.md](docs/CONTROLFLOW.md)

### Speed

`garble build` should take about twice as long as `go build`, as it needs to
Expand All @@ -92,6 +122,10 @@ Note that the first call to `garble build` may be comparatively slow,
as it has to obfuscate each package for the first time. This is akin to clearing
`GOCACHE` with `go clean -cache` and running a `go build` from scratch.

Garble also makes use of its own cache to reuse work, akin to Go's `GOCACHE`.
It defaults to a directory under your user's cache directory,
such as `~/.cache/garble`, and can be placed elsewhere by setting `GARBLE_CACHE`.

### Determinism and seeds

Just like Go, garble builds are deterministic and reproducible in nature.
Expand Down Expand Up @@ -123,11 +157,11 @@ to document the current shortcomings of this tool.
be required by interfaces. This area is a work in progress; see
[#3](https://github.com/burrowers/garble/issues/3).

* Garble aims to automatically detect which Go types are used with reflection,
as obfuscating those types might break your program.
* Garble automatically detects which Go types are used with reflection
to avoid obfuscating them, as that might break your program.
Note that Garble obfuscates [one package at a time](#speed),
so if your reflection code inspects a type from an imported package,
and your program broke, you may need to add a "hint" in the imported package:
you may need to add a "hint" in the imported package to exclude obfuscating it:
```go
type Message struct {
Command string
Expand All @@ -138,8 +172,20 @@ to document the current shortcomings of this tool.
var _ = reflect.TypeOf(Message{})
```

* Aside from `GOGARBLE` to select patterns of packages to obfuscate,
and the hint above with `reflect.TypeOf` to exclude obfuscating particular types,
there is no supported way to exclude obfuscating a selection of files or packages.
More often than not, a user would want to do this to work around a bug; please file the bug instead.

* Go programs [are initialized](https://go.dev/ref/spec#Program_initialization) one package at a time,
where imported packages are always initialized before their importers,
and otherwise they are initialized in the lexical order of their import paths.
Since garble obfuscates import paths, this lexical order may change arbitrarily.

* Go plugins are not currently supported; see [#87](https://github.com/burrowers/garble/issues/87).
* Garble requires `git` to patch the linker. That can be avoided once go-gitdiff supports [non-strict patches](https://github.com/bluekeyes/go-gitdiff/issues/30).

* Garble requires `git` to patch the linker. That can be avoided once go-gitdiff
supports [non-strict patches](https://github.com/bluekeyes/go-gitdiff/issues/30).

### Contributing

Expand Down
Loading

0 comments on commit 09822ab

Please sign in to comment.