Skip to content

Commit 6105722

Browse files
committed
Moved to wiki, for now
1 parent 627b144 commit 6105722

File tree

1 file changed

+1
-69
lines changed

1 file changed

+1
-69
lines changed

README.md

Lines changed: 1 addition & 69 deletions
Original file line numberDiff line numberDiff line change
@@ -4,75 +4,7 @@ The purpose is to separate big-file caching from revision-control.
44
From the author of **git-fat**:
55
> Checking large binary files into a source repository (Git or otherwise) is a bad idea because repository size quickly becomes unreasonable. Even if the instantaneous working tree stays manageable, preserving repository integrity requires all binary files in the entire project history, which given the typically poor compression of binary diffs, implies that the repository size will become impractically large. Some people recommend checking binaries into different repositories or even not versioning them at all, but these are not satisfying solutions for most workflows.
66
7-
## How is **git-sym** different?
8-
There are several alternatives:
9-
* https://github.com/jedbrown/git-fat
10-
* https://github.com/schacon/git-media
11-
* http://git-annex.branchable.com/
12-
* https://github.com/github/git-lfs
13-
14-
But all those impose the penalty of checksums on the large files. We assert that check-summing is not absolutely requried to guarantee integrity. These large resources can be uniquely derived from stable URLs (e.g. versioned in Amazon S3 or stored by unique filename somewhere). **git-sym** stores only symlinks in the git repo, not the checksums themselves.
15-
16-
In addition to some faster git operations, **git-sym** also allows symlinking *directories*. We think that is more appropriate for a wide variety of use-cases.
17-
18-
## How are these large files retrieved and cached?
19-
**git-sym** leaves that up to you. The repo author is responsible for providing a **makefile** with a rule for every unique target.
20-
21-
Some of the tools listed above provide a lot of out-of-the-box caching functionality, but we prefer to decouple caching from revision-control.
22-
23-
## Features
24-
These are features which **git-sym** shares in common with **git-fat** (which we also like for some use-cases). *Differences are in italics.*
25-
* Clones of the source repository are small and fast because no binaries are transferred, yet fully functional with complete metadata and incremental retrieval (git clone --depth has limited granularity and couples metadata to content).
26-
* git-bisect works properly even when versions of the binary files change over time, *as long as they have been cached already*.
27-
* selective control of which large files to pull into the local store
28-
* Local fat object stores can be shared between multiple clones, even by different users.
29-
* can easily support fat object stores distributed across multiple hosts, *and potentially anywhere in the world*
30-
* depends only on stock Python and rsync
31-
32-
This feature of **git-fat** is *not* yet in **git-sym**. (It depends on `.gitattributes`. We are looking into that.)
33-
* **git-fat** supports the same workflow for large binaries and traditionally versioned files, but internally manages the "fat" files separately.
34-
35-
These are features of **git-sym** which are missing from some or all of the alternatives.
36-
* **git-sym** can fetch via rsync, curl, wget, an s3 client, a google-drive client, or anything else available in your system.
37-
* **git-sym** allows you to symlink directories, which can simplify your data-management.
38-
* **git-sym** never slows down for check-summing (except for the initial caching).
39-
* **git-sym** works fine in git-submodules.
40-
* **git-sym** is easy to understand.
41-
42-
## Installing
43-
You can run this as a **git** command by calling it `git-sym`
44-
in your `$PATH`. Here is one way:
45-
```
46-
ln -sf `pwd`/git-sym ~/bin/git-sym
47-
```
48-
Alternatively, you can run it directly:
49-
```
50-
python git-sym -h
51-
```
52-
53-
## Basic usage
54-
### For repo users
55-
```
56-
git-sym show
57-
git-sym update
58-
```
59-
### For repo owners
60-
```
61-
git-sym add my_big_file.gif
62-
git commit -m 'git-sym added'
63-
ls -l my_big_file.gif
64-
```
65-
Or more explicitly, and with a rule for retrieval:
66-
```
67-
ln -sf .git_sym/my_big_data.v123.db my_big_data.db
68-
git add my_big_data.db
69-
git commit
70-
cat <<EOF >> git_sym.makefile
71-
my_big_data.v123.db:
72-
wget http://www.somewhere.com/my_big_data.v123.db
73-
EOF
74-
git-sym update
75-
```
7+
* https://github.com/cdunn2001/git-sym/wiki/Rationale
768

779
## Examples
7810
These examples explain the basics:

0 commit comments

Comments
 (0)