Pasta 🍝

Copy files between repositories without pain.

About

Pasta is a tool that allows copying files between repositories in a robust way: Specify a list of sources in pasta.yaml, run pasta, and keep your sanity.

You can use it as a more predictable alternative to git submodules, or easily extended it to support other sources, such as Gitlab, Google Drive, or FTP servers.

Installation

The easiest way to install pasta is to install go, then do go install github.com/audiotool/pasta@latest.

Make sure you have $(go env GOPATH)/bin in your PATH variable, and invoke pasta with with pasta.

Alternatively, download a binary from the releases page, and install it manually.

pasta.yaml

The following is an example pasta.yaml, making pasta:

• copying from:
  • repo https://github.com/audiotool/manual:
  • branch main
• files:
  • from directory images/ matching regex audiotrack\.png, into pics/
  • LICENSE from topevel, next to pasta.yaml

keep_dirs: true
deps:
  - url: https://github.com/audiotool/manual
    from: images/
    to: pics/
    include: audiotrack\.png
    options:
      ref: heads/main

  - url: https://github.com/audiotool/manual
    from: .
    to: .
    files:
      - LICENSE
    options:
      ref: heads/main

Specification

keep_dirs specifies whether the target directories should first be deleted before new files are copied there.

deps is a list of dependencies, each with the following options:

Name	Meaning	Default
url	The URL of the source repository/directory	required
from	Path from which files are copied, relative to root of what url points to	required
to	Where the files should be copied to, relative to pasta.yaml	required
options	More information for the copier in use	{}
files	Information on what should be copied	(empty)
include	Information on what should be copied	include everything
exclude	Information on what should be copied	(empty)

Selecting files and directories

Each dependency can either filter copied files using regexes include and exclude, or list a fixed list of files using files.

Using include/exclude

Name	Meaning	Default
include	Only copy files matching this regex.	.* (everything)
exclude	Only copy files that don't match this regex. Takes precedence over include.	$^ (nothing)

Behaviour:

• the to directory has to be a subdirectory relative to pasta.yaml
• the to directory is cleared in case keep_dirs is false

Using files

Name	Meaning	Default
files	A list of files to copy, relative to from	[]

Behaviour:

• the to directory can also be the top level dir "."
• the target file are not deleted, even if keep_dirs is false

pasta.result.yaml

Once the copy takes place using ./pasta, a new file called pasta.result.yaml is generated. It contains metadata on what exactly is copied, for later reference.

For example, after executing pasta with the config file above, a file like this might be generated:

deps:
  - url: https://github.com/audiotool/manual
    source_info:
      sha: 61a667b266a6f0d5f05eb4db592858073c1a473c
      message: added sitemap to deploy
      author:
          date: 2023-06-19T13:14:12Z
          name: Silas Gyger
          email: silasgyge@gmail.com

(What the source_info value contains is up to the See Copier)

CLI

Pasta can be invoked with pasta. It will look for a config file in the current directory, and if it doesn't find one, travel upwards until it finds one.

Argument	Meaning
--help, -h	Show help and exit
--dry-run	Don't do anything, only show what would be done
--version, -v	Show the pasta version in use and exit

Copiers

Depending on what url is, a different Copier-plugin is used to copy the files. Additional options can be added to the specific copier using the options property.

The following copiers exist currently:

Name	Matches Urls	Description
Github	.*github\.com.*	Download files from a github repository

See Copiers below for more information on each copier.

You can also implement your own copier to accept URLs from e.g. Google Drive, Gitlab, Dropbox, IPFS, FTP, HTTP, or anything else. Read Contribute for more information on how to do that.

Best practices

We recommend to both check in the copied files and pasta.result.yaml, due to the following reasons:

• new developers can work with the repository even before setting up pasta
• pasta.yaml doesn't necessarily specify what files should be copied exactly, in which case pasta.result.yaml can provide helpful information on which files were present in a repo for a specific commit. For example, the pasta.yaml might specify that files should be copied from a specific github branch; which commit was chosen during copy is written into pasta.result.yaml.
• the copied files might differ between branches. When switching branch in git, it's easy to forget to run pasta again every time if the files aren't checked in.

Contribute

You can easily add your own Copier plugin that handles specific URLs.

All you need to do is:

• Implement the Copier interface found here: pkg/copier/copier.go,
• Add your copier to the list of all copiers here: pkg/pasta/run.go

Note that all copies are executed in parallel.

Copier Plugins

Github Copier

Matches URLs: github.com\<repo-owner>\<repo-name>\

The github copier downloads files from github repositories using the Github API. It can download files from public repos without authentication, and from private repo if supplied with an API access key through the GITHUB_TOKEN environment variable.

You can retriever your token from from https://github.com/settings/tokens

Options

• ref: the reference or commit sha to fetch from, can be one of:
  • heads/<branch-name>: explicitly reference a branch
  • tags/<tag-name>: explicilty reference a tag
  • commit/<commit-sha>: explicilty reference a commit
  • branch, tag, commit- or ref-sha, without prefix

If no prefix is given, the copier tries to resolve the ref in the order shown above.

If ref is left out, the default branch is used.

Authentication

For public repositories, no authentication is required, unless you're running into rate limits.

For private repos and to increase the rate limit, create a (classic) oauth token here, and set the environment variable GITHUB_TOKEN to its value before executing pasta.

For example on linux, if your token is ghp_foobar, execute pasta like

GITHUB_TOKEN=ghp_foobar pasta

or add the line

export GITHUB_TOKEN=ghp_foobar

to your .bashrc / .zshrc / etc.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Silas Gyger 💻 📖 🚇 ⚠️

Felix 💻 📖 🚇 ⚠️

jujurocket 📖

This project follows the all-contributors specification. Contributions of any kind welcome!

LICENSE

MIT