Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deps: bump it-stream-types from 1.0.5 to 2.0.1 #63

Merged
merged 5 commits into from
Feb 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ updates:
schedule:
interval: daily
time: "10:00"
open-pull-requests-limit: 10
open-pull-requests-limit: 20
commit-message:
prefix: "deps"
prefix-development: "deps(dev)"
2 changes: 2 additions & 0 deletions .github/workflows/js-test-and-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ on:

permissions:
contents: write
id-token: write
packages: write
pull-requests: write

concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}
Expand Down
12 changes: 12 additions & 0 deletions .github/workflows/semantic-pull-request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: Semantic PR

on:
pull_request_target:
types:
- opened
- edited
- synchronize

jobs:
main:
uses: pl-strflt/.github/.github/workflows/reusable-semantic-pull-request.yml@v0.3
13 changes: 13 additions & 0 deletions .github/workflows/stale.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: Close and mark stale issue

on:
schedule:
- cron: '0 0 * * *'

permissions:
issues: write
pull-requests: write

jobs:
stale:
uses: pl-strflt/.github/.github/workflows/reusable-stale-issue.yml@v0.3
10 changes: 7 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
node_modules
sandbox.js
package-lock.json
build
dist
.coverage
.docs
.coverage
node_modules
package-lock.json
yarn.lock
.vscode
129 changes: 68 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,43 +1,41 @@
# it-tar <!-- omit in toc -->
# it-tar

[![codecov](https://img.shields.io/codecov/c/github/alanshaw/it-tar.svg?style=flat-square)](https://codecov.io/gh/alanshaw/it-tar)
[![CI](https://img.shields.io/github/workflow/status/alanshaw/it-tar/test%20&%20maybe%20release/master?style=flat-square)](https://github.com/alanshaw/it-tar/actions/workflows/js-test-and-release.yml)
[![CI](https://img.shields.io/github/actions/workflow/status/alanshaw/it-tar/js-test-and-release.yml?branch=master\&style=flat-square)](https://github.com/alanshaw/it-tar/actions/workflows/js-test-and-release.yml?query=branch%3Amaster)

> it-tar is a streaming tar parser (and maybe a generator in the future) and nothing else. It operates purely using async iterables which means you can easily extract/parse tarballs without ever hitting the file system.
> it-tar is a streaming tar parser and generator. It operates purely using async iterables which means you can easily extract/parse tarballs without ever hitting the file system.

## Table of contents <!-- omit in toc -->
# About

- [Install](#install)
- [Usage](#usage)
- [Packing](#packing)
- [Extracting](#extracting)
- [Headers](#headers)
- [Modifying existing tarballs](#modifying-existing-tarballs)
- [Related](#related)
- [Contribute](#contribute)
- [License](#license)
- [Contribution](#contribution)
<!--

## Install
!IMPORTANT!

```console
$ npm i it-tar
```
Everything in this README between "# About" and "# Install" is automatically
generated and will be overwritten the next time the doc generator is run.

To make changes to this section, please update the @packageDocumentation section
of src/index.js or src/index.ts

## Usage
To experiment with formatting, please run "npm run docs" from the root of this
repo and examine the changes made.

-->

`it-tar` [packs](#packing) and [extracts](#extracts) tarballs.

It implementes USTAR with additional support for pax extended headers. It should be compatible with all popular tar distributions out there (gnutar, bsdtar etc)
It implements USTAR with additional support for pax extended headers. It should be compatible with all popular tar distributions out there (gnutar, bsdtar etc)

### Packing
## Example - Packing

To create a pack stream use `tar.pack()` and pipe entries to it.

```js
const Tar = require('it-tar')
```TypeScript
import fs from 'node:fs'
import Tar from 'it-tar'
import { pipe } from 'it-pipe'
const toIterable = require('stream-to-it')
// @ts-expect-error no types
import { sink } from 'stream-to-it'

await pipe(
[
Expand All @@ -51,25 +49,25 @@ await pipe(
header: { name: 'my-stream-test.txt', size: 11 },
body: fs.createReadStream('./my-stream-test.txt')
}
]
],
Tar.pack(),
// pipe the pack stream somewhere
toIterable.sink(process.stdout)
sink(process.stdout)
)
```

### Extracting
## Example - Extracting

To extract a stream use `tar.extract()` and pipe a [source iterable](https://gist.github.com/alanshaw/591dc7dd54e4f99338a347ef568d6ee9#source-it) to it.

```js
const Tar = require('it-tar')
```TypeScript
import Tar from 'it-tar'
import { pipe } from 'it-pipe'

await pipe(
source, // An async iterable (for example a Node.js readable stream)
[Uint8Array.from([0, 1, 2, 3, 4])], // An async iterable (for example a Node.js readable stream)
Tar.extract(),
source => {
async source => {
for await (const entry of source) {
// entry.header is the tar header (see below)
// entry.body is the content body (might be an empty async iterable)
Expand All @@ -84,7 +82,35 @@ await pipe(

The tar archive is streamed sequentially, meaning you **must** drain each entry's body as you get them or else the main extract stream will receive backpressure and stop reading.

Note that the body stream yields [`BufferList`](https://npm.im/bl) objects **not** `Buffer`s.
Note that the body stream yields [`Uint8ArrayList`](https://npm.im/uint8arraylist) objects **not** `Uint8Arrays`s.

## Example - Modifying existing tarballs

Using tar-stream it is easy to rewrite paths / change modes etc in an existing tarball.

```TypeScript
import Tar from 'it-tar'
import { pipe } from 'it-pipe'
// @ts-expect-error no types
import { sink } from 'stream-to-it'
import fs from 'node:fs'
import path from 'node:path'

await pipe(
fs.createReadStream('./old-tarball.tar'),
Tar.extract(),
async function * (source) {
for await (const entry of source) {
// let's prefix all names with 'tmp'
entry.header.name = path.join('tmp', entry.header.name)
// write the new entry to the pack stream
yield entry
}
},
Tar.pack(),
sink(fs.createWriteStream('./new-tarball.tar'))
)
```

#### Headers

Expand All @@ -110,48 +136,29 @@ Most of these values can be found by stat'ing a file.
}
```

## Modifying existing tarballs

Using tar-stream it is easy to rewrite paths / change modes etc in an existing tarball.

```js
const Tar = require('it-tar')
import { pipe } from 'it-pipe'
const toIterable = require('stream-to-it')

await pipe(
fs.createReadStream('./old-tarball.tar'),
Tar.extract(),
async function * (source) {
for await (const entry of source) {
// let's prefix all names with 'tmp'
entry.header.name = path.join('tmp', entry.header.name)
// write the new entry to the pack stream
yield entry
}
},
Tar.pack(),
toIterable.sink(fs.createWriteStream('./new-tarball.tar'))
)
```

## Related

- [`it-pipe`](https://www.npmjs.com/package/it-pipe) Utility to "pipe" async iterables together
- [`it-reader`](https://www.npmjs.com/package/it-reader) Read an exact number of bytes from a binary (async) iterable
- [`stream-to-it`](https://www.npmjs.com/package/stream-to-it) Convert Node.js streams to streaming iterables

## Contribute
# Install

```console
$ npm i it-tar
```

# API Docs

Feel free to dive in! [Open an issue](https://github.com/alanshaw/it-tar/issues/new) or submit PRs.
- <https://alanshaw.github.io/it-tar>

## License
# License

Licensed under either of

- Apache 2.0, ([LICENSE-APACHE](LICENSE-APACHE) / <http://www.apache.org/licenses/LICENSE-2.0>)
- MIT ([LICENSE-MIT](LICENSE-MIT) / <http://opensource.org/licenses/MIT>)

## Contribution
# Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
22 changes: 14 additions & 8 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@
"bugs": {
"url": "https://github.com/alanshaw/it-tar/issues"
},
"publishConfig": {
"access": "public",
"provenance": true
},
"keywords": [
"extract",
"generate",
Expand All @@ -28,15 +32,11 @@
"tar",
"tarball"
],
"engines": {
"node": ">=16.0.0",
"npm": ">=7.0.0"
},
"type": "module",
"types": "./dist/src/index.d.ts",
"files": [
"src",
"dist/src",
"dist",
"!dist/test",
"!**/*.tsbuildinfo"
],
Expand All @@ -49,6 +49,7 @@
"eslintConfig": {
"extends": "ipfs",
"parserOptions": {
"project": true,
"sourceType": "module"
}
},
Expand Down Expand Up @@ -140,7 +141,9 @@
"scripts": {
"clean": "aegir clean",
"lint": "aegir lint",
"docs": "aegir docs",
"dep-check": "aegir dep-check",
"doc-check": "aegir doc-check",
"build": "aegir build",
"postbuild": "cp ./test/fixtures/*.tar ./test/fixtures/*.tar.gz ./test/fixtures/*.tgz ./dist/test/fixtures",
"test": "aegir test -t node",
Expand All @@ -151,16 +154,19 @@
"dependencies": {
"iso-constants": "^0.1.2",
"it-reader": "^6.0.1",
"it-stream-types": "^1.0.4",
"it-stream-types": "^2.0.1",
"it-to-buffer": "^4.0.5",
"p-defer": "^4.0.0",
"uint8arraylist": "^2.3.2",
"uint8arrays": "^5.0.2"
},
"devDependencies": {
"aegir": "^37.5.1",
"concat-stream": "^2.0.0",
"aegir": "^42.2.4",
"it-pipe": "^3.0.1",
"stream-to-it": "^0.2.0"
},
"engines": {
"node": ">=16.0.0",
"npm": ">=7.0.0"
}
}
Loading
Loading