Last commit july 5th

This commit is contained in:
2024-07-05 13:46:23 +02:00
parent dad0d86e8c
commit b0e4dfbb76
24982 changed files with 2621219 additions and 413 deletions

20
spa/node_modules/npm-registry-fetch/LICENSE.md generated vendored Normal file
View File

@@ -0,0 +1,20 @@
<!-- This file is automatically added by @npmcli/template-oss. Do not edit. -->
ISC License
Copyright npm, Inc.
Permission to use, copy, modify, and/or distribute this
software for any purpose with or without fee is hereby
granted, provided that the above copyright notice and this
permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND NPM DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO
EVENT SHALL NPM BE LIABLE FOR ANY SPECIAL, DIRECT,
INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE
USE OR PERFORMANCE OF THIS SOFTWARE.

616
spa/node_modules/npm-registry-fetch/README.md generated vendored Normal file
View File

@@ -0,0 +1,616 @@
# npm-registry-fetch
[`npm-registry-fetch`](https://github.com/npm/npm-registry-fetch) is a Node.js
library that implements a `fetch`-like API for accessing npm registry APIs
consistently. It's able to consume npm-style configuration values and has all
the necessary logic for picking registries, handling scopes, and dealing with
authentication details built-in.
This package is meant to replace the older
[`npm-registry-client`](https://npm.im/npm-registry-client).
## Example
```javascript
const npmFetch = require('npm-registry-fetch')
console.log(
await npmFetch.json('/-/ping')
)
```
## Table of Contents
* [Installing](#install)
* [Example](#example)
* [Contributing](#contributing)
* [API](#api)
* [`fetch`](#fetch)
* [`fetch.json`](#fetch-json)
* [`fetch` options](#fetch-opts)
### Install
`$ npm install npm-registry-fetch`
### Contributing
The npm team enthusiastically welcomes contributions and project participation!
There's a bunch of things you can do if you want to contribute! The [Contributor
Guide](CONTRIBUTING.md) has all the information you need for everything from
reporting bugs to contributing entire new features. Please don't hesitate to
jump in if you'd like to, or even ask us questions if something isn't clear.
All participants and maintainers in this project are expected to follow [Code of
Conduct](CODE_OF_CONDUCT.md), and just generally be excellent to each other.
Please refer to the [Changelog](CHANGELOG.md) for project history details, too.
Happy hacking!
### API
#### Caching and `write=true` query strings
Before performing any PUT or DELETE operation, npm clients first make a
GET request to the registry resource being updated, which includes
the query string `?write=true`.
The semantics of this are, effectively, "I intend to write to this thing,
and need to know the latest current value, so that my write can land
cleanly".
The public npm registry handles these `?write=true` requests by ensuring
that the cache is re-validated before sending a response. In order to
maintain the same behavior on the client, and not get tripped up by an
overeager local cache when we intend to write data to the registry, any
request that comes through `npm-registry-fetch` that contains `write=true`
in the query string will forcibly set the `prefer-online` option to `true`,
and set both `prefer-offline` and `offline` to false, so that any local
cached value will be revalidated.
#### <a name="fetch"></a> `> fetch(url, [opts]) -> Promise<Response>`
Performs a request to a given URL.
The URL can be either a full URL, or a path to one. The appropriate registry
will be automatically picked if only a URL path is given.
For available options, please see the section on [`fetch` options](#fetch-opts).
##### Example
```javascript
const res = await fetch('/-/ping')
console.log(res.headers)
res.on('data', d => console.log(d.toString('utf8')))
```
#### <a name="fetch-json"></a> `> fetch.json(url, [opts]) -> Promise<ResponseJSON>`
Performs a request to a given registry URL, parses the body of the response as
JSON, and returns it as its final value. This is a utility shorthand for
`fetch(url).then(res => res.json())`.
For available options, please see the section on [`fetch` options](#fetch-opts).
##### Example
```javascript
const res = await fetch.json('/-/ping')
console.log(res) // Body parsed as JSON
```
#### <a name="fetch-json-stream"></a> `> fetch.json.stream(url, jsonPath, [opts]) -> Stream`
Performs a request to a given registry URL and parses the body of the response
as JSON, with each entry being emitted through the stream.
The `jsonPath` argument is a [`JSONStream.parse()`
path](https://github.com/dominictarr/JSONStream#jsonstreamparsepath), and the
returned stream (unlike default `JSONStream`s), has a valid
`Symbol.asyncIterator` implementation.
For available options, please see the section on [`fetch` options](#fetch-opts).
##### Example
```javascript
console.log('https://npm.im/~zkat has access to the following packages:')
for await (let {key, value} of fetch.json.stream('/-/user/zkat/package', '$*')) {
console.log(`https://npm.im/${key} (perms: ${value})`)
}
```
#### <a name="fetch-opts"></a> `fetch` Options
Fetch options are optional, and can be passed in as either a Map-like object
(one with a `.get()` method), a plain javascript object, or a
[`figgy-pudding`](https://npm.im/figgy-pudding) instance.
##### <a name="opts-agent"></a> `opts.agent`
* Type: http.Agent
* Default: an appropriate agent based on URL protocol and proxy settings
An [`Agent`](https://nodejs.org/api/http.html#http_class_http_agent) instance to
be shared across requests. This allows multiple concurrent `fetch` requests to
happen on the same socket.
You do _not_ need to provide this option unless you want something particularly
specialized, since proxy configurations and http/https agents are already
automatically managed internally when this option is not passed through.
##### <a name="opts-body"></a> `opts.body`
* Type: Buffer | Stream | Object
* Default: null
Request body to send through the outgoing request. Buffers and Streams will be
passed through as-is, with a default `content-type` of
`application/octet-stream`. Plain JavaScript objects will be `JSON.stringify`ed
and the `content-type` will default to `application/json`.
Use [`opts.headers`](#opts-headers) to set the content-type to something else.
##### <a name="opts-ca"></a> `opts.ca`
* Type: String, Array, or null
* Default: null
The Certificate Authority signing certificate that is trusted for SSL
connections to the registry. Values should be in PEM format (Windows calls it
"Base-64 encoded X.509 (.CER)") with newlines replaced by the string `'\n'`. For
example:
```
{
ca: '-----BEGIN CERTIFICATE-----\nXXXX\nXXXX\n-----END CERTIFICATE-----'
}
```
Set to `null` to only allow "known" registrars, or to a specific CA cert
to trust only that specific signing authority.
Multiple CAs can be trusted by specifying an array of certificates instead of a
single string.
See also [`opts.strictSSL`](#opts-strictSSL), [`opts.ca`](#opts-ca) and
[`opts.key`](#opts-key)
##### <a name="opts-cache"></a> `opts.cache`
* Type: path
* Default: null
The location of the http cache directory. If provided, certain cachable requests
will be cached according to [IETF RFC 7234](https://tools.ietf.org/html/rfc7234)
rules. This will speed up future requests, as well as make the cached data
available offline if necessary/requested.
See also [`offline`](#opts-offline), [`preferOffline`](#opts-preferOffline),
and [`preferOnline`](#opts-preferOnline).
##### <a name="opts-cert"></a> `opts.cert`
* Type: String
* Default: null
A client certificate to pass when accessing the registry. Values should be in
PEM format (Windows calls it "Base-64 encoded X.509 (.CER)") with newlines
replaced by the string `'\n'`. For example:
```
{
cert: '-----BEGIN CERTIFICATE-----\nXXXX\nXXXX\n-----END CERTIFICATE-----'
}
```
It is _not_ the path to a certificate file (and there is no "certfile" option).
See also: [`opts.ca`](#opts-ca) and [`opts.key`](#opts-key)
##### <a name="opts-fetchRetries"></a> `opts.fetchRetries`
* Type: Number
* Default: 2
The "retries" config for [`retry`](https://npm.im/retry) to use when fetching
packages from the registry.
See also [`opts.retry`](#opts-retry) to provide all retry options as a single
object.
##### <a name="opts-fetchRetryFactor"></a> `opts.fetchRetryFactor`
* Type: Number
* Default: 10
The "factor" config for [`retry`](https://npm.im/retry) to use when fetching
packages.
See also [`opts.retry`](#opts-retry) to provide all retry options as a single
object.
##### <a name="opts-fetchRetryMintimeout"></a> `opts.fetchRetryMintimeout`
* Type: Number
* Default: 10000 (10 seconds)
The "minTimeout" config for [`retry`](https://npm.im/retry) to use when fetching
packages.
See also [`opts.retry`](#opts-retry) to provide all retry options as a single
object.
##### <a name="opts-fetchRetryMaxtimeout"></a> `opts.fetchRetryMaxtimeout`
* Type: Number
* Default: 60000 (1 minute)
The "maxTimeout" config for [`retry`](https://npm.im/retry) to use when fetching
packages.
See also [`opts.retry`](#opts-retry) to provide all retry options as a single
object.
##### <a name="opts-forceAuth"></a> `opts.forceAuth`
* Type: Object
* Default: null
If present, other auth-related values in `opts` will be completely ignored,
including `alwaysAuth`, `email`, and `otp`, when calculating auth for a request,
and the auth details in `opts.forceAuth` will be used instead.
##### <a name="opts-gzip"></a> `opts.gzip`
* Type: Boolean
* Default: false
If true, `npm-registry-fetch` will set the `Content-Encoding` header to `gzip`
and use `zlib.gzip()` or `zlib.createGzip()` to gzip-encode
[`opts.body`](#opts-body).
##### <a name="opts-headers"></a> `opts.headers`
* Type: Object
* Default: null
Additional headers for the outgoing request. This option can also be used to
override headers automatically generated by `npm-registry-fetch`, such as
`Content-Type`.
##### <a name="opts-ignoreBody"></a> `opts.ignoreBody`
* Type: Boolean
* Default: false
If true, the **response body** will be thrown away and `res.body` set to `null`.
This will prevent dangling response sockets for requests where you don't usually
care what the response body is.
##### <a name="opts-integrity"></a> `opts.integrity`
* Type: String | [SRI object](https://npm.im/ssri)
* Default: null
If provided, the response body's will be verified against this integrity string,
using [`ssri`](https://npm.im/ssri). If verification succeeds, the response will
complete as normal. If verification fails, the response body will error with an
`EINTEGRITY` error.
Body integrity is only verified if the body is actually consumed to completion --
that is, if you use `res.json()`/`res.buffer()`, or if you consume the default
`res` stream data to its end.
Cached data will have its integrity automatically verified using the
previously-generated integrity hash for the saved request information, so
`EINTEGRITY` errors can happen if [`opts.cache`](#opts-cache) is used, even if
`opts.integrity` is not passed in.
##### <a name="opts-key"></a> `opts.key`
* Type: String
* Default: null
A client key to pass when accessing the registry. Values should be in PEM
format with newlines replaced by the string `'\n'`. For example:
```
{
key: '-----BEGIN PRIVATE KEY-----\nXXXX\nXXXX\n-----END PRIVATE KEY-----'
}
```
It is _not_ the path to a key file (and there is no "keyfile" option).
See also: [`opts.ca`](#opts-ca) and [`opts.cert`](#opts-cert)
##### <a name="opts-localAddress"></a> `opts.localAddress`
* Type: IP Address String
* Default: null
The IP address of the local interface to use when making connections
to the registry.
See also [`opts.proxy`](#opts-proxy)
##### <a name="opts-mapJSON"></a> `opts.mapJSON`
* Type: Function
* Default: undefined
When using `fetch.json.stream()` (NOT `fetch.json()`), this will be passed down
to [`JSONStream`](https://npm.im/JSONStream) as the second argument to
`JSONStream.parse`, and can be used to transform stream data before output.
##### <a name="opts-maxSockets"></a> `opts.maxSockets`
* Type: Integer
* Default: 12
Maximum number of sockets to keep open during requests. Has no effect if
[`opts.agent`](#opts-agent) is used.
##### <a name="opts-method"></a> `opts.method`
* Type: String
* Default: 'GET'
HTTP method to use for the outgoing request. Case-insensitive.
##### <a name="opts-noProxy"></a> `opts.noProxy`
* Type: String | String[]
* Default: process.env.NOPROXY
If present, should be a comma-separated string or an array of domain extensions
that a proxy should _not_ be used for.
##### <a name="opts-npmSession"></a> `opts.npmSession`
* Type: String
* Default: null
If provided, will be sent in the `npm-session` header. This header is used by
the npm registry to identify individual user sessions (usually individual
invocations of the CLI).
##### <a name="opts-npmCommand"></a> `opts.npmCommand`
* Type: String
* Default: null
If provided, it will be sent in the `npm-command` header. This header is
used by the npm registry to identify the npm command that caused this
request to be made.
##### <a name="opts-offline"></a> `opts.offline`
* Type: Boolean
* Default: false
Force offline mode: no network requests will be done during install. To allow
`npm-registry-fetch` to fill in missing cache data, see
[`opts.preferOffline`](#opts-preferOffline).
This option is only really useful if you're also using
[`opts.cache`](#opts-cache).
This option is set to `true` when the request includes `write=true` in the
query string.
##### <a name="opts-otp"></a> `opts.otp`
* Type: Number | String
* Default: null
This is a one-time password from a two-factor authenticator. It is required for
certain registry interactions when two-factor auth is enabled for a user
account.
##### <a name="opts-otpPrompt"></a> `opts.otpPrompt`
* Type: Function
* Default: null
This is a method which will be called to provide an OTP if the server
responds with a 401 response indicating that a one-time-password is
required.
It may return a promise, which must resolve to the OTP value to be used.
If the method fails to provide an OTP value, then the fetch will fail with
the auth error that indicated an OTP was needed.
##### <a name="opts-password"></a> `opts.password`
* Alias: `_password`
* Type: String
* Default: null
Password used for basic authentication. For the more modern authentication
method, please use the (more secure) [`opts.token`](#opts-token)
Can optionally be scoped to a registry by using a "nerf dart" for that registry.
That is:
```
{
'//registry.npmjs.org/:password': 't0k3nH34r'
}
```
See also [`opts.username`](#opts-username)
##### <a name="opts-preferOffline"></a> `opts.preferOffline`
* Type: Boolean
* Default: false
If true, staleness checks for cached data will be bypassed, but missing data
will be requested from the server. To force full offline mode, use
[`opts.offline`](#opts-offline).
This option is generally only useful if you're also using
[`opts.cache`](#opts-cache).
This option is set to `false` when the request includes `write=true` in the
query string.
##### <a name="opts-preferOnline"></a> `opts.preferOnline`
* Type: Boolean
* Default: false
If true, staleness checks for cached data will be forced, making the CLI look
for updates immediately even for fresh package data.
This option is generally only useful if you're also using
[`opts.cache`](#opts-cache).
This option is set to `true` when the request includes `write=true` in the
query string.
##### <a name="opts-scope"></a> `opts.scope`
* Type: String
* Default: null
If provided, will be sent in the `npm-scope` header. This header is used by the
npm registry to identify the toplevel package scope that a particular project
installation is using.
##### <a name="opts-proxy"></a> `opts.proxy`
* Type: url
* Default: null
A proxy to use for outgoing http requests. If not passed in, the `HTTP(S)_PROXY`
environment variable will be used.
##### <a name="opts-query"></a> `opts.query`
* Type: String | Object
* Default: null
If provided, the request URI will have a query string appended to it using this
query. If `opts.query` is an object, it will be converted to a query string
using
[`querystring.stringify()`](https://nodejs.org/api/querystring.html#querystring_querystring_stringify_obj_sep_eq_options).
If the request URI already has a query string, it will be merged with
`opts.query`, preferring `opts.query` values.
##### <a name="opts-registry"></a> `opts.registry`
* Type: URL
* Default: `'https://registry.npmjs.org'`
Registry configuration for a request. If a request URL only includes the URL
path, this registry setting will be prepended.
See also [`opts.scope`](#opts-scope), [`opts.spec`](#opts-spec), and
[`opts.<scope>:registry`](#opts-scope-registry) which can all affect the actual
registry URL used by the outgoing request.
##### <a name="opts-retry"></a> `opts.retry`
* Type: Object
* Default: null
Single-object configuration for request retry settings. If passed in, will
override individually-passed `fetch-retry-*` settings.
##### <a name="opts-scope"></a> `opts.scope`
* Type: String
* Default: null
Associate an operation with a scope for a scoped registry. This option can force
lookup of scope-specific registries and authentication.
See also [`opts.<scope>:registry`](#opts-scope-registry) and
[`opts.spec`](#opts-spec) for interactions with this option.
##### <a name="opts-scope-registry"></a> `opts.<scope>:registry`
* Type: String
* Default: null
This option type can be used to configure the registry used for requests
involving a particular scope. For example, `opts['@myscope:registry'] =
'https://scope-specific.registry/'` will make it so requests go out to this
registry instead of [`opts.registry`](#opts-registry) when
[`opts.scope`](#opts-scope) is used, or when [`opts.spec`](#opts-spec) is a
scoped package spec.
The `@` before the scope name is optional, but recommended.
##### <a name="opts-spec"></a> `opts.spec`
* Type: String | [`npm-registry-arg`](https://npm.im/npm-registry-arg) object.
* Default: null
If provided, can be used to automatically configure [`opts.scope`](#opts-scope)
based on a specific package name. Non-registry package specs will throw an
error.
##### <a name="opts-strictSSL"></a> `opts.strictSSL`
* Type: Boolean
* Default: true
Whether or not to do SSL key validation when making requests to the
registry via https.
See also [`opts.ca`](#opts-ca).
##### <a name="opts-timeout"></a> `opts.timeout`
* Type: Milliseconds
* Default: 300000 (5 minutes)
Time before a hanging request times out.
##### <a name="opts-authtoken"></a> `opts._authToken`
* Type: String
* Default: null
Authentication token string.
Can be scoped to a registry by using a "nerf dart" for that registry. That is:
```
{
'//registry.npmjs.org/:_authToken': 't0k3nH34r'
}
```
##### <a name="opts-userAgent"></a> `opts.userAgent`
* Type: String
* Default: `'npm-registry-fetch@<version>/node@<node-version>+<arch> (<platform>)'`
User agent string to send in the `User-Agent` header.
##### <a name="opts-username"></a> `opts.username`
* Type: String
* Default: null
Username used for basic authentication. For the more modern authentication
method, please use the (more secure) [`opts.authtoken`](#opts-authtoken)
Can optionally be scoped to a registry by using a "nerf dart" for that registry.
That is:
```
{
'//registry.npmjs.org/:username': 't0k3nH34r'
}
```
See also [`opts.password`](#opts-password)

145
spa/node_modules/npm-registry-fetch/lib/auth.js generated vendored Normal file
View File

@@ -0,0 +1,145 @@
'use strict'
const fs = require('fs')
const npa = require('npm-package-arg')
const { URL } = require('url')
// Find the longest registry key that is used for some kind of auth
// in the options.
const regKeyFromURI = (uri, opts) => {
const parsed = new URL(uri)
// try to find a config key indicating we have auth for this registry
// can be one of :_authToken, :_auth, :_password and :username, or
// :certfile and :keyfile
// We walk up the "path" until we're left with just //<host>[:<port>],
// stopping when we reach '//'.
let regKey = `//${parsed.host}${parsed.pathname}`
while (regKey.length > '//'.length) {
// got some auth for this URI
if (hasAuth(regKey, opts)) {
return regKey
}
// can be either //host/some/path/:_auth or //host/some/path:_auth
// walk up by removing EITHER what's after the slash OR the slash itself
regKey = regKey.replace(/([^/]+|\/)$/, '')
}
}
const hasAuth = (regKey, opts) => (
opts[`${regKey}:_authToken`] ||
opts[`${regKey}:_auth`] ||
opts[`${regKey}:username`] && opts[`${regKey}:_password`] ||
opts[`${regKey}:certfile`] && opts[`${regKey}:keyfile`]
)
const sameHost = (a, b) => {
const parsedA = new URL(a)
const parsedB = new URL(b)
return parsedA.host === parsedB.host
}
const getRegistry = opts => {
const { spec } = opts
const { scope: specScope, subSpec } = spec ? npa(spec) : {}
const subSpecScope = subSpec && subSpec.scope
const scope = subSpec ? subSpecScope : specScope
const scopeReg = scope && opts[`${scope}:registry`]
return scopeReg || opts.registry
}
const maybeReadFile = file => {
try {
return fs.readFileSync(file, 'utf8')
} catch (er) {
if (er.code !== 'ENOENT') {
throw er
}
return null
}
}
const getAuth = (uri, opts = {}) => {
const { forceAuth } = opts
if (!uri) {
throw new Error('URI is required')
}
const regKey = regKeyFromURI(uri, forceAuth || opts)
// we are only allowed to use what's in forceAuth if specified
if (forceAuth && !regKey) {
return new Auth({
scopeAuthKey: null,
token: forceAuth._authToken || forceAuth.token,
username: forceAuth.username,
password: forceAuth._password || forceAuth.password,
auth: forceAuth._auth || forceAuth.auth,
certfile: forceAuth.certfile,
keyfile: forceAuth.keyfile,
})
}
// no auth for this URI, but might have it for the registry
if (!regKey) {
const registry = getRegistry(opts)
if (registry && uri !== registry && sameHost(uri, registry)) {
return getAuth(registry, opts)
} else if (registry !== opts.registry) {
// If making a tarball request to a different base URI than the
// registry where we logged in, but the same auth SHOULD be sent
// to that artifact host, then we track where it was coming in from,
// and warn the user if we get a 4xx error on it.
const scopeAuthKey = regKeyFromURI(registry, opts)
return new Auth({ scopeAuthKey })
}
}
const {
[`${regKey}:_authToken`]: token,
[`${regKey}:username`]: username,
[`${regKey}:_password`]: password,
[`${regKey}:_auth`]: auth,
[`${regKey}:certfile`]: certfile,
[`${regKey}:keyfile`]: keyfile,
} = opts
return new Auth({
scopeAuthKey: null,
token,
auth,
username,
password,
certfile,
keyfile,
})
}
class Auth {
constructor ({ token, auth, username, password, scopeAuthKey, certfile, keyfile }) {
this.scopeAuthKey = scopeAuthKey
this.token = null
this.auth = null
this.isBasicAuth = false
this.cert = null
this.key = null
if (token) {
this.token = token
} else if (auth) {
this.auth = auth
} else if (username && password) {
const p = Buffer.from(password, 'base64').toString('utf8')
this.auth = Buffer.from(`${username}:${p}`, 'utf8').toString('base64')
this.isBasicAuth = true
}
// mTLS may be used in conjunction with another auth method above
if (certfile && keyfile) {
const cert = maybeReadFile(certfile, 'utf-8')
const key = maybeReadFile(keyfile, 'utf-8')
if (cert && key) {
this.cert = cert
this.key = key
}
}
}
}
module.exports = getAuth

View File

@@ -0,0 +1,100 @@
'use strict'
const errors = require('./errors.js')
const { Response } = require('minipass-fetch')
const defaultOpts = require('./default-opts.js')
const log = require('proc-log')
const cleanUrl = require('./clean-url.js')
/* eslint-disable-next-line max-len */
const moreInfoUrl = 'https://github.com/npm/cli/wiki/No-auth-for-URI,-but-auth-present-for-scoped-registry'
const checkResponse =
async ({ method, uri, res, startTime, auth, opts }) => {
opts = { ...defaultOpts, ...opts }
if (res.headers.has('npm-notice') && !res.headers.has('x-local-cache')) {
log.notice('', res.headers.get('npm-notice'))
}
if (res.status >= 400) {
logRequest(method, res, startTime)
if (auth && auth.scopeAuthKey && !auth.token && !auth.auth) {
// we didn't have auth for THIS request, but we do have auth for
// requests to the registry indicated by the spec's scope value.
// Warn the user.
log.warn('registry', `No auth for URI, but auth present for scoped registry.
URI: ${uri}
Scoped Registry Key: ${auth.scopeAuthKey}
More info here: ${moreInfoUrl}`)
}
return checkErrors(method, res, startTime, opts)
} else {
res.body.on('end', () => logRequest(method, res, startTime, opts))
if (opts.ignoreBody) {
res.body.resume()
return new Response(null, res)
}
return res
}
}
module.exports = checkResponse
function logRequest (method, res, startTime) {
const elapsedTime = Date.now() - startTime
const attempt = res.headers.get('x-fetch-attempts')
const attemptStr = attempt && attempt > 1 ? ` attempt #${attempt}` : ''
const cacheStatus = res.headers.get('x-local-cache-status')
const cacheStr = cacheStatus ? ` (cache ${cacheStatus})` : ''
const urlStr = cleanUrl(res.url)
log.http(
'fetch',
`${method.toUpperCase()} ${res.status} ${urlStr} ${elapsedTime}ms${attemptStr}${cacheStr}`
)
}
function checkErrors (method, res, startTime, opts) {
return res.buffer()
.catch(() => null)
.then(body => {
let parsed = body
try {
parsed = JSON.parse(body.toString('utf8'))
} catch {
// ignore errors
}
if (res.status === 401 && res.headers.get('www-authenticate')) {
const auth = res.headers.get('www-authenticate')
.split(/,\s*/)
.map(s => s.toLowerCase())
if (auth.indexOf('ipaddress') !== -1) {
throw new errors.HttpErrorAuthIPAddress(
method, res, parsed, opts.spec
)
} else if (auth.indexOf('otp') !== -1) {
throw new errors.HttpErrorAuthOTP(
method, res, parsed, opts.spec
)
} else {
throw new errors.HttpErrorAuthUnknown(
method, res, parsed, opts.spec
)
}
} else if (
res.status === 401 &&
body != null &&
/one-time pass/.test(body.toString('utf8'))
) {
// Heuristic for malformed OTP responses that don't include the
// www-authenticate header.
throw new errors.HttpErrorAuthOTP(
method, res, parsed, opts.spec
)
} else {
throw new errors.HttpErrorGeneral(
method, res, parsed, opts.spec
)
}
})
}

27
spa/node_modules/npm-registry-fetch/lib/clean-url.js generated vendored Normal file
View File

@@ -0,0 +1,27 @@
const { URL } = require('url')
const replace = '***'
const tokenRegex = /\bnpm_[a-zA-Z0-9]{36}\b/g
const guidRegex = /\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\b/g
const cleanUrl = (str) => {
if (typeof str !== 'string' || !str) {
return str
}
try {
const url = new URL(str)
if (url.password) {
url.password = replace
str = url.toString()
}
} catch {
// ignore errors
}
return str
.replace(tokenRegex, `npm_${replace}`)
.replace(guidRegex, `npm_${replace}`)
}
module.exports = cleanUrl

View File

@@ -0,0 +1,19 @@
const pkg = require('../package.json')
module.exports = {
maxSockets: 12,
method: 'GET',
registry: 'https://registry.npmjs.org/',
timeout: 5 * 60 * 1000, // 5 minutes
strictSSL: true,
noProxy: process.env.NOPROXY,
userAgent: `${pkg.name
}@${
pkg.version
}/node@${
process.version
}+${
process.arch
} (${
process.platform
})`,
}

80
spa/node_modules/npm-registry-fetch/lib/errors.js generated vendored Normal file
View File

@@ -0,0 +1,80 @@
'use strict'
const url = require('url')
function packageName (href) {
try {
let basePath = new url.URL(href).pathname.slice(1)
if (!basePath.match(/^-/)) {
basePath = basePath.split('/')
var index = basePath.indexOf('_rewrite')
if (index === -1) {
index = basePath.length - 1
} else {
index++
}
return decodeURIComponent(basePath[index])
}
} catch (_) {
// this is ok
}
}
class HttpErrorBase extends Error {
constructor (method, res, body, spec) {
super()
this.name = this.constructor.name
this.headers = res.headers.raw()
this.statusCode = res.status
this.code = `E${res.status}`
this.method = method
this.uri = res.url
this.body = body
this.pkgid = spec ? spec.toString() : packageName(res.url)
}
}
module.exports.HttpErrorBase = HttpErrorBase
class HttpErrorGeneral extends HttpErrorBase {
constructor (method, res, body, spec) {
super(method, res, body, spec)
this.message = `${res.status} ${res.statusText} - ${
this.method.toUpperCase()
} ${
this.spec || this.uri
}${
(body && body.error) ? ' - ' + body.error : ''
}`
Error.captureStackTrace(this, HttpErrorGeneral)
}
}
module.exports.HttpErrorGeneral = HttpErrorGeneral
class HttpErrorAuthOTP extends HttpErrorBase {
constructor (method, res, body, spec) {
super(method, res, body, spec)
this.message = 'OTP required for authentication'
this.code = 'EOTP'
Error.captureStackTrace(this, HttpErrorAuthOTP)
}
}
module.exports.HttpErrorAuthOTP = HttpErrorAuthOTP
class HttpErrorAuthIPAddress extends HttpErrorBase {
constructor (method, res, body, spec) {
super(method, res, body, spec)
this.message = 'Login is not allowed from your IP address'
this.code = 'EAUTHIP'
Error.captureStackTrace(this, HttpErrorAuthIPAddress)
}
}
module.exports.HttpErrorAuthIPAddress = HttpErrorAuthIPAddress
class HttpErrorAuthUnknown extends HttpErrorBase {
constructor (method, res, body, spec) {
super(method, res, body, spec)
this.message = 'Unable to authenticate, need: ' + res.headers.get('www-authenticate')
Error.captureStackTrace(this, HttpErrorAuthUnknown)
}
}
module.exports.HttpErrorAuthUnknown = HttpErrorAuthUnknown

247
spa/node_modules/npm-registry-fetch/lib/index.js generated vendored Normal file
View File

@@ -0,0 +1,247 @@
'use strict'
const { HttpErrorAuthOTP } = require('./errors.js')
const checkResponse = require('./check-response.js')
const getAuth = require('./auth.js')
const fetch = require('make-fetch-happen')
const JSONStream = require('minipass-json-stream')
const npa = require('npm-package-arg')
const qs = require('querystring')
const url = require('url')
const zlib = require('minizlib')
const { Minipass } = require('minipass')
const defaultOpts = require('./default-opts.js')
// WhatWG URL throws if it's not fully resolved
const urlIsValid = u => {
try {
return !!new url.URL(u)
} catch (_) {
return false
}
}
module.exports = regFetch
function regFetch (uri, /* istanbul ignore next */ opts_ = {}) {
const opts = {
...defaultOpts,
...opts_,
}
// if we did not get a fully qualified URI, then we look at the registry
// config or relevant scope to resolve it.
const uriValid = urlIsValid(uri)
let registry = opts.registry || defaultOpts.registry
if (!uriValid) {
registry = opts.registry = (
(opts.spec && pickRegistry(opts.spec, opts)) ||
opts.registry ||
registry
)
uri = `${
registry.trim().replace(/\/?$/g, '')
}/${
uri.trim().replace(/^\//, '')
}`
// asserts that this is now valid
new url.URL(uri)
}
const method = opts.method || 'GET'
// through that takes into account the scope, the prefix of `uri`, etc
const startTime = Date.now()
const auth = getAuth(uri, opts)
const headers = getHeaders(uri, auth, opts)
let body = opts.body
const bodyIsStream = Minipass.isStream(body)
const bodyIsPromise = body &&
typeof body === 'object' &&
typeof body.then === 'function'
if (
body && !bodyIsStream && !bodyIsPromise && typeof body !== 'string' && !Buffer.isBuffer(body)
) {
headers['content-type'] = headers['content-type'] || 'application/json'
body = JSON.stringify(body)
} else if (body && !headers['content-type']) {
headers['content-type'] = 'application/octet-stream'
}
if (opts.gzip) {
headers['content-encoding'] = 'gzip'
if (bodyIsStream) {
const gz = new zlib.Gzip()
body.on('error', /* istanbul ignore next: unlikely and hard to test */
err => gz.emit('error', err))
body = body.pipe(gz)
} else if (!bodyIsPromise) {
body = new zlib.Gzip().end(body).concat()
}
}
const parsed = new url.URL(uri)
if (opts.query) {
const q = typeof opts.query === 'string' ? qs.parse(opts.query)
: opts.query
Object.keys(q).forEach(key => {
if (q[key] !== undefined) {
parsed.searchParams.set(key, q[key])
}
})
uri = url.format(parsed)
}
if (parsed.searchParams.get('write') === 'true' && method === 'GET') {
// do not cache, because this GET is fetching a rev that will be
// used for a subsequent PUT or DELETE, so we need to conditionally
// update cache.
opts.offline = false
opts.preferOffline = false
opts.preferOnline = true
}
const doFetch = async fetchBody => {
const p = fetch(uri, {
agent: opts.agent,
algorithms: opts.algorithms,
body: fetchBody,
cache: getCacheMode(opts),
cachePath: opts.cache,
ca: opts.ca,
cert: auth.cert || opts.cert,
headers,
integrity: opts.integrity,
key: auth.key || opts.key,
localAddress: opts.localAddress,
maxSockets: opts.maxSockets,
memoize: opts.memoize,
method: method,
noProxy: opts.noProxy,
proxy: opts.httpsProxy || opts.proxy,
retry: opts.retry ? opts.retry : {
retries: opts.fetchRetries,
factor: opts.fetchRetryFactor,
minTimeout: opts.fetchRetryMintimeout,
maxTimeout: opts.fetchRetryMaxtimeout,
},
strictSSL: opts.strictSSL,
timeout: opts.timeout || 30 * 1000,
}).then(res => checkResponse({
method,
uri,
res,
registry,
startTime,
auth,
opts,
}))
if (typeof opts.otpPrompt === 'function') {
return p.catch(async er => {
if (er instanceof HttpErrorAuthOTP) {
let otp
// if otp fails to complete, we fail with that failure
try {
otp = await opts.otpPrompt()
} catch (_) {
// ignore this error
}
// if no otp provided, or otpPrompt errored, throw the original HTTP error
if (!otp) {
throw er
}
return regFetch(uri, { ...opts, otp })
}
throw er
})
} else {
return p
}
}
return Promise.resolve(body).then(doFetch)
}
module.exports.json = fetchJSON
function fetchJSON (uri, opts) {
return regFetch(uri, opts).then(res => res.json())
}
module.exports.json.stream = fetchJSONStream
function fetchJSONStream (uri, jsonPath,
/* istanbul ignore next */ opts_ = {}) {
const opts = { ...defaultOpts, ...opts_ }
const parser = JSONStream.parse(jsonPath, opts.mapJSON)
regFetch(uri, opts).then(res =>
res.body.on('error',
/* istanbul ignore next: unlikely and difficult to test */
er => parser.emit('error', er)).pipe(parser)
).catch(er => parser.emit('error', er))
return parser
}
module.exports.pickRegistry = pickRegistry
function pickRegistry (spec, opts = {}) {
spec = npa(spec)
let registry = spec.scope &&
opts[spec.scope.replace(/^@?/, '@') + ':registry']
if (!registry && opts.scope) {
registry = opts[opts.scope.replace(/^@?/, '@') + ':registry']
}
if (!registry) {
registry = opts.registry || defaultOpts.registry
}
return registry
}
function getCacheMode (opts) {
return opts.offline ? 'only-if-cached'
: opts.preferOffline ? 'force-cache'
: opts.preferOnline ? 'no-cache'
: 'default'
}
function getHeaders (uri, auth, opts) {
const headers = Object.assign({
'user-agent': opts.userAgent,
}, opts.headers || {})
if (opts.authType) {
headers['npm-auth-type'] = opts.authType
}
if (opts.scope) {
headers['npm-scope'] = opts.scope
}
if (opts.npmSession) {
headers['npm-session'] = opts.npmSession
}
if (opts.npmCommand) {
headers['npm-command'] = opts.npmCommand
}
// If a tarball is hosted on a different place than the manifest, only send
// credentials on `alwaysAuth`
if (auth.token) {
headers.authorization = `Bearer ${auth.token}`
} else if (auth.auth) {
headers.authorization = `Basic ${auth.auth}`
}
if (opts.otp) {
headers['npm-otp'] = opts.otp
}
return headers
}
module.exports.cleanUrl = require('./clean-url.js')

View File

@@ -0,0 +1 @@
../glob/dist/esm/bin.mjs

View File

@@ -0,0 +1 @@
../semver/bin/semver.js

View File

@@ -0,0 +1,20 @@
<!-- This file is automatically added by @npmcli/template-oss. Do not edit. -->
ISC License
Copyright npm, Inc.
Permission to use, copy, modify, and/or distribute this
software for any purpose with or without fee is hereby
granted, provided that the above copyright notice and this
permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND NPM DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO
EVENT SHALL NPM BE LIABLE FOR ANY SPECIAL, DIRECT,
INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER
TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE
USE OR PERFORMANCE OF THIS SOFTWARE.

View File

@@ -0,0 +1,97 @@
# @npmcli/fs
polyfills, and extensions, of the core `fs` module.
## Features
- `fs.cp` polyfill for node < 16.7.0
- `fs.withTempDir` added
- `fs.readdirScoped` added
- `fs.moveFile` added
## `fs.withTempDir(root, fn, options) -> Promise`
### Parameters
- `root`: the directory in which to create the temporary directory
- `fn`: a function that will be called with the path to the temporary directory
- `options`
- `tmpPrefix`: a prefix to be used in the generated directory name
### Usage
The `withTempDir` function creates a temporary directory, runs the provided
function (`fn`), then removes the temporary directory and resolves or rejects
based on the result of `fn`.
```js
const fs = require('@npmcli/fs')
const os = require('os')
// this function will be called with the full path to the temporary directory
// it is called with `await` behind the scenes, so can be async if desired.
const myFunction = async (tempPath) => {
return 'done!'
}
const main = async () => {
const result = await fs.withTempDir(os.tmpdir(), myFunction)
// result === 'done!'
}
main()
```
## `fs.readdirScoped(root) -> Promise`
### Parameters
- `root`: the directory to read
### Usage
Like `fs.readdir` but handling `@org/module` dirs as if they were
a single entry.
```javascript
const { readdirScoped } = require('@npmcli/fs')
const entries = await readdirScoped('node_modules')
// entries will be something like: ['a', '@org/foo', '@org/bar']
```
## `fs.moveFile(source, dest, options) -> Promise`
A fork of [move-file](https://github.com/sindresorhus/move-file) with
support for Common JS.
### Highlights
- Promise API.
- Supports moving a file across partitions and devices.
- Optionally prevent overwriting an existing file.
- Creates non-existent destination directories for you.
- Automatically recurses when source is a directory.
### Parameters
- `source`: File, or directory, you want to move.
- `dest`: Where you want the file or directory moved.
- `options`
- `overwrite` (`boolean`, default: `true`): Overwrite existing destination file(s).
### Usage
The built-in
[`fs.rename()`](https://nodejs.org/api/fs.html#fs_fs_rename_oldpath_newpath_callback)
is just a JavaScript wrapper for the C `rename(2)` function, which doesn't
support moving files across partitions or devices. This module is what you
would have expected `fs.rename()` to be.
```js
const { moveFile } = require('@npmcli/fs');
(async () => {
await moveFile('source/unicorn.png', 'destination/unicorn.png');
console.log('The file has been moved');
})();
```

View File

@@ -0,0 +1,20 @@
// given an input that may or may not be an object, return an object that has
// a copy of every defined property listed in 'copy'. if the input is not an
// object, assign it to the property named by 'wrap'
const getOptions = (input, { copy, wrap }) => {
const result = {}
if (input && typeof input === 'object') {
for (const prop of copy) {
if (input[prop] !== undefined) {
result[prop] = input[prop]
}
}
} else {
result[wrap] = input
}
return result
}
module.exports = getOptions

View File

@@ -0,0 +1,9 @@
const semver = require('semver')
const satisfies = (range) => {
return semver.satisfies(process.version, range, { includePrerelease: true })
}
module.exports = {
satisfies,
}

View File

@@ -0,0 +1,15 @@
(The MIT License)
Copyright (c) 2011-2017 JP Richardson
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files
(the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS
OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -0,0 +1,129 @@
'use strict'
const { inspect } = require('util')
// adapted from node's internal/errors
// https://github.com/nodejs/node/blob/c8a04049/lib/internal/errors.js
// close copy of node's internal SystemError class.
class SystemError {
constructor (code, prefix, context) {
// XXX context.code is undefined in all constructors used in cp/polyfill
// that may be a bug copied from node, maybe the constructor should use
// `code` not `errno`? nodejs/node#41104
let message = `${prefix}: ${context.syscall} returned ` +
`${context.code} (${context.message})`
if (context.path !== undefined) {
message += ` ${context.path}`
}
if (context.dest !== undefined) {
message += ` => ${context.dest}`
}
this.code = code
Object.defineProperties(this, {
name: {
value: 'SystemError',
enumerable: false,
writable: true,
configurable: true,
},
message: {
value: message,
enumerable: false,
writable: true,
configurable: true,
},
info: {
value: context,
enumerable: true,
configurable: true,
writable: false,
},
errno: {
get () {
return context.errno
},
set (value) {
context.errno = value
},
enumerable: true,
configurable: true,
},
syscall: {
get () {
return context.syscall
},
set (value) {
context.syscall = value
},
enumerable: true,
configurable: true,
},
})
if (context.path !== undefined) {
Object.defineProperty(this, 'path', {
get () {
return context.path
},
set (value) {
context.path = value
},
enumerable: true,
configurable: true,
})
}
if (context.dest !== undefined) {
Object.defineProperty(this, 'dest', {
get () {
return context.dest
},
set (value) {
context.dest = value
},
enumerable: true,
configurable: true,
})
}
}
toString () {
return `${this.name} [${this.code}]: ${this.message}`
}
[Symbol.for('nodejs.util.inspect.custom')] (_recurseTimes, ctx) {
return inspect(this, {
...ctx,
getters: true,
customInspect: false,
})
}
}
function E (code, message) {
module.exports[code] = class NodeError extends SystemError {
constructor (ctx) {
super(code, message, ctx)
}
}
}
E('ERR_FS_CP_DIR_TO_NON_DIR', 'Cannot overwrite directory with non-directory')
E('ERR_FS_CP_EEXIST', 'Target already exists')
E('ERR_FS_CP_EINVAL', 'Invalid src or dest')
E('ERR_FS_CP_FIFO_PIPE', 'Cannot copy a FIFO pipe')
E('ERR_FS_CP_NON_DIR_TO_DIR', 'Cannot overwrite non-directory with directory')
E('ERR_FS_CP_SOCKET', 'Cannot copy a socket file')
E('ERR_FS_CP_SYMLINK_TO_SUBDIRECTORY', 'Cannot overwrite symlink in subdirectory of self')
E('ERR_FS_CP_UNKNOWN', 'Cannot copy an unknown file type')
E('ERR_FS_EISDIR', 'Path is a directory')
module.exports.ERR_INVALID_ARG_TYPE = class ERR_INVALID_ARG_TYPE extends Error {
constructor (name, expected, actual) {
super()
this.code = 'ERR_INVALID_ARG_TYPE'
this.message = `The ${name} argument must be ${expected}. Received ${typeof actual}`
}
}

View File

@@ -0,0 +1,22 @@
const fs = require('fs/promises')
const getOptions = require('../common/get-options.js')
const node = require('../common/node.js')
const polyfill = require('./polyfill.js')
// node 16.7.0 added fs.cp
const useNative = node.satisfies('>=16.7.0')
const cp = async (src, dest, opts) => {
const options = getOptions(opts, {
copy: ['dereference', 'errorOnExist', 'filter', 'force', 'preserveTimestamps', 'recursive'],
})
// the polyfill is tested separately from this module, no need to hack
// process.version to try to trigger it just for coverage
// istanbul ignore next
return useNative
? fs.cp(src, dest, options)
: polyfill(src, dest, options)
}
module.exports = cp

View File

@@ -0,0 +1,428 @@
// this file is a modified version of the code in node 17.2.0
// which is, in turn, a modified version of the fs-extra module on npm
// node core changes:
// - Use of the assert module has been replaced with core's error system.
// - All code related to the glob dependency has been removed.
// - Bring your own custom fs module is not currently supported.
// - Some basic code cleanup.
// changes here:
// - remove all callback related code
// - drop sync support
// - change assertions back to non-internal methods (see options.js)
// - throws ENOTDIR when rmdir gets an ENOENT for a path that exists in Windows
'use strict'
const {
ERR_FS_CP_DIR_TO_NON_DIR,
ERR_FS_CP_EEXIST,
ERR_FS_CP_EINVAL,
ERR_FS_CP_FIFO_PIPE,
ERR_FS_CP_NON_DIR_TO_DIR,
ERR_FS_CP_SOCKET,
ERR_FS_CP_SYMLINK_TO_SUBDIRECTORY,
ERR_FS_CP_UNKNOWN,
ERR_FS_EISDIR,
ERR_INVALID_ARG_TYPE,
} = require('./errors.js')
const {
constants: {
errno: {
EEXIST,
EISDIR,
EINVAL,
ENOTDIR,
},
},
} = require('os')
const {
chmod,
copyFile,
lstat,
mkdir,
readdir,
readlink,
stat,
symlink,
unlink,
utimes,
} = require('fs/promises')
const {
dirname,
isAbsolute,
join,
parse,
resolve,
sep,
toNamespacedPath,
} = require('path')
const { fileURLToPath } = require('url')
const defaultOptions = {
dereference: false,
errorOnExist: false,
filter: undefined,
force: true,
preserveTimestamps: false,
recursive: false,
}
async function cp (src, dest, opts) {
if (opts != null && typeof opts !== 'object') {
throw new ERR_INVALID_ARG_TYPE('options', ['Object'], opts)
}
return cpFn(
toNamespacedPath(getValidatedPath(src)),
toNamespacedPath(getValidatedPath(dest)),
{ ...defaultOptions, ...opts })
}
function getValidatedPath (fileURLOrPath) {
const path = fileURLOrPath != null && fileURLOrPath.href
&& fileURLOrPath.origin
? fileURLToPath(fileURLOrPath)
: fileURLOrPath
return path
}
async function cpFn (src, dest, opts) {
// Warn about using preserveTimestamps on 32-bit node
// istanbul ignore next
if (opts.preserveTimestamps && process.arch === 'ia32') {
const warning = 'Using the preserveTimestamps option in 32-bit ' +
'node is not recommended'
process.emitWarning(warning, 'TimestampPrecisionWarning')
}
const stats = await checkPaths(src, dest, opts)
const { srcStat, destStat } = stats
await checkParentPaths(src, srcStat, dest)
if (opts.filter) {
return handleFilter(checkParentDir, destStat, src, dest, opts)
}
return checkParentDir(destStat, src, dest, opts)
}
async function checkPaths (src, dest, opts) {
const { 0: srcStat, 1: destStat } = await getStats(src, dest, opts)
if (destStat) {
if (areIdentical(srcStat, destStat)) {
throw new ERR_FS_CP_EINVAL({
message: 'src and dest cannot be the same',
path: dest,
syscall: 'cp',
errno: EINVAL,
})
}
if (srcStat.isDirectory() && !destStat.isDirectory()) {
throw new ERR_FS_CP_DIR_TO_NON_DIR({
message: `cannot overwrite directory ${src} ` +
`with non-directory ${dest}`,
path: dest,
syscall: 'cp',
errno: EISDIR,
})
}
if (!srcStat.isDirectory() && destStat.isDirectory()) {
throw new ERR_FS_CP_NON_DIR_TO_DIR({
message: `cannot overwrite non-directory ${src} ` +
`with directory ${dest}`,
path: dest,
syscall: 'cp',
errno: ENOTDIR,
})
}
}
if (srcStat.isDirectory() && isSrcSubdir(src, dest)) {
throw new ERR_FS_CP_EINVAL({
message: `cannot copy ${src} to a subdirectory of self ${dest}`,
path: dest,
syscall: 'cp',
errno: EINVAL,
})
}
return { srcStat, destStat }
}
function areIdentical (srcStat, destStat) {
return destStat.ino && destStat.dev && destStat.ino === srcStat.ino &&
destStat.dev === srcStat.dev
}
function getStats (src, dest, opts) {
const statFunc = opts.dereference ?
(file) => stat(file, { bigint: true }) :
(file) => lstat(file, { bigint: true })
return Promise.all([
statFunc(src),
statFunc(dest).catch((err) => {
// istanbul ignore next: unsure how to cover.
if (err.code === 'ENOENT') {
return null
}
// istanbul ignore next: unsure how to cover.
throw err
}),
])
}
async function checkParentDir (destStat, src, dest, opts) {
const destParent = dirname(dest)
const dirExists = await pathExists(destParent)
if (dirExists) {
return getStatsForCopy(destStat, src, dest, opts)
}
await mkdir(destParent, { recursive: true })
return getStatsForCopy(destStat, src, dest, opts)
}
function pathExists (dest) {
return stat(dest).then(
() => true,
// istanbul ignore next: not sure when this would occur
(err) => (err.code === 'ENOENT' ? false : Promise.reject(err)))
}
// Recursively check if dest parent is a subdirectory of src.
// It works for all file types including symlinks since it
// checks the src and dest inodes. It starts from the deepest
// parent and stops once it reaches the src parent or the root path.
async function checkParentPaths (src, srcStat, dest) {
const srcParent = resolve(dirname(src))
const destParent = resolve(dirname(dest))
if (destParent === srcParent || destParent === parse(destParent).root) {
return
}
let destStat
try {
destStat = await stat(destParent, { bigint: true })
} catch (err) {
// istanbul ignore else: not sure when this would occur
if (err.code === 'ENOENT') {
return
}
// istanbul ignore next: not sure when this would occur
throw err
}
if (areIdentical(srcStat, destStat)) {
throw new ERR_FS_CP_EINVAL({
message: `cannot copy ${src} to a subdirectory of self ${dest}`,
path: dest,
syscall: 'cp',
errno: EINVAL,
})
}
return checkParentPaths(src, srcStat, destParent)
}
const normalizePathToArray = (path) =>
resolve(path).split(sep).filter(Boolean)
// Return true if dest is a subdir of src, otherwise false.
// It only checks the path strings.
function isSrcSubdir (src, dest) {
const srcArr = normalizePathToArray(src)
const destArr = normalizePathToArray(dest)
return srcArr.every((cur, i) => destArr[i] === cur)
}
async function handleFilter (onInclude, destStat, src, dest, opts, cb) {
const include = await opts.filter(src, dest)
if (include) {
return onInclude(destStat, src, dest, opts, cb)
}
}
function startCopy (destStat, src, dest, opts) {
if (opts.filter) {
return handleFilter(getStatsForCopy, destStat, src, dest, opts)
}
return getStatsForCopy(destStat, src, dest, opts)
}
async function getStatsForCopy (destStat, src, dest, opts) {
const statFn = opts.dereference ? stat : lstat
const srcStat = await statFn(src)
// istanbul ignore else: can't portably test FIFO
if (srcStat.isDirectory() && opts.recursive) {
return onDir(srcStat, destStat, src, dest, opts)
} else if (srcStat.isDirectory()) {
throw new ERR_FS_EISDIR({
message: `${src} is a directory (not copied)`,
path: src,
syscall: 'cp',
errno: EINVAL,
})
} else if (srcStat.isFile() ||
srcStat.isCharacterDevice() ||
srcStat.isBlockDevice()) {
return onFile(srcStat, destStat, src, dest, opts)
} else if (srcStat.isSymbolicLink()) {
return onLink(destStat, src, dest)
} else if (srcStat.isSocket()) {
throw new ERR_FS_CP_SOCKET({
message: `cannot copy a socket file: ${dest}`,
path: dest,
syscall: 'cp',
errno: EINVAL,
})
} else if (srcStat.isFIFO()) {
throw new ERR_FS_CP_FIFO_PIPE({
message: `cannot copy a FIFO pipe: ${dest}`,
path: dest,
syscall: 'cp',
errno: EINVAL,
})
}
// istanbul ignore next: should be unreachable
throw new ERR_FS_CP_UNKNOWN({
message: `cannot copy an unknown file type: ${dest}`,
path: dest,
syscall: 'cp',
errno: EINVAL,
})
}
function onFile (srcStat, destStat, src, dest, opts) {
if (!destStat) {
return _copyFile(srcStat, src, dest, opts)
}
return mayCopyFile(srcStat, src, dest, opts)
}
async function mayCopyFile (srcStat, src, dest, opts) {
if (opts.force) {
await unlink(dest)
return _copyFile(srcStat, src, dest, opts)
} else if (opts.errorOnExist) {
throw new ERR_FS_CP_EEXIST({
message: `${dest} already exists`,
path: dest,
syscall: 'cp',
errno: EEXIST,
})
}
}
async function _copyFile (srcStat, src, dest, opts) {
await copyFile(src, dest)
if (opts.preserveTimestamps) {
return handleTimestampsAndMode(srcStat.mode, src, dest)
}
return setDestMode(dest, srcStat.mode)
}
async function handleTimestampsAndMode (srcMode, src, dest) {
// Make sure the file is writable before setting the timestamp
// otherwise open fails with EPERM when invoked with 'r+'
// (through utimes call)
if (fileIsNotWritable(srcMode)) {
await makeFileWritable(dest, srcMode)
return setDestTimestampsAndMode(srcMode, src, dest)
}
return setDestTimestampsAndMode(srcMode, src, dest)
}
function fileIsNotWritable (srcMode) {
return (srcMode & 0o200) === 0
}
function makeFileWritable (dest, srcMode) {
return setDestMode(dest, srcMode | 0o200)
}
async function setDestTimestampsAndMode (srcMode, src, dest) {
await setDestTimestamps(src, dest)
return setDestMode(dest, srcMode)
}
function setDestMode (dest, srcMode) {
return chmod(dest, srcMode)
}
async function setDestTimestamps (src, dest) {
// The initial srcStat.atime cannot be trusted
// because it is modified by the read(2) system call
// (See https://nodejs.org/api/fs.html#fs_stat_time_values)
const updatedSrcStat = await stat(src)
return utimes(dest, updatedSrcStat.atime, updatedSrcStat.mtime)
}
function onDir (srcStat, destStat, src, dest, opts) {
if (!destStat) {
return mkDirAndCopy(srcStat.mode, src, dest, opts)
}
return copyDir(src, dest, opts)
}
async function mkDirAndCopy (srcMode, src, dest, opts) {
await mkdir(dest)
await copyDir(src, dest, opts)
return setDestMode(dest, srcMode)
}
async function copyDir (src, dest, opts) {
const dir = await readdir(src)
for (let i = 0; i < dir.length; i++) {
const item = dir[i]
const srcItem = join(src, item)
const destItem = join(dest, item)
const { destStat } = await checkPaths(srcItem, destItem, opts)
await startCopy(destStat, srcItem, destItem, opts)
}
}
async function onLink (destStat, src, dest) {
let resolvedSrc = await readlink(src)
if (!isAbsolute(resolvedSrc)) {
resolvedSrc = resolve(dirname(src), resolvedSrc)
}
if (!destStat) {
return symlink(resolvedSrc, dest)
}
let resolvedDest
try {
resolvedDest = await readlink(dest)
} catch (err) {
// Dest exists and is a regular file or directory,
// Windows may throw UNKNOWN error. If dest already exists,
// fs throws error anyway, so no need to guard against it here.
// istanbul ignore next: can only test on windows
if (err.code === 'EINVAL' || err.code === 'UNKNOWN') {
return symlink(resolvedSrc, dest)
}
// istanbul ignore next: should not be possible
throw err
}
if (!isAbsolute(resolvedDest)) {
resolvedDest = resolve(dirname(dest), resolvedDest)
}
if (isSrcSubdir(resolvedSrc, resolvedDest)) {
throw new ERR_FS_CP_EINVAL({
message: `cannot copy ${resolvedSrc} to a subdirectory of self ` +
`${resolvedDest}`,
path: dest,
syscall: 'cp',
errno: EINVAL,
})
}
// Do not copy if src is a subdir of dest since unlinking
// dest in this case would result in removing src contents
// and therefore a broken symlink would be created.
const srcStat = await stat(src)
if (srcStat.isDirectory() && isSrcSubdir(resolvedDest, resolvedSrc)) {
throw new ERR_FS_CP_SYMLINK_TO_SUBDIRECTORY({
message: `cannot overwrite ${resolvedDest} with ${resolvedSrc}`,
path: dest,
syscall: 'cp',
errno: EINVAL,
})
}
return copyLink(resolvedSrc, dest)
}
async function copyLink (resolvedSrc, dest) {
await unlink(dest)
return symlink(resolvedSrc, dest)
}
module.exports = cp

View File

@@ -0,0 +1,13 @@
'use strict'
const cp = require('./cp/index.js')
const withTempDir = require('./with-temp-dir.js')
const readdirScoped = require('./readdir-scoped.js')
const moveFile = require('./move-file.js')
module.exports = {
cp,
withTempDir,
readdirScoped,
moveFile,
}

View File

@@ -0,0 +1,78 @@
const { dirname, join, resolve, relative, isAbsolute } = require('path')
const fs = require('fs/promises')
const pathExists = async path => {
try {
await fs.access(path)
return true
} catch (er) {
return er.code !== 'ENOENT'
}
}
const moveFile = async (source, destination, options = {}, root = true, symlinks = []) => {
if (!source || !destination) {
throw new TypeError('`source` and `destination` file required')
}
options = {
overwrite: true,
...options,
}
if (!options.overwrite && await pathExists(destination)) {
throw new Error(`The destination file exists: ${destination}`)
}
await fs.mkdir(dirname(destination), { recursive: true })
try {
await fs.rename(source, destination)
} catch (error) {
if (error.code === 'EXDEV' || error.code === 'EPERM') {
const sourceStat = await fs.lstat(source)
if (sourceStat.isDirectory()) {
const files = await fs.readdir(source)
await Promise.all(files.map((file) =>
moveFile(join(source, file), join(destination, file), options, false, symlinks)
))
} else if (sourceStat.isSymbolicLink()) {
symlinks.push({ source, destination })
} else {
await fs.copyFile(source, destination)
}
} else {
throw error
}
}
if (root) {
await Promise.all(symlinks.map(async ({ source: symSource, destination: symDestination }) => {
let target = await fs.readlink(symSource)
// junction symlinks in windows will be absolute paths, so we need to
// make sure they point to the symlink destination
if (isAbsolute(target)) {
target = resolve(symDestination, relative(symSource, target))
}
// try to determine what the actual file is so we can create the correct
// type of symlink in windows
let targetStat = 'file'
try {
targetStat = await fs.stat(resolve(dirname(symSource), target))
if (targetStat.isDirectory()) {
targetStat = 'junction'
}
} catch {
// targetStat remains 'file'
}
await fs.symlink(
target,
symDestination,
targetStat
)
}))
await fs.rm(source, { recursive: true, force: true })
}
}
module.exports = moveFile

View File

@@ -0,0 +1,20 @@
const { readdir } = require('fs/promises')
const { join } = require('path')
const readdirScoped = async (dir) => {
const results = []
for (const item of await readdir(dir)) {
if (item.startsWith('@')) {
for (const scopedItem of await readdir(join(dir, item))) {
results.push(join(item, scopedItem))
}
} else {
results.push(item)
}
}
return results
}
module.exports = readdirScoped

View File

@@ -0,0 +1,39 @@
const { join, sep } = require('path')
const getOptions = require('./common/get-options.js')
const { mkdir, mkdtemp, rm } = require('fs/promises')
// create a temp directory, ensure its permissions match its parent, then call
// the supplied function passing it the path to the directory. clean up after
// the function finishes, whether it throws or not
const withTempDir = async (root, fn, opts) => {
const options = getOptions(opts, {
copy: ['tmpPrefix'],
})
// create the directory
await mkdir(root, { recursive: true })
const target = await mkdtemp(join(`${root}${sep}`, options.tmpPrefix || ''))
let err
let result
try {
result = await fn(target)
} catch (_err) {
err = _err
}
try {
await rm(target, { force: true, recursive: true })
} catch {
// ignore errors
}
if (err) {
throw err
}
return result
}
module.exports = withTempDir

View File

@@ -0,0 +1,52 @@
{
"name": "@npmcli/fs",
"version": "3.1.1",
"description": "filesystem utilities for the npm cli",
"main": "lib/index.js",
"files": [
"bin/",
"lib/"
],
"scripts": {
"snap": "tap",
"test": "tap",
"npmclilint": "npmcli-lint",
"lint": "eslint \"**/*.{js,cjs,ts,mjs,jsx,tsx}\"",
"lintfix": "npm run lint -- --fix",
"posttest": "npm run lint",
"postsnap": "npm run lintfix --",
"postlint": "template-oss-check",
"template-oss-apply": "template-oss-apply --force"
},
"repository": {
"type": "git",
"url": "git+https://github.com/npm/fs.git"
},
"keywords": [
"npm",
"oss"
],
"author": "GitHub Inc.",
"license": "ISC",
"devDependencies": {
"@npmcli/eslint-config": "^4.0.0",
"@npmcli/template-oss": "4.22.0",
"tap": "^16.0.1"
},
"dependencies": {
"semver": "^7.3.5"
},
"engines": {
"node": "^14.17.0 || ^16.13.0 || >=18.0.0"
},
"templateOSS": {
"//@npmcli/template-oss": "This file is partially managed by @npmcli/template-oss. Edits may be overwritten.",
"version": "4.22.0"
},
"tap": {
"nyc-arg": [
"--exclude",
"tap-snapshots/**"
]
}
}

View File

@@ -0,0 +1,2 @@
tidelift: "npm/brace-expansion"
patreon: juliangruber

View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2013 Julian Gruber <julian@juliangruber.com>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -0,0 +1,135 @@
# brace-expansion
[Brace expansion](https://www.gnu.org/software/bash/manual/html_node/Brace-Expansion.html),
as known from sh/bash, in JavaScript.
[![build status](https://secure.travis-ci.org/juliangruber/brace-expansion.svg)](http://travis-ci.org/juliangruber/brace-expansion)
[![downloads](https://img.shields.io/npm/dm/brace-expansion.svg)](https://www.npmjs.org/package/brace-expansion)
[![Greenkeeper badge](https://badges.greenkeeper.io/juliangruber/brace-expansion.svg)](https://greenkeeper.io/)
[![testling badge](https://ci.testling.com/juliangruber/brace-expansion.png)](https://ci.testling.com/juliangruber/brace-expansion)
## Example
```js
var expand = require('brace-expansion');
expand('file-{a,b,c}.jpg')
// => ['file-a.jpg', 'file-b.jpg', 'file-c.jpg']
expand('-v{,,}')
// => ['-v', '-v', '-v']
expand('file{0..2}.jpg')
// => ['file0.jpg', 'file1.jpg', 'file2.jpg']
expand('file-{a..c}.jpg')
// => ['file-a.jpg', 'file-b.jpg', 'file-c.jpg']
expand('file{2..0}.jpg')
// => ['file2.jpg', 'file1.jpg', 'file0.jpg']
expand('file{0..4..2}.jpg')
// => ['file0.jpg', 'file2.jpg', 'file4.jpg']
expand('file-{a..e..2}.jpg')
// => ['file-a.jpg', 'file-c.jpg', 'file-e.jpg']
expand('file{00..10..5}.jpg')
// => ['file00.jpg', 'file05.jpg', 'file10.jpg']
expand('{{A..C},{a..c}}')
// => ['A', 'B', 'C', 'a', 'b', 'c']
expand('ppp{,config,oe{,conf}}')
// => ['ppp', 'pppconfig', 'pppoe', 'pppoeconf']
```
## API
```js
var expand = require('brace-expansion');
```
### var expanded = expand(str)
Return an array of all possible and valid expansions of `str`. If none are
found, `[str]` is returned.
Valid expansions are:
```js
/^(.*,)+(.+)?$/
// {a,b,...}
```
A comma separated list of options, like `{a,b}` or `{a,{b,c}}` or `{,a,}`.
```js
/^-?\d+\.\.-?\d+(\.\.-?\d+)?$/
// {x..y[..incr]}
```
A numeric sequence from `x` to `y` inclusive, with optional increment.
If `x` or `y` start with a leading `0`, all the numbers will be padded
to have equal length. Negative numbers and backwards iteration work too.
```js
/^-?\d+\.\.-?\d+(\.\.-?\d+)?$/
// {x..y[..incr]}
```
An alphabetic sequence from `x` to `y` inclusive, with optional increment.
`x` and `y` must be exactly one character, and if given, `incr` must be a
number.
For compatibility reasons, the string `${` is not eligible for brace expansion.
## Installation
With [npm](https://npmjs.org) do:
```bash
npm install brace-expansion
```
## Contributors
- [Julian Gruber](https://github.com/juliangruber)
- [Isaac Z. Schlueter](https://github.com/isaacs)
## Sponsors
This module is proudly supported by my [Sponsors](https://github.com/juliangruber/sponsors)!
Do you want to support modules like this to improve their quality, stability and weigh in on new features? Then please consider donating to my [Patreon](https://www.patreon.com/juliangruber). Not sure how much of my modules you're using? Try [feross/thanks](https://github.com/feross/thanks)!
## Security contact information
To report a security vulnerability, please use the
[Tidelift security contact](https://tidelift.com/security).
Tidelift will coordinate the fix and disclosure.
## License
(MIT)
Copyright (c) 2013 Julian Gruber &lt;julian@juliangruber.com&gt;
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -0,0 +1,203 @@
var balanced = require('balanced-match');
module.exports = expandTop;
var escSlash = '\0SLASH'+Math.random()+'\0';
var escOpen = '\0OPEN'+Math.random()+'\0';
var escClose = '\0CLOSE'+Math.random()+'\0';
var escComma = '\0COMMA'+Math.random()+'\0';
var escPeriod = '\0PERIOD'+Math.random()+'\0';
function numeric(str) {
return parseInt(str, 10) == str
? parseInt(str, 10)
: str.charCodeAt(0);
}
function escapeBraces(str) {
return str.split('\\\\').join(escSlash)
.split('\\{').join(escOpen)
.split('\\}').join(escClose)
.split('\\,').join(escComma)
.split('\\.').join(escPeriod);
}
function unescapeBraces(str) {
return str.split(escSlash).join('\\')
.split(escOpen).join('{')
.split(escClose).join('}')
.split(escComma).join(',')
.split(escPeriod).join('.');
}
// Basically just str.split(","), but handling cases
// where we have nested braced sections, which should be
// treated as individual members, like {a,{b,c},d}
function parseCommaParts(str) {
if (!str)
return [''];
var parts = [];
var m = balanced('{', '}', str);
if (!m)
return str.split(',');
var pre = m.pre;
var body = m.body;
var post = m.post;
var p = pre.split(',');
p[p.length-1] += '{' + body + '}';
var postParts = parseCommaParts(post);
if (post.length) {
p[p.length-1] += postParts.shift();
p.push.apply(p, postParts);
}
parts.push.apply(parts, p);
return parts;
}
function expandTop(str) {
if (!str)
return [];
// I don't know why Bash 4.3 does this, but it does.
// Anything starting with {} will have the first two bytes preserved
// but *only* at the top level, so {},a}b will not expand to anything,
// but a{},b}c will be expanded to [a}c,abc].
// One could argue that this is a bug in Bash, but since the goal of
// this module is to match Bash's rules, we escape a leading {}
if (str.substr(0, 2) === '{}') {
str = '\\{\\}' + str.substr(2);
}
return expand(escapeBraces(str), true).map(unescapeBraces);
}
function embrace(str) {
return '{' + str + '}';
}
function isPadded(el) {
return /^-?0\d/.test(el);
}
function lte(i, y) {
return i <= y;
}
function gte(i, y) {
return i >= y;
}
function expand(str, isTop) {
var expansions = [];
var m = balanced('{', '}', str);
if (!m) return [str];
// no need to expand pre, since it is guaranteed to be free of brace-sets
var pre = m.pre;
var post = m.post.length
? expand(m.post, false)
: [''];
if (/\$$/.test(m.pre)) {
for (var k = 0; k < post.length; k++) {
var expansion = pre+ '{' + m.body + '}' + post[k];
expansions.push(expansion);
}
} else {
var isNumericSequence = /^-?\d+\.\.-?\d+(?:\.\.-?\d+)?$/.test(m.body);
var isAlphaSequence = /^[a-zA-Z]\.\.[a-zA-Z](?:\.\.-?\d+)?$/.test(m.body);
var isSequence = isNumericSequence || isAlphaSequence;
var isOptions = m.body.indexOf(',') >= 0;
if (!isSequence && !isOptions) {
// {a},b}
if (m.post.match(/,.*\}/)) {
str = m.pre + '{' + m.body + escClose + m.post;
return expand(str);
}
return [str];
}
var n;
if (isSequence) {
n = m.body.split(/\.\./);
} else {
n = parseCommaParts(m.body);
if (n.length === 1) {
// x{{a,b}}y ==> x{a}y x{b}y
n = expand(n[0], false).map(embrace);
if (n.length === 1) {
return post.map(function(p) {
return m.pre + n[0] + p;
});
}
}
}
// at this point, n is the parts, and we know it's not a comma set
// with a single entry.
var N;
if (isSequence) {
var x = numeric(n[0]);
var y = numeric(n[1]);
var width = Math.max(n[0].length, n[1].length)
var incr = n.length == 3
? Math.abs(numeric(n[2]))
: 1;
var test = lte;
var reverse = y < x;
if (reverse) {
incr *= -1;
test = gte;
}
var pad = n.some(isPadded);
N = [];
for (var i = x; test(i, y); i += incr) {
var c;
if (isAlphaSequence) {
c = String.fromCharCode(i);
if (c === '\\')
c = '';
} else {
c = String(i);
if (pad) {
var need = width - c.length;
if (need > 0) {
var z = new Array(need + 1).join('0');
if (i < 0)
c = '-' + z + c.slice(1);
else
c = z + c;
}
}
}
N.push(c);
}
} else {
N = [];
for (var j = 0; j < n.length; j++) {
N.push.apply(N, expand(n[j], false));
}
}
for (var j = 0; j < N.length; j++) {
for (var k = 0; k < post.length; k++) {
var expansion = pre + N[j] + post[k];
if (!isTop || isSequence || expansion)
expansions.push(expansion);
}
}
}
return expansions;
}

View File

@@ -0,0 +1,46 @@
{
"name": "brace-expansion",
"description": "Brace expansion as known from sh/bash",
"version": "2.0.1",
"repository": {
"type": "git",
"url": "git://github.com/juliangruber/brace-expansion.git"
},
"homepage": "https://github.com/juliangruber/brace-expansion",
"main": "index.js",
"scripts": {
"test": "tape test/*.js",
"gentest": "bash test/generate.sh",
"bench": "matcha test/perf/bench.js"
},
"dependencies": {
"balanced-match": "^1.0.0"
},
"devDependencies": {
"@c4312/matcha": "^1.3.1",
"tape": "^4.6.0"
},
"keywords": [],
"author": {
"name": "Julian Gruber",
"email": "mail@juliangruber.com",
"url": "http://juliangruber.com"
},
"license": "MIT",
"testling": {
"files": "test/*.js",
"browsers": [
"ie/8..latest",
"firefox/20..latest",
"firefox/nightly",
"chrome/25..latest",
"chrome/canary",
"opera/12..latest",
"opera/next",
"safari/5.1..latest",
"ipad/6.0..latest",
"iphone/6.0..latest",
"android-browser/4.2..latest"
]
}
}

View File

@@ -0,0 +1,16 @@
ISC License
Copyright (c) npm, Inc.
Permission to use, copy, modify, and/or distribute this software for
any purpose with or without fee is hereby granted, provided that the
above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE COPYRIGHT HOLDER DISCLAIMS
ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE
USE OR PERFORMANCE OF THIS SOFTWARE.

View File

@@ -0,0 +1,716 @@
# cacache [![npm version](https://img.shields.io/npm/v/cacache.svg)](https://npm.im/cacache) [![license](https://img.shields.io/npm/l/cacache.svg)](https://npm.im/cacache) [![Travis](https://img.shields.io/travis/npm/cacache.svg)](https://travis-ci.org/npm/cacache) [![AppVeyor](https://ci.appveyor.com/api/projects/status/github/npm/cacache?svg=true)](https://ci.appveyor.com/project/npm/cacache) [![Coverage Status](https://coveralls.io/repos/github/npm/cacache/badge.svg?branch=latest)](https://coveralls.io/github/npm/cacache?branch=latest)
[`cacache`](https://github.com/npm/cacache) is a Node.js library for managing
local key and content address caches. It's really fast, really good at
concurrency, and it will never give you corrupted data, even if cache files
get corrupted or manipulated.
On systems that support user and group settings on files, cacache will
match the `uid` and `gid` values to the folder where the cache lives, even
when running as `root`.
It was written to be used as [npm](https://npm.im)'s local cache, but can
just as easily be used on its own.
## Install
`$ npm install --save cacache`
## Table of Contents
* [Example](#example)
* [Features](#features)
* [Contributing](#contributing)
* [API](#api)
* [Using localized APIs](#localized-api)
* Reading
* [`ls`](#ls)
* [`ls.stream`](#ls-stream)
* [`get`](#get-data)
* [`get.stream`](#get-stream)
* [`get.info`](#get-info)
* [`get.hasContent`](#get-hasContent)
* Writing
* [`put`](#put-data)
* [`put.stream`](#put-stream)
* [`rm.all`](#rm-all)
* [`rm.entry`](#rm-entry)
* [`rm.content`](#rm-content)
* [`index.compact`](#index-compact)
* [`index.insert`](#index-insert)
* Utilities
* [`clearMemoized`](#clear-memoized)
* [`tmp.mkdir`](#tmp-mkdir)
* [`tmp.withTmp`](#with-tmp)
* Integrity
* [Subresource Integrity](#integrity)
* [`verify`](#verify)
* [`verify.lastRun`](#verify-last-run)
### Example
```javascript
const cacache = require('cacache')
const fs = require('fs')
const tarball = '/path/to/mytar.tgz'
const cachePath = '/tmp/my-toy-cache'
const key = 'my-unique-key-1234'
// Cache it! Use `cachePath` as the root of the content cache
cacache.put(cachePath, key, '10293801983029384').then(integrity => {
console.log(`Saved content to ${cachePath}.`)
})
const destination = '/tmp/mytar.tgz'
// Copy the contents out of the cache and into their destination!
// But this time, use stream instead!
cacache.get.stream(
cachePath, key
).pipe(
fs.createWriteStream(destination)
).on('finish', () => {
console.log('done extracting!')
})
// The same thing, but skip the key index.
cacache.get.byDigest(cachePath, integrityHash).then(data => {
fs.writeFile(destination, data, err => {
console.log('tarball data fetched based on its sha512sum and written out!')
})
})
```
### Features
* Extraction by key or by content address (shasum, etc)
* [Subresource Integrity](#integrity) web standard support
* Multi-hash support - safely host sha1, sha512, etc, in a single cache
* Automatic content deduplication
* Fault tolerance (immune to corruption, partial writes, process races, etc)
* Consistency guarantees on read and write (full data verification)
* Lockless, high-concurrency cache access
* Streaming support
* Promise support
* Fast -- sub-millisecond reads and writes including verification
* Arbitrary metadata storage
* Garbage collection and additional offline verification
* Thorough test coverage
* There's probably a bloom filter in there somewhere. Those are cool, right? 🤔
### Contributing
The cacache team enthusiastically welcomes contributions and project participation! There's a bunch of things you can do if you want to contribute! Please don't hesitate to jump in if you'd like to, or even ask us questions if something isn't clear.
All participants and maintainers in this project are expected to follow [Code of Conduct](CODE_OF_CONDUCT.md), and just generally be excellent to each other.
Please refer to the [Changelog](CHANGELOG.md) for project history details, too.
Happy hacking!
### API
#### <a name="ls"></a> `> cacache.ls(cache) -> Promise<Object>`
Lists info for all entries currently in the cache as a single large object. Each
entry in the object will be keyed by the unique index key, with corresponding
[`get.info`](#get-info) objects as the values.
##### Example
```javascript
cacache.ls(cachePath).then(console.log)
// Output
{
'my-thing': {
key: 'my-thing',
integrity: 'sha512-BaSe64/EnCoDED+HAsh=='
path: '.testcache/content/deadbeef', // joined with `cachePath`
time: 12345698490,
size: 4023948,
metadata: {
name: 'blah',
version: '1.2.3',
description: 'this was once a package but now it is my-thing'
}
},
'other-thing': {
key: 'other-thing',
integrity: 'sha1-ANothER+hasH=',
path: '.testcache/content/bada55',
time: 11992309289,
size: 111112
}
}
```
#### <a name="ls-stream"></a> `> cacache.ls.stream(cache) -> Readable`
Lists info for all entries currently in the cache as a single large object.
This works just like [`ls`](#ls), except [`get.info`](#get-info) entries are
returned as `'data'` events on the returned stream.
##### Example
```javascript
cacache.ls.stream(cachePath).on('data', console.log)
// Output
{
key: 'my-thing',
integrity: 'sha512-BaSe64HaSh',
path: '.testcache/content/deadbeef', // joined with `cachePath`
time: 12345698490,
size: 13423,
metadata: {
name: 'blah',
version: '1.2.3',
description: 'this was once a package but now it is my-thing'
}
}
{
key: 'other-thing',
integrity: 'whirlpool-WoWSoMuchSupport',
path: '.testcache/content/bada55',
time: 11992309289,
size: 498023984029
}
{
...
}
```
#### <a name="get-data"></a> `> cacache.get(cache, key, [opts]) -> Promise({data, metadata, integrity})`
Returns an object with the cached data, digest, and metadata identified by
`key`. The `data` property of this object will be a `Buffer` instance that
presumably holds some data that means something to you. I'm sure you know what
to do with it! cacache just won't care.
`integrity` is a [Subresource
Integrity](#integrity)
string. That is, a string that can be used to verify `data`, which looks like
`<hash-algorithm>-<base64-integrity-hash>`.
If there is no content identified by `key`, or if the locally-stored data does
not pass the validity checksum, the promise will be rejected.
A sub-function, `get.byDigest` may be used for identical behavior, except lookup
will happen by integrity hash, bypassing the index entirely. This version of the
function *only* returns `data` itself, without any wrapper.
See: [options](#get-options)
##### Note
This function loads the entire cache entry into memory before returning it. If
you're dealing with Very Large data, consider using [`get.stream`](#get-stream)
instead.
##### Example
```javascript
// Look up by key
cache.get(cachePath, 'my-thing').then(console.log)
// Output:
{
metadata: {
thingName: 'my'
},
integrity: 'sha512-BaSe64HaSh',
data: Buffer#<deadbeef>,
size: 9320
}
// Look up by digest
cache.get.byDigest(cachePath, 'sha512-BaSe64HaSh').then(console.log)
// Output:
Buffer#<deadbeef>
```
#### <a name="get-stream"></a> `> cacache.get.stream(cache, key, [opts]) -> Readable`
Returns a [Readable Stream](https://nodejs.org/api/stream.html#stream_readable_streams) of the cached data identified by `key`.
If there is no content identified by `key`, or if the locally-stored data does
not pass the validity checksum, an error will be emitted.
`metadata` and `integrity` events will be emitted before the stream closes, if
you need to collect that extra data about the cached entry.
A sub-function, `get.stream.byDigest` may be used for identical behavior,
except lookup will happen by integrity hash, bypassing the index entirely. This
version does not emit the `metadata` and `integrity` events at all.
See: [options](#get-options)
##### Example
```javascript
// Look up by key
cache.get.stream(
cachePath, 'my-thing'
).on('metadata', metadata => {
console.log('metadata:', metadata)
}).on('integrity', integrity => {
console.log('integrity:', integrity)
}).pipe(
fs.createWriteStream('./x.tgz')
)
// Outputs:
metadata: { ... }
integrity: 'sha512-SoMeDIGest+64=='
// Look up by digest
cache.get.stream.byDigest(
cachePath, 'sha512-SoMeDIGest+64=='
).pipe(
fs.createWriteStream('./x.tgz')
)
```
#### <a name="get-info"></a> `> cacache.get.info(cache, key) -> Promise`
Looks up `key` in the cache index, returning information about the entry if
one exists.
##### Fields
* `key` - Key the entry was looked up under. Matches the `key` argument.
* `integrity` - [Subresource Integrity hash](#integrity) for the content this entry refers to.
* `path` - Filesystem path where content is stored, joined with `cache` argument.
* `time` - Timestamp the entry was first added on.
* `metadata` - User-assigned metadata associated with the entry/content.
##### Example
```javascript
cacache.get.info(cachePath, 'my-thing').then(console.log)
// Output
{
key: 'my-thing',
integrity: 'sha256-MUSTVERIFY+ALL/THINGS=='
path: '.testcache/content/deadbeef',
time: 12345698490,
size: 849234,
metadata: {
name: 'blah',
version: '1.2.3',
description: 'this was once a package but now it is my-thing'
}
}
```
#### <a name="get-hasContent"></a> `> cacache.get.hasContent(cache, integrity) -> Promise`
Looks up a [Subresource Integrity hash](#integrity) in the cache. If content
exists for this `integrity`, it will return an object, with the specific single integrity hash
that was found in `sri` key, and the size of the found content as `size`. If no content exists for this integrity, it will return `false`.
##### Example
```javascript
cacache.get.hasContent(cachePath, 'sha256-MUSTVERIFY+ALL/THINGS==').then(console.log)
// Output
{
sri: {
source: 'sha256-MUSTVERIFY+ALL/THINGS==',
algorithm: 'sha256',
digest: 'MUSTVERIFY+ALL/THINGS==',
options: []
},
size: 9001
}
cacache.get.hasContent(cachePath, 'sha521-NOT+IN/CACHE==').then(console.log)
// Output
false
```
##### <a name="get-options"></a> Options
##### `opts.integrity`
If present, the pre-calculated digest for the inserted content. If this option
is provided and does not match the post-insertion digest, insertion will fail
with an `EINTEGRITY` error.
##### `opts.memoize`
Default: null
If explicitly truthy, cacache will read from memory and memoize data on bulk read. If `false`, cacache will read from disk data. Reader functions by default read from in-memory cache.
##### `opts.size`
If provided, the data stream will be verified to check that enough data was
passed through. If there's more or less data than expected, insertion will fail
with an `EBADSIZE` error.
#### <a name="put-data"></a> `> cacache.put(cache, key, data, [opts]) -> Promise`
Inserts data passed to it into the cache. The returned Promise resolves with a
digest (generated according to [`opts.algorithms`](#optsalgorithms)) after the
cache entry has been successfully written.
See: [options](#put-options)
##### Example
```javascript
fetch(
'https://registry.npmjs.org/cacache/-/cacache-1.0.0.tgz'
).then(data => {
return cacache.put(cachePath, 'registry.npmjs.org|cacache@1.0.0', data)
}).then(integrity => {
console.log('integrity hash is', integrity)
})
```
#### <a name="put-stream"></a> `> cacache.put.stream(cache, key, [opts]) -> Writable`
Returns a [Writable
Stream](https://nodejs.org/api/stream.html#stream_writable_streams) that inserts
data written to it into the cache. Emits an `integrity` event with the digest of
written contents when it succeeds.
See: [options](#put-options)
##### Example
```javascript
request.get(
'https://registry.npmjs.org/cacache/-/cacache-1.0.0.tgz'
).pipe(
cacache.put.stream(
cachePath, 'registry.npmjs.org|cacache@1.0.0'
).on('integrity', d => console.log(`integrity digest is ${d}`))
)
```
##### <a name="put-options"></a> Options
##### `opts.metadata`
Arbitrary metadata to be attached to the inserted key.
##### `opts.size`
If provided, the data stream will be verified to check that enough data was
passed through. If there's more or less data than expected, insertion will fail
with an `EBADSIZE` error.
##### `opts.integrity`
If present, the pre-calculated digest for the inserted content. If this option
is provided and does not match the post-insertion digest, insertion will fail
with an `EINTEGRITY` error.
`algorithms` has no effect if this option is present.
##### `opts.integrityEmitter`
*Streaming only* If present, uses the provided event emitter as a source of
truth for both integrity and size. This allows use cases where integrity is
already being calculated outside of cacache to reuse that data instead of
calculating it a second time.
The emitter must emit both the `'integrity'` and `'size'` events.
NOTE: If this option is provided, you must verify that you receive the correct
integrity value yourself and emit an `'error'` event if there is a mismatch.
[ssri Integrity Streams](https://github.com/npm/ssri#integrity-stream) do this for you when given an expected integrity.
##### `opts.algorithms`
Default: ['sha512']
Hashing algorithms to use when calculating the [subresource integrity
digest](#integrity)
for inserted data. Can use any algorithm listed in `crypto.getHashes()` or
`'omakase'`/`'お任せします'` to pick a random hash algorithm on each insertion. You
may also use any anagram of `'modnar'` to use this feature.
Currently only supports one algorithm at a time (i.e., an array length of
exactly `1`). Has no effect if `opts.integrity` is present.
##### `opts.memoize`
Default: null
If provided, cacache will memoize the given cache insertion in memory, bypassing
any filesystem checks for that key or digest in future cache fetches. Nothing
will be written to the in-memory cache unless this option is explicitly truthy.
If `opts.memoize` is an object or a `Map`-like (that is, an object with `get`
and `set` methods), it will be written to instead of the global memoization
cache.
Reading from disk data can be forced by explicitly passing `memoize: false` to
the reader functions, but their default will be to read from memory.
##### `opts.tmpPrefix`
Default: null
Prefix to append on the temporary directory name inside the cache's tmp dir.
#### <a name="rm-all"></a> `> cacache.rm.all(cache) -> Promise`
Clears the entire cache. Mainly by blowing away the cache directory itself.
##### Example
```javascript
cacache.rm.all(cachePath).then(() => {
console.log('THE APOCALYPSE IS UPON US 😱')
})
```
#### <a name="rm-entry"></a> `> cacache.rm.entry(cache, key, [opts]) -> Promise`
Alias: `cacache.rm`
Removes the index entry for `key`. Content will still be accessible if
requested directly by content address ([`get.stream.byDigest`](#get-stream)).
By default, this appends a new entry to the index with an integrity of `null`.
If `opts.removeFully` is set to `true` then the index file itself will be
physically deleted rather than appending a `null`.
To remove the content itself (which might still be used by other entries), use
[`rm.content`](#rm-content). Or, to safely vacuum any unused content, use
[`verify`](#verify).
##### Example
```javascript
cacache.rm.entry(cachePath, 'my-thing').then(() => {
console.log('I did not like it anyway')
})
```
#### <a name="rm-content"></a> `> cacache.rm.content(cache, integrity) -> Promise`
Removes the content identified by `integrity`. Any index entries referring to it
will not be usable again until the content is re-added to the cache with an
identical digest.
##### Example
```javascript
cacache.rm.content(cachePath, 'sha512-SoMeDIGest/IN+BaSE64==').then(() => {
console.log('data for my-thing is gone!')
})
```
#### <a name="index-compact"></a> `> cacache.index.compact(cache, key, matchFn, [opts]) -> Promise`
Uses `matchFn`, which must be a synchronous function that accepts two entries
and returns a boolean indicating whether or not the two entries match, to
deduplicate all entries in the cache for the given `key`.
If `opts.validateEntry` is provided, it will be called as a function with the
only parameter being a single index entry. The function must return a Boolean,
if it returns `true` the entry is considered valid and will be kept in the index,
if it returns `false` the entry will be removed from the index.
If `opts.validateEntry` is not provided, however, every entry in the index will
be deduplicated and kept until the first `null` integrity is reached, removing
all entries that were written before the `null`.
The deduplicated list of entries is both written to the index, replacing the
existing content, and returned in the Promise.
#### <a name="index-insert"></a> `> cacache.index.insert(cache, key, integrity, opts) -> Promise`
Writes an index entry to the cache for the given `key` without writing content.
It is assumed if you are using this method, you have already stored the content
some other way and you only wish to add a new index to that content. The `metadata`
and `size` properties are read from `opts` and used as part of the index entry.
Returns a Promise resolving to the newly added entry.
#### <a name="clear-memoized"></a> `> cacache.clearMemoized()`
Completely resets the in-memory entry cache.
#### <a name="tmp-mkdir"></a> `> tmp.mkdir(cache, opts) -> Promise<Path>`
Returns a unique temporary directory inside the cache's `tmp` dir. This
directory will use the same safe user assignment that all the other stuff use.
Once the directory is made, it's the user's responsibility that all files
within are given the appropriate `gid`/`uid` ownership settings to match
the rest of the cache. If not, you can ask cacache to do it for you by
calling [`tmp.fix()`](#tmp-fix), which will fix all tmp directory
permissions.
If you want automatic cleanup of this directory, use
[`tmp.withTmp()`](#with-tpm)
See: [options](#tmp-options)
##### Example
```javascript
cacache.tmp.mkdir(cache).then(dir => {
fs.writeFile(path.join(dir, 'blablabla'), Buffer#<1234>, ...)
})
```
#### <a name="tmp-fix"></a> `> tmp.fix(cache) -> Promise`
Sets the `uid` and `gid` properties on all files and folders within the tmp
folder to match the rest of the cache.
Use this after manually writing files into [`tmp.mkdir`](#tmp-mkdir) or
[`tmp.withTmp`](#with-tmp).
##### Example
```javascript
cacache.tmp.mkdir(cache).then(dir => {
writeFile(path.join(dir, 'file'), someData).then(() => {
// make sure we didn't just put a root-owned file in the cache
cacache.tmp.fix().then(() => {
// all uids and gids match now
})
})
})
```
#### <a name="with-tmp"></a> `> tmp.withTmp(cache, opts, cb) -> Promise`
Creates a temporary directory with [`tmp.mkdir()`](#tmp-mkdir) and calls `cb`
with it. The created temporary directory will be removed when the return value
of `cb()` resolves, the tmp directory will be automatically deleted once that
promise completes.
The same caveats apply when it comes to managing permissions for the tmp dir's
contents.
See: [options](#tmp-options)
##### Example
```javascript
cacache.tmp.withTmp(cache, dir => {
return fs.writeFile(path.join(dir, 'blablabla'), 'blabla contents', { encoding: 'utf8' })
}).then(() => {
// `dir` no longer exists
})
```
##### <a name="tmp-options"></a> Options
##### `opts.tmpPrefix`
Default: null
Prefix to append on the temporary directory name inside the cache's tmp dir.
#### <a name="integrity"></a> Subresource Integrity Digests
For content verification and addressing, cacache uses strings following the
[Subresource
Integrity spec](https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity).
That is, any time cacache expects an `integrity` argument or option, it
should be in the format `<hashAlgorithm>-<base64-hash>`.
One deviation from the current spec is that cacache will support any hash
algorithms supported by the underlying Node.js process. You can use
`crypto.getHashes()` to see which ones you can use.
##### Generating Digests Yourself
If you have an existing content shasum, they are generally formatted as a
hexadecimal string (that is, a sha1 would look like:
`5f5513f8822fdbe5145af33b64d8d970dcf95c6e`). In order to be compatible with
cacache, you'll need to convert this to an equivalent subresource integrity
string. For this example, the corresponding hash would be:
`sha1-X1UT+IIv2+UUWvM7ZNjZcNz5XG4=`.
If you want to generate an integrity string yourself for existing data, you can
use something like this:
```javascript
const crypto = require('crypto')
const hashAlgorithm = 'sha512'
const data = 'foobarbaz'
const integrity = (
hashAlgorithm +
'-' +
crypto.createHash(hashAlgorithm).update(data).digest('base64')
)
```
You can also use [`ssri`](https://npm.im/ssri) to have a richer set of functionality
around SRI strings, including generation, parsing, and translating from existing
hex-formatted strings.
#### <a name="verify"></a> `> cacache.verify(cache, opts) -> Promise`
Checks out and fixes up your cache:
* Cleans up corrupted or invalid index entries.
* Custom entry filtering options.
* Garbage collects any content entries not referenced by the index.
* Checks integrity for all content entries and removes invalid content.
* Fixes cache ownership.
* Removes the `tmp` directory in the cache and all its contents.
When it's done, it'll return an object with various stats about the verification
process, including amount of storage reclaimed, number of valid entries, number
of entries removed, etc.
##### <a name="verify-options"></a> Options
##### `opts.concurrency`
Default: 20
Number of concurrently read files in the filesystem while doing clean up.
##### `opts.filter`
Receives a formatted entry. Return false to remove it.
Note: might be called more than once on the same entry.
##### `opts.log`
Custom logger function:
```
log: { silly () {} }
log.silly('verify', 'verifying cache at', cache)
```
##### Example
```sh
echo somegarbage >> $CACHEPATH/content/deadbeef
```
```javascript
cacache.verify(cachePath).then(stats => {
// deadbeef collected, because of invalid checksum.
console.log('cache is much nicer now! stats:', stats)
})
```
#### <a name="verify-last-run"></a> `> cacache.verify.lastRun(cache) -> Promise`
Returns a `Date` representing the last time `cacache.verify` was run on `cache`.
##### Example
```javascript
cacache.verify(cachePath).then(() => {
cacache.verify.lastRun(cachePath).then(lastTime => {
console.log('cacache.verify was last called on' + lastTime)
})
})
```

View File

@@ -0,0 +1,29 @@
'use strict'
const contentVer = require('../../package.json')['cache-version'].content
const hashToSegments = require('../util/hash-to-segments')
const path = require('path')
const ssri = require('ssri')
// Current format of content file path:
//
// sha512-BaSE64Hex= ->
// ~/.my-cache/content-v2/sha512/ba/da/55deadbeefc0ffee
//
module.exports = contentPath
function contentPath (cache, integrity) {
const sri = ssri.parse(integrity, { single: true })
// contentPath is the *strongest* algo given
return path.join(
contentDir(cache),
sri.algorithm,
...hashToSegments(sri.hexDigest())
)
}
module.exports.contentDir = contentDir
function contentDir (cache) {
return path.join(cache, `content-v${contentVer}`)
}

View File

@@ -0,0 +1,166 @@
'use strict'
const fs = require('fs/promises')
const fsm = require('fs-minipass')
const ssri = require('ssri')
const contentPath = require('./path')
const Pipeline = require('minipass-pipeline')
module.exports = read
const MAX_SINGLE_READ_SIZE = 64 * 1024 * 1024
async function read (cache, integrity, opts = {}) {
const { size } = opts
const { stat, cpath, sri } = await withContentSri(cache, integrity, async (cpath, sri) => {
// get size
const stat = await fs.stat(cpath)
return { stat, cpath, sri }
})
if (typeof size === 'number' && stat.size !== size) {
throw sizeError(size, stat.size)
}
if (stat.size > MAX_SINGLE_READ_SIZE) {
return readPipeline(cpath, stat.size, sri, new Pipeline()).concat()
}
const data = await fs.readFile(cpath, { encoding: null })
if (!ssri.checkData(data, sri)) {
throw integrityError(sri, cpath)
}
return data
}
const readPipeline = (cpath, size, sri, stream) => {
stream.push(
new fsm.ReadStream(cpath, {
size,
readSize: MAX_SINGLE_READ_SIZE,
}),
ssri.integrityStream({
integrity: sri,
size,
})
)
return stream
}
module.exports.stream = readStream
module.exports.readStream = readStream
function readStream (cache, integrity, opts = {}) {
const { size } = opts
const stream = new Pipeline()
// Set all this up to run on the stream and then just return the stream
Promise.resolve().then(async () => {
const { stat, cpath, sri } = await withContentSri(cache, integrity, async (cpath, sri) => {
// just stat to ensure it exists
const stat = await fs.stat(cpath)
return { stat, cpath, sri }
})
if (typeof size === 'number' && size !== stat.size) {
return stream.emit('error', sizeError(size, stat.size))
}
return readPipeline(cpath, stat.size, sri, stream)
}).catch(err => stream.emit('error', err))
return stream
}
module.exports.copy = copy
function copy (cache, integrity, dest) {
return withContentSri(cache, integrity, (cpath, sri) => {
return fs.copyFile(cpath, dest)
})
}
module.exports.hasContent = hasContent
async function hasContent (cache, integrity) {
if (!integrity) {
return false
}
try {
return await withContentSri(cache, integrity, async (cpath, sri) => {
const stat = await fs.stat(cpath)
return { size: stat.size, sri, stat }
})
} catch (err) {
if (err.code === 'ENOENT') {
return false
}
if (err.code === 'EPERM') {
/* istanbul ignore else */
if (process.platform !== 'win32') {
throw err
} else {
return false
}
}
}
}
async function withContentSri (cache, integrity, fn) {
const sri = ssri.parse(integrity)
// If `integrity` has multiple entries, pick the first digest
// with available local data.
const algo = sri.pickAlgorithm()
const digests = sri[algo]
if (digests.length <= 1) {
const cpath = contentPath(cache, digests[0])
return fn(cpath, digests[0])
} else {
// Can't use race here because a generic error can happen before
// a ENOENT error, and can happen before a valid result
const results = await Promise.all(digests.map(async (meta) => {
try {
return await withContentSri(cache, meta, fn)
} catch (err) {
if (err.code === 'ENOENT') {
return Object.assign(
new Error('No matching content found for ' + sri.toString()),
{ code: 'ENOENT' }
)
}
return err
}
}))
// Return the first non error if it is found
const result = results.find((r) => !(r instanceof Error))
if (result) {
return result
}
// Throw the No matching content found error
const enoentError = results.find((r) => r.code === 'ENOENT')
if (enoentError) {
throw enoentError
}
// Throw generic error
throw results.find((r) => r instanceof Error)
}
}
function sizeError (expected, found) {
/* eslint-disable-next-line max-len */
const err = new Error(`Bad data size: expected inserted data to be ${expected} bytes, but got ${found} instead`)
err.expected = expected
err.found = found
err.code = 'EBADSIZE'
return err
}
function integrityError (sri, path) {
const err = new Error(`Integrity verification failed for ${sri} (${path})`)
err.code = 'EINTEGRITY'
err.sri = sri
err.path = path
return err
}

View File

@@ -0,0 +1,18 @@
'use strict'
const fs = require('fs/promises')
const contentPath = require('./path')
const { hasContent } = require('./read')
module.exports = rm
async function rm (cache, integrity) {
const content = await hasContent(cache, integrity)
// ~pretty~ sure we can't end up with a content lacking sri, but be safe
if (content && content.sri) {
await fs.rm(contentPath(cache, content.sri), { recursive: true, force: true })
return true
} else {
return false
}
}

View File

@@ -0,0 +1,205 @@
'use strict'
const events = require('events')
const contentPath = require('./path')
const fs = require('fs/promises')
const { moveFile } = require('@npmcli/fs')
const { Minipass } = require('minipass')
const Pipeline = require('minipass-pipeline')
const Flush = require('minipass-flush')
const path = require('path')
const ssri = require('ssri')
const uniqueFilename = require('unique-filename')
const fsm = require('fs-minipass')
module.exports = write
// Cache of move operations in process so we don't duplicate
const moveOperations = new Map()
async function write (cache, data, opts = {}) {
const { algorithms, size, integrity } = opts
if (typeof size === 'number' && data.length !== size) {
throw sizeError(size, data.length)
}
const sri = ssri.fromData(data, algorithms ? { algorithms } : {})
if (integrity && !ssri.checkData(data, integrity, opts)) {
throw checksumError(integrity, sri)
}
for (const algo in sri) {
const tmp = await makeTmp(cache, opts)
const hash = sri[algo].toString()
try {
await fs.writeFile(tmp.target, data, { flag: 'wx' })
await moveToDestination(tmp, cache, hash, opts)
} finally {
if (!tmp.moved) {
await fs.rm(tmp.target, { recursive: true, force: true })
}
}
}
return { integrity: sri, size: data.length }
}
module.exports.stream = writeStream
// writes proxied to the 'inputStream' that is passed to the Promise
// 'end' is deferred until content is handled.
class CacacheWriteStream extends Flush {
constructor (cache, opts) {
super()
this.opts = opts
this.cache = cache
this.inputStream = new Minipass()
this.inputStream.on('error', er => this.emit('error', er))
this.inputStream.on('drain', () => this.emit('drain'))
this.handleContentP = null
}
write (chunk, encoding, cb) {
if (!this.handleContentP) {
this.handleContentP = handleContent(
this.inputStream,
this.cache,
this.opts
)
}
return this.inputStream.write(chunk, encoding, cb)
}
flush (cb) {
this.inputStream.end(() => {
if (!this.handleContentP) {
const e = new Error('Cache input stream was empty')
e.code = 'ENODATA'
// empty streams are probably emitting end right away.
// defer this one tick by rejecting a promise on it.
return Promise.reject(e).catch(cb)
}
// eslint-disable-next-line promise/catch-or-return
this.handleContentP.then(
(res) => {
res.integrity && this.emit('integrity', res.integrity)
// eslint-disable-next-line promise/always-return
res.size !== null && this.emit('size', res.size)
cb()
},
(er) => cb(er)
)
})
}
}
function writeStream (cache, opts = {}) {
return new CacacheWriteStream(cache, opts)
}
async function handleContent (inputStream, cache, opts) {
const tmp = await makeTmp(cache, opts)
try {
const res = await pipeToTmp(inputStream, cache, tmp.target, opts)
await moveToDestination(
tmp,
cache,
res.integrity,
opts
)
return res
} finally {
if (!tmp.moved) {
await fs.rm(tmp.target, { recursive: true, force: true })
}
}
}
async function pipeToTmp (inputStream, cache, tmpTarget, opts) {
const outStream = new fsm.WriteStream(tmpTarget, {
flags: 'wx',
})
if (opts.integrityEmitter) {
// we need to create these all simultaneously since they can fire in any order
const [integrity, size] = await Promise.all([
events.once(opts.integrityEmitter, 'integrity').then(res => res[0]),
events.once(opts.integrityEmitter, 'size').then(res => res[0]),
new Pipeline(inputStream, outStream).promise(),
])
return { integrity, size }
}
let integrity
let size
const hashStream = ssri.integrityStream({
integrity: opts.integrity,
algorithms: opts.algorithms,
size: opts.size,
})
hashStream.on('integrity', i => {
integrity = i
})
hashStream.on('size', s => {
size = s
})
const pipeline = new Pipeline(inputStream, hashStream, outStream)
await pipeline.promise()
return { integrity, size }
}
async function makeTmp (cache, opts) {
const tmpTarget = uniqueFilename(path.join(cache, 'tmp'), opts.tmpPrefix)
await fs.mkdir(path.dirname(tmpTarget), { recursive: true })
return {
target: tmpTarget,
moved: false,
}
}
async function moveToDestination (tmp, cache, sri, opts) {
const destination = contentPath(cache, sri)
const destDir = path.dirname(destination)
if (moveOperations.has(destination)) {
return moveOperations.get(destination)
}
moveOperations.set(
destination,
fs.mkdir(destDir, { recursive: true })
.then(async () => {
await moveFile(tmp.target, destination, { overwrite: false })
tmp.moved = true
return tmp.moved
})
.catch(err => {
if (!err.message.startsWith('The destination file exists')) {
throw Object.assign(err, { code: 'EEXIST' })
}
}).finally(() => {
moveOperations.delete(destination)
})
)
return moveOperations.get(destination)
}
function sizeError (expected, found) {
/* eslint-disable-next-line max-len */
const err = new Error(`Bad data size: expected inserted data to be ${expected} bytes, but got ${found} instead`)
err.expected = expected
err.found = found
err.code = 'EBADSIZE'
return err
}
function checksumError (expected, found) {
const err = new Error(`Integrity check failed:
Wanted: ${expected}
Found: ${found}`)
err.code = 'EINTEGRITY'
err.expected = expected
err.found = found
return err
}

View File

@@ -0,0 +1,330 @@
'use strict'
const crypto = require('crypto')
const {
appendFile,
mkdir,
readFile,
readdir,
rm,
writeFile,
} = require('fs/promises')
const { Minipass } = require('minipass')
const path = require('path')
const ssri = require('ssri')
const uniqueFilename = require('unique-filename')
const contentPath = require('./content/path')
const hashToSegments = require('./util/hash-to-segments')
const indexV = require('../package.json')['cache-version'].index
const { moveFile } = require('@npmcli/fs')
module.exports.NotFoundError = class NotFoundError extends Error {
constructor (cache, key) {
super(`No cache entry for ${key} found in ${cache}`)
this.code = 'ENOENT'
this.cache = cache
this.key = key
}
}
module.exports.compact = compact
async function compact (cache, key, matchFn, opts = {}) {
const bucket = bucketPath(cache, key)
const entries = await bucketEntries(bucket)
const newEntries = []
// we loop backwards because the bottom-most result is the newest
// since we add new entries with appendFile
for (let i = entries.length - 1; i >= 0; --i) {
const entry = entries[i]
// a null integrity could mean either a delete was appended
// or the user has simply stored an index that does not map
// to any content. we determine if the user wants to keep the
// null integrity based on the validateEntry function passed in options.
// if the integrity is null and no validateEntry is provided, we break
// as we consider the null integrity to be a deletion of everything
// that came before it.
if (entry.integrity === null && !opts.validateEntry) {
break
}
// if this entry is valid, and it is either the first entry or
// the newEntries array doesn't already include an entry that
// matches this one based on the provided matchFn, then we add
// it to the beginning of our list
if ((!opts.validateEntry || opts.validateEntry(entry) === true) &&
(newEntries.length === 0 ||
!newEntries.find((oldEntry) => matchFn(oldEntry, entry)))) {
newEntries.unshift(entry)
}
}
const newIndex = '\n' + newEntries.map((entry) => {
const stringified = JSON.stringify(entry)
const hash = hashEntry(stringified)
return `${hash}\t${stringified}`
}).join('\n')
const setup = async () => {
const target = uniqueFilename(path.join(cache, 'tmp'), opts.tmpPrefix)
await mkdir(path.dirname(target), { recursive: true })
return {
target,
moved: false,
}
}
const teardown = async (tmp) => {
if (!tmp.moved) {
return rm(tmp.target, { recursive: true, force: true })
}
}
const write = async (tmp) => {
await writeFile(tmp.target, newIndex, { flag: 'wx' })
await mkdir(path.dirname(bucket), { recursive: true })
// we use @npmcli/move-file directly here because we
// want to overwrite the existing file
await moveFile(tmp.target, bucket)
tmp.moved = true
}
// write the file atomically
const tmp = await setup()
try {
await write(tmp)
} finally {
await teardown(tmp)
}
// we reverse the list we generated such that the newest
// entries come first in order to make looping through them easier
// the true passed to formatEntry tells it to keep null
// integrity values, if they made it this far it's because
// validateEntry returned true, and as such we should return it
return newEntries.reverse().map((entry) => formatEntry(cache, entry, true))
}
module.exports.insert = insert
async function insert (cache, key, integrity, opts = {}) {
const { metadata, size, time } = opts
const bucket = bucketPath(cache, key)
const entry = {
key,
integrity: integrity && ssri.stringify(integrity),
time: time || Date.now(),
size,
metadata,
}
try {
await mkdir(path.dirname(bucket), { recursive: true })
const stringified = JSON.stringify(entry)
// NOTE - Cleverness ahoy!
//
// This works because it's tremendously unlikely for an entry to corrupt
// another while still preserving the string length of the JSON in
// question. So, we just slap the length in there and verify it on read.
//
// Thanks to @isaacs for the whiteboarding session that ended up with
// this.
await appendFile(bucket, `\n${hashEntry(stringified)}\t${stringified}`)
} catch (err) {
if (err.code === 'ENOENT') {
return undefined
}
throw err
}
return formatEntry(cache, entry)
}
module.exports.find = find
async function find (cache, key) {
const bucket = bucketPath(cache, key)
try {
const entries = await bucketEntries(bucket)
return entries.reduce((latest, next) => {
if (next && next.key === key) {
return formatEntry(cache, next)
} else {
return latest
}
}, null)
} catch (err) {
if (err.code === 'ENOENT') {
return null
} else {
throw err
}
}
}
module.exports.delete = del
function del (cache, key, opts = {}) {
if (!opts.removeFully) {
return insert(cache, key, null, opts)
}
const bucket = bucketPath(cache, key)
return rm(bucket, { recursive: true, force: true })
}
module.exports.lsStream = lsStream
function lsStream (cache) {
const indexDir = bucketDir(cache)
const stream = new Minipass({ objectMode: true })
// Set all this up to run on the stream and then just return the stream
Promise.resolve().then(async () => {
const buckets = await readdirOrEmpty(indexDir)
await Promise.all(buckets.map(async (bucket) => {
const bucketPath = path.join(indexDir, bucket)
const subbuckets = await readdirOrEmpty(bucketPath)
await Promise.all(subbuckets.map(async (subbucket) => {
const subbucketPath = path.join(bucketPath, subbucket)
// "/cachename/<bucket 0xFF>/<bucket 0xFF>./*"
const subbucketEntries = await readdirOrEmpty(subbucketPath)
await Promise.all(subbucketEntries.map(async (entry) => {
const entryPath = path.join(subbucketPath, entry)
try {
const entries = await bucketEntries(entryPath)
// using a Map here prevents duplicate keys from showing up
// twice, I guess?
const reduced = entries.reduce((acc, entry) => {
acc.set(entry.key, entry)
return acc
}, new Map())
// reduced is a map of key => entry
for (const entry of reduced.values()) {
const formatted = formatEntry(cache, entry)
if (formatted) {
stream.write(formatted)
}
}
} catch (err) {
if (err.code === 'ENOENT') {
return undefined
}
throw err
}
}))
}))
}))
stream.end()
return stream
}).catch(err => stream.emit('error', err))
return stream
}
module.exports.ls = ls
async function ls (cache) {
const entries = await lsStream(cache).collect()
return entries.reduce((acc, xs) => {
acc[xs.key] = xs
return acc
}, {})
}
module.exports.bucketEntries = bucketEntries
async function bucketEntries (bucket, filter) {
const data = await readFile(bucket, 'utf8')
return _bucketEntries(data, filter)
}
function _bucketEntries (data, filter) {
const entries = []
data.split('\n').forEach((entry) => {
if (!entry) {
return
}
const pieces = entry.split('\t')
if (!pieces[1] || hashEntry(pieces[1]) !== pieces[0]) {
// Hash is no good! Corruption or malice? Doesn't matter!
// EJECT EJECT
return
}
let obj
try {
obj = JSON.parse(pieces[1])
} catch (_) {
// eslint-ignore-next-line no-empty-block
}
// coverage disabled here, no need to test with an entry that parses to something falsey
// istanbul ignore else
if (obj) {
entries.push(obj)
}
})
return entries
}
module.exports.bucketDir = bucketDir
function bucketDir (cache) {
return path.join(cache, `index-v${indexV}`)
}
module.exports.bucketPath = bucketPath
function bucketPath (cache, key) {
const hashed = hashKey(key)
return path.join.apply(
path,
[bucketDir(cache)].concat(hashToSegments(hashed))
)
}
module.exports.hashKey = hashKey
function hashKey (key) {
return hash(key, 'sha256')
}
module.exports.hashEntry = hashEntry
function hashEntry (str) {
return hash(str, 'sha1')
}
function hash (str, digest) {
return crypto
.createHash(digest)
.update(str)
.digest('hex')
}
function formatEntry (cache, entry, keepAll) {
// Treat null digests as deletions. They'll shadow any previous entries.
if (!entry.integrity && !keepAll) {
return null
}
return {
key: entry.key,
integrity: entry.integrity,
path: entry.integrity ? contentPath(cache, entry.integrity) : undefined,
size: entry.size,
time: entry.time,
metadata: entry.metadata,
}
}
function readdirOrEmpty (dir) {
return readdir(dir).catch((err) => {
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') {
return []
}
throw err
})
}

View File

@@ -0,0 +1,170 @@
'use strict'
const Collect = require('minipass-collect')
const { Minipass } = require('minipass')
const Pipeline = require('minipass-pipeline')
const index = require('./entry-index')
const memo = require('./memoization')
const read = require('./content/read')
async function getData (cache, key, opts = {}) {
const { integrity, memoize, size } = opts
const memoized = memo.get(cache, key, opts)
if (memoized && memoize !== false) {
return {
metadata: memoized.entry.metadata,
data: memoized.data,
integrity: memoized.entry.integrity,
size: memoized.entry.size,
}
}
const entry = await index.find(cache, key, opts)
if (!entry) {
throw new index.NotFoundError(cache, key)
}
const data = await read(cache, entry.integrity, { integrity, size })
if (memoize) {
memo.put(cache, entry, data, opts)
}
return {
data,
metadata: entry.metadata,
size: entry.size,
integrity: entry.integrity,
}
}
module.exports = getData
async function getDataByDigest (cache, key, opts = {}) {
const { integrity, memoize, size } = opts
const memoized = memo.get.byDigest(cache, key, opts)
if (memoized && memoize !== false) {
return memoized
}
const res = await read(cache, key, { integrity, size })
if (memoize) {
memo.put.byDigest(cache, key, res, opts)
}
return res
}
module.exports.byDigest = getDataByDigest
const getMemoizedStream = (memoized) => {
const stream = new Minipass()
stream.on('newListener', function (ev, cb) {
ev === 'metadata' && cb(memoized.entry.metadata)
ev === 'integrity' && cb(memoized.entry.integrity)
ev === 'size' && cb(memoized.entry.size)
})
stream.end(memoized.data)
return stream
}
function getStream (cache, key, opts = {}) {
const { memoize, size } = opts
const memoized = memo.get(cache, key, opts)
if (memoized && memoize !== false) {
return getMemoizedStream(memoized)
}
const stream = new Pipeline()
// Set all this up to run on the stream and then just return the stream
Promise.resolve().then(async () => {
const entry = await index.find(cache, key)
if (!entry) {
throw new index.NotFoundError(cache, key)
}
stream.emit('metadata', entry.metadata)
stream.emit('integrity', entry.integrity)
stream.emit('size', entry.size)
stream.on('newListener', function (ev, cb) {
ev === 'metadata' && cb(entry.metadata)
ev === 'integrity' && cb(entry.integrity)
ev === 'size' && cb(entry.size)
})
const src = read.readStream(
cache,
entry.integrity,
{ ...opts, size: typeof size !== 'number' ? entry.size : size }
)
if (memoize) {
const memoStream = new Collect.PassThrough()
memoStream.on('collect', data => memo.put(cache, entry, data, opts))
stream.unshift(memoStream)
}
stream.unshift(src)
return stream
}).catch((err) => stream.emit('error', err))
return stream
}
module.exports.stream = getStream
function getStreamDigest (cache, integrity, opts = {}) {
const { memoize } = opts
const memoized = memo.get.byDigest(cache, integrity, opts)
if (memoized && memoize !== false) {
const stream = new Minipass()
stream.end(memoized)
return stream
} else {
const stream = read.readStream(cache, integrity, opts)
if (!memoize) {
return stream
}
const memoStream = new Collect.PassThrough()
memoStream.on('collect', data => memo.put.byDigest(
cache,
integrity,
data,
opts
))
return new Pipeline(stream, memoStream)
}
}
module.exports.stream.byDigest = getStreamDigest
function info (cache, key, opts = {}) {
const { memoize } = opts
const memoized = memo.get(cache, key, opts)
if (memoized && memoize !== false) {
return Promise.resolve(memoized.entry)
} else {
return index.find(cache, key)
}
}
module.exports.info = info
async function copy (cache, key, dest, opts = {}) {
const entry = await index.find(cache, key, opts)
if (!entry) {
throw new index.NotFoundError(cache, key)
}
await read.copy(cache, entry.integrity, dest, opts)
return {
metadata: entry.metadata,
size: entry.size,
integrity: entry.integrity,
}
}
module.exports.copy = copy
async function copyByDigest (cache, key, dest, opts = {}) {
await read.copy(cache, key, dest, opts)
return key
}
module.exports.copy.byDigest = copyByDigest
module.exports.hasContent = read.hasContent

View File

@@ -0,0 +1,42 @@
'use strict'
const get = require('./get.js')
const put = require('./put.js')
const rm = require('./rm.js')
const verify = require('./verify.js')
const { clearMemoized } = require('./memoization.js')
const tmp = require('./util/tmp.js')
const index = require('./entry-index.js')
module.exports.index = {}
module.exports.index.compact = index.compact
module.exports.index.insert = index.insert
module.exports.ls = index.ls
module.exports.ls.stream = index.lsStream
module.exports.get = get
module.exports.get.byDigest = get.byDigest
module.exports.get.stream = get.stream
module.exports.get.stream.byDigest = get.stream.byDigest
module.exports.get.copy = get.copy
module.exports.get.copy.byDigest = get.copy.byDigest
module.exports.get.info = get.info
module.exports.get.hasContent = get.hasContent
module.exports.put = put
module.exports.put.stream = put.stream
module.exports.rm = rm.entry
module.exports.rm.all = rm.all
module.exports.rm.entry = module.exports.rm
module.exports.rm.content = rm.content
module.exports.clearMemoized = clearMemoized
module.exports.tmp = {}
module.exports.tmp.mkdir = tmp.mkdir
module.exports.tmp.withTmp = tmp.withTmp
module.exports.verify = verify
module.exports.verify.lastRun = verify.lastRun

View File

@@ -0,0 +1,72 @@
'use strict'
const LRU = require('lru-cache')
const MEMOIZED = new LRU({
max: 500,
maxSize: 50 * 1024 * 1024, // 50MB
ttl: 3 * 60 * 1000, // 3 minutes
sizeCalculation: (entry, key) => key.startsWith('key:') ? entry.data.length : entry.length,
})
module.exports.clearMemoized = clearMemoized
function clearMemoized () {
const old = {}
MEMOIZED.forEach((v, k) => {
old[k] = v
})
MEMOIZED.clear()
return old
}
module.exports.put = put
function put (cache, entry, data, opts) {
pickMem(opts).set(`key:${cache}:${entry.key}`, { entry, data })
putDigest(cache, entry.integrity, data, opts)
}
module.exports.put.byDigest = putDigest
function putDigest (cache, integrity, data, opts) {
pickMem(opts).set(`digest:${cache}:${integrity}`, data)
}
module.exports.get = get
function get (cache, key, opts) {
return pickMem(opts).get(`key:${cache}:${key}`)
}
module.exports.get.byDigest = getDigest
function getDigest (cache, integrity, opts) {
return pickMem(opts).get(`digest:${cache}:${integrity}`)
}
class ObjProxy {
constructor (obj) {
this.obj = obj
}
get (key) {
return this.obj[key]
}
set (key, val) {
this.obj[key] = val
}
}
function pickMem (opts) {
if (!opts || !opts.memoize) {
return MEMOIZED
} else if (opts.memoize.get && opts.memoize.set) {
return opts.memoize
} else if (typeof opts.memoize === 'object') {
return new ObjProxy(opts.memoize)
} else {
return MEMOIZED
}
}

View File

@@ -0,0 +1,80 @@
'use strict'
const index = require('./entry-index')
const memo = require('./memoization')
const write = require('./content/write')
const Flush = require('minipass-flush')
const { PassThrough } = require('minipass-collect')
const Pipeline = require('minipass-pipeline')
const putOpts = (opts) => ({
algorithms: ['sha512'],
...opts,
})
module.exports = putData
async function putData (cache, key, data, opts = {}) {
const { memoize } = opts
opts = putOpts(opts)
const res = await write(cache, data, opts)
const entry = await index.insert(cache, key, res.integrity, { ...opts, size: res.size })
if (memoize) {
memo.put(cache, entry, data, opts)
}
return res.integrity
}
module.exports.stream = putStream
function putStream (cache, key, opts = {}) {
const { memoize } = opts
opts = putOpts(opts)
let integrity
let size
let error
let memoData
const pipeline = new Pipeline()
// first item in the pipeline is the memoizer, because we need
// that to end first and get the collected data.
if (memoize) {
const memoizer = new PassThrough().on('collect', data => {
memoData = data
})
pipeline.push(memoizer)
}
// contentStream is a write-only, not a passthrough
// no data comes out of it.
const contentStream = write.stream(cache, opts)
.on('integrity', (int) => {
integrity = int
})
.on('size', (s) => {
size = s
})
.on('error', (err) => {
error = err
})
pipeline.push(contentStream)
// last but not least, we write the index and emit hash and size,
// and memoize if we're doing that
pipeline.push(new Flush({
async flush () {
if (!error) {
const entry = await index.insert(cache, key, integrity, { ...opts, size })
if (memoize && memoData) {
memo.put(cache, entry, memoData, opts)
}
pipeline.emit('integrity', integrity)
pipeline.emit('size', size)
}
},
}))
return pipeline
}

View File

@@ -0,0 +1,31 @@
'use strict'
const { rm } = require('fs/promises')
const glob = require('./util/glob.js')
const index = require('./entry-index')
const memo = require('./memoization')
const path = require('path')
const rmContent = require('./content/rm')
module.exports = entry
module.exports.entry = entry
function entry (cache, key, opts) {
memo.clearMemoized()
return index.delete(cache, key, opts)
}
module.exports.content = content
function content (cache, integrity) {
memo.clearMemoized()
return rmContent(cache, integrity)
}
module.exports.all = all
async function all (cache) {
memo.clearMemoized()
const paths = await glob(path.join(cache, '*(content-*|index-*)'), { silent: true, nosort: true })
return Promise.all(paths.map((p) => rm(p, { recursive: true, force: true })))
}

View File

@@ -0,0 +1,7 @@
'use strict'
const { glob } = require('glob')
const path = require('path')
const globify = (pattern) => pattern.split(path.win32.sep).join(path.posix.sep)
module.exports = (path, options) => glob(globify(path), options)

View File

@@ -0,0 +1,7 @@
'use strict'
module.exports = hashToSegments
function hashToSegments (hash) {
return [hash.slice(0, 2), hash.slice(2, 4), hash.slice(4)]
}

View File

@@ -0,0 +1,26 @@
'use strict'
const { withTempDir } = require('@npmcli/fs')
const fs = require('fs/promises')
const path = require('path')
module.exports.mkdir = mktmpdir
async function mktmpdir (cache, opts = {}) {
const { tmpPrefix } = opts
const tmpDir = path.join(cache, 'tmp')
await fs.mkdir(tmpDir, { recursive: true, owner: 'inherit' })
// do not use path.join(), it drops the trailing / if tmpPrefix is unset
const target = `${tmpDir}${path.sep}${tmpPrefix || ''}`
return fs.mkdtemp(target, { owner: 'inherit' })
}
module.exports.withTmp = withTmp
function withTmp (cache, opts, cb) {
if (!cb) {
cb = opts
opts = {}
}
return withTempDir(path.join(cache, 'tmp'), cb, opts)
}

View File

@@ -0,0 +1,257 @@
'use strict'
const {
mkdir,
readFile,
rm,
stat,
truncate,
writeFile,
} = require('fs/promises')
const pMap = require('p-map')
const contentPath = require('./content/path')
const fsm = require('fs-minipass')
const glob = require('./util/glob.js')
const index = require('./entry-index')
const path = require('path')
const ssri = require('ssri')
const hasOwnProperty = (obj, key) =>
Object.prototype.hasOwnProperty.call(obj, key)
const verifyOpts = (opts) => ({
concurrency: 20,
log: { silly () {} },
...opts,
})
module.exports = verify
async function verify (cache, opts) {
opts = verifyOpts(opts)
opts.log.silly('verify', 'verifying cache at', cache)
const steps = [
markStartTime,
fixPerms,
garbageCollect,
rebuildIndex,
cleanTmp,
writeVerifile,
markEndTime,
]
const stats = {}
for (const step of steps) {
const label = step.name
const start = new Date()
const s = await step(cache, opts)
if (s) {
Object.keys(s).forEach((k) => {
stats[k] = s[k]
})
}
const end = new Date()
if (!stats.runTime) {
stats.runTime = {}
}
stats.runTime[label] = end - start
}
stats.runTime.total = stats.endTime - stats.startTime
opts.log.silly(
'verify',
'verification finished for',
cache,
'in',
`${stats.runTime.total}ms`
)
return stats
}
async function markStartTime (cache, opts) {
return { startTime: new Date() }
}
async function markEndTime (cache, opts) {
return { endTime: new Date() }
}
async function fixPerms (cache, opts) {
opts.log.silly('verify', 'fixing cache permissions')
await mkdir(cache, { recursive: true })
return null
}
// Implements a naive mark-and-sweep tracing garbage collector.
//
// The algorithm is basically as follows:
// 1. Read (and filter) all index entries ("pointers")
// 2. Mark each integrity value as "live"
// 3. Read entire filesystem tree in `content-vX/` dir
// 4. If content is live, verify its checksum and delete it if it fails
// 5. If content is not marked as live, rm it.
//
async function garbageCollect (cache, opts) {
opts.log.silly('verify', 'garbage collecting content')
const indexStream = index.lsStream(cache)
const liveContent = new Set()
indexStream.on('data', (entry) => {
if (opts.filter && !opts.filter(entry)) {
return
}
// integrity is stringified, re-parse it so we can get each hash
const integrity = ssri.parse(entry.integrity)
for (const algo in integrity) {
liveContent.add(integrity[algo].toString())
}
})
await new Promise((resolve, reject) => {
indexStream.on('end', resolve).on('error', reject)
})
const contentDir = contentPath.contentDir(cache)
const files = await glob(path.join(contentDir, '**'), {
follow: false,
nodir: true,
nosort: true,
})
const stats = {
verifiedContent: 0,
reclaimedCount: 0,
reclaimedSize: 0,
badContentCount: 0,
keptSize: 0,
}
await pMap(
files,
async (f) => {
const split = f.split(/[/\\]/)
const digest = split.slice(split.length - 3).join('')
const algo = split[split.length - 4]
const integrity = ssri.fromHex(digest, algo)
if (liveContent.has(integrity.toString())) {
const info = await verifyContent(f, integrity)
if (!info.valid) {
stats.reclaimedCount++
stats.badContentCount++
stats.reclaimedSize += info.size
} else {
stats.verifiedContent++
stats.keptSize += info.size
}
} else {
// No entries refer to this content. We can delete.
stats.reclaimedCount++
const s = await stat(f)
await rm(f, { recursive: true, force: true })
stats.reclaimedSize += s.size
}
return stats
},
{ concurrency: opts.concurrency }
)
return stats
}
async function verifyContent (filepath, sri) {
const contentInfo = {}
try {
const { size } = await stat(filepath)
contentInfo.size = size
contentInfo.valid = true
await ssri.checkStream(new fsm.ReadStream(filepath), sri)
} catch (err) {
if (err.code === 'ENOENT') {
return { size: 0, valid: false }
}
if (err.code !== 'EINTEGRITY') {
throw err
}
await rm(filepath, { recursive: true, force: true })
contentInfo.valid = false
}
return contentInfo
}
async function rebuildIndex (cache, opts) {
opts.log.silly('verify', 'rebuilding index')
const entries = await index.ls(cache)
const stats = {
missingContent: 0,
rejectedEntries: 0,
totalEntries: 0,
}
const buckets = {}
for (const k in entries) {
/* istanbul ignore else */
if (hasOwnProperty(entries, k)) {
const hashed = index.hashKey(k)
const entry = entries[k]
const excluded = opts.filter && !opts.filter(entry)
excluded && stats.rejectedEntries++
if (buckets[hashed] && !excluded) {
buckets[hashed].push(entry)
} else if (buckets[hashed] && excluded) {
// skip
} else if (excluded) {
buckets[hashed] = []
buckets[hashed]._path = index.bucketPath(cache, k)
} else {
buckets[hashed] = [entry]
buckets[hashed]._path = index.bucketPath(cache, k)
}
}
}
await pMap(
Object.keys(buckets),
(key) => {
return rebuildBucket(cache, buckets[key], stats, opts)
},
{ concurrency: opts.concurrency }
)
return stats
}
async function rebuildBucket (cache, bucket, stats, opts) {
await truncate(bucket._path)
// This needs to be serialized because cacache explicitly
// lets very racy bucket conflicts clobber each other.
for (const entry of bucket) {
const content = contentPath(cache, entry.integrity)
try {
await stat(content)
await index.insert(cache, entry.key, entry.integrity, {
metadata: entry.metadata,
size: entry.size,
time: entry.time,
})
stats.totalEntries++
} catch (err) {
if (err.code === 'ENOENT') {
stats.rejectedEntries++
stats.missingContent++
} else {
throw err
}
}
}
}
function cleanTmp (cache, opts) {
opts.log.silly('verify', 'cleaning tmp directory')
return rm(path.join(cache, 'tmp'), { recursive: true, force: true })
}
async function writeVerifile (cache, opts) {
const verifile = path.join(cache, '_lastverified')
opts.log.silly('verify', 'writing verifile to ' + verifile)
return writeFile(verifile, `${Date.now()}`)
}
module.exports.lastRun = lastRun
async function lastRun (cache) {
const data = await readFile(path.join(cache, '_lastverified'), { encoding: 'utf8' })
return new Date(+data)
}

View File

@@ -0,0 +1,15 @@
The ISC License
Copyright (c) 2017-2023 npm, Inc., Isaac Z. Schlueter, and Contributors
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

View File

@@ -0,0 +1,825 @@
# minipass
A _very_ minimal implementation of a [PassThrough
stream](https://nodejs.org/api/stream.html#stream_class_stream_passthrough)
[It's very
fast](https://docs.google.com/spreadsheets/d/1K_HR5oh3r80b8WVMWCPPjfuWXUgfkmhlX7FGI6JJ8tY/edit?usp=sharing)
for objects, strings, and buffers.
Supports `pipe()`ing (including multi-`pipe()` and backpressure
transmission), buffering data until either a `data` event handler
or `pipe()` is added (so you don't lose the first chunk), and
most other cases where PassThrough is a good idea.
There is a `read()` method, but it's much more efficient to
consume data from this stream via `'data'` events or by calling
`pipe()` into some other stream. Calling `read()` requires the
buffer to be flattened in some cases, which requires copying
memory.
If you set `objectMode: true` in the options, then whatever is
written will be emitted. Otherwise, it'll do a minimal amount of
Buffer copying to ensure proper Streams semantics when `read(n)`
is called.
`objectMode` can only be set at instantiation. Attempting to
write something other than a String or Buffer without having set
`objectMode` in the options will throw an error.
This is not a `through` or `through2` stream. It doesn't
transform the data, it just passes it right through. If you want
to transform the data, extend the class, and override the
`write()` method. Once you're done transforming the data however
you want, call `super.write()` with the transform output.
For some examples of streams that extend Minipass in various
ways, check out:
- [minizlib](http://npm.im/minizlib)
- [fs-minipass](http://npm.im/fs-minipass)
- [tar](http://npm.im/tar)
- [minipass-collect](http://npm.im/minipass-collect)
- [minipass-flush](http://npm.im/minipass-flush)
- [minipass-pipeline](http://npm.im/minipass-pipeline)
- [tap](http://npm.im/tap)
- [tap-parser](http://npm.im/tap-parser)
- [treport](http://npm.im/treport)
- [minipass-fetch](http://npm.im/minipass-fetch)
- [pacote](http://npm.im/pacote)
- [make-fetch-happen](http://npm.im/make-fetch-happen)
- [cacache](http://npm.im/cacache)
- [ssri](http://npm.im/ssri)
- [npm-registry-fetch](http://npm.im/npm-registry-fetch)
- [minipass-json-stream](http://npm.im/minipass-json-stream)
- [minipass-sized](http://npm.im/minipass-sized)
## Usage in TypeScript
The `Minipass` class takes three type template definitions:
- `RType` the type being read, which defaults to `Buffer`. If
`RType` is `string`, then the constructor _must_ get an options
object specifying either an `encoding` or `objectMode: true`.
If it's anything other than `string` or `Buffer`, then it
_must_ get an options object specifying `objectMode: true`.
- `WType` the type being written. If `RType` is `Buffer` or
`string`, then this defaults to `ContiguousData` (Buffer,
string, ArrayBuffer, or ArrayBufferView). Otherwise, it
defaults to `RType`.
- `Events` type mapping event names to the arguments emitted
with that event, which extends `Minipass.Events`.
To declare types for custom events in subclasses, extend the
third parameter with your own event signatures. For example:
```js
import { Minipass } from 'minipass'
// a NDJSON stream that emits 'jsonError' when it can't stringify
export interface Events extends Minipass.Events {
jsonError: [e: Error]
}
export class NDJSONStream extends Minipass<string, any, Events> {
constructor() {
super({ objectMode: true })
}
// data is type `any` because that's WType
write(data, encoding, cb) {
try {
const json = JSON.stringify(data)
return super.write(json + '\n', encoding, cb)
} catch (er) {
if (!er instanceof Error) {
er = Object.assign(new Error('json stringify failed'), {
cause: er,
})
}
// trying to emit with something OTHER than an error will
// fail, because we declared the event arguments type.
this.emit('jsonError', er)
}
}
}
const s = new NDJSONStream()
s.on('jsonError', e => {
// here, TS knows that e is an Error
})
```
Emitting/handling events that aren't declared in this way is
fine, but the arguments will be typed as `unknown`.
## Differences from Node.js Streams
There are several things that make Minipass streams different
from (and in some ways superior to) Node.js core streams.
Please read these caveats if you are familiar with node-core
streams and intend to use Minipass streams in your programs.
You can avoid most of these differences entirely (for a very
small performance penalty) by setting `{async: true}` in the
constructor options.
### Timing
Minipass streams are designed to support synchronous use-cases.
Thus, data is emitted as soon as it is available, always. It is
buffered until read, but no longer. Another way to look at it is
that Minipass streams are exactly as synchronous as the logic
that writes into them.
This can be surprising if your code relies on
`PassThrough.write()` always providing data on the next tick
rather than the current one, or being able to call `resume()` and
not have the entire buffer disappear immediately.
However, without this synchronicity guarantee, there would be no
way for Minipass to achieve the speeds it does, or support the
synchronous use cases that it does. Simply put, waiting takes
time.
This non-deferring approach makes Minipass streams much easier to
reason about, especially in the context of Promises and other
flow-control mechanisms.
Example:
```js
// hybrid module, either works
import { Minipass } from 'minipass'
// or:
const { Minipass } = require('minipass')
const stream = new Minipass()
stream.on('data', () => console.log('data event'))
console.log('before write')
stream.write('hello')
console.log('after write')
// output:
// before write
// data event
// after write
```
### Exception: Async Opt-In
If you wish to have a Minipass stream with behavior that more
closely mimics Node.js core streams, you can set the stream in
async mode either by setting `async: true` in the constructor
options, or by setting `stream.async = true` later on.
```js
// hybrid module, either works
import { Minipass } from 'minipass'
// or:
const { Minipass } = require('minipass')
const asyncStream = new Minipass({ async: true })
asyncStream.on('data', () => console.log('data event'))
console.log('before write')
asyncStream.write('hello')
console.log('after write')
// output:
// before write
// after write
// data event <-- this is deferred until the next tick
```
Switching _out_ of async mode is unsafe, as it could cause data
corruption, and so is not enabled. Example:
```js
import { Minipass } from 'minipass'
const stream = new Minipass({ encoding: 'utf8' })
stream.on('data', chunk => console.log(chunk))
stream.async = true
console.log('before writes')
stream.write('hello')
setStreamSyncAgainSomehow(stream) // <-- this doesn't actually exist!
stream.write('world')
console.log('after writes')
// hypothetical output would be:
// before writes
// world
// after writes
// hello
// NOT GOOD!
```
To avoid this problem, once set into async mode, any attempt to
make the stream sync again will be ignored.
```js
const { Minipass } = require('minipass')
const stream = new Minipass({ encoding: 'utf8' })
stream.on('data', chunk => console.log(chunk))
stream.async = true
console.log('before writes')
stream.write('hello')
stream.async = false // <-- no-op, stream already async
stream.write('world')
console.log('after writes')
// actual output:
// before writes
// after writes
// hello
// world
```
### No High/Low Water Marks
Node.js core streams will optimistically fill up a buffer,
returning `true` on all writes until the limit is hit, even if
the data has nowhere to go. Then, they will not attempt to draw
more data in until the buffer size dips below a minimum value.
Minipass streams are much simpler. The `write()` method will
return `true` if the data has somewhere to go (which is to say,
given the timing guarantees, that the data is already there by
the time `write()` returns).
If the data has nowhere to go, then `write()` returns false, and
the data sits in a buffer, to be drained out immediately as soon
as anyone consumes it.
Since nothing is ever buffered unnecessarily, there is much less
copying data, and less bookkeeping about buffer capacity levels.
### Hazards of Buffering (or: Why Minipass Is So Fast)
Since data written to a Minipass stream is immediately written
all the way through the pipeline, and `write()` always returns
true/false based on whether the data was fully flushed,
backpressure is communicated immediately to the upstream caller.
This minimizes buffering.
Consider this case:
```js
const { PassThrough } = require('stream')
const p1 = new PassThrough({ highWaterMark: 1024 })
const p2 = new PassThrough({ highWaterMark: 1024 })
const p3 = new PassThrough({ highWaterMark: 1024 })
const p4 = new PassThrough({ highWaterMark: 1024 })
p1.pipe(p2).pipe(p3).pipe(p4)
p4.on('data', () => console.log('made it through'))
// this returns false and buffers, then writes to p2 on next tick (1)
// p2 returns false and buffers, pausing p1, then writes to p3 on next tick (2)
// p3 returns false and buffers, pausing p2, then writes to p4 on next tick (3)
// p4 returns false and buffers, pausing p3, then emits 'data' and 'drain'
// on next tick (4)
// p3 sees p4's 'drain' event, and calls resume(), emitting 'resume' and
// 'drain' on next tick (5)
// p2 sees p3's 'drain', calls resume(), emits 'resume' and 'drain' on next tick (6)
// p1 sees p2's 'drain', calls resume(), emits 'resume' and 'drain' on next
// tick (7)
p1.write(Buffer.alloc(2048)) // returns false
```
Along the way, the data was buffered and deferred at each stage,
and multiple event deferrals happened, for an unblocked pipeline
where it was perfectly safe to write all the way through!
Furthermore, setting a `highWaterMark` of `1024` might lead
someone reading the code to think an advisory maximum of 1KiB is
being set for the pipeline. However, the actual advisory
buffering level is the _sum_ of `highWaterMark` values, since
each one has its own bucket.
Consider the Minipass case:
```js
const m1 = new Minipass()
const m2 = new Minipass()
const m3 = new Minipass()
const m4 = new Minipass()
m1.pipe(m2).pipe(m3).pipe(m4)
m4.on('data', () => console.log('made it through'))
// m1 is flowing, so it writes the data to m2 immediately
// m2 is flowing, so it writes the data to m3 immediately
// m3 is flowing, so it writes the data to m4 immediately
// m4 is flowing, so it fires the 'data' event immediately, returns true
// m4's write returned true, so m3 is still flowing, returns true
// m3's write returned true, so m2 is still flowing, returns true
// m2's write returned true, so m1 is still flowing, returns true
// No event deferrals or buffering along the way!
m1.write(Buffer.alloc(2048)) // returns true
```
It is extremely unlikely that you _don't_ want to buffer any data
written, or _ever_ buffer data that can be flushed all the way
through. Neither node-core streams nor Minipass ever fail to
buffer written data, but node-core streams do a lot of
unnecessary buffering and pausing.
As always, the faster implementation is the one that does less
stuff and waits less time to do it.
### Immediately emit `end` for empty streams (when not paused)
If a stream is not paused, and `end()` is called before writing
any data into it, then it will emit `end` immediately.
If you have logic that occurs on the `end` event which you don't
want to potentially happen immediately (for example, closing file
descriptors, moving on to the next entry in an archive parse
stream, etc.) then be sure to call `stream.pause()` on creation,
and then `stream.resume()` once you are ready to respond to the
`end` event.
However, this is _usually_ not a problem because:
### Emit `end` When Asked
One hazard of immediately emitting `'end'` is that you may not
yet have had a chance to add a listener. In order to avoid this
hazard, Minipass streams safely re-emit the `'end'` event if a
new listener is added after `'end'` has been emitted.
Ie, if you do `stream.on('end', someFunction)`, and the stream
has already emitted `end`, then it will call the handler right
away. (You can think of this somewhat like attaching a new
`.then(fn)` to a previously-resolved Promise.)
To prevent calling handlers multiple times who would not expect
multiple ends to occur, all listeners are removed from the
`'end'` event whenever it is emitted.
### Emit `error` When Asked
The most recent error object passed to the `'error'` event is
stored on the stream. If a new `'error'` event handler is added,
and an error was previously emitted, then the event handler will
be called immediately (or on `process.nextTick` in the case of
async streams).
This makes it much more difficult to end up trying to interact
with a broken stream, if the error handler is added after an
error was previously emitted.
### Impact of "immediate flow" on Tee-streams
A "tee stream" is a stream piping to multiple destinations:
```js
const tee = new Minipass()
t.pipe(dest1)
t.pipe(dest2)
t.write('foo') // goes to both destinations
```
Since Minipass streams _immediately_ process any pending data
through the pipeline when a new pipe destination is added, this
can have surprising effects, especially when a stream comes in
from some other function and may or may not have data in its
buffer.
```js
// WARNING! WILL LOSE DATA!
const src = new Minipass()
src.write('foo')
src.pipe(dest1) // 'foo' chunk flows to dest1 immediately, and is gone
src.pipe(dest2) // gets nothing!
```
One solution is to create a dedicated tee-stream junction that
pipes to both locations, and then pipe to _that_ instead.
```js
// Safe example: tee to both places
const src = new Minipass()
src.write('foo')
const tee = new Minipass()
tee.pipe(dest1)
tee.pipe(dest2)
src.pipe(tee) // tee gets 'foo', pipes to both locations
```
The same caveat applies to `on('data')` event listeners. The
first one added will _immediately_ receive all of the data,
leaving nothing for the second:
```js
// WARNING! WILL LOSE DATA!
const src = new Minipass()
src.write('foo')
src.on('data', handler1) // receives 'foo' right away
src.on('data', handler2) // nothing to see here!
```
Using a dedicated tee-stream can be used in this case as well:
```js
// Safe example: tee to both data handlers
const src = new Minipass()
src.write('foo')
const tee = new Minipass()
tee.on('data', handler1)
tee.on('data', handler2)
src.pipe(tee)
```
All of the hazards in this section are avoided by setting `{
async: true }` in the Minipass constructor, or by setting
`stream.async = true` afterwards. Note that this does add some
overhead, so should only be done in cases where you are willing
to lose a bit of performance in order to avoid having to refactor
program logic.
## USAGE
It's a stream! Use it like a stream and it'll most likely do what
you want.
```js
import { Minipass } from 'minipass'
const mp = new Minipass(options) // options is optional
mp.write('foo')
mp.pipe(someOtherStream)
mp.end('bar')
```
### OPTIONS
- `encoding` How would you like the data coming _out_ of the
stream to be encoded? Accepts any values that can be passed to
`Buffer.toString()`.
- `objectMode` Emit data exactly as it comes in. This will be
flipped on by default if you write() something other than a
string or Buffer at any point. Setting `objectMode: true` will
prevent setting any encoding value.
- `async` Defaults to `false`. Set to `true` to defer data
emission until next tick. This reduces performance slightly,
but makes Minipass streams use timing behavior closer to Node
core streams. See [Timing](#timing) for more details.
- `signal` An `AbortSignal` that will cause the stream to unhook
itself from everything and become as inert as possible. Note
that providing a `signal` parameter will make `'error'` events
no longer throw if they are unhandled, but they will still be
emitted to handlers if any are attached.
### API
Implements the user-facing portions of Node.js's `Readable` and
`Writable` streams.
### Methods
- `write(chunk, [encoding], [callback])` - Put data in. (Note
that, in the base Minipass class, the same data will come out.)
Returns `false` if the stream will buffer the next write, or
true if it's still in "flowing" mode.
- `end([chunk, [encoding]], [callback])` - Signal that you have
no more data to write. This will queue an `end` event to be
fired when all the data has been consumed.
- `pause()` - No more data for a while, please. This also
prevents `end` from being emitted for empty streams until the
stream is resumed.
- `resume()` - Resume the stream. If there's data in the buffer,
it is all discarded. Any buffered events are immediately
emitted.
- `pipe(dest)` - Send all output to the stream provided. When
data is emitted, it is immediately written to any and all pipe
destinations. (Or written on next tick in `async` mode.)
- `unpipe(dest)` - Stop piping to the destination stream. This is
immediate, meaning that any asynchronously queued data will
_not_ make it to the destination when running in `async` mode.
- `options.end` - Boolean, end the destination stream when the
source stream ends. Default `true`.
- `options.proxyErrors` - Boolean, proxy `error` events from
the source stream to the destination stream. Note that errors
are _not_ proxied after the pipeline terminates, either due
to the source emitting `'end'` or manually unpiping with
`src.unpipe(dest)`. Default `false`.
- `on(ev, fn)`, `emit(ev, fn)` - Minipass streams are
EventEmitters. Some events are given special treatment,
however. (See below under "events".)
- `promise()` - Returns a Promise that resolves when the stream
emits `end`, or rejects if the stream emits `error`.
- `collect()` - Return a Promise that resolves on `end` with an
array containing each chunk of data that was emitted, or
rejects if the stream emits `error`. Note that this consumes
the stream data.
- `concat()` - Same as `collect()`, but concatenates the data
into a single Buffer object. Will reject the returned promise
if the stream is in objectMode, or if it goes into objectMode
by the end of the data.
- `read(n)` - Consume `n` bytes of data out of the buffer. If `n`
is not provided, then consume all of it. If `n` bytes are not
available, then it returns null. **Note** consuming streams in
this way is less efficient, and can lead to unnecessary Buffer
copying.
- `destroy([er])` - Destroy the stream. If an error is provided,
then an `'error'` event is emitted. If the stream has a
`close()` method, and has not emitted a `'close'` event yet,
then `stream.close()` will be called. Any Promises returned by
`.promise()`, `.collect()` or `.concat()` will be rejected.
After being destroyed, writing to the stream will emit an
error. No more data will be emitted if the stream is destroyed,
even if it was previously buffered.
### Properties
- `bufferLength` Read-only. Total number of bytes buffered, or in
the case of objectMode, the total number of objects.
- `encoding` Read-only. The encoding that has been set.
- `flowing` Read-only. Boolean indicating whether a chunk written
to the stream will be immediately emitted.
- `emittedEnd` Read-only. Boolean indicating whether the end-ish
events (ie, `end`, `prefinish`, `finish`) have been emitted.
Note that listening on any end-ish event will immediateyl
re-emit it if it has already been emitted.
- `writable` Whether the stream is writable. Default `true`. Set
to `false` when `end()`
- `readable` Whether the stream is readable. Default `true`.
- `pipes` An array of Pipe objects referencing streams that this
stream is piping into.
- `destroyed` A getter that indicates whether the stream was
destroyed.
- `paused` True if the stream has been explicitly paused,
otherwise false.
- `objectMode` Indicates whether the stream is in `objectMode`.
- `aborted` Readonly property set when the `AbortSignal`
dispatches an `abort` event.
### Events
- `data` Emitted when there's data to read. Argument is the data
to read. This is never emitted while not flowing. If a listener
is attached, that will resume the stream.
- `end` Emitted when there's no more data to read. This will be
emitted immediately for empty streams when `end()` is called.
If a listener is attached, and `end` was already emitted, then
it will be emitted again. All listeners are removed when `end`
is emitted.
- `prefinish` An end-ish event that follows the same logic as
`end` and is emitted in the same conditions where `end` is
emitted. Emitted after `'end'`.
- `finish` An end-ish event that follows the same logic as `end`
and is emitted in the same conditions where `end` is emitted.
Emitted after `'prefinish'`.
- `close` An indication that an underlying resource has been
released. Minipass does not emit this event, but will defer it
until after `end` has been emitted, since it throws off some
stream libraries otherwise.
- `drain` Emitted when the internal buffer empties, and it is
again suitable to `write()` into the stream.
- `readable` Emitted when data is buffered and ready to be read
by a consumer.
- `resume` Emitted when stream changes state from buffering to
flowing mode. (Ie, when `resume` is called, `pipe` is called,
or a `data` event listener is added.)
### Static Methods
- `Minipass.isStream(stream)` Returns `true` if the argument is a
stream, and false otherwise. To be considered a stream, the
object must be either an instance of Minipass, or an
EventEmitter that has either a `pipe()` method, or both
`write()` and `end()` methods. (Pretty much any stream in
node-land will return `true` for this.)
## EXAMPLES
Here are some examples of things you can do with Minipass
streams.
### simple "are you done yet" promise
```js
mp.promise().then(
() => {
// stream is finished
},
er => {
// stream emitted an error
}
)
```
### collecting
```js
mp.collect().then(all => {
// all is an array of all the data emitted
// encoding is supported in this case, so
// so the result will be a collection of strings if
// an encoding is specified, or buffers/objects if not.
//
// In an async function, you may do
// const data = await stream.collect()
})
```
### collecting into a single blob
This is a bit slower because it concatenates the data into one
chunk for you, but if you're going to do it yourself anyway, it's
convenient this way:
```js
mp.concat().then(onebigchunk => {
// onebigchunk is a string if the stream
// had an encoding set, or a buffer otherwise.
})
```
### iteration
You can iterate over streams synchronously or asynchronously in
platforms that support it.
Synchronous iteration will end when the currently available data
is consumed, even if the `end` event has not been reached. In
string and buffer mode, the data is concatenated, so unless
multiple writes are occurring in the same tick as the `read()`,
sync iteration loops will generally only have a single iteration.
To consume chunks in this way exactly as they have been written,
with no flattening, create the stream with the `{ objectMode:
true }` option.
```js
const mp = new Minipass({ objectMode: true })
mp.write('a')
mp.write('b')
for (let letter of mp) {
console.log(letter) // a, b
}
mp.write('c')
mp.write('d')
for (let letter of mp) {
console.log(letter) // c, d
}
mp.write('e')
mp.end()
for (let letter of mp) {
console.log(letter) // e
}
for (let letter of mp) {
console.log(letter) // nothing
}
```
Asynchronous iteration will continue until the end event is reached,
consuming all of the data.
```js
const mp = new Minipass({ encoding: 'utf8' })
// some source of some data
let i = 5
const inter = setInterval(() => {
if (i-- > 0) mp.write(Buffer.from('foo\n', 'utf8'))
else {
mp.end()
clearInterval(inter)
}
}, 100)
// consume the data with asynchronous iteration
async function consume() {
for await (let chunk of mp) {
console.log(chunk)
}
return 'ok'
}
consume().then(res => console.log(res))
// logs `foo\n` 5 times, and then `ok`
```
### subclass that `console.log()`s everything written into it
```js
class Logger extends Minipass {
write(chunk, encoding, callback) {
console.log('WRITE', chunk, encoding)
return super.write(chunk, encoding, callback)
}
end(chunk, encoding, callback) {
console.log('END', chunk, encoding)
return super.end(chunk, encoding, callback)
}
}
someSource.pipe(new Logger()).pipe(someDest)
```
### same thing, but using an inline anonymous class
```js
// js classes are fun
someSource
.pipe(
new (class extends Minipass {
emit(ev, ...data) {
// let's also log events, because debugging some weird thing
console.log('EMIT', ev)
return super.emit(ev, ...data)
}
write(chunk, encoding, callback) {
console.log('WRITE', chunk, encoding)
return super.write(chunk, encoding, callback)
}
end(chunk, encoding, callback) {
console.log('END', chunk, encoding)
return super.end(chunk, encoding, callback)
}
})()
)
.pipe(someDest)
```
### subclass that defers 'end' for some reason
```js
class SlowEnd extends Minipass {
emit(ev, ...args) {
if (ev === 'end') {
console.log('going to end, hold on a sec')
setTimeout(() => {
console.log('ok, ready to end now')
super.emit('end', ...args)
}, 100)
return true
} else {
return super.emit(ev, ...args)
}
}
}
```
### transform that creates newline-delimited JSON
```js
class NDJSONEncode extends Minipass {
write(obj, cb) {
try {
// JSON.stringify can throw, emit an error on that
return super.write(JSON.stringify(obj) + '\n', 'utf8', cb)
} catch (er) {
this.emit('error', er)
}
}
end(obj, cb) {
if (typeof obj === 'function') {
cb = obj
obj = undefined
}
if (obj !== undefined) {
this.write(obj)
}
return super.end(cb)
}
}
```
### transform that parses newline-delimited JSON
```js
class NDJSONDecode extends Minipass {
constructor(options) {
// always be in object mode, as far as Minipass is concerned
super({ objectMode: true })
this._jsonBuffer = ''
}
write(chunk, encoding, cb) {
if (
typeof chunk === 'string' &&
typeof encoding === 'string' &&
encoding !== 'utf8'
) {
chunk = Buffer.from(chunk, encoding).toString()
} else if (Buffer.isBuffer(chunk)) {
chunk = chunk.toString()
}
if (typeof encoding === 'function') {
cb = encoding
}
const jsonData = (this._jsonBuffer + chunk).split('\n')
this._jsonBuffer = jsonData.pop()
for (let i = 0; i < jsonData.length; i++) {
try {
// JSON.parse can throw, emit an error on that
super.write(JSON.parse(jsonData[i]))
} catch (er) {
this.emit('error', er)
continue
}
}
if (cb) cb()
}
}
```

View File

@@ -0,0 +1,549 @@
/// <reference types="node" />
/// <reference types="node" />
/// <reference types="node" />
/// <reference types="node" />
import { EventEmitter } from 'node:events';
import { StringDecoder } from 'node:string_decoder';
/**
* Same as StringDecoder, but exposing the `lastNeed` flag on the type
*/
type SD = StringDecoder & {
lastNeed: boolean;
};
export type { SD, Pipe, PipeProxyErrors };
/**
* Return true if the argument is a Minipass stream, Node stream, or something
* else that Minipass can interact with.
*/
export declare const isStream: (s: any) => s is NodeJS.WriteStream | NodeJS.ReadStream | Minipass<any, any, any> | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
}) | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
/**
* Return true if the argument is a valid {@link Minipass.Readable}
*/
export declare const isReadable: (s: any) => s is Minipass.Readable;
/**
* Return true if the argument is a valid {@link Minipass.Writable}
*/
export declare const isWritable: (s: any) => s is Minipass.Readable;
declare const EOF: unique symbol;
declare const MAYBE_EMIT_END: unique symbol;
declare const EMITTED_END: unique symbol;
declare const EMITTING_END: unique symbol;
declare const EMITTED_ERROR: unique symbol;
declare const CLOSED: unique symbol;
declare const READ: unique symbol;
declare const FLUSH: unique symbol;
declare const FLUSHCHUNK: unique symbol;
declare const ENCODING: unique symbol;
declare const DECODER: unique symbol;
declare const FLOWING: unique symbol;
declare const PAUSED: unique symbol;
declare const RESUME: unique symbol;
declare const BUFFER: unique symbol;
declare const PIPES: unique symbol;
declare const BUFFERLENGTH: unique symbol;
declare const BUFFERPUSH: unique symbol;
declare const BUFFERSHIFT: unique symbol;
declare const OBJECTMODE: unique symbol;
declare const DESTROYED: unique symbol;
declare const ERROR: unique symbol;
declare const EMITDATA: unique symbol;
declare const EMITEND: unique symbol;
declare const EMITEND2: unique symbol;
declare const ASYNC: unique symbol;
declare const ABORT: unique symbol;
declare const ABORTED: unique symbol;
declare const SIGNAL: unique symbol;
declare const DATALISTENERS: unique symbol;
declare const DISCARDED: unique symbol;
/**
* Options that may be passed to stream.pipe()
*/
export interface PipeOptions {
/**
* end the destination stream when the source stream ends
*/
end?: boolean;
/**
* proxy errors from the source stream to the destination stream
*/
proxyErrors?: boolean;
}
/**
* Internal class representing a pipe to a destination stream.
*
* @internal
*/
declare class Pipe<T extends unknown> {
src: Minipass<T>;
dest: Minipass<any, T>;
opts: PipeOptions;
ondrain: () => any;
constructor(src: Minipass<T>, dest: Minipass.Writable, opts: PipeOptions);
unpipe(): void;
proxyErrors(_er: any): void;
end(): void;
}
/**
* Internal class representing a pipe to a destination stream where
* errors are proxied.
*
* @internal
*/
declare class PipeProxyErrors<T> extends Pipe<T> {
unpipe(): void;
constructor(src: Minipass<T>, dest: Minipass.Writable, opts: PipeOptions);
}
export declare namespace Minipass {
/**
* Encoding used to create a stream that outputs strings rather than
* Buffer objects.
*/
export type Encoding = BufferEncoding | 'buffer' | null;
/**
* Any stream that Minipass can pipe into
*/
export type Writable = Minipass<any, any, any> | NodeJS.WriteStream | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
/**
* Any stream that can be read from
*/
export type Readable = Minipass<any, any, any> | NodeJS.ReadStream | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
});
/**
* Utility type that can be iterated sync or async
*/
export type DualIterable<T> = Iterable<T> & AsyncIterable<T>;
type EventArguments = Record<string | symbol, unknown[]>;
/**
* The listing of events that a Minipass class can emit.
* Extend this when extending the Minipass class, and pass as
* the third template argument. The key is the name of the event,
* and the value is the argument list.
*
* Any undeclared events will still be allowed, but the handler will get
* arguments as `unknown[]`.
*/
export interface Events<RType extends any = Buffer> extends EventArguments {
readable: [];
data: [chunk: RType];
error: [er: unknown];
abort: [reason: unknown];
drain: [];
resume: [];
end: [];
finish: [];
prefinish: [];
close: [];
[DESTROYED]: [er?: unknown];
[ERROR]: [er: unknown];
}
/**
* String or buffer-like data that can be joined and sliced
*/
export type ContiguousData = Buffer | ArrayBufferLike | ArrayBufferView | string;
export type BufferOrString = Buffer | string;
/**
* Options passed to the Minipass constructor.
*/
export type SharedOptions = {
/**
* Defer all data emission and other events until the end of the
* current tick, similar to Node core streams
*/
async?: boolean;
/**
* A signal which will abort the stream
*/
signal?: AbortSignal;
/**
* Output string encoding. Set to `null` or `'buffer'` (or omit) to
* emit Buffer objects rather than strings.
*
* Conflicts with `objectMode`
*/
encoding?: BufferEncoding | null | 'buffer';
/**
* Output data exactly as it was written, supporting non-buffer/string
* data (such as arbitrary objects, falsey values, etc.)
*
* Conflicts with `encoding`
*/
objectMode?: boolean;
};
/**
* Options for a string encoded output
*/
export type EncodingOptions = SharedOptions & {
encoding: BufferEncoding;
objectMode?: false;
};
/**
* Options for contiguous data buffer output
*/
export type BufferOptions = SharedOptions & {
encoding?: null | 'buffer';
objectMode?: false;
};
/**
* Options for objectMode arbitrary output
*/
export type ObjectModeOptions = SharedOptions & {
objectMode: true;
encoding?: null;
};
/**
* Utility type to determine allowed options based on read type
*/
export type Options<T> = ObjectModeOptions | (T extends string ? EncodingOptions : T extends Buffer ? BufferOptions : SharedOptions);
export {};
}
/**
* Main export, the Minipass class
*
* `RType` is the type of data emitted, defaults to Buffer
*
* `WType` is the type of data to be written, if RType is buffer or string,
* then any {@link Minipass.ContiguousData} is allowed.
*
* `Events` is the set of event handler signatures that this object
* will emit, see {@link Minipass.Events}
*/
export declare class Minipass<RType extends unknown = Buffer, WType extends unknown = RType extends Minipass.BufferOrString ? Minipass.ContiguousData : RType, Events extends Minipass.Events<RType> = Minipass.Events<RType>> extends EventEmitter implements Minipass.DualIterable<RType> {
[FLOWING]: boolean;
[PAUSED]: boolean;
[PIPES]: Pipe<RType>[];
[BUFFER]: RType[];
[OBJECTMODE]: boolean;
[ENCODING]: BufferEncoding | null;
[ASYNC]: boolean;
[DECODER]: SD | null;
[EOF]: boolean;
[EMITTED_END]: boolean;
[EMITTING_END]: boolean;
[CLOSED]: boolean;
[EMITTED_ERROR]: unknown;
[BUFFERLENGTH]: number;
[DESTROYED]: boolean;
[SIGNAL]?: AbortSignal;
[ABORTED]: boolean;
[DATALISTENERS]: number;
[DISCARDED]: boolean;
/**
* true if the stream can be written
*/
writable: boolean;
/**
* true if the stream can be read
*/
readable: boolean;
/**
* If `RType` is Buffer, then options do not need to be provided.
* Otherwise, an options object must be provided to specify either
* {@link Minipass.SharedOptions.objectMode} or
* {@link Minipass.SharedOptions.encoding}, as appropriate.
*/
constructor(...args: [Minipass.ObjectModeOptions] | (RType extends Buffer ? [] | [Minipass.Options<RType>] : [Minipass.Options<RType>]));
/**
* The amount of data stored in the buffer waiting to be read.
*
* For Buffer strings, this will be the total byte length.
* For string encoding streams, this will be the string character length,
* according to JavaScript's `string.length` logic.
* For objectMode streams, this is a count of the items waiting to be
* emitted.
*/
get bufferLength(): number;
/**
* The `BufferEncoding` currently in use, or `null`
*/
get encoding(): BufferEncoding | null;
/**
* @deprecated - This is a read only property
*/
set encoding(_enc: BufferEncoding | null);
/**
* @deprecated - Encoding may only be set at instantiation time
*/
setEncoding(_enc: Minipass.Encoding): void;
/**
* True if this is an objectMode stream
*/
get objectMode(): boolean;
/**
* @deprecated - This is a read-only property
*/
set objectMode(_om: boolean);
/**
* true if this is an async stream
*/
get ['async'](): boolean;
/**
* Set to true to make this stream async.
*
* Once set, it cannot be unset, as this would potentially cause incorrect
* behavior. Ie, a sync stream can be made async, but an async stream
* cannot be safely made sync.
*/
set ['async'](a: boolean);
[ABORT](): void;
/**
* True if the stream has been aborted.
*/
get aborted(): boolean;
/**
* No-op setter. Stream aborted status is set via the AbortSignal provided
* in the constructor options.
*/
set aborted(_: boolean);
/**
* Write data into the stream
*
* If the chunk written is a string, and encoding is not specified, then
* `utf8` will be assumed. If the stream encoding matches the encoding of
* a written string, and the state of the string decoder allows it, then
* the string will be passed through to either the output or the internal
* buffer without any processing. Otherwise, it will be turned into a
* Buffer object for processing into the desired encoding.
*
* If provided, `cb` function is called immediately before return for
* sync streams, or on next tick for async streams, because for this
* base class, a chunk is considered "processed" once it is accepted
* and either emitted or buffered. That is, the callback does not indicate
* that the chunk has been eventually emitted, though of course child
* classes can override this function to do whatever processing is required
* and call `super.write(...)` only once processing is completed.
*/
write(chunk: WType, cb?: () => void): boolean;
write(chunk: WType, encoding?: Minipass.Encoding, cb?: () => void): boolean;
/**
* Low-level explicit read method.
*
* In objectMode, the argument is ignored, and one item is returned if
* available.
*
* `n` is the number of bytes (or in the case of encoding streams,
* characters) to consume. If `n` is not provided, then the entire buffer
* is returned, or `null` is returned if no data is available.
*
* If `n` is greater that the amount of data in the internal buffer,
* then `null` is returned.
*/
read(n?: number | null): RType | null;
[READ](n: number | null, chunk: RType): RType;
/**
* End the stream, optionally providing a final write.
*
* See {@link Minipass#write} for argument descriptions
*/
end(cb?: () => void): this;
end(chunk: WType, cb?: () => void): this;
end(chunk: WType, encoding?: Minipass.Encoding, cb?: () => void): this;
[RESUME](): void;
/**
* Resume the stream if it is currently in a paused state
*
* If called when there are no pipe destinations or `data` event listeners,
* this will place the stream in a "discarded" state, where all data will
* be thrown away. The discarded state is removed if a pipe destination or
* data handler is added, if pause() is called, or if any synchronous or
* asynchronous iteration is started.
*/
resume(): void;
/**
* Pause the stream
*/
pause(): void;
/**
* true if the stream has been forcibly destroyed
*/
get destroyed(): boolean;
/**
* true if the stream is currently in a flowing state, meaning that
* any writes will be immediately emitted.
*/
get flowing(): boolean;
/**
* true if the stream is currently in a paused state
*/
get paused(): boolean;
[BUFFERPUSH](chunk: RType): void;
[BUFFERSHIFT](): RType;
[FLUSH](noDrain?: boolean): void;
[FLUSHCHUNK](chunk: RType): boolean;
/**
* Pipe all data emitted by this stream into the destination provided.
*
* Triggers the flow of data.
*/
pipe<W extends Minipass.Writable>(dest: W, opts?: PipeOptions): W;
/**
* Fully unhook a piped destination stream.
*
* If the destination stream was the only consumer of this stream (ie,
* there are no other piped destinations or `'data'` event listeners)
* then the flow of data will stop until there is another consumer or
* {@link Minipass#resume} is explicitly called.
*/
unpipe<W extends Minipass.Writable>(dest: W): void;
/**
* Alias for {@link Minipass#on}
*/
addListener<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.on`, with the following
* behavior differences to prevent data loss and unnecessary hangs:
*
* - Adding a 'data' event handler will trigger the flow of data
*
* - Adding a 'readable' event handler when there is data waiting to be read
* will cause 'readable' to be emitted immediately.
*
* - Adding an 'endish' event handler ('end', 'finish', etc.) which has
* already passed will cause the event to be emitted immediately and all
* handlers removed.
*
* - Adding an 'error' event handler after an error has been emitted will
* cause the event to be re-emitted immediately with the error previously
* raised.
*/
on<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Alias for {@link Minipass#off}
*/
removeListener<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.off`
*
* If a 'data' event handler is removed, and it was the last consumer
* (ie, there are no pipe destinations or other 'data' event listeners),
* then the flow of data will stop until there is another consumer or
* {@link Minipass#resume} is explicitly called.
*/
off<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.removeAllListeners`
*
* If all 'data' event handlers are removed, and they were the last consumer
* (ie, there are no pipe destinations), then the flow of data will stop
* until there is another consumer or {@link Minipass#resume} is explicitly
* called.
*/
removeAllListeners<Event extends keyof Events>(ev?: Event): this;
/**
* true if the 'end' event has been emitted
*/
get emittedEnd(): boolean;
[MAYBE_EMIT_END](): void;
/**
* Mostly identical to `EventEmitter.emit`, with the following
* behavior differences to prevent data loss and unnecessary hangs:
*
* If the stream has been destroyed, and the event is something other
* than 'close' or 'error', then `false` is returned and no handlers
* are called.
*
* If the event is 'end', and has already been emitted, then the event
* is ignored. If the stream is in a paused or non-flowing state, then
* the event will be deferred until data flow resumes. If the stream is
* async, then handlers will be called on the next tick rather than
* immediately.
*
* If the event is 'close', and 'end' has not yet been emitted, then
* the event will be deferred until after 'end' is emitted.
*
* If the event is 'error', and an AbortSignal was provided for the stream,
* and there are no listeners, then the event is ignored, matching the
* behavior of node core streams in the presense of an AbortSignal.
*
* If the event is 'finish' or 'prefinish', then all listeners will be
* removed after emitting the event, to prevent double-firing.
*/
emit<Event extends keyof Events>(ev: Event, ...args: Events[Event]): boolean;
[EMITDATA](data: RType): boolean;
[EMITEND](): boolean;
[EMITEND2](): boolean;
/**
* Return a Promise that resolves to an array of all emitted data once
* the stream ends.
*/
collect(): Promise<RType[] & {
dataLength: number;
}>;
/**
* Return a Promise that resolves to the concatenation of all emitted data
* once the stream ends.
*
* Not allowed on objectMode streams.
*/
concat(): Promise<RType>;
/**
* Return a void Promise that resolves once the stream ends.
*/
promise(): Promise<void>;
/**
* Asynchronous `for await of` iteration.
*
* This will continue emitting all chunks until the stream terminates.
*/
[Symbol.asyncIterator](): AsyncGenerator<RType, void, void>;
/**
* Synchronous `for of` iteration.
*
* The iteration will terminate when the internal buffer runs out, even
* if the stream has not yet terminated.
*/
[Symbol.iterator](): Generator<RType, void, void>;
/**
* Destroy a stream, preventing it from being used for any further purpose.
*
* If the stream has a `close()` method, then it will be called on
* destruction.
*
* After destruction, any attempt to write data, read data, or emit most
* events will be ignored.
*
* If an error argument is provided, then it will be emitted in an
* 'error' event.
*/
destroy(er?: unknown): this;
/**
* Alias for {@link isStream}
*
* Former export location, maintained for backwards compatibility.
*
* @deprecated
*/
static get isStream(): (s: any) => s is NodeJS.WriteStream | NodeJS.ReadStream | Minipass<any, any, any> | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
}) | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
}
//# sourceMappingURL=index.d.ts.map

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,3 @@
{
"type": "commonjs"
}

View File

@@ -0,0 +1,549 @@
/// <reference types="node" resolution-mode="require"/>
/// <reference types="node" resolution-mode="require"/>
/// <reference types="node" resolution-mode="require"/>
/// <reference types="node" resolution-mode="require"/>
import { EventEmitter } from 'node:events';
import { StringDecoder } from 'node:string_decoder';
/**
* Same as StringDecoder, but exposing the `lastNeed` flag on the type
*/
type SD = StringDecoder & {
lastNeed: boolean;
};
export type { SD, Pipe, PipeProxyErrors };
/**
* Return true if the argument is a Minipass stream, Node stream, or something
* else that Minipass can interact with.
*/
export declare const isStream: (s: any) => s is NodeJS.WriteStream | NodeJS.ReadStream | Minipass<any, any, any> | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
}) | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
/**
* Return true if the argument is a valid {@link Minipass.Readable}
*/
export declare const isReadable: (s: any) => s is Minipass.Readable;
/**
* Return true if the argument is a valid {@link Minipass.Writable}
*/
export declare const isWritable: (s: any) => s is Minipass.Readable;
declare const EOF: unique symbol;
declare const MAYBE_EMIT_END: unique symbol;
declare const EMITTED_END: unique symbol;
declare const EMITTING_END: unique symbol;
declare const EMITTED_ERROR: unique symbol;
declare const CLOSED: unique symbol;
declare const READ: unique symbol;
declare const FLUSH: unique symbol;
declare const FLUSHCHUNK: unique symbol;
declare const ENCODING: unique symbol;
declare const DECODER: unique symbol;
declare const FLOWING: unique symbol;
declare const PAUSED: unique symbol;
declare const RESUME: unique symbol;
declare const BUFFER: unique symbol;
declare const PIPES: unique symbol;
declare const BUFFERLENGTH: unique symbol;
declare const BUFFERPUSH: unique symbol;
declare const BUFFERSHIFT: unique symbol;
declare const OBJECTMODE: unique symbol;
declare const DESTROYED: unique symbol;
declare const ERROR: unique symbol;
declare const EMITDATA: unique symbol;
declare const EMITEND: unique symbol;
declare const EMITEND2: unique symbol;
declare const ASYNC: unique symbol;
declare const ABORT: unique symbol;
declare const ABORTED: unique symbol;
declare const SIGNAL: unique symbol;
declare const DATALISTENERS: unique symbol;
declare const DISCARDED: unique symbol;
/**
* Options that may be passed to stream.pipe()
*/
export interface PipeOptions {
/**
* end the destination stream when the source stream ends
*/
end?: boolean;
/**
* proxy errors from the source stream to the destination stream
*/
proxyErrors?: boolean;
}
/**
* Internal class representing a pipe to a destination stream.
*
* @internal
*/
declare class Pipe<T extends unknown> {
src: Minipass<T>;
dest: Minipass<any, T>;
opts: PipeOptions;
ondrain: () => any;
constructor(src: Minipass<T>, dest: Minipass.Writable, opts: PipeOptions);
unpipe(): void;
proxyErrors(_er: any): void;
end(): void;
}
/**
* Internal class representing a pipe to a destination stream where
* errors are proxied.
*
* @internal
*/
declare class PipeProxyErrors<T> extends Pipe<T> {
unpipe(): void;
constructor(src: Minipass<T>, dest: Minipass.Writable, opts: PipeOptions);
}
export declare namespace Minipass {
/**
* Encoding used to create a stream that outputs strings rather than
* Buffer objects.
*/
export type Encoding = BufferEncoding | 'buffer' | null;
/**
* Any stream that Minipass can pipe into
*/
export type Writable = Minipass<any, any, any> | NodeJS.WriteStream | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
/**
* Any stream that can be read from
*/
export type Readable = Minipass<any, any, any> | NodeJS.ReadStream | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
});
/**
* Utility type that can be iterated sync or async
*/
export type DualIterable<T> = Iterable<T> & AsyncIterable<T>;
type EventArguments = Record<string | symbol, unknown[]>;
/**
* The listing of events that a Minipass class can emit.
* Extend this when extending the Minipass class, and pass as
* the third template argument. The key is the name of the event,
* and the value is the argument list.
*
* Any undeclared events will still be allowed, but the handler will get
* arguments as `unknown[]`.
*/
export interface Events<RType extends any = Buffer> extends EventArguments {
readable: [];
data: [chunk: RType];
error: [er: unknown];
abort: [reason: unknown];
drain: [];
resume: [];
end: [];
finish: [];
prefinish: [];
close: [];
[DESTROYED]: [er?: unknown];
[ERROR]: [er: unknown];
}
/**
* String or buffer-like data that can be joined and sliced
*/
export type ContiguousData = Buffer | ArrayBufferLike | ArrayBufferView | string;
export type BufferOrString = Buffer | string;
/**
* Options passed to the Minipass constructor.
*/
export type SharedOptions = {
/**
* Defer all data emission and other events until the end of the
* current tick, similar to Node core streams
*/
async?: boolean;
/**
* A signal which will abort the stream
*/
signal?: AbortSignal;
/**
* Output string encoding. Set to `null` or `'buffer'` (or omit) to
* emit Buffer objects rather than strings.
*
* Conflicts with `objectMode`
*/
encoding?: BufferEncoding | null | 'buffer';
/**
* Output data exactly as it was written, supporting non-buffer/string
* data (such as arbitrary objects, falsey values, etc.)
*
* Conflicts with `encoding`
*/
objectMode?: boolean;
};
/**
* Options for a string encoded output
*/
export type EncodingOptions = SharedOptions & {
encoding: BufferEncoding;
objectMode?: false;
};
/**
* Options for contiguous data buffer output
*/
export type BufferOptions = SharedOptions & {
encoding?: null | 'buffer';
objectMode?: false;
};
/**
* Options for objectMode arbitrary output
*/
export type ObjectModeOptions = SharedOptions & {
objectMode: true;
encoding?: null;
};
/**
* Utility type to determine allowed options based on read type
*/
export type Options<T> = ObjectModeOptions | (T extends string ? EncodingOptions : T extends Buffer ? BufferOptions : SharedOptions);
export {};
}
/**
* Main export, the Minipass class
*
* `RType` is the type of data emitted, defaults to Buffer
*
* `WType` is the type of data to be written, if RType is buffer or string,
* then any {@link Minipass.ContiguousData} is allowed.
*
* `Events` is the set of event handler signatures that this object
* will emit, see {@link Minipass.Events}
*/
export declare class Minipass<RType extends unknown = Buffer, WType extends unknown = RType extends Minipass.BufferOrString ? Minipass.ContiguousData : RType, Events extends Minipass.Events<RType> = Minipass.Events<RType>> extends EventEmitter implements Minipass.DualIterable<RType> {
[FLOWING]: boolean;
[PAUSED]: boolean;
[PIPES]: Pipe<RType>[];
[BUFFER]: RType[];
[OBJECTMODE]: boolean;
[ENCODING]: BufferEncoding | null;
[ASYNC]: boolean;
[DECODER]: SD | null;
[EOF]: boolean;
[EMITTED_END]: boolean;
[EMITTING_END]: boolean;
[CLOSED]: boolean;
[EMITTED_ERROR]: unknown;
[BUFFERLENGTH]: number;
[DESTROYED]: boolean;
[SIGNAL]?: AbortSignal;
[ABORTED]: boolean;
[DATALISTENERS]: number;
[DISCARDED]: boolean;
/**
* true if the stream can be written
*/
writable: boolean;
/**
* true if the stream can be read
*/
readable: boolean;
/**
* If `RType` is Buffer, then options do not need to be provided.
* Otherwise, an options object must be provided to specify either
* {@link Minipass.SharedOptions.objectMode} or
* {@link Minipass.SharedOptions.encoding}, as appropriate.
*/
constructor(...args: [Minipass.ObjectModeOptions] | (RType extends Buffer ? [] | [Minipass.Options<RType>] : [Minipass.Options<RType>]));
/**
* The amount of data stored in the buffer waiting to be read.
*
* For Buffer strings, this will be the total byte length.
* For string encoding streams, this will be the string character length,
* according to JavaScript's `string.length` logic.
* For objectMode streams, this is a count of the items waiting to be
* emitted.
*/
get bufferLength(): number;
/**
* The `BufferEncoding` currently in use, or `null`
*/
get encoding(): BufferEncoding | null;
/**
* @deprecated - This is a read only property
*/
set encoding(_enc: BufferEncoding | null);
/**
* @deprecated - Encoding may only be set at instantiation time
*/
setEncoding(_enc: Minipass.Encoding): void;
/**
* True if this is an objectMode stream
*/
get objectMode(): boolean;
/**
* @deprecated - This is a read-only property
*/
set objectMode(_om: boolean);
/**
* true if this is an async stream
*/
get ['async'](): boolean;
/**
* Set to true to make this stream async.
*
* Once set, it cannot be unset, as this would potentially cause incorrect
* behavior. Ie, a sync stream can be made async, but an async stream
* cannot be safely made sync.
*/
set ['async'](a: boolean);
[ABORT](): void;
/**
* True if the stream has been aborted.
*/
get aborted(): boolean;
/**
* No-op setter. Stream aborted status is set via the AbortSignal provided
* in the constructor options.
*/
set aborted(_: boolean);
/**
* Write data into the stream
*
* If the chunk written is a string, and encoding is not specified, then
* `utf8` will be assumed. If the stream encoding matches the encoding of
* a written string, and the state of the string decoder allows it, then
* the string will be passed through to either the output or the internal
* buffer without any processing. Otherwise, it will be turned into a
* Buffer object for processing into the desired encoding.
*
* If provided, `cb` function is called immediately before return for
* sync streams, or on next tick for async streams, because for this
* base class, a chunk is considered "processed" once it is accepted
* and either emitted or buffered. That is, the callback does not indicate
* that the chunk has been eventually emitted, though of course child
* classes can override this function to do whatever processing is required
* and call `super.write(...)` only once processing is completed.
*/
write(chunk: WType, cb?: () => void): boolean;
write(chunk: WType, encoding?: Minipass.Encoding, cb?: () => void): boolean;
/**
* Low-level explicit read method.
*
* In objectMode, the argument is ignored, and one item is returned if
* available.
*
* `n` is the number of bytes (or in the case of encoding streams,
* characters) to consume. If `n` is not provided, then the entire buffer
* is returned, or `null` is returned if no data is available.
*
* If `n` is greater that the amount of data in the internal buffer,
* then `null` is returned.
*/
read(n?: number | null): RType | null;
[READ](n: number | null, chunk: RType): RType;
/**
* End the stream, optionally providing a final write.
*
* See {@link Minipass#write} for argument descriptions
*/
end(cb?: () => void): this;
end(chunk: WType, cb?: () => void): this;
end(chunk: WType, encoding?: Minipass.Encoding, cb?: () => void): this;
[RESUME](): void;
/**
* Resume the stream if it is currently in a paused state
*
* If called when there are no pipe destinations or `data` event listeners,
* this will place the stream in a "discarded" state, where all data will
* be thrown away. The discarded state is removed if a pipe destination or
* data handler is added, if pause() is called, or if any synchronous or
* asynchronous iteration is started.
*/
resume(): void;
/**
* Pause the stream
*/
pause(): void;
/**
* true if the stream has been forcibly destroyed
*/
get destroyed(): boolean;
/**
* true if the stream is currently in a flowing state, meaning that
* any writes will be immediately emitted.
*/
get flowing(): boolean;
/**
* true if the stream is currently in a paused state
*/
get paused(): boolean;
[BUFFERPUSH](chunk: RType): void;
[BUFFERSHIFT](): RType;
[FLUSH](noDrain?: boolean): void;
[FLUSHCHUNK](chunk: RType): boolean;
/**
* Pipe all data emitted by this stream into the destination provided.
*
* Triggers the flow of data.
*/
pipe<W extends Minipass.Writable>(dest: W, opts?: PipeOptions): W;
/**
* Fully unhook a piped destination stream.
*
* If the destination stream was the only consumer of this stream (ie,
* there are no other piped destinations or `'data'` event listeners)
* then the flow of data will stop until there is another consumer or
* {@link Minipass#resume} is explicitly called.
*/
unpipe<W extends Minipass.Writable>(dest: W): void;
/**
* Alias for {@link Minipass#on}
*/
addListener<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.on`, with the following
* behavior differences to prevent data loss and unnecessary hangs:
*
* - Adding a 'data' event handler will trigger the flow of data
*
* - Adding a 'readable' event handler when there is data waiting to be read
* will cause 'readable' to be emitted immediately.
*
* - Adding an 'endish' event handler ('end', 'finish', etc.) which has
* already passed will cause the event to be emitted immediately and all
* handlers removed.
*
* - Adding an 'error' event handler after an error has been emitted will
* cause the event to be re-emitted immediately with the error previously
* raised.
*/
on<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Alias for {@link Minipass#off}
*/
removeListener<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.off`
*
* If a 'data' event handler is removed, and it was the last consumer
* (ie, there are no pipe destinations or other 'data' event listeners),
* then the flow of data will stop until there is another consumer or
* {@link Minipass#resume} is explicitly called.
*/
off<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.removeAllListeners`
*
* If all 'data' event handlers are removed, and they were the last consumer
* (ie, there are no pipe destinations), then the flow of data will stop
* until there is another consumer or {@link Minipass#resume} is explicitly
* called.
*/
removeAllListeners<Event extends keyof Events>(ev?: Event): this;
/**
* true if the 'end' event has been emitted
*/
get emittedEnd(): boolean;
[MAYBE_EMIT_END](): void;
/**
* Mostly identical to `EventEmitter.emit`, with the following
* behavior differences to prevent data loss and unnecessary hangs:
*
* If the stream has been destroyed, and the event is something other
* than 'close' or 'error', then `false` is returned and no handlers
* are called.
*
* If the event is 'end', and has already been emitted, then the event
* is ignored. If the stream is in a paused or non-flowing state, then
* the event will be deferred until data flow resumes. If the stream is
* async, then handlers will be called on the next tick rather than
* immediately.
*
* If the event is 'close', and 'end' has not yet been emitted, then
* the event will be deferred until after 'end' is emitted.
*
* If the event is 'error', and an AbortSignal was provided for the stream,
* and there are no listeners, then the event is ignored, matching the
* behavior of node core streams in the presense of an AbortSignal.
*
* If the event is 'finish' or 'prefinish', then all listeners will be
* removed after emitting the event, to prevent double-firing.
*/
emit<Event extends keyof Events>(ev: Event, ...args: Events[Event]): boolean;
[EMITDATA](data: RType): boolean;
[EMITEND](): boolean;
[EMITEND2](): boolean;
/**
* Return a Promise that resolves to an array of all emitted data once
* the stream ends.
*/
collect(): Promise<RType[] & {
dataLength: number;
}>;
/**
* Return a Promise that resolves to the concatenation of all emitted data
* once the stream ends.
*
* Not allowed on objectMode streams.
*/
concat(): Promise<RType>;
/**
* Return a void Promise that resolves once the stream ends.
*/
promise(): Promise<void>;
/**
* Asynchronous `for await of` iteration.
*
* This will continue emitting all chunks until the stream terminates.
*/
[Symbol.asyncIterator](): AsyncGenerator<RType, void, void>;
/**
* Synchronous `for of` iteration.
*
* The iteration will terminate when the internal buffer runs out, even
* if the stream has not yet terminated.
*/
[Symbol.iterator](): Generator<RType, void, void>;
/**
* Destroy a stream, preventing it from being used for any further purpose.
*
* If the stream has a `close()` method, then it will be called on
* destruction.
*
* After destruction, any attempt to write data, read data, or emit most
* events will be ignored.
*
* If an error argument is provided, then it will be emitted in an
* 'error' event.
*/
destroy(er?: unknown): this;
/**
* Alias for {@link isStream}
*
* Former export location, maintained for backwards compatibility.
*
* @deprecated
*/
static get isStream(): (s: any) => s is NodeJS.WriteStream | NodeJS.ReadStream | Minipass<any, any, any> | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
}) | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
}
//# sourceMappingURL=index.d.ts.map

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,3 @@
{
"type": "module"
}

View File

@@ -0,0 +1,82 @@
{
"name": "minipass",
"version": "7.1.2",
"description": "minimal implementation of a PassThrough stream",
"main": "./dist/commonjs/index.js",
"types": "./dist/commonjs/index.d.ts",
"type": "module",
"tshy": {
"selfLink": false,
"main": true,
"exports": {
"./package.json": "./package.json",
".": "./src/index.ts"
}
},
"exports": {
"./package.json": "./package.json",
".": {
"import": {
"types": "./dist/esm/index.d.ts",
"default": "./dist/esm/index.js"
},
"require": {
"types": "./dist/commonjs/index.d.ts",
"default": "./dist/commonjs/index.js"
}
}
},
"files": [
"dist"
],
"scripts": {
"preversion": "npm test",
"postversion": "npm publish",
"prepublishOnly": "git push origin --follow-tags",
"prepare": "tshy",
"pretest": "npm run prepare",
"presnap": "npm run prepare",
"test": "tap",
"snap": "tap",
"format": "prettier --write . --loglevel warn",
"typedoc": "typedoc --tsconfig .tshy/esm.json ./src/*.ts"
},
"prettier": {
"semi": false,
"printWidth": 75,
"tabWidth": 2,
"useTabs": false,
"singleQuote": true,
"jsxSingleQuote": false,
"bracketSameLine": true,
"arrowParens": "avoid",
"endOfLine": "lf"
},
"devDependencies": {
"@types/end-of-stream": "^1.4.2",
"@types/node": "^20.1.2",
"end-of-stream": "^1.4.0",
"node-abort-controller": "^3.1.1",
"prettier": "^2.6.2",
"tap": "^19.0.0",
"through2": "^2.0.3",
"tshy": "^1.14.0",
"typedoc": "^0.25.1"
},
"repository": "https://github.com/isaacs/minipass",
"keywords": [
"passthrough",
"stream"
],
"author": "Isaac Z. Schlueter <i@izs.me> (http://blog.izs.me/)",
"license": "ISC",
"engines": {
"node": ">=16 || 14 >=14.17"
},
"tap": {
"typecheck": true,
"include": [
"test/*.ts"
]
}
}

View File

@@ -0,0 +1,82 @@
{
"name": "cacache",
"version": "17.1.4",
"cache-version": {
"content": "2",
"index": "5"
},
"description": "Fast, fault-tolerant, cross-platform, disk-based, data-agnostic, content-addressable cache.",
"main": "lib/index.js",
"files": [
"bin/",
"lib/"
],
"scripts": {
"test": "tap",
"snap": "tap",
"coverage": "tap",
"test-docker": "docker run -it --rm --name pacotest -v \"$PWD\":/tmp -w /tmp node:latest npm test",
"lint": "eslint \"**/*.js\"",
"npmclilint": "npmcli-lint",
"lintfix": "npm run lint -- --fix",
"postsnap": "npm run lintfix --",
"postlint": "template-oss-check",
"posttest": "npm run lint",
"template-oss-apply": "template-oss-apply --force"
},
"repository": {
"type": "git",
"url": "https://github.com/npm/cacache.git"
},
"keywords": [
"cache",
"caching",
"content-addressable",
"sri",
"sri hash",
"subresource integrity",
"cache",
"storage",
"store",
"file store",
"filesystem",
"disk cache",
"disk storage"
],
"license": "ISC",
"dependencies": {
"@npmcli/fs": "^3.1.0",
"fs-minipass": "^3.0.0",
"glob": "^10.2.2",
"lru-cache": "^7.7.1",
"minipass": "^7.0.3",
"minipass-collect": "^1.0.2",
"minipass-flush": "^1.0.5",
"minipass-pipeline": "^1.2.4",
"p-map": "^4.0.0",
"ssri": "^10.0.0",
"tar": "^6.1.11",
"unique-filename": "^3.0.0"
},
"devDependencies": {
"@npmcli/eslint-config": "^4.0.0",
"@npmcli/template-oss": "4.18.0",
"tap": "^16.0.0"
},
"engines": {
"node": "^14.17.0 || ^16.13.0 || >=18.0.0"
},
"templateOSS": {
"//@npmcli/template-oss": "This file is partially managed by @npmcli/template-oss. Edits may be overwritten.",
"windowsCI": false,
"version": "4.18.0",
"publish": "true"
},
"author": "GitHub Inc.",
"tap": {
"nyc-arg": [
"--exclude",
"tap-snapshots/**"
]
}
}

View File

@@ -0,0 +1,15 @@
The ISC License
Copyright (c) Isaac Z. Schlueter and Contributors
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

View File

@@ -0,0 +1,70 @@
# fs-minipass
Filesystem streams based on [minipass](http://npm.im/minipass).
4 classes are exported:
- ReadStream
- ReadStreamSync
- WriteStream
- WriteStreamSync
When using `ReadStreamSync`, all of the data is made available
immediately upon consuming the stream. Nothing is buffered in memory
when the stream is constructed. If the stream is piped to a writer,
then it will synchronously `read()` and emit data into the writer as
fast as the writer can consume it. (That is, it will respect
backpressure.) If you call `stream.read()` then it will read the
entire file and return the contents.
When using `WriteStreamSync`, every write is flushed to the file
synchronously. If your writes all come in a single tick, then it'll
write it all out in a single tick. It's as synchronous as you are.
The async versions work much like their node builtin counterparts,
with the exception of introducing significantly less Stream machinery
overhead.
## USAGE
It's just streams, you pipe them or read() them or write() to them.
```js
const fsm = require('fs-minipass')
const readStream = new fsm.ReadStream('file.txt')
const writeStream = new fsm.WriteStream('output.txt')
writeStream.write('some file header or whatever\n')
readStream.pipe(writeStream)
```
## ReadStream(path, options)
Path string is required, but somewhat irrelevant if an open file
descriptor is passed in as an option.
Options:
- `fd` Pass in a numeric file descriptor, if the file is already open.
- `readSize` The size of reads to do, defaults to 16MB
- `size` The size of the file, if known. Prevents zero-byte read()
call at the end.
- `autoClose` Set to `false` to prevent the file descriptor from being
closed when the file is done being read.
## WriteStream(path, options)
Path string is required, but somewhat irrelevant if an open file
descriptor is passed in as an option.
Options:
- `fd` Pass in a numeric file descriptor, if the file is already open.
- `mode` The mode to create the file with. Defaults to `0o666`.
- `start` The position in the file to start reading. If not
specified, then the file will start writing at position zero, and be
truncated by default.
- `autoClose` Set to `false` to prevent the file descriptor from being
closed when the stream is ended.
- `flags` Flags to use when opening the file. Irrelevant if `fd` is
passed in, since file won't be opened in that case. Defaults to
`'a'` if a `pos` is specified, or `'w'` otherwise.

View File

@@ -0,0 +1,443 @@
'use strict'
const { Minipass } = require('minipass')
const EE = require('events').EventEmitter
const fs = require('fs')
const writev = fs.writev
const _autoClose = Symbol('_autoClose')
const _close = Symbol('_close')
const _ended = Symbol('_ended')
const _fd = Symbol('_fd')
const _finished = Symbol('_finished')
const _flags = Symbol('_flags')
const _flush = Symbol('_flush')
const _handleChunk = Symbol('_handleChunk')
const _makeBuf = Symbol('_makeBuf')
const _mode = Symbol('_mode')
const _needDrain = Symbol('_needDrain')
const _onerror = Symbol('_onerror')
const _onopen = Symbol('_onopen')
const _onread = Symbol('_onread')
const _onwrite = Symbol('_onwrite')
const _open = Symbol('_open')
const _path = Symbol('_path')
const _pos = Symbol('_pos')
const _queue = Symbol('_queue')
const _read = Symbol('_read')
const _readSize = Symbol('_readSize')
const _reading = Symbol('_reading')
const _remain = Symbol('_remain')
const _size = Symbol('_size')
const _write = Symbol('_write')
const _writing = Symbol('_writing')
const _defaultFlag = Symbol('_defaultFlag')
const _errored = Symbol('_errored')
class ReadStream extends Minipass {
constructor (path, opt) {
opt = opt || {}
super(opt)
this.readable = true
this.writable = false
if (typeof path !== 'string') {
throw new TypeError('path must be a string')
}
this[_errored] = false
this[_fd] = typeof opt.fd === 'number' ? opt.fd : null
this[_path] = path
this[_readSize] = opt.readSize || 16 * 1024 * 1024
this[_reading] = false
this[_size] = typeof opt.size === 'number' ? opt.size : Infinity
this[_remain] = this[_size]
this[_autoClose] = typeof opt.autoClose === 'boolean' ?
opt.autoClose : true
if (typeof this[_fd] === 'number') {
this[_read]()
} else {
this[_open]()
}
}
get fd () {
return this[_fd]
}
get path () {
return this[_path]
}
write () {
throw new TypeError('this is a readable stream')
}
end () {
throw new TypeError('this is a readable stream')
}
[_open] () {
fs.open(this[_path], 'r', (er, fd) => this[_onopen](er, fd))
}
[_onopen] (er, fd) {
if (er) {
this[_onerror](er)
} else {
this[_fd] = fd
this.emit('open', fd)
this[_read]()
}
}
[_makeBuf] () {
return Buffer.allocUnsafe(Math.min(this[_readSize], this[_remain]))
}
[_read] () {
if (!this[_reading]) {
this[_reading] = true
const buf = this[_makeBuf]()
/* istanbul ignore if */
if (buf.length === 0) {
return process.nextTick(() => this[_onread](null, 0, buf))
}
fs.read(this[_fd], buf, 0, buf.length, null, (er, br, b) =>
this[_onread](er, br, b))
}
}
[_onread] (er, br, buf) {
this[_reading] = false
if (er) {
this[_onerror](er)
} else if (this[_handleChunk](br, buf)) {
this[_read]()
}
}
[_close] () {
if (this[_autoClose] && typeof this[_fd] === 'number') {
const fd = this[_fd]
this[_fd] = null
fs.close(fd, er => er ? this.emit('error', er) : this.emit('close'))
}
}
[_onerror] (er) {
this[_reading] = true
this[_close]()
this.emit('error', er)
}
[_handleChunk] (br, buf) {
let ret = false
// no effect if infinite
this[_remain] -= br
if (br > 0) {
ret = super.write(br < buf.length ? buf.slice(0, br) : buf)
}
if (br === 0 || this[_remain] <= 0) {
ret = false
this[_close]()
super.end()
}
return ret
}
emit (ev, data) {
switch (ev) {
case 'prefinish':
case 'finish':
break
case 'drain':
if (typeof this[_fd] === 'number') {
this[_read]()
}
break
case 'error':
if (this[_errored]) {
return
}
this[_errored] = true
return super.emit(ev, data)
default:
return super.emit(ev, data)
}
}
}
class ReadStreamSync extends ReadStream {
[_open] () {
let threw = true
try {
this[_onopen](null, fs.openSync(this[_path], 'r'))
threw = false
} finally {
if (threw) {
this[_close]()
}
}
}
[_read] () {
let threw = true
try {
if (!this[_reading]) {
this[_reading] = true
do {
const buf = this[_makeBuf]()
/* istanbul ignore next */
const br = buf.length === 0 ? 0
: fs.readSync(this[_fd], buf, 0, buf.length, null)
if (!this[_handleChunk](br, buf)) {
break
}
} while (true)
this[_reading] = false
}
threw = false
} finally {
if (threw) {
this[_close]()
}
}
}
[_close] () {
if (this[_autoClose] && typeof this[_fd] === 'number') {
const fd = this[_fd]
this[_fd] = null
fs.closeSync(fd)
this.emit('close')
}
}
}
class WriteStream extends EE {
constructor (path, opt) {
opt = opt || {}
super(opt)
this.readable = false
this.writable = true
this[_errored] = false
this[_writing] = false
this[_ended] = false
this[_needDrain] = false
this[_queue] = []
this[_path] = path
this[_fd] = typeof opt.fd === 'number' ? opt.fd : null
this[_mode] = opt.mode === undefined ? 0o666 : opt.mode
this[_pos] = typeof opt.start === 'number' ? opt.start : null
this[_autoClose] = typeof opt.autoClose === 'boolean' ?
opt.autoClose : true
// truncating makes no sense when writing into the middle
const defaultFlag = this[_pos] !== null ? 'r+' : 'w'
this[_defaultFlag] = opt.flags === undefined
this[_flags] = this[_defaultFlag] ? defaultFlag : opt.flags
if (this[_fd] === null) {
this[_open]()
}
}
emit (ev, data) {
if (ev === 'error') {
if (this[_errored]) {
return
}
this[_errored] = true
}
return super.emit(ev, data)
}
get fd () {
return this[_fd]
}
get path () {
return this[_path]
}
[_onerror] (er) {
this[_close]()
this[_writing] = true
this.emit('error', er)
}
[_open] () {
fs.open(this[_path], this[_flags], this[_mode],
(er, fd) => this[_onopen](er, fd))
}
[_onopen] (er, fd) {
if (this[_defaultFlag] &&
this[_flags] === 'r+' &&
er && er.code === 'ENOENT') {
this[_flags] = 'w'
this[_open]()
} else if (er) {
this[_onerror](er)
} else {
this[_fd] = fd
this.emit('open', fd)
if (!this[_writing]) {
this[_flush]()
}
}
}
end (buf, enc) {
if (buf) {
this.write(buf, enc)
}
this[_ended] = true
// synthetic after-write logic, where drain/finish live
if (!this[_writing] && !this[_queue].length &&
typeof this[_fd] === 'number') {
this[_onwrite](null, 0)
}
return this
}
write (buf, enc) {
if (typeof buf === 'string') {
buf = Buffer.from(buf, enc)
}
if (this[_ended]) {
this.emit('error', new Error('write() after end()'))
return false
}
if (this[_fd] === null || this[_writing] || this[_queue].length) {
this[_queue].push(buf)
this[_needDrain] = true
return false
}
this[_writing] = true
this[_write](buf)
return true
}
[_write] (buf) {
fs.write(this[_fd], buf, 0, buf.length, this[_pos], (er, bw) =>
this[_onwrite](er, bw))
}
[_onwrite] (er, bw) {
if (er) {
this[_onerror](er)
} else {
if (this[_pos] !== null) {
this[_pos] += bw
}
if (this[_queue].length) {
this[_flush]()
} else {
this[_writing] = false
if (this[_ended] && !this[_finished]) {
this[_finished] = true
this[_close]()
this.emit('finish')
} else if (this[_needDrain]) {
this[_needDrain] = false
this.emit('drain')
}
}
}
}
[_flush] () {
if (this[_queue].length === 0) {
if (this[_ended]) {
this[_onwrite](null, 0)
}
} else if (this[_queue].length === 1) {
this[_write](this[_queue].pop())
} else {
const iovec = this[_queue]
this[_queue] = []
writev(this[_fd], iovec, this[_pos],
(er, bw) => this[_onwrite](er, bw))
}
}
[_close] () {
if (this[_autoClose] && typeof this[_fd] === 'number') {
const fd = this[_fd]
this[_fd] = null
fs.close(fd, er => er ? this.emit('error', er) : this.emit('close'))
}
}
}
class WriteStreamSync extends WriteStream {
[_open] () {
let fd
// only wrap in a try{} block if we know we'll retry, to avoid
// the rethrow obscuring the error's source frame in most cases.
if (this[_defaultFlag] && this[_flags] === 'r+') {
try {
fd = fs.openSync(this[_path], this[_flags], this[_mode])
} catch (er) {
if (er.code === 'ENOENT') {
this[_flags] = 'w'
return this[_open]()
} else {
throw er
}
}
} else {
fd = fs.openSync(this[_path], this[_flags], this[_mode])
}
this[_onopen](null, fd)
}
[_close] () {
if (this[_autoClose] && typeof this[_fd] === 'number') {
const fd = this[_fd]
this[_fd] = null
fs.closeSync(fd)
this.emit('close')
}
}
[_write] (buf) {
// throw the original, but try to close if it fails
let threw = true
try {
this[_onwrite](null,
fs.writeSync(this[_fd], buf, 0, buf.length, this[_pos]))
threw = false
} finally {
if (threw) {
try {
this[_close]()
} catch {
// ok error
}
}
}
}
}
exports.ReadStream = ReadStream
exports.ReadStreamSync = ReadStreamSync
exports.WriteStream = WriteStream
exports.WriteStreamSync = WriteStreamSync

View File

@@ -0,0 +1,15 @@
The ISC License
Copyright (c) 2017-2023 npm, Inc., Isaac Z. Schlueter, and Contributors
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

View File

@@ -0,0 +1,825 @@
# minipass
A _very_ minimal implementation of a [PassThrough
stream](https://nodejs.org/api/stream.html#stream_class_stream_passthrough)
[It's very
fast](https://docs.google.com/spreadsheets/d/1K_HR5oh3r80b8WVMWCPPjfuWXUgfkmhlX7FGI6JJ8tY/edit?usp=sharing)
for objects, strings, and buffers.
Supports `pipe()`ing (including multi-`pipe()` and backpressure
transmission), buffering data until either a `data` event handler
or `pipe()` is added (so you don't lose the first chunk), and
most other cases where PassThrough is a good idea.
There is a `read()` method, but it's much more efficient to
consume data from this stream via `'data'` events or by calling
`pipe()` into some other stream. Calling `read()` requires the
buffer to be flattened in some cases, which requires copying
memory.
If you set `objectMode: true` in the options, then whatever is
written will be emitted. Otherwise, it'll do a minimal amount of
Buffer copying to ensure proper Streams semantics when `read(n)`
is called.
`objectMode` can only be set at instantiation. Attempting to
write something other than a String or Buffer without having set
`objectMode` in the options will throw an error.
This is not a `through` or `through2` stream. It doesn't
transform the data, it just passes it right through. If you want
to transform the data, extend the class, and override the
`write()` method. Once you're done transforming the data however
you want, call `super.write()` with the transform output.
For some examples of streams that extend Minipass in various
ways, check out:
- [minizlib](http://npm.im/minizlib)
- [fs-minipass](http://npm.im/fs-minipass)
- [tar](http://npm.im/tar)
- [minipass-collect](http://npm.im/minipass-collect)
- [minipass-flush](http://npm.im/minipass-flush)
- [minipass-pipeline](http://npm.im/minipass-pipeline)
- [tap](http://npm.im/tap)
- [tap-parser](http://npm.im/tap-parser)
- [treport](http://npm.im/treport)
- [minipass-fetch](http://npm.im/minipass-fetch)
- [pacote](http://npm.im/pacote)
- [make-fetch-happen](http://npm.im/make-fetch-happen)
- [cacache](http://npm.im/cacache)
- [ssri](http://npm.im/ssri)
- [npm-registry-fetch](http://npm.im/npm-registry-fetch)
- [minipass-json-stream](http://npm.im/minipass-json-stream)
- [minipass-sized](http://npm.im/minipass-sized)
## Usage in TypeScript
The `Minipass` class takes three type template definitions:
- `RType` the type being read, which defaults to `Buffer`. If
`RType` is `string`, then the constructor _must_ get an options
object specifying either an `encoding` or `objectMode: true`.
If it's anything other than `string` or `Buffer`, then it
_must_ get an options object specifying `objectMode: true`.
- `WType` the type being written. If `RType` is `Buffer` or
`string`, then this defaults to `ContiguousData` (Buffer,
string, ArrayBuffer, or ArrayBufferView). Otherwise, it
defaults to `RType`.
- `Events` type mapping event names to the arguments emitted
with that event, which extends `Minipass.Events`.
To declare types for custom events in subclasses, extend the
third parameter with your own event signatures. For example:
```js
import { Minipass } from 'minipass'
// a NDJSON stream that emits 'jsonError' when it can't stringify
export interface Events extends Minipass.Events {
jsonError: [e: Error]
}
export class NDJSONStream extends Minipass<string, any, Events> {
constructor() {
super({ objectMode: true })
}
// data is type `any` because that's WType
write(data, encoding, cb) {
try {
const json = JSON.stringify(data)
return super.write(json + '\n', encoding, cb)
} catch (er) {
if (!er instanceof Error) {
er = Object.assign(new Error('json stringify failed'), {
cause: er,
})
}
// trying to emit with something OTHER than an error will
// fail, because we declared the event arguments type.
this.emit('jsonError', er)
}
}
}
const s = new NDJSONStream()
s.on('jsonError', e => {
// here, TS knows that e is an Error
})
```
Emitting/handling events that aren't declared in this way is
fine, but the arguments will be typed as `unknown`.
## Differences from Node.js Streams
There are several things that make Minipass streams different
from (and in some ways superior to) Node.js core streams.
Please read these caveats if you are familiar with node-core
streams and intend to use Minipass streams in your programs.
You can avoid most of these differences entirely (for a very
small performance penalty) by setting `{async: true}` in the
constructor options.
### Timing
Minipass streams are designed to support synchronous use-cases.
Thus, data is emitted as soon as it is available, always. It is
buffered until read, but no longer. Another way to look at it is
that Minipass streams are exactly as synchronous as the logic
that writes into them.
This can be surprising if your code relies on
`PassThrough.write()` always providing data on the next tick
rather than the current one, or being able to call `resume()` and
not have the entire buffer disappear immediately.
However, without this synchronicity guarantee, there would be no
way for Minipass to achieve the speeds it does, or support the
synchronous use cases that it does. Simply put, waiting takes
time.
This non-deferring approach makes Minipass streams much easier to
reason about, especially in the context of Promises and other
flow-control mechanisms.
Example:
```js
// hybrid module, either works
import { Minipass } from 'minipass'
// or:
const { Minipass } = require('minipass')
const stream = new Minipass()
stream.on('data', () => console.log('data event'))
console.log('before write')
stream.write('hello')
console.log('after write')
// output:
// before write
// data event
// after write
```
### Exception: Async Opt-In
If you wish to have a Minipass stream with behavior that more
closely mimics Node.js core streams, you can set the stream in
async mode either by setting `async: true` in the constructor
options, or by setting `stream.async = true` later on.
```js
// hybrid module, either works
import { Minipass } from 'minipass'
// or:
const { Minipass } = require('minipass')
const asyncStream = new Minipass({ async: true })
asyncStream.on('data', () => console.log('data event'))
console.log('before write')
asyncStream.write('hello')
console.log('after write')
// output:
// before write
// after write
// data event <-- this is deferred until the next tick
```
Switching _out_ of async mode is unsafe, as it could cause data
corruption, and so is not enabled. Example:
```js
import { Minipass } from 'minipass'
const stream = new Minipass({ encoding: 'utf8' })
stream.on('data', chunk => console.log(chunk))
stream.async = true
console.log('before writes')
stream.write('hello')
setStreamSyncAgainSomehow(stream) // <-- this doesn't actually exist!
stream.write('world')
console.log('after writes')
// hypothetical output would be:
// before writes
// world
// after writes
// hello
// NOT GOOD!
```
To avoid this problem, once set into async mode, any attempt to
make the stream sync again will be ignored.
```js
const { Minipass } = require('minipass')
const stream = new Minipass({ encoding: 'utf8' })
stream.on('data', chunk => console.log(chunk))
stream.async = true
console.log('before writes')
stream.write('hello')
stream.async = false // <-- no-op, stream already async
stream.write('world')
console.log('after writes')
// actual output:
// before writes
// after writes
// hello
// world
```
### No High/Low Water Marks
Node.js core streams will optimistically fill up a buffer,
returning `true` on all writes until the limit is hit, even if
the data has nowhere to go. Then, they will not attempt to draw
more data in until the buffer size dips below a minimum value.
Minipass streams are much simpler. The `write()` method will
return `true` if the data has somewhere to go (which is to say,
given the timing guarantees, that the data is already there by
the time `write()` returns).
If the data has nowhere to go, then `write()` returns false, and
the data sits in a buffer, to be drained out immediately as soon
as anyone consumes it.
Since nothing is ever buffered unnecessarily, there is much less
copying data, and less bookkeeping about buffer capacity levels.
### Hazards of Buffering (or: Why Minipass Is So Fast)
Since data written to a Minipass stream is immediately written
all the way through the pipeline, and `write()` always returns
true/false based on whether the data was fully flushed,
backpressure is communicated immediately to the upstream caller.
This minimizes buffering.
Consider this case:
```js
const { PassThrough } = require('stream')
const p1 = new PassThrough({ highWaterMark: 1024 })
const p2 = new PassThrough({ highWaterMark: 1024 })
const p3 = new PassThrough({ highWaterMark: 1024 })
const p4 = new PassThrough({ highWaterMark: 1024 })
p1.pipe(p2).pipe(p3).pipe(p4)
p4.on('data', () => console.log('made it through'))
// this returns false and buffers, then writes to p2 on next tick (1)
// p2 returns false and buffers, pausing p1, then writes to p3 on next tick (2)
// p3 returns false and buffers, pausing p2, then writes to p4 on next tick (3)
// p4 returns false and buffers, pausing p3, then emits 'data' and 'drain'
// on next tick (4)
// p3 sees p4's 'drain' event, and calls resume(), emitting 'resume' and
// 'drain' on next tick (5)
// p2 sees p3's 'drain', calls resume(), emits 'resume' and 'drain' on next tick (6)
// p1 sees p2's 'drain', calls resume(), emits 'resume' and 'drain' on next
// tick (7)
p1.write(Buffer.alloc(2048)) // returns false
```
Along the way, the data was buffered and deferred at each stage,
and multiple event deferrals happened, for an unblocked pipeline
where it was perfectly safe to write all the way through!
Furthermore, setting a `highWaterMark` of `1024` might lead
someone reading the code to think an advisory maximum of 1KiB is
being set for the pipeline. However, the actual advisory
buffering level is the _sum_ of `highWaterMark` values, since
each one has its own bucket.
Consider the Minipass case:
```js
const m1 = new Minipass()
const m2 = new Minipass()
const m3 = new Minipass()
const m4 = new Minipass()
m1.pipe(m2).pipe(m3).pipe(m4)
m4.on('data', () => console.log('made it through'))
// m1 is flowing, so it writes the data to m2 immediately
// m2 is flowing, so it writes the data to m3 immediately
// m3 is flowing, so it writes the data to m4 immediately
// m4 is flowing, so it fires the 'data' event immediately, returns true
// m4's write returned true, so m3 is still flowing, returns true
// m3's write returned true, so m2 is still flowing, returns true
// m2's write returned true, so m1 is still flowing, returns true
// No event deferrals or buffering along the way!
m1.write(Buffer.alloc(2048)) // returns true
```
It is extremely unlikely that you _don't_ want to buffer any data
written, or _ever_ buffer data that can be flushed all the way
through. Neither node-core streams nor Minipass ever fail to
buffer written data, but node-core streams do a lot of
unnecessary buffering and pausing.
As always, the faster implementation is the one that does less
stuff and waits less time to do it.
### Immediately emit `end` for empty streams (when not paused)
If a stream is not paused, and `end()` is called before writing
any data into it, then it will emit `end` immediately.
If you have logic that occurs on the `end` event which you don't
want to potentially happen immediately (for example, closing file
descriptors, moving on to the next entry in an archive parse
stream, etc.) then be sure to call `stream.pause()` on creation,
and then `stream.resume()` once you are ready to respond to the
`end` event.
However, this is _usually_ not a problem because:
### Emit `end` When Asked
One hazard of immediately emitting `'end'` is that you may not
yet have had a chance to add a listener. In order to avoid this
hazard, Minipass streams safely re-emit the `'end'` event if a
new listener is added after `'end'` has been emitted.
Ie, if you do `stream.on('end', someFunction)`, and the stream
has already emitted `end`, then it will call the handler right
away. (You can think of this somewhat like attaching a new
`.then(fn)` to a previously-resolved Promise.)
To prevent calling handlers multiple times who would not expect
multiple ends to occur, all listeners are removed from the
`'end'` event whenever it is emitted.
### Emit `error` When Asked
The most recent error object passed to the `'error'` event is
stored on the stream. If a new `'error'` event handler is added,
and an error was previously emitted, then the event handler will
be called immediately (or on `process.nextTick` in the case of
async streams).
This makes it much more difficult to end up trying to interact
with a broken stream, if the error handler is added after an
error was previously emitted.
### Impact of "immediate flow" on Tee-streams
A "tee stream" is a stream piping to multiple destinations:
```js
const tee = new Minipass()
t.pipe(dest1)
t.pipe(dest2)
t.write('foo') // goes to both destinations
```
Since Minipass streams _immediately_ process any pending data
through the pipeline when a new pipe destination is added, this
can have surprising effects, especially when a stream comes in
from some other function and may or may not have data in its
buffer.
```js
// WARNING! WILL LOSE DATA!
const src = new Minipass()
src.write('foo')
src.pipe(dest1) // 'foo' chunk flows to dest1 immediately, and is gone
src.pipe(dest2) // gets nothing!
```
One solution is to create a dedicated tee-stream junction that
pipes to both locations, and then pipe to _that_ instead.
```js
// Safe example: tee to both places
const src = new Minipass()
src.write('foo')
const tee = new Minipass()
tee.pipe(dest1)
tee.pipe(dest2)
src.pipe(tee) // tee gets 'foo', pipes to both locations
```
The same caveat applies to `on('data')` event listeners. The
first one added will _immediately_ receive all of the data,
leaving nothing for the second:
```js
// WARNING! WILL LOSE DATA!
const src = new Minipass()
src.write('foo')
src.on('data', handler1) // receives 'foo' right away
src.on('data', handler2) // nothing to see here!
```
Using a dedicated tee-stream can be used in this case as well:
```js
// Safe example: tee to both data handlers
const src = new Minipass()
src.write('foo')
const tee = new Minipass()
tee.on('data', handler1)
tee.on('data', handler2)
src.pipe(tee)
```
All of the hazards in this section are avoided by setting `{
async: true }` in the Minipass constructor, or by setting
`stream.async = true` afterwards. Note that this does add some
overhead, so should only be done in cases where you are willing
to lose a bit of performance in order to avoid having to refactor
program logic.
## USAGE
It's a stream! Use it like a stream and it'll most likely do what
you want.
```js
import { Minipass } from 'minipass'
const mp = new Minipass(options) // options is optional
mp.write('foo')
mp.pipe(someOtherStream)
mp.end('bar')
```
### OPTIONS
- `encoding` How would you like the data coming _out_ of the
stream to be encoded? Accepts any values that can be passed to
`Buffer.toString()`.
- `objectMode` Emit data exactly as it comes in. This will be
flipped on by default if you write() something other than a
string or Buffer at any point. Setting `objectMode: true` will
prevent setting any encoding value.
- `async` Defaults to `false`. Set to `true` to defer data
emission until next tick. This reduces performance slightly,
but makes Minipass streams use timing behavior closer to Node
core streams. See [Timing](#timing) for more details.
- `signal` An `AbortSignal` that will cause the stream to unhook
itself from everything and become as inert as possible. Note
that providing a `signal` parameter will make `'error'` events
no longer throw if they are unhandled, but they will still be
emitted to handlers if any are attached.
### API
Implements the user-facing portions of Node.js's `Readable` and
`Writable` streams.
### Methods
- `write(chunk, [encoding], [callback])` - Put data in. (Note
that, in the base Minipass class, the same data will come out.)
Returns `false` if the stream will buffer the next write, or
true if it's still in "flowing" mode.
- `end([chunk, [encoding]], [callback])` - Signal that you have
no more data to write. This will queue an `end` event to be
fired when all the data has been consumed.
- `pause()` - No more data for a while, please. This also
prevents `end` from being emitted for empty streams until the
stream is resumed.
- `resume()` - Resume the stream. If there's data in the buffer,
it is all discarded. Any buffered events are immediately
emitted.
- `pipe(dest)` - Send all output to the stream provided. When
data is emitted, it is immediately written to any and all pipe
destinations. (Or written on next tick in `async` mode.)
- `unpipe(dest)` - Stop piping to the destination stream. This is
immediate, meaning that any asynchronously queued data will
_not_ make it to the destination when running in `async` mode.
- `options.end` - Boolean, end the destination stream when the
source stream ends. Default `true`.
- `options.proxyErrors` - Boolean, proxy `error` events from
the source stream to the destination stream. Note that errors
are _not_ proxied after the pipeline terminates, either due
to the source emitting `'end'` or manually unpiping with
`src.unpipe(dest)`. Default `false`.
- `on(ev, fn)`, `emit(ev, fn)` - Minipass streams are
EventEmitters. Some events are given special treatment,
however. (See below under "events".)
- `promise()` - Returns a Promise that resolves when the stream
emits `end`, or rejects if the stream emits `error`.
- `collect()` - Return a Promise that resolves on `end` with an
array containing each chunk of data that was emitted, or
rejects if the stream emits `error`. Note that this consumes
the stream data.
- `concat()` - Same as `collect()`, but concatenates the data
into a single Buffer object. Will reject the returned promise
if the stream is in objectMode, or if it goes into objectMode
by the end of the data.
- `read(n)` - Consume `n` bytes of data out of the buffer. If `n`
is not provided, then consume all of it. If `n` bytes are not
available, then it returns null. **Note** consuming streams in
this way is less efficient, and can lead to unnecessary Buffer
copying.
- `destroy([er])` - Destroy the stream. If an error is provided,
then an `'error'` event is emitted. If the stream has a
`close()` method, and has not emitted a `'close'` event yet,
then `stream.close()` will be called. Any Promises returned by
`.promise()`, `.collect()` or `.concat()` will be rejected.
After being destroyed, writing to the stream will emit an
error. No more data will be emitted if the stream is destroyed,
even if it was previously buffered.
### Properties
- `bufferLength` Read-only. Total number of bytes buffered, or in
the case of objectMode, the total number of objects.
- `encoding` Read-only. The encoding that has been set.
- `flowing` Read-only. Boolean indicating whether a chunk written
to the stream will be immediately emitted.
- `emittedEnd` Read-only. Boolean indicating whether the end-ish
events (ie, `end`, `prefinish`, `finish`) have been emitted.
Note that listening on any end-ish event will immediateyl
re-emit it if it has already been emitted.
- `writable` Whether the stream is writable. Default `true`. Set
to `false` when `end()`
- `readable` Whether the stream is readable. Default `true`.
- `pipes` An array of Pipe objects referencing streams that this
stream is piping into.
- `destroyed` A getter that indicates whether the stream was
destroyed.
- `paused` True if the stream has been explicitly paused,
otherwise false.
- `objectMode` Indicates whether the stream is in `objectMode`.
- `aborted` Readonly property set when the `AbortSignal`
dispatches an `abort` event.
### Events
- `data` Emitted when there's data to read. Argument is the data
to read. This is never emitted while not flowing. If a listener
is attached, that will resume the stream.
- `end` Emitted when there's no more data to read. This will be
emitted immediately for empty streams when `end()` is called.
If a listener is attached, and `end` was already emitted, then
it will be emitted again. All listeners are removed when `end`
is emitted.
- `prefinish` An end-ish event that follows the same logic as
`end` and is emitted in the same conditions where `end` is
emitted. Emitted after `'end'`.
- `finish` An end-ish event that follows the same logic as `end`
and is emitted in the same conditions where `end` is emitted.
Emitted after `'prefinish'`.
- `close` An indication that an underlying resource has been
released. Minipass does not emit this event, but will defer it
until after `end` has been emitted, since it throws off some
stream libraries otherwise.
- `drain` Emitted when the internal buffer empties, and it is
again suitable to `write()` into the stream.
- `readable` Emitted when data is buffered and ready to be read
by a consumer.
- `resume` Emitted when stream changes state from buffering to
flowing mode. (Ie, when `resume` is called, `pipe` is called,
or a `data` event listener is added.)
### Static Methods
- `Minipass.isStream(stream)` Returns `true` if the argument is a
stream, and false otherwise. To be considered a stream, the
object must be either an instance of Minipass, or an
EventEmitter that has either a `pipe()` method, or both
`write()` and `end()` methods. (Pretty much any stream in
node-land will return `true` for this.)
## EXAMPLES
Here are some examples of things you can do with Minipass
streams.
### simple "are you done yet" promise
```js
mp.promise().then(
() => {
// stream is finished
},
er => {
// stream emitted an error
}
)
```
### collecting
```js
mp.collect().then(all => {
// all is an array of all the data emitted
// encoding is supported in this case, so
// so the result will be a collection of strings if
// an encoding is specified, or buffers/objects if not.
//
// In an async function, you may do
// const data = await stream.collect()
})
```
### collecting into a single blob
This is a bit slower because it concatenates the data into one
chunk for you, but if you're going to do it yourself anyway, it's
convenient this way:
```js
mp.concat().then(onebigchunk => {
// onebigchunk is a string if the stream
// had an encoding set, or a buffer otherwise.
})
```
### iteration
You can iterate over streams synchronously or asynchronously in
platforms that support it.
Synchronous iteration will end when the currently available data
is consumed, even if the `end` event has not been reached. In
string and buffer mode, the data is concatenated, so unless
multiple writes are occurring in the same tick as the `read()`,
sync iteration loops will generally only have a single iteration.
To consume chunks in this way exactly as they have been written,
with no flattening, create the stream with the `{ objectMode:
true }` option.
```js
const mp = new Minipass({ objectMode: true })
mp.write('a')
mp.write('b')
for (let letter of mp) {
console.log(letter) // a, b
}
mp.write('c')
mp.write('d')
for (let letter of mp) {
console.log(letter) // c, d
}
mp.write('e')
mp.end()
for (let letter of mp) {
console.log(letter) // e
}
for (let letter of mp) {
console.log(letter) // nothing
}
```
Asynchronous iteration will continue until the end event is reached,
consuming all of the data.
```js
const mp = new Minipass({ encoding: 'utf8' })
// some source of some data
let i = 5
const inter = setInterval(() => {
if (i-- > 0) mp.write(Buffer.from('foo\n', 'utf8'))
else {
mp.end()
clearInterval(inter)
}
}, 100)
// consume the data with asynchronous iteration
async function consume() {
for await (let chunk of mp) {
console.log(chunk)
}
return 'ok'
}
consume().then(res => console.log(res))
// logs `foo\n` 5 times, and then `ok`
```
### subclass that `console.log()`s everything written into it
```js
class Logger extends Minipass {
write(chunk, encoding, callback) {
console.log('WRITE', chunk, encoding)
return super.write(chunk, encoding, callback)
}
end(chunk, encoding, callback) {
console.log('END', chunk, encoding)
return super.end(chunk, encoding, callback)
}
}
someSource.pipe(new Logger()).pipe(someDest)
```
### same thing, but using an inline anonymous class
```js
// js classes are fun
someSource
.pipe(
new (class extends Minipass {
emit(ev, ...data) {
// let's also log events, because debugging some weird thing
console.log('EMIT', ev)
return super.emit(ev, ...data)
}
write(chunk, encoding, callback) {
console.log('WRITE', chunk, encoding)
return super.write(chunk, encoding, callback)
}
end(chunk, encoding, callback) {
console.log('END', chunk, encoding)
return super.end(chunk, encoding, callback)
}
})()
)
.pipe(someDest)
```
### subclass that defers 'end' for some reason
```js
class SlowEnd extends Minipass {
emit(ev, ...args) {
if (ev === 'end') {
console.log('going to end, hold on a sec')
setTimeout(() => {
console.log('ok, ready to end now')
super.emit('end', ...args)
}, 100)
return true
} else {
return super.emit(ev, ...args)
}
}
}
```
### transform that creates newline-delimited JSON
```js
class NDJSONEncode extends Minipass {
write(obj, cb) {
try {
// JSON.stringify can throw, emit an error on that
return super.write(JSON.stringify(obj) + '\n', 'utf8', cb)
} catch (er) {
this.emit('error', er)
}
}
end(obj, cb) {
if (typeof obj === 'function') {
cb = obj
obj = undefined
}
if (obj !== undefined) {
this.write(obj)
}
return super.end(cb)
}
}
```
### transform that parses newline-delimited JSON
```js
class NDJSONDecode extends Minipass {
constructor(options) {
// always be in object mode, as far as Minipass is concerned
super({ objectMode: true })
this._jsonBuffer = ''
}
write(chunk, encoding, cb) {
if (
typeof chunk === 'string' &&
typeof encoding === 'string' &&
encoding !== 'utf8'
) {
chunk = Buffer.from(chunk, encoding).toString()
} else if (Buffer.isBuffer(chunk)) {
chunk = chunk.toString()
}
if (typeof encoding === 'function') {
cb = encoding
}
const jsonData = (this._jsonBuffer + chunk).split('\n')
this._jsonBuffer = jsonData.pop()
for (let i = 0; i < jsonData.length; i++) {
try {
// JSON.parse can throw, emit an error on that
super.write(JSON.parse(jsonData[i]))
} catch (er) {
this.emit('error', er)
continue
}
}
if (cb) cb()
}
}
```

View File

@@ -0,0 +1,549 @@
/// <reference types="node" />
/// <reference types="node" />
/// <reference types="node" />
/// <reference types="node" />
import { EventEmitter } from 'node:events';
import { StringDecoder } from 'node:string_decoder';
/**
* Same as StringDecoder, but exposing the `lastNeed` flag on the type
*/
type SD = StringDecoder & {
lastNeed: boolean;
};
export type { SD, Pipe, PipeProxyErrors };
/**
* Return true if the argument is a Minipass stream, Node stream, or something
* else that Minipass can interact with.
*/
export declare const isStream: (s: any) => s is NodeJS.WriteStream | NodeJS.ReadStream | Minipass<any, any, any> | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
}) | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
/**
* Return true if the argument is a valid {@link Minipass.Readable}
*/
export declare const isReadable: (s: any) => s is Minipass.Readable;
/**
* Return true if the argument is a valid {@link Minipass.Writable}
*/
export declare const isWritable: (s: any) => s is Minipass.Readable;
declare const EOF: unique symbol;
declare const MAYBE_EMIT_END: unique symbol;
declare const EMITTED_END: unique symbol;
declare const EMITTING_END: unique symbol;
declare const EMITTED_ERROR: unique symbol;
declare const CLOSED: unique symbol;
declare const READ: unique symbol;
declare const FLUSH: unique symbol;
declare const FLUSHCHUNK: unique symbol;
declare const ENCODING: unique symbol;
declare const DECODER: unique symbol;
declare const FLOWING: unique symbol;
declare const PAUSED: unique symbol;
declare const RESUME: unique symbol;
declare const BUFFER: unique symbol;
declare const PIPES: unique symbol;
declare const BUFFERLENGTH: unique symbol;
declare const BUFFERPUSH: unique symbol;
declare const BUFFERSHIFT: unique symbol;
declare const OBJECTMODE: unique symbol;
declare const DESTROYED: unique symbol;
declare const ERROR: unique symbol;
declare const EMITDATA: unique symbol;
declare const EMITEND: unique symbol;
declare const EMITEND2: unique symbol;
declare const ASYNC: unique symbol;
declare const ABORT: unique symbol;
declare const ABORTED: unique symbol;
declare const SIGNAL: unique symbol;
declare const DATALISTENERS: unique symbol;
declare const DISCARDED: unique symbol;
/**
* Options that may be passed to stream.pipe()
*/
export interface PipeOptions {
/**
* end the destination stream when the source stream ends
*/
end?: boolean;
/**
* proxy errors from the source stream to the destination stream
*/
proxyErrors?: boolean;
}
/**
* Internal class representing a pipe to a destination stream.
*
* @internal
*/
declare class Pipe<T extends unknown> {
src: Minipass<T>;
dest: Minipass<any, T>;
opts: PipeOptions;
ondrain: () => any;
constructor(src: Minipass<T>, dest: Minipass.Writable, opts: PipeOptions);
unpipe(): void;
proxyErrors(_er: any): void;
end(): void;
}
/**
* Internal class representing a pipe to a destination stream where
* errors are proxied.
*
* @internal
*/
declare class PipeProxyErrors<T> extends Pipe<T> {
unpipe(): void;
constructor(src: Minipass<T>, dest: Minipass.Writable, opts: PipeOptions);
}
export declare namespace Minipass {
/**
* Encoding used to create a stream that outputs strings rather than
* Buffer objects.
*/
export type Encoding = BufferEncoding | 'buffer' | null;
/**
* Any stream that Minipass can pipe into
*/
export type Writable = Minipass<any, any, any> | NodeJS.WriteStream | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
/**
* Any stream that can be read from
*/
export type Readable = Minipass<any, any, any> | NodeJS.ReadStream | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
});
/**
* Utility type that can be iterated sync or async
*/
export type DualIterable<T> = Iterable<T> & AsyncIterable<T>;
type EventArguments = Record<string | symbol, unknown[]>;
/**
* The listing of events that a Minipass class can emit.
* Extend this when extending the Minipass class, and pass as
* the third template argument. The key is the name of the event,
* and the value is the argument list.
*
* Any undeclared events will still be allowed, but the handler will get
* arguments as `unknown[]`.
*/
export interface Events<RType extends any = Buffer> extends EventArguments {
readable: [];
data: [chunk: RType];
error: [er: unknown];
abort: [reason: unknown];
drain: [];
resume: [];
end: [];
finish: [];
prefinish: [];
close: [];
[DESTROYED]: [er?: unknown];
[ERROR]: [er: unknown];
}
/**
* String or buffer-like data that can be joined and sliced
*/
export type ContiguousData = Buffer | ArrayBufferLike | ArrayBufferView | string;
export type BufferOrString = Buffer | string;
/**
* Options passed to the Minipass constructor.
*/
export type SharedOptions = {
/**
* Defer all data emission and other events until the end of the
* current tick, similar to Node core streams
*/
async?: boolean;
/**
* A signal which will abort the stream
*/
signal?: AbortSignal;
/**
* Output string encoding. Set to `null` or `'buffer'` (or omit) to
* emit Buffer objects rather than strings.
*
* Conflicts with `objectMode`
*/
encoding?: BufferEncoding | null | 'buffer';
/**
* Output data exactly as it was written, supporting non-buffer/string
* data (such as arbitrary objects, falsey values, etc.)
*
* Conflicts with `encoding`
*/
objectMode?: boolean;
};
/**
* Options for a string encoded output
*/
export type EncodingOptions = SharedOptions & {
encoding: BufferEncoding;
objectMode?: false;
};
/**
* Options for contiguous data buffer output
*/
export type BufferOptions = SharedOptions & {
encoding?: null | 'buffer';
objectMode?: false;
};
/**
* Options for objectMode arbitrary output
*/
export type ObjectModeOptions = SharedOptions & {
objectMode: true;
encoding?: null;
};
/**
* Utility type to determine allowed options based on read type
*/
export type Options<T> = ObjectModeOptions | (T extends string ? EncodingOptions : T extends Buffer ? BufferOptions : SharedOptions);
export {};
}
/**
* Main export, the Minipass class
*
* `RType` is the type of data emitted, defaults to Buffer
*
* `WType` is the type of data to be written, if RType is buffer or string,
* then any {@link Minipass.ContiguousData} is allowed.
*
* `Events` is the set of event handler signatures that this object
* will emit, see {@link Minipass.Events}
*/
export declare class Minipass<RType extends unknown = Buffer, WType extends unknown = RType extends Minipass.BufferOrString ? Minipass.ContiguousData : RType, Events extends Minipass.Events<RType> = Minipass.Events<RType>> extends EventEmitter implements Minipass.DualIterable<RType> {
[FLOWING]: boolean;
[PAUSED]: boolean;
[PIPES]: Pipe<RType>[];
[BUFFER]: RType[];
[OBJECTMODE]: boolean;
[ENCODING]: BufferEncoding | null;
[ASYNC]: boolean;
[DECODER]: SD | null;
[EOF]: boolean;
[EMITTED_END]: boolean;
[EMITTING_END]: boolean;
[CLOSED]: boolean;
[EMITTED_ERROR]: unknown;
[BUFFERLENGTH]: number;
[DESTROYED]: boolean;
[SIGNAL]?: AbortSignal;
[ABORTED]: boolean;
[DATALISTENERS]: number;
[DISCARDED]: boolean;
/**
* true if the stream can be written
*/
writable: boolean;
/**
* true if the stream can be read
*/
readable: boolean;
/**
* If `RType` is Buffer, then options do not need to be provided.
* Otherwise, an options object must be provided to specify either
* {@link Minipass.SharedOptions.objectMode} or
* {@link Minipass.SharedOptions.encoding}, as appropriate.
*/
constructor(...args: [Minipass.ObjectModeOptions] | (RType extends Buffer ? [] | [Minipass.Options<RType>] : [Minipass.Options<RType>]));
/**
* The amount of data stored in the buffer waiting to be read.
*
* For Buffer strings, this will be the total byte length.
* For string encoding streams, this will be the string character length,
* according to JavaScript's `string.length` logic.
* For objectMode streams, this is a count of the items waiting to be
* emitted.
*/
get bufferLength(): number;
/**
* The `BufferEncoding` currently in use, or `null`
*/
get encoding(): BufferEncoding | null;
/**
* @deprecated - This is a read only property
*/
set encoding(_enc: BufferEncoding | null);
/**
* @deprecated - Encoding may only be set at instantiation time
*/
setEncoding(_enc: Minipass.Encoding): void;
/**
* True if this is an objectMode stream
*/
get objectMode(): boolean;
/**
* @deprecated - This is a read-only property
*/
set objectMode(_om: boolean);
/**
* true if this is an async stream
*/
get ['async'](): boolean;
/**
* Set to true to make this stream async.
*
* Once set, it cannot be unset, as this would potentially cause incorrect
* behavior. Ie, a sync stream can be made async, but an async stream
* cannot be safely made sync.
*/
set ['async'](a: boolean);
[ABORT](): void;
/**
* True if the stream has been aborted.
*/
get aborted(): boolean;
/**
* No-op setter. Stream aborted status is set via the AbortSignal provided
* in the constructor options.
*/
set aborted(_: boolean);
/**
* Write data into the stream
*
* If the chunk written is a string, and encoding is not specified, then
* `utf8` will be assumed. If the stream encoding matches the encoding of
* a written string, and the state of the string decoder allows it, then
* the string will be passed through to either the output or the internal
* buffer without any processing. Otherwise, it will be turned into a
* Buffer object for processing into the desired encoding.
*
* If provided, `cb` function is called immediately before return for
* sync streams, or on next tick for async streams, because for this
* base class, a chunk is considered "processed" once it is accepted
* and either emitted or buffered. That is, the callback does not indicate
* that the chunk has been eventually emitted, though of course child
* classes can override this function to do whatever processing is required
* and call `super.write(...)` only once processing is completed.
*/
write(chunk: WType, cb?: () => void): boolean;
write(chunk: WType, encoding?: Minipass.Encoding, cb?: () => void): boolean;
/**
* Low-level explicit read method.
*
* In objectMode, the argument is ignored, and one item is returned if
* available.
*
* `n` is the number of bytes (or in the case of encoding streams,
* characters) to consume. If `n` is not provided, then the entire buffer
* is returned, or `null` is returned if no data is available.
*
* If `n` is greater that the amount of data in the internal buffer,
* then `null` is returned.
*/
read(n?: number | null): RType | null;
[READ](n: number | null, chunk: RType): RType;
/**
* End the stream, optionally providing a final write.
*
* See {@link Minipass#write} for argument descriptions
*/
end(cb?: () => void): this;
end(chunk: WType, cb?: () => void): this;
end(chunk: WType, encoding?: Minipass.Encoding, cb?: () => void): this;
[RESUME](): void;
/**
* Resume the stream if it is currently in a paused state
*
* If called when there are no pipe destinations or `data` event listeners,
* this will place the stream in a "discarded" state, where all data will
* be thrown away. The discarded state is removed if a pipe destination or
* data handler is added, if pause() is called, or if any synchronous or
* asynchronous iteration is started.
*/
resume(): void;
/**
* Pause the stream
*/
pause(): void;
/**
* true if the stream has been forcibly destroyed
*/
get destroyed(): boolean;
/**
* true if the stream is currently in a flowing state, meaning that
* any writes will be immediately emitted.
*/
get flowing(): boolean;
/**
* true if the stream is currently in a paused state
*/
get paused(): boolean;
[BUFFERPUSH](chunk: RType): void;
[BUFFERSHIFT](): RType;
[FLUSH](noDrain?: boolean): void;
[FLUSHCHUNK](chunk: RType): boolean;
/**
* Pipe all data emitted by this stream into the destination provided.
*
* Triggers the flow of data.
*/
pipe<W extends Minipass.Writable>(dest: W, opts?: PipeOptions): W;
/**
* Fully unhook a piped destination stream.
*
* If the destination stream was the only consumer of this stream (ie,
* there are no other piped destinations or `'data'` event listeners)
* then the flow of data will stop until there is another consumer or
* {@link Minipass#resume} is explicitly called.
*/
unpipe<W extends Minipass.Writable>(dest: W): void;
/**
* Alias for {@link Minipass#on}
*/
addListener<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.on`, with the following
* behavior differences to prevent data loss and unnecessary hangs:
*
* - Adding a 'data' event handler will trigger the flow of data
*
* - Adding a 'readable' event handler when there is data waiting to be read
* will cause 'readable' to be emitted immediately.
*
* - Adding an 'endish' event handler ('end', 'finish', etc.) which has
* already passed will cause the event to be emitted immediately and all
* handlers removed.
*
* - Adding an 'error' event handler after an error has been emitted will
* cause the event to be re-emitted immediately with the error previously
* raised.
*/
on<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Alias for {@link Minipass#off}
*/
removeListener<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.off`
*
* If a 'data' event handler is removed, and it was the last consumer
* (ie, there are no pipe destinations or other 'data' event listeners),
* then the flow of data will stop until there is another consumer or
* {@link Minipass#resume} is explicitly called.
*/
off<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.removeAllListeners`
*
* If all 'data' event handlers are removed, and they were the last consumer
* (ie, there are no pipe destinations), then the flow of data will stop
* until there is another consumer or {@link Minipass#resume} is explicitly
* called.
*/
removeAllListeners<Event extends keyof Events>(ev?: Event): this;
/**
* true if the 'end' event has been emitted
*/
get emittedEnd(): boolean;
[MAYBE_EMIT_END](): void;
/**
* Mostly identical to `EventEmitter.emit`, with the following
* behavior differences to prevent data loss and unnecessary hangs:
*
* If the stream has been destroyed, and the event is something other
* than 'close' or 'error', then `false` is returned and no handlers
* are called.
*
* If the event is 'end', and has already been emitted, then the event
* is ignored. If the stream is in a paused or non-flowing state, then
* the event will be deferred until data flow resumes. If the stream is
* async, then handlers will be called on the next tick rather than
* immediately.
*
* If the event is 'close', and 'end' has not yet been emitted, then
* the event will be deferred until after 'end' is emitted.
*
* If the event is 'error', and an AbortSignal was provided for the stream,
* and there are no listeners, then the event is ignored, matching the
* behavior of node core streams in the presense of an AbortSignal.
*
* If the event is 'finish' or 'prefinish', then all listeners will be
* removed after emitting the event, to prevent double-firing.
*/
emit<Event extends keyof Events>(ev: Event, ...args: Events[Event]): boolean;
[EMITDATA](data: RType): boolean;
[EMITEND](): boolean;
[EMITEND2](): boolean;
/**
* Return a Promise that resolves to an array of all emitted data once
* the stream ends.
*/
collect(): Promise<RType[] & {
dataLength: number;
}>;
/**
* Return a Promise that resolves to the concatenation of all emitted data
* once the stream ends.
*
* Not allowed on objectMode streams.
*/
concat(): Promise<RType>;
/**
* Return a void Promise that resolves once the stream ends.
*/
promise(): Promise<void>;
/**
* Asynchronous `for await of` iteration.
*
* This will continue emitting all chunks until the stream terminates.
*/
[Symbol.asyncIterator](): AsyncGenerator<RType, void, void>;
/**
* Synchronous `for of` iteration.
*
* The iteration will terminate when the internal buffer runs out, even
* if the stream has not yet terminated.
*/
[Symbol.iterator](): Generator<RType, void, void>;
/**
* Destroy a stream, preventing it from being used for any further purpose.
*
* If the stream has a `close()` method, then it will be called on
* destruction.
*
* After destruction, any attempt to write data, read data, or emit most
* events will be ignored.
*
* If an error argument is provided, then it will be emitted in an
* 'error' event.
*/
destroy(er?: unknown): this;
/**
* Alias for {@link isStream}
*
* Former export location, maintained for backwards compatibility.
*
* @deprecated
*/
static get isStream(): (s: any) => s is NodeJS.WriteStream | NodeJS.ReadStream | Minipass<any, any, any> | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
}) | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
}
//# sourceMappingURL=index.d.ts.map

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,3 @@
{
"type": "commonjs"
}

View File

@@ -0,0 +1,549 @@
/// <reference types="node" resolution-mode="require"/>
/// <reference types="node" resolution-mode="require"/>
/// <reference types="node" resolution-mode="require"/>
/// <reference types="node" resolution-mode="require"/>
import { EventEmitter } from 'node:events';
import { StringDecoder } from 'node:string_decoder';
/**
* Same as StringDecoder, but exposing the `lastNeed` flag on the type
*/
type SD = StringDecoder & {
lastNeed: boolean;
};
export type { SD, Pipe, PipeProxyErrors };
/**
* Return true if the argument is a Minipass stream, Node stream, or something
* else that Minipass can interact with.
*/
export declare const isStream: (s: any) => s is NodeJS.WriteStream | NodeJS.ReadStream | Minipass<any, any, any> | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
}) | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
/**
* Return true if the argument is a valid {@link Minipass.Readable}
*/
export declare const isReadable: (s: any) => s is Minipass.Readable;
/**
* Return true if the argument is a valid {@link Minipass.Writable}
*/
export declare const isWritable: (s: any) => s is Minipass.Readable;
declare const EOF: unique symbol;
declare const MAYBE_EMIT_END: unique symbol;
declare const EMITTED_END: unique symbol;
declare const EMITTING_END: unique symbol;
declare const EMITTED_ERROR: unique symbol;
declare const CLOSED: unique symbol;
declare const READ: unique symbol;
declare const FLUSH: unique symbol;
declare const FLUSHCHUNK: unique symbol;
declare const ENCODING: unique symbol;
declare const DECODER: unique symbol;
declare const FLOWING: unique symbol;
declare const PAUSED: unique symbol;
declare const RESUME: unique symbol;
declare const BUFFER: unique symbol;
declare const PIPES: unique symbol;
declare const BUFFERLENGTH: unique symbol;
declare const BUFFERPUSH: unique symbol;
declare const BUFFERSHIFT: unique symbol;
declare const OBJECTMODE: unique symbol;
declare const DESTROYED: unique symbol;
declare const ERROR: unique symbol;
declare const EMITDATA: unique symbol;
declare const EMITEND: unique symbol;
declare const EMITEND2: unique symbol;
declare const ASYNC: unique symbol;
declare const ABORT: unique symbol;
declare const ABORTED: unique symbol;
declare const SIGNAL: unique symbol;
declare const DATALISTENERS: unique symbol;
declare const DISCARDED: unique symbol;
/**
* Options that may be passed to stream.pipe()
*/
export interface PipeOptions {
/**
* end the destination stream when the source stream ends
*/
end?: boolean;
/**
* proxy errors from the source stream to the destination stream
*/
proxyErrors?: boolean;
}
/**
* Internal class representing a pipe to a destination stream.
*
* @internal
*/
declare class Pipe<T extends unknown> {
src: Minipass<T>;
dest: Minipass<any, T>;
opts: PipeOptions;
ondrain: () => any;
constructor(src: Minipass<T>, dest: Minipass.Writable, opts: PipeOptions);
unpipe(): void;
proxyErrors(_er: any): void;
end(): void;
}
/**
* Internal class representing a pipe to a destination stream where
* errors are proxied.
*
* @internal
*/
declare class PipeProxyErrors<T> extends Pipe<T> {
unpipe(): void;
constructor(src: Minipass<T>, dest: Minipass.Writable, opts: PipeOptions);
}
export declare namespace Minipass {
/**
* Encoding used to create a stream that outputs strings rather than
* Buffer objects.
*/
export type Encoding = BufferEncoding | 'buffer' | null;
/**
* Any stream that Minipass can pipe into
*/
export type Writable = Minipass<any, any, any> | NodeJS.WriteStream | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
/**
* Any stream that can be read from
*/
export type Readable = Minipass<any, any, any> | NodeJS.ReadStream | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
});
/**
* Utility type that can be iterated sync or async
*/
export type DualIterable<T> = Iterable<T> & AsyncIterable<T>;
type EventArguments = Record<string | symbol, unknown[]>;
/**
* The listing of events that a Minipass class can emit.
* Extend this when extending the Minipass class, and pass as
* the third template argument. The key is the name of the event,
* and the value is the argument list.
*
* Any undeclared events will still be allowed, but the handler will get
* arguments as `unknown[]`.
*/
export interface Events<RType extends any = Buffer> extends EventArguments {
readable: [];
data: [chunk: RType];
error: [er: unknown];
abort: [reason: unknown];
drain: [];
resume: [];
end: [];
finish: [];
prefinish: [];
close: [];
[DESTROYED]: [er?: unknown];
[ERROR]: [er: unknown];
}
/**
* String or buffer-like data that can be joined and sliced
*/
export type ContiguousData = Buffer | ArrayBufferLike | ArrayBufferView | string;
export type BufferOrString = Buffer | string;
/**
* Options passed to the Minipass constructor.
*/
export type SharedOptions = {
/**
* Defer all data emission and other events until the end of the
* current tick, similar to Node core streams
*/
async?: boolean;
/**
* A signal which will abort the stream
*/
signal?: AbortSignal;
/**
* Output string encoding. Set to `null` or `'buffer'` (or omit) to
* emit Buffer objects rather than strings.
*
* Conflicts with `objectMode`
*/
encoding?: BufferEncoding | null | 'buffer';
/**
* Output data exactly as it was written, supporting non-buffer/string
* data (such as arbitrary objects, falsey values, etc.)
*
* Conflicts with `encoding`
*/
objectMode?: boolean;
};
/**
* Options for a string encoded output
*/
export type EncodingOptions = SharedOptions & {
encoding: BufferEncoding;
objectMode?: false;
};
/**
* Options for contiguous data buffer output
*/
export type BufferOptions = SharedOptions & {
encoding?: null | 'buffer';
objectMode?: false;
};
/**
* Options for objectMode arbitrary output
*/
export type ObjectModeOptions = SharedOptions & {
objectMode: true;
encoding?: null;
};
/**
* Utility type to determine allowed options based on read type
*/
export type Options<T> = ObjectModeOptions | (T extends string ? EncodingOptions : T extends Buffer ? BufferOptions : SharedOptions);
export {};
}
/**
* Main export, the Minipass class
*
* `RType` is the type of data emitted, defaults to Buffer
*
* `WType` is the type of data to be written, if RType is buffer or string,
* then any {@link Minipass.ContiguousData} is allowed.
*
* `Events` is the set of event handler signatures that this object
* will emit, see {@link Minipass.Events}
*/
export declare class Minipass<RType extends unknown = Buffer, WType extends unknown = RType extends Minipass.BufferOrString ? Minipass.ContiguousData : RType, Events extends Minipass.Events<RType> = Minipass.Events<RType>> extends EventEmitter implements Minipass.DualIterable<RType> {
[FLOWING]: boolean;
[PAUSED]: boolean;
[PIPES]: Pipe<RType>[];
[BUFFER]: RType[];
[OBJECTMODE]: boolean;
[ENCODING]: BufferEncoding | null;
[ASYNC]: boolean;
[DECODER]: SD | null;
[EOF]: boolean;
[EMITTED_END]: boolean;
[EMITTING_END]: boolean;
[CLOSED]: boolean;
[EMITTED_ERROR]: unknown;
[BUFFERLENGTH]: number;
[DESTROYED]: boolean;
[SIGNAL]?: AbortSignal;
[ABORTED]: boolean;
[DATALISTENERS]: number;
[DISCARDED]: boolean;
/**
* true if the stream can be written
*/
writable: boolean;
/**
* true if the stream can be read
*/
readable: boolean;
/**
* If `RType` is Buffer, then options do not need to be provided.
* Otherwise, an options object must be provided to specify either
* {@link Minipass.SharedOptions.objectMode} or
* {@link Minipass.SharedOptions.encoding}, as appropriate.
*/
constructor(...args: [Minipass.ObjectModeOptions] | (RType extends Buffer ? [] | [Minipass.Options<RType>] : [Minipass.Options<RType>]));
/**
* The amount of data stored in the buffer waiting to be read.
*
* For Buffer strings, this will be the total byte length.
* For string encoding streams, this will be the string character length,
* according to JavaScript's `string.length` logic.
* For objectMode streams, this is a count of the items waiting to be
* emitted.
*/
get bufferLength(): number;
/**
* The `BufferEncoding` currently in use, or `null`
*/
get encoding(): BufferEncoding | null;
/**
* @deprecated - This is a read only property
*/
set encoding(_enc: BufferEncoding | null);
/**
* @deprecated - Encoding may only be set at instantiation time
*/
setEncoding(_enc: Minipass.Encoding): void;
/**
* True if this is an objectMode stream
*/
get objectMode(): boolean;
/**
* @deprecated - This is a read-only property
*/
set objectMode(_om: boolean);
/**
* true if this is an async stream
*/
get ['async'](): boolean;
/**
* Set to true to make this stream async.
*
* Once set, it cannot be unset, as this would potentially cause incorrect
* behavior. Ie, a sync stream can be made async, but an async stream
* cannot be safely made sync.
*/
set ['async'](a: boolean);
[ABORT](): void;
/**
* True if the stream has been aborted.
*/
get aborted(): boolean;
/**
* No-op setter. Stream aborted status is set via the AbortSignal provided
* in the constructor options.
*/
set aborted(_: boolean);
/**
* Write data into the stream
*
* If the chunk written is a string, and encoding is not specified, then
* `utf8` will be assumed. If the stream encoding matches the encoding of
* a written string, and the state of the string decoder allows it, then
* the string will be passed through to either the output or the internal
* buffer without any processing. Otherwise, it will be turned into a
* Buffer object for processing into the desired encoding.
*
* If provided, `cb` function is called immediately before return for
* sync streams, or on next tick for async streams, because for this
* base class, a chunk is considered "processed" once it is accepted
* and either emitted or buffered. That is, the callback does not indicate
* that the chunk has been eventually emitted, though of course child
* classes can override this function to do whatever processing is required
* and call `super.write(...)` only once processing is completed.
*/
write(chunk: WType, cb?: () => void): boolean;
write(chunk: WType, encoding?: Minipass.Encoding, cb?: () => void): boolean;
/**
* Low-level explicit read method.
*
* In objectMode, the argument is ignored, and one item is returned if
* available.
*
* `n` is the number of bytes (or in the case of encoding streams,
* characters) to consume. If `n` is not provided, then the entire buffer
* is returned, or `null` is returned if no data is available.
*
* If `n` is greater that the amount of data in the internal buffer,
* then `null` is returned.
*/
read(n?: number | null): RType | null;
[READ](n: number | null, chunk: RType): RType;
/**
* End the stream, optionally providing a final write.
*
* See {@link Minipass#write} for argument descriptions
*/
end(cb?: () => void): this;
end(chunk: WType, cb?: () => void): this;
end(chunk: WType, encoding?: Minipass.Encoding, cb?: () => void): this;
[RESUME](): void;
/**
* Resume the stream if it is currently in a paused state
*
* If called when there are no pipe destinations or `data` event listeners,
* this will place the stream in a "discarded" state, where all data will
* be thrown away. The discarded state is removed if a pipe destination or
* data handler is added, if pause() is called, or if any synchronous or
* asynchronous iteration is started.
*/
resume(): void;
/**
* Pause the stream
*/
pause(): void;
/**
* true if the stream has been forcibly destroyed
*/
get destroyed(): boolean;
/**
* true if the stream is currently in a flowing state, meaning that
* any writes will be immediately emitted.
*/
get flowing(): boolean;
/**
* true if the stream is currently in a paused state
*/
get paused(): boolean;
[BUFFERPUSH](chunk: RType): void;
[BUFFERSHIFT](): RType;
[FLUSH](noDrain?: boolean): void;
[FLUSHCHUNK](chunk: RType): boolean;
/**
* Pipe all data emitted by this stream into the destination provided.
*
* Triggers the flow of data.
*/
pipe<W extends Minipass.Writable>(dest: W, opts?: PipeOptions): W;
/**
* Fully unhook a piped destination stream.
*
* If the destination stream was the only consumer of this stream (ie,
* there are no other piped destinations or `'data'` event listeners)
* then the flow of data will stop until there is another consumer or
* {@link Minipass#resume} is explicitly called.
*/
unpipe<W extends Minipass.Writable>(dest: W): void;
/**
* Alias for {@link Minipass#on}
*/
addListener<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.on`, with the following
* behavior differences to prevent data loss and unnecessary hangs:
*
* - Adding a 'data' event handler will trigger the flow of data
*
* - Adding a 'readable' event handler when there is data waiting to be read
* will cause 'readable' to be emitted immediately.
*
* - Adding an 'endish' event handler ('end', 'finish', etc.) which has
* already passed will cause the event to be emitted immediately and all
* handlers removed.
*
* - Adding an 'error' event handler after an error has been emitted will
* cause the event to be re-emitted immediately with the error previously
* raised.
*/
on<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Alias for {@link Minipass#off}
*/
removeListener<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.off`
*
* If a 'data' event handler is removed, and it was the last consumer
* (ie, there are no pipe destinations or other 'data' event listeners),
* then the flow of data will stop until there is another consumer or
* {@link Minipass#resume} is explicitly called.
*/
off<Event extends keyof Events>(ev: Event, handler: (...args: Events[Event]) => any): this;
/**
* Mostly identical to `EventEmitter.removeAllListeners`
*
* If all 'data' event handlers are removed, and they were the last consumer
* (ie, there are no pipe destinations), then the flow of data will stop
* until there is another consumer or {@link Minipass#resume} is explicitly
* called.
*/
removeAllListeners<Event extends keyof Events>(ev?: Event): this;
/**
* true if the 'end' event has been emitted
*/
get emittedEnd(): boolean;
[MAYBE_EMIT_END](): void;
/**
* Mostly identical to `EventEmitter.emit`, with the following
* behavior differences to prevent data loss and unnecessary hangs:
*
* If the stream has been destroyed, and the event is something other
* than 'close' or 'error', then `false` is returned and no handlers
* are called.
*
* If the event is 'end', and has already been emitted, then the event
* is ignored. If the stream is in a paused or non-flowing state, then
* the event will be deferred until data flow resumes. If the stream is
* async, then handlers will be called on the next tick rather than
* immediately.
*
* If the event is 'close', and 'end' has not yet been emitted, then
* the event will be deferred until after 'end' is emitted.
*
* If the event is 'error', and an AbortSignal was provided for the stream,
* and there are no listeners, then the event is ignored, matching the
* behavior of node core streams in the presense of an AbortSignal.
*
* If the event is 'finish' or 'prefinish', then all listeners will be
* removed after emitting the event, to prevent double-firing.
*/
emit<Event extends keyof Events>(ev: Event, ...args: Events[Event]): boolean;
[EMITDATA](data: RType): boolean;
[EMITEND](): boolean;
[EMITEND2](): boolean;
/**
* Return a Promise that resolves to an array of all emitted data once
* the stream ends.
*/
collect(): Promise<RType[] & {
dataLength: number;
}>;
/**
* Return a Promise that resolves to the concatenation of all emitted data
* once the stream ends.
*
* Not allowed on objectMode streams.
*/
concat(): Promise<RType>;
/**
* Return a void Promise that resolves once the stream ends.
*/
promise(): Promise<void>;
/**
* Asynchronous `for await of` iteration.
*
* This will continue emitting all chunks until the stream terminates.
*/
[Symbol.asyncIterator](): AsyncGenerator<RType, void, void>;
/**
* Synchronous `for of` iteration.
*
* The iteration will terminate when the internal buffer runs out, even
* if the stream has not yet terminated.
*/
[Symbol.iterator](): Generator<RType, void, void>;
/**
* Destroy a stream, preventing it from being used for any further purpose.
*
* If the stream has a `close()` method, then it will be called on
* destruction.
*
* After destruction, any attempt to write data, read data, or emit most
* events will be ignored.
*
* If an error argument is provided, then it will be emitted in an
* 'error' event.
*/
destroy(er?: unknown): this;
/**
* Alias for {@link isStream}
*
* Former export location, maintained for backwards compatibility.
*
* @deprecated
*/
static get isStream(): (s: any) => s is NodeJS.WriteStream | NodeJS.ReadStream | Minipass<any, any, any> | (NodeJS.ReadStream & {
fd: number;
}) | (EventEmitter & {
pause(): any;
resume(): any;
pipe(...destArgs: any[]): any;
}) | (NodeJS.WriteStream & {
fd: number;
}) | (EventEmitter & {
end(): any;
write(chunk: any, ...args: any[]): any;
});
}
//# sourceMappingURL=index.d.ts.map

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,3 @@
{
"type": "module"
}

View File

@@ -0,0 +1,82 @@
{
"name": "minipass",
"version": "7.1.2",
"description": "minimal implementation of a PassThrough stream",
"main": "./dist/commonjs/index.js",
"types": "./dist/commonjs/index.d.ts",
"type": "module",
"tshy": {
"selfLink": false,
"main": true,
"exports": {
"./package.json": "./package.json",
".": "./src/index.ts"
}
},
"exports": {
"./package.json": "./package.json",
".": {
"import": {
"types": "./dist/esm/index.d.ts",
"default": "./dist/esm/index.js"
},
"require": {
"types": "./dist/commonjs/index.d.ts",
"default": "./dist/commonjs/index.js"
}
}
},
"files": [
"dist"
],
"scripts": {
"preversion": "npm test",
"postversion": "npm publish",
"prepublishOnly": "git push origin --follow-tags",
"prepare": "tshy",
"pretest": "npm run prepare",
"presnap": "npm run prepare",
"test": "tap",
"snap": "tap",
"format": "prettier --write . --loglevel warn",
"typedoc": "typedoc --tsconfig .tshy/esm.json ./src/*.ts"
},
"prettier": {
"semi": false,
"printWidth": 75,
"tabWidth": 2,
"useTabs": false,
"singleQuote": true,
"jsxSingleQuote": false,
"bracketSameLine": true,
"arrowParens": "avoid",
"endOfLine": "lf"
},
"devDependencies": {
"@types/end-of-stream": "^1.4.2",
"@types/node": "^20.1.2",
"end-of-stream": "^1.4.0",
"node-abort-controller": "^3.1.1",
"prettier": "^2.6.2",
"tap": "^19.0.0",
"through2": "^2.0.3",
"tshy": "^1.14.0",
"typedoc": "^0.25.1"
},
"repository": "https://github.com/isaacs/minipass",
"keywords": [
"passthrough",
"stream"
],
"author": "Isaac Z. Schlueter <i@izs.me> (http://blog.izs.me/)",
"license": "ISC",
"engines": {
"node": ">=16 || 14 >=14.17"
},
"tap": {
"typecheck": true,
"include": [
"test/*.ts"
]
}
}

View File

@@ -0,0 +1,54 @@
{
"name": "fs-minipass",
"version": "3.0.3",
"main": "lib/index.js",
"scripts": {
"test": "tap",
"lint": "eslint \"**/*.js\"",
"postlint": "template-oss-check",
"template-oss-apply": "template-oss-apply --force",
"lintfix": "npm run lint -- --fix",
"snap": "tap",
"posttest": "npm run lint"
},
"keywords": [],
"author": "GitHub Inc.",
"license": "ISC",
"repository": {
"type": "git",
"url": "https://github.com/npm/fs-minipass.git"
},
"bugs": {
"url": "https://github.com/npm/fs-minipass/issues"
},
"homepage": "https://github.com/npm/fs-minipass#readme",
"description": "fs read and write streams based on minipass",
"dependencies": {
"minipass": "^7.0.3"
},
"devDependencies": {
"@npmcli/eslint-config": "^4.0.1",
"@npmcli/template-oss": "4.18.0",
"mutate-fs": "^2.1.1",
"tap": "^16.3.2"
},
"files": [
"bin/",
"lib/"
],
"tap": {
"check-coverage": true,
"nyc-arg": [
"--exclude",
"tap-snapshots/**"
]
},
"engines": {
"node": "^14.17.0 || ^16.13.0 || >=18.0.0"
},
"templateOSS": {
"//@npmcli/template-oss": "This file is partially managed by @npmcli/template-oss. Edits may be overwritten.",
"version": "4.18.0",
"publish": "true"
}
}

View File

@@ -0,0 +1,15 @@
The ISC License
Copyright (c) 2009-2023 Isaac Z. Schlueter and Contributors
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,389 @@
/// <reference types="node" />
import { Minimatch } from 'minimatch';
import { Minipass } from 'minipass';
import { FSOption, Path, PathScurry } from 'path-scurry';
import { IgnoreLike } from './ignore.js';
import { Pattern } from './pattern.js';
export type MatchSet = Minimatch['set'];
export type GlobParts = Exclude<Minimatch['globParts'], undefined>;
/**
* A `GlobOptions` object may be provided to any of the exported methods, and
* must be provided to the `Glob` constructor.
*
* All options are optional, boolean, and false by default, unless otherwise
* noted.
*
* All resolved options are added to the Glob object as properties.
*
* If you are running many `glob` operations, you can pass a Glob object as the
* `options` argument to a subsequent operation to share the previously loaded
* cache.
*/
export interface GlobOptions {
/**
* Set to `true` to always receive absolute paths for
* matched files. Set to `false` to always return relative paths.
*
* When this option is not set, absolute paths are returned for patterns
* that are absolute, and otherwise paths are returned that are relative
* to the `cwd` setting.
*
* This does _not_ make an extra system call to get
* the realpath, it only does string path resolution.
*
* Conflicts with {@link withFileTypes}
*/
absolute?: boolean;
/**
* Set to false to enable {@link windowsPathsNoEscape}
*
* @deprecated
*/
allowWindowsEscape?: boolean;
/**
* The current working directory in which to search. Defaults to
* `process.cwd()`.
*
* May be eiher a string path or a `file://` URL object or string.
*/
cwd?: string | URL;
/**
* Include `.dot` files in normal matches and `globstar`
* matches. Note that an explicit dot in a portion of the pattern
* will always match dot files.
*/
dot?: boolean;
/**
* Prepend all relative path strings with `./` (or `.\` on Windows).
*
* Without this option, returned relative paths are "bare", so instead of
* returning `'./foo/bar'`, they are returned as `'foo/bar'`.
*
* Relative patterns starting with `'../'` are not prepended with `./`, even
* if this option is set.
*/
dotRelative?: boolean;
/**
* Follow symlinked directories when expanding `**`
* patterns. This can result in a lot of duplicate references in
* the presence of cyclic links, and make performance quite bad.
*
* By default, a `**` in a pattern will follow 1 symbolic link if
* it is not the first item in the pattern, or none if it is the
* first item in the pattern, following the same behavior as Bash.
*/
follow?: boolean;
/**
* string or string[], or an object with `ignore` and `ignoreChildren`
* methods.
*
* If a string or string[] is provided, then this is treated as a glob
* pattern or array of glob patterns to exclude from matches. To ignore all
* children within a directory, as well as the entry itself, append `'/**'`
* to the ignore pattern.
*
* **Note** `ignore` patterns are _always_ in `dot:true` mode, regardless of
* any other settings.
*
* If an object is provided that has `ignored(path)` and/or
* `childrenIgnored(path)` methods, then these methods will be called to
* determine whether any Path is a match or if its children should be
* traversed, respectively.
*/
ignore?: string | string[] | IgnoreLike;
/**
* Treat brace expansion like `{a,b}` as a "magic" pattern. Has no
* effect if {@link nobrace} is set.
*
* Only has effect on the {@link hasMagic} function.
*/
magicalBraces?: boolean;
/**
* Add a `/` character to directory matches. Note that this requires
* additional stat calls in some cases.
*/
mark?: boolean;
/**
* Perform a basename-only match if the pattern does not contain any slash
* characters. That is, `*.js` would be treated as equivalent to
* `**\/*.js`, matching all js files in all directories.
*/
matchBase?: boolean;
/**
* Limit the directory traversal to a given depth below the cwd.
* Note that this does NOT prevent traversal to sibling folders,
* root patterns, and so on. It only limits the maximum folder depth
* that the walk will descend, relative to the cwd.
*/
maxDepth?: number;
/**
* Do not expand `{a,b}` and `{1..3}` brace sets.
*/
nobrace?: boolean;
/**
* Perform a case-insensitive match. This defaults to `true` on macOS and
* Windows systems, and `false` on all others.
*
* **Note** `nocase` should only be explicitly set when it is
* known that the filesystem's case sensitivity differs from the
* platform default. If set `true` on case-sensitive file
* systems, or `false` on case-insensitive file systems, then the
* walk may return more or less results than expected.
*/
nocase?: boolean;
/**
* Do not match directories, only files. (Note: to match
* _only_ directories, put a `/` at the end of the pattern.)
*/
nodir?: boolean;
/**
* Do not match "extglob" patterns such as `+(a|b)`.
*/
noext?: boolean;
/**
* Do not match `**` against multiple filenames. (Ie, treat it as a normal
* `*` instead.)
*
* Conflicts with {@link matchBase}
*/
noglobstar?: boolean;
/**
* Defaults to value of `process.platform` if available, or `'linux'` if
* not. Setting `platform:'win32'` on non-Windows systems may cause strange
* behavior.
*/
platform?: NodeJS.Platform;
/**
* Set to true to call `fs.realpath` on all of the
* results. In the case of an entry that cannot be resolved, the
* entry is omitted. This incurs a slight performance penalty, of
* course, because of the added system calls.
*/
realpath?: boolean;
/**
*
* A string path resolved against the `cwd` option, which
* is used as the starting point for absolute patterns that start
* with `/`, (but not drive letters or UNC paths on Windows).
*
* Note that this _doesn't_ necessarily limit the walk to the
* `root` directory, and doesn't affect the cwd starting point for
* non-absolute patterns. A pattern containing `..` will still be
* able to traverse out of the root directory, if it is not an
* actual root directory on the filesystem, and any non-absolute
* patterns will be matched in the `cwd`. For example, the
* pattern `/../*` with `{root:'/some/path'}` will return all
* files in `/some`, not all files in `/some/path`. The pattern
* `*` with `{root:'/some/path'}` will return all the entries in
* the cwd, not the entries in `/some/path`.
*
* To start absolute and non-absolute patterns in the same
* path, you can use `{root:''}`. However, be aware that on
* Windows systems, a pattern like `x:/*` or `//host/share/*` will
* _always_ start in the `x:/` or `//host/share` directory,
* regardless of the `root` setting.
*/
root?: string;
/**
* A [PathScurry](http://npm.im/path-scurry) object used
* to traverse the file system. If the `nocase` option is set
* explicitly, then any provided `scurry` object must match this
* setting.
*/
scurry?: PathScurry;
/**
* Call `lstat()` on all entries, whether required or not to determine
* if it's a valid match. When used with {@link withFileTypes}, this means
* that matches will include data such as modified time, permissions, and
* so on. Note that this will incur a performance cost due to the added
* system calls.
*/
stat?: boolean;
/**
* An AbortSignal which will cancel the Glob walk when
* triggered.
*/
signal?: AbortSignal;
/**
* Use `\\` as a path separator _only_, and
* _never_ as an escape character. If set, all `\\` characters are
* replaced with `/` in the pattern.
*
* Note that this makes it **impossible** to match against paths
* containing literal glob pattern characters, but allows matching
* with patterns constructed using `path.join()` and
* `path.resolve()` on Windows platforms, mimicking the (buggy!)
* behavior of Glob v7 and before on Windows. Please use with
* caution, and be mindful of [the caveat below about Windows
* paths](#windows). (For legacy reasons, this is also set if
* `allowWindowsEscape` is set to the exact value `false`.)
*/
windowsPathsNoEscape?: boolean;
/**
* Return [PathScurry](http://npm.im/path-scurry)
* `Path` objects instead of strings. These are similar to a
* NodeJS `Dirent` object, but with additional methods and
* properties.
*
* Conflicts with {@link absolute}
*/
withFileTypes?: boolean;
/**
* An fs implementation to override some or all of the defaults. See
* http://npm.im/path-scurry for details about what can be overridden.
*/
fs?: FSOption;
/**
* Just passed along to Minimatch. Note that this makes all pattern
* matching operations slower and *extremely* noisy.
*/
debug?: boolean;
/**
* Return `/` delimited paths, even on Windows.
*
* On posix systems, this has no effect. But, on Windows, it means that
* paths will be `/` delimited, and absolute paths will be their full
* resolved UNC forms, eg instead of `'C:\\foo\\bar'`, it would return
* `'//?/C:/foo/bar'`
*/
posix?: boolean;
/**
* Do not match any children of any matches. For example, the pattern
* `**\/foo` would match `a/foo`, but not `a/foo/b/foo` in this mode.
*
* This is especially useful for cases like "find all `node_modules`
* folders, but not the ones in `node_modules`".
*
* In order to support this, the `Ignore` implementation must support an
* `add(pattern: string)` method. If using the default `Ignore` class, then
* this is fine, but if this is set to `false`, and a custom `Ignore` is
* provided that does not have an `add()` method, then it will throw an
* error.
*
* **Caveat** It *only* ignores matches that would be a descendant of a
* previous match, and only if that descendant is matched *after* the
* ancestor is encountered. Since the file system walk happens in
* indeterminate order, it's possible that a match will already be added
* before its ancestor, if multiple or braced patterns are used.
*
* For example:
*
* ```ts
* const results = await glob([
* // likely to match first, since it's just a stat
* 'a/b/c/d/e/f',
*
* // this pattern is more complicated! It must to various readdir()
* // calls and test the results against a regular expression, and that
* // is certainly going to take a little bit longer.
* //
* // So, later on, it encounters a match at 'a/b/c/d/e', but it's too
* // late to ignore a/b/c/d/e/f, because it's already been emitted.
* 'a/[bdf]/?/[a-z]/*',
* ], { includeChildMatches: false })
* ```
*
* It's best to only set this to `false` if you can be reasonably sure that
* no components of the pattern will potentially match one another's file
* system descendants, or if the occasional included child entry will not
* cause problems.
*
* @default true
*/
includeChildMatches?: boolean;
}
export type GlobOptionsWithFileTypesTrue = GlobOptions & {
withFileTypes: true;
absolute?: undefined;
mark?: undefined;
posix?: undefined;
};
export type GlobOptionsWithFileTypesFalse = GlobOptions & {
withFileTypes?: false;
};
export type GlobOptionsWithFileTypesUnset = GlobOptions & {
withFileTypes?: undefined;
};
export type Result<Opts> = Opts extends GlobOptionsWithFileTypesTrue ? Path : Opts extends GlobOptionsWithFileTypesFalse ? string : Opts extends GlobOptionsWithFileTypesUnset ? string : string | Path;
export type Results<Opts> = Result<Opts>[];
export type FileTypes<Opts> = Opts extends GlobOptionsWithFileTypesTrue ? true : Opts extends GlobOptionsWithFileTypesFalse ? false : Opts extends GlobOptionsWithFileTypesUnset ? false : boolean;
/**
* An object that can perform glob pattern traversals.
*/
export declare class Glob<Opts extends GlobOptions> implements GlobOptions {
absolute?: boolean;
cwd: string;
root?: string;
dot: boolean;
dotRelative: boolean;
follow: boolean;
ignore?: string | string[] | IgnoreLike;
magicalBraces: boolean;
mark?: boolean;
matchBase: boolean;
maxDepth: number;
nobrace: boolean;
nocase: boolean;
nodir: boolean;
noext: boolean;
noglobstar: boolean;
pattern: string[];
platform: NodeJS.Platform;
realpath: boolean;
scurry: PathScurry;
stat: boolean;
signal?: AbortSignal;
windowsPathsNoEscape: boolean;
withFileTypes: FileTypes<Opts>;
includeChildMatches: boolean;
/**
* The options provided to the constructor.
*/
opts: Opts;
/**
* An array of parsed immutable {@link Pattern} objects.
*/
patterns: Pattern[];
/**
* All options are stored as properties on the `Glob` object.
*
* See {@link GlobOptions} for full options descriptions.
*
* Note that a previous `Glob` object can be passed as the
* `GlobOptions` to another `Glob` instantiation to re-use settings
* and caches with a new pattern.
*
* Traversal functions can be called multiple times to run the walk
* again.
*/
constructor(pattern: string | string[], opts: Opts);
/**
* Returns a Promise that resolves to the results array.
*/
walk(): Promise<Results<Opts>>;
/**
* synchronous {@link Glob.walk}
*/
walkSync(): Results<Opts>;
/**
* Stream results asynchronously.
*/
stream(): Minipass<Result<Opts>, Result<Opts>>;
/**
* Stream results synchronously.
*/
streamSync(): Minipass<Result<Opts>, Result<Opts>>;
/**
* Default sync iteration function. Returns a Generator that
* iterates over the results.
*/
iterateSync(): Generator<Result<Opts>, void, void>;
[Symbol.iterator](): Generator<Result<Opts>, void, void>;
/**
* Default async iteration function. Returns an AsyncGenerator that
* iterates over the results.
*/
iterate(): AsyncGenerator<Result<Opts>, void, void>;
[Symbol.asyncIterator](): AsyncGenerator<Result<Opts>, void, void>;
}
//# sourceMappingURL=glob.d.ts.map

View File

@@ -0,0 +1 @@
{"version":3,"file":"glob.d.ts","sourceRoot":"","sources":["../../src/glob.ts"],"names":[],"mappings":";AAAA,OAAO,EAAE,SAAS,EAAoB,MAAM,WAAW,CAAA;AACvD,OAAO,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAA;AAEnC,OAAO,EACL,QAAQ,EACR,IAAI,EACJ,UAAU,EAIX,MAAM,aAAa,CAAA;AACpB,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAA;AACxC,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAA;AAGtC,MAAM,MAAM,QAAQ,GAAG,SAAS,CAAC,KAAK,CAAC,CAAA;AACvC,MAAM,MAAM,SAAS,GAAG,OAAO,CAAC,SAAS,CAAC,WAAW,CAAC,EAAE,SAAS,CAAC,CAAA;AAalE;;;;;;;;;;;;GAYG;AACH,MAAM,WAAW,WAAW;IAC1B;;;;;;;;;;;;OAYG;IACH,QAAQ,CAAC,EAAE,OAAO,CAAA;IAElB;;;;OAIG;IACH,kBAAkB,CAAC,EAAE,OAAO,CAAA;IAE5B;;;;;OAKG;IACH,GAAG,CAAC,EAAE,MAAM,GAAG,GAAG,CAAA;IAElB;;;;OAIG;IACH,GAAG,CAAC,EAAE,OAAO,CAAA;IAEb;;;;;;;;OAQG;IACH,WAAW,CAAC,EAAE,OAAO,CAAA;IAErB;;;;;;;;OAQG;IACH,MAAM,CAAC,EAAE,OAAO,CAAA;IAEhB;;;;;;;;;;;;;;;;OAgBG;IACH,MAAM,CAAC,EAAE,MAAM,GAAG,MAAM,EAAE,GAAG,UAAU,CAAA;IAEvC;;;;;OAKG;IACH,aAAa,CAAC,EAAE,OAAO,CAAA;IAEvB;;;OAGG;IACH,IAAI,CAAC,EAAE,OAAO,CAAA;IAEd;;;;OAIG;IACH,SAAS,CAAC,EAAE,OAAO,CAAA;IAEnB;;;;;OAKG;IACH,QAAQ,CAAC,EAAE,MAAM,CAAA;IAEjB;;OAEG;IACH,OAAO,CAAC,EAAE,OAAO,CAAA;IAEjB;;;;;;;;;OASG;IACH,MAAM,CAAC,EAAE,OAAO,CAAA;IAEhB;;;OAGG;IACH,KAAK,CAAC,EAAE,OAAO,CAAA;IAEf;;OAEG;IACH,KAAK,CAAC,EAAE,OAAO,CAAA;IAEf;;;;;OAKG;IACH,UAAU,CAAC,EAAE,OAAO,CAAA;IAEpB;;;;OAIG;IACH,QAAQ,CAAC,EAAE,MAAM,CAAC,QAAQ,CAAA;IAE1B;;;;;OAKG;IACH,QAAQ,CAAC,EAAE,OAAO,CAAA;IAElB;;;;;;;;;;;;;;;;;;;;;;OAsBG;IACH,IAAI,CAAC,EAAE,MAAM,CAAA;IAEb;;;;;OAKG;IACH,MAAM,CAAC,EAAE,UAAU,CAAA;IAEnB;;;;;;OAMG;IACH,IAAI,CAAC,EAAE,OAAO,CAAA;IAEd;;;OAGG;IACH,MAAM,CAAC,EAAE,WAAW,CAAA;IAEpB;;;;;;;;;;;;;OAaG;IACH,oBAAoB,CAAC,EAAE,OAAO,CAAA;IAE9B;;;;;;;OAOG;IACH,aAAa,CAAC,EAAE,OAAO,CAAA;IAEvB;;;OAGG;IACH,EAAE,CAAC,EAAE,QAAQ,CAAA;IAEb;;;OAGG;IACH,KAAK,CAAC,EAAE,OAAO,CAAA;IAEf;;;;;;;OAOG;IACH,KAAK,CAAC,EAAE,OAAO,CAAA;IAEf;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OA0CG;IACH,mBAAmB,CAAC,EAAE,OAAO,CAAA;CAC9B;AAED,MAAM,MAAM,4BAA4B,GAAG,WAAW,GAAG;IACvD,aAAa,EAAE,IAAI,CAAA;IAEnB,QAAQ,CAAC,EAAE,SAAS,CAAA;IACpB,IAAI,CAAC,EAAE,SAAS,CAAA;IAChB,KAAK,CAAC,EAAE,SAAS,CAAA;CAClB,CAAA;AAED,MAAM,MAAM,6BAA6B,GAAG,WAAW,GAAG;IACxD,aAAa,CAAC,EAAE,KAAK,CAAA;CACtB,CAAA;AAED,MAAM,MAAM,6BAA6B,GAAG,WAAW,GAAG;IACxD,aAAa,CAAC,EAAE,SAAS,CAAA;CAC1B,CAAA;AAED,MAAM,MAAM,MAAM,CAAC,IAAI,IACrB,IAAI,SAAS,4BAA4B,GAAG,IAAI,GAC9C,IAAI,SAAS,6BAA6B,GAAG,MAAM,GACnD,IAAI,SAAS,6BAA6B,GAAG,MAAM,GACnD,MAAM,GAAG,IAAI,CAAA;AACjB,MAAM,MAAM,OAAO,CAAC,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,EAAE,CAAA;AAE1C,MAAM,MAAM,SAAS,CAAC,IAAI,IACxB,IAAI,SAAS,4BAA4B,GAAG,IAAI,GAC9C,IAAI,SAAS,6BAA6B,GAAG,KAAK,GAClD,IAAI,SAAS,6BAA6B,GAAG,KAAK,GAClD,OAAO,CAAA;AAEX;;GAEG;AACH,qBAAa,IAAI,CAAC,IAAI,SAAS,WAAW,CAAE,YAAW,WAAW;IAChE,QAAQ,CAAC,EAAE,OAAO,CAAA;IAClB,GAAG,EAAE,MAAM,CAAA;IACX,IAAI,CAAC,EAAE,MAAM,CAAA;IACb,GAAG,EAAE,OAAO,CAAA;IACZ,WAAW,EAAE,OAAO,CAAA;IACpB,MAAM,EAAE,OAAO,CAAA;IACf,MAAM,CAAC,EAAE,MAAM,GAAG,MAAM,EAAE,GAAG,UAAU,CAAA;IACvC,aAAa,EAAE,OAAO,CAAA;IACtB,IAAI,CAAC,EAAE,OAAO,CAAA;IACd,SAAS,EAAE,OAAO,CAAA;IAClB,QAAQ,EAAE,MAAM,CAAA;IAChB,OAAO,EAAE,OAAO,CAAA;IAChB,MAAM,EAAE,OAAO,CAAA;IACf,KAAK,EAAE,OAAO,CAAA;IACd,KAAK,EAAE,OAAO,CAAA;IACd,UAAU,EAAE,OAAO,CAAA;IACnB,OAAO,EAAE,MAAM,EAAE,CAAA;IACjB,QAAQ,EAAE,MAAM,CAAC,QAAQ,CAAA;IACzB,QAAQ,EAAE,OAAO,CAAA;IACjB,MAAM,EAAE,UAAU,CAAA;IAClB,IAAI,EAAE,OAAO,CAAA;IACb,MAAM,CAAC,EAAE,WAAW,CAAA;IACpB,oBAAoB,EAAE,OAAO,CAAA;IAC7B,aAAa,EAAE,SAAS,CAAC,IAAI,CAAC,CAAA;IAC9B,mBAAmB,EAAE,OAAO,CAAA;IAE5B;;OAEG;IACH,IAAI,EAAE,IAAI,CAAA;IAEV;;OAEG;IACH,QAAQ,EAAE,OAAO,EAAE,CAAA;IAEnB;;;;;;;;;;;OAWG;gBACS,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAAE,IAAI,EAAE,IAAI;IA2HlD;;OAEG;IACG,IAAI,IAAI,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;IAoBpC;;OAEG;IACH,QAAQ,IAAI,OAAO,CAAC,IAAI,CAAC;IAgBzB;;OAEG;IACH,MAAM,IAAI,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC,IAAI,CAAC,CAAC;IAc9C;;OAEG;IACH,UAAU,IAAI,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC,IAAI,CAAC,CAAC;IAclD;;;OAGG;IACH,WAAW,IAAI,SAAS,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC;IAGlD,CAAC,MAAM,CAAC,QAAQ,CAAC;IAIjB;;;OAGG;IACH,OAAO,IAAI,cAAc,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC;IAGnD,CAAC,MAAM,CAAC,aAAa,CAAC;CAGvB"}

View File

@@ -0,0 +1,247 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.Glob = void 0;
const minimatch_1 = require("minimatch");
const node_url_1 = require("node:url");
const path_scurry_1 = require("path-scurry");
const pattern_js_1 = require("./pattern.js");
const walker_js_1 = require("./walker.js");
// if no process global, just call it linux.
// so we default to case-sensitive, / separators
const defaultPlatform = (typeof process === 'object' &&
process &&
typeof process.platform === 'string') ?
process.platform
: 'linux';
/**
* An object that can perform glob pattern traversals.
*/
class Glob {
absolute;
cwd;
root;
dot;
dotRelative;
follow;
ignore;
magicalBraces;
mark;
matchBase;
maxDepth;
nobrace;
nocase;
nodir;
noext;
noglobstar;
pattern;
platform;
realpath;
scurry;
stat;
signal;
windowsPathsNoEscape;
withFileTypes;
includeChildMatches;
/**
* The options provided to the constructor.
*/
opts;
/**
* An array of parsed immutable {@link Pattern} objects.
*/
patterns;
/**
* All options are stored as properties on the `Glob` object.
*
* See {@link GlobOptions} for full options descriptions.
*
* Note that a previous `Glob` object can be passed as the
* `GlobOptions` to another `Glob` instantiation to re-use settings
* and caches with a new pattern.
*
* Traversal functions can be called multiple times to run the walk
* again.
*/
constructor(pattern, opts) {
/* c8 ignore start */
if (!opts)
throw new TypeError('glob options required');
/* c8 ignore stop */
this.withFileTypes = !!opts.withFileTypes;
this.signal = opts.signal;
this.follow = !!opts.follow;
this.dot = !!opts.dot;
this.dotRelative = !!opts.dotRelative;
this.nodir = !!opts.nodir;
this.mark = !!opts.mark;
if (!opts.cwd) {
this.cwd = '';
}
else if (opts.cwd instanceof URL || opts.cwd.startsWith('file://')) {
opts.cwd = (0, node_url_1.fileURLToPath)(opts.cwd);
}
this.cwd = opts.cwd || '';
this.root = opts.root;
this.magicalBraces = !!opts.magicalBraces;
this.nobrace = !!opts.nobrace;
this.noext = !!opts.noext;
this.realpath = !!opts.realpath;
this.absolute = opts.absolute;
this.includeChildMatches = opts.includeChildMatches !== false;
this.noglobstar = !!opts.noglobstar;
this.matchBase = !!opts.matchBase;
this.maxDepth =
typeof opts.maxDepth === 'number' ? opts.maxDepth : Infinity;
this.stat = !!opts.stat;
this.ignore = opts.ignore;
if (this.withFileTypes && this.absolute !== undefined) {
throw new Error('cannot set absolute and withFileTypes:true');
}
if (typeof pattern === 'string') {
pattern = [pattern];
}
this.windowsPathsNoEscape =
!!opts.windowsPathsNoEscape ||
opts.allowWindowsEscape ===
false;
if (this.windowsPathsNoEscape) {
pattern = pattern.map(p => p.replace(/\\/g, '/'));
}
if (this.matchBase) {
if (opts.noglobstar) {
throw new TypeError('base matching requires globstar');
}
pattern = pattern.map(p => (p.includes('/') ? p : `./**/${p}`));
}
this.pattern = pattern;
this.platform = opts.platform || defaultPlatform;
this.opts = { ...opts, platform: this.platform };
if (opts.scurry) {
this.scurry = opts.scurry;
if (opts.nocase !== undefined &&
opts.nocase !== opts.scurry.nocase) {
throw new Error('nocase option contradicts provided scurry option');
}
}
else {
const Scurry = opts.platform === 'win32' ? path_scurry_1.PathScurryWin32
: opts.platform === 'darwin' ? path_scurry_1.PathScurryDarwin
: opts.platform ? path_scurry_1.PathScurryPosix
: path_scurry_1.PathScurry;
this.scurry = new Scurry(this.cwd, {
nocase: opts.nocase,
fs: opts.fs,
});
}
this.nocase = this.scurry.nocase;
// If you do nocase:true on a case-sensitive file system, then
// we need to use regexps instead of strings for non-magic
// path portions, because statting `aBc` won't return results
// for the file `AbC` for example.
const nocaseMagicOnly = this.platform === 'darwin' || this.platform === 'win32';
const mmo = {
// default nocase based on platform
...opts,
dot: this.dot,
matchBase: this.matchBase,
nobrace: this.nobrace,
nocase: this.nocase,
nocaseMagicOnly,
nocomment: true,
noext: this.noext,
nonegate: true,
optimizationLevel: 2,
platform: this.platform,
windowsPathsNoEscape: this.windowsPathsNoEscape,
debug: !!this.opts.debug,
};
const mms = this.pattern.map(p => new minimatch_1.Minimatch(p, mmo));
const [matchSet, globParts] = mms.reduce((set, m) => {
set[0].push(...m.set);
set[1].push(...m.globParts);
return set;
}, [[], []]);
this.patterns = matchSet.map((set, i) => {
const g = globParts[i];
/* c8 ignore start */
if (!g)
throw new Error('invalid pattern object');
/* c8 ignore stop */
return new pattern_js_1.Pattern(set, g, 0, this.platform);
});
}
async walk() {
// Walkers always return array of Path objects, so we just have to
// coerce them into the right shape. It will have already called
// realpath() if the option was set to do so, so we know that's cached.
// start out knowing the cwd, at least
return [
...(await new walker_js_1.GlobWalker(this.patterns, this.scurry.cwd, {
...this.opts,
maxDepth: this.maxDepth !== Infinity ?
this.maxDepth + this.scurry.cwd.depth()
: Infinity,
platform: this.platform,
nocase: this.nocase,
includeChildMatches: this.includeChildMatches,
}).walk()),
];
}
walkSync() {
return [
...new walker_js_1.GlobWalker(this.patterns, this.scurry.cwd, {
...this.opts,
maxDepth: this.maxDepth !== Infinity ?
this.maxDepth + this.scurry.cwd.depth()
: Infinity,
platform: this.platform,
nocase: this.nocase,
includeChildMatches: this.includeChildMatches,
}).walkSync(),
];
}
stream() {
return new walker_js_1.GlobStream(this.patterns, this.scurry.cwd, {
...this.opts,
maxDepth: this.maxDepth !== Infinity ?
this.maxDepth + this.scurry.cwd.depth()
: Infinity,
platform: this.platform,
nocase: this.nocase,
includeChildMatches: this.includeChildMatches,
}).stream();
}
streamSync() {
return new walker_js_1.GlobStream(this.patterns, this.scurry.cwd, {
...this.opts,
maxDepth: this.maxDepth !== Infinity ?
this.maxDepth + this.scurry.cwd.depth()
: Infinity,
platform: this.platform,
nocase: this.nocase,
includeChildMatches: this.includeChildMatches,
}).streamSync();
}
/**
* Default sync iteration function. Returns a Generator that
* iterates over the results.
*/
iterateSync() {
return this.streamSync()[Symbol.iterator]();
}
[Symbol.iterator]() {
return this.iterateSync();
}
/**
* Default async iteration function. Returns an AsyncGenerator that
* iterates over the results.
*/
iterate() {
return this.stream()[Symbol.asyncIterator]();
}
[Symbol.asyncIterator]() {
return this.iterate();
}
}
exports.Glob = Glob;
//# sourceMappingURL=glob.js.map

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,14 @@
import { GlobOptions } from './glob.js';
/**
* Return true if the patterns provided contain any magic glob characters,
* given the options provided.
*
* Brace expansion is not considered "magic" unless the `magicalBraces` option
* is set, as brace expansion just turns one string into an array of strings.
* So a pattern like `'x{a,b}y'` would return `false`, because `'xay'` and
* `'xby'` both do not contain any magic glob characters, and it's treated the
* same as if you had called it on `['xay', 'xby']`. When `magicalBraces:true`
* is in the options, brace expansion _is_ treated as a pattern having magic.
*/
export declare const hasMagic: (pattern: string | string[], options?: GlobOptions) => boolean;
//# sourceMappingURL=has-magic.d.ts.map

View File

@@ -0,0 +1 @@
{"version":3,"file":"has-magic.d.ts","sourceRoot":"","sources":["../../src/has-magic.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,WAAW,EAAE,MAAM,WAAW,CAAA;AAEvC;;;;;;;;;;GAUG;AACH,eAAO,MAAM,QAAQ,YACV,MAAM,GAAG,MAAM,EAAE,YACjB,WAAW,KACnB,OAQF,CAAA"}

View File

@@ -0,0 +1,27 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.hasMagic = void 0;
const minimatch_1 = require("minimatch");
/**
* Return true if the patterns provided contain any magic glob characters,
* given the options provided.
*
* Brace expansion is not considered "magic" unless the `magicalBraces` option
* is set, as brace expansion just turns one string into an array of strings.
* So a pattern like `'x{a,b}y'` would return `false`, because `'xay'` and
* `'xby'` both do not contain any magic glob characters, and it's treated the
* same as if you had called it on `['xay', 'xby']`. When `magicalBraces:true`
* is in the options, brace expansion _is_ treated as a pattern having magic.
*/
const hasMagic = (pattern, options = {}) => {
if (!Array.isArray(pattern)) {
pattern = [pattern];
}
for (const p of pattern) {
if (new minimatch_1.Minimatch(p, options).hasMagic())
return true;
}
return false;
};
exports.hasMagic = hasMagic;
//# sourceMappingURL=has-magic.js.map

View File

@@ -0,0 +1 @@
{"version":3,"file":"has-magic.js","sourceRoot":"","sources":["../../src/has-magic.ts"],"names":[],"mappings":";;;AAAA,yCAAqC;AAGrC;;;;;;;;;;GAUG;AACI,MAAM,QAAQ,GAAG,CACtB,OAA0B,EAC1B,UAAuB,EAAE,EAChB,EAAE;IACX,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC;QAC5B,OAAO,GAAG,CAAC,OAAO,CAAC,CAAA;IACrB,CAAC;IACD,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;QACxB,IAAI,IAAI,qBAAS,CAAC,CAAC,EAAE,OAAO,CAAC,CAAC,QAAQ,EAAE;YAAE,OAAO,IAAI,CAAA;IACvD,CAAC;IACD,OAAO,KAAK,CAAA;AACd,CAAC,CAAA;AAXY,QAAA,QAAQ,YAWpB","sourcesContent":["import { Minimatch } from 'minimatch'\nimport { GlobOptions } from './glob.js'\n\n/**\n * Return true if the patterns provided contain any magic glob characters,\n * given the options provided.\n *\n * Brace expansion is not considered \"magic\" unless the `magicalBraces` option\n * is set, as brace expansion just turns one string into an array of strings.\n * So a pattern like `'x{a,b}y'` would return `false`, because `'xay'` and\n * `'xby'` both do not contain any magic glob characters, and it's treated the\n * same as if you had called it on `['xay', 'xby']`. When `magicalBraces:true`\n * is in the options, brace expansion _is_ treated as a pattern having magic.\n */\nexport const hasMagic = (\n pattern: string | string[],\n options: GlobOptions = {},\n): boolean => {\n if (!Array.isArray(pattern)) {\n pattern = [pattern]\n }\n for (const p of pattern) {\n if (new Minimatch(p, options).hasMagic()) return true\n }\n return false\n}\n"]}

View File

@@ -0,0 +1,25 @@
/// <reference types="node" />
import { Minimatch, MinimatchOptions } from 'minimatch';
import { Path } from 'path-scurry';
import { GlobWalkerOpts } from './walker.js';
export interface IgnoreLike {
ignored?: (p: Path) => boolean;
childrenIgnored?: (p: Path) => boolean;
add?: (ignore: string) => void;
}
/**
* Class used to process ignored patterns
*/
export declare class Ignore implements IgnoreLike {
relative: Minimatch[];
relativeChildren: Minimatch[];
absolute: Minimatch[];
absoluteChildren: Minimatch[];
platform: NodeJS.Platform;
mmopts: MinimatchOptions;
constructor(ignored: string[], { nobrace, nocase, noext, noglobstar, platform, }: GlobWalkerOpts);
add(ign: string): void;
ignored(p: Path): boolean;
childrenIgnored(p: Path): boolean;
}
//# sourceMappingURL=ignore.d.ts.map

View File

@@ -0,0 +1 @@
{"version":3,"file":"ignore.d.ts","sourceRoot":"","sources":["../../src/ignore.ts"],"names":[],"mappings":";AAKA,OAAO,EAAE,SAAS,EAAE,gBAAgB,EAAE,MAAM,WAAW,CAAA;AACvD,OAAO,EAAE,IAAI,EAAE,MAAM,aAAa,CAAA;AAElC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAA;AAE5C,MAAM,WAAW,UAAU;IACzB,OAAO,CAAC,EAAE,CAAC,CAAC,EAAE,IAAI,KAAK,OAAO,CAAA;IAC9B,eAAe,CAAC,EAAE,CAAC,CAAC,EAAE,IAAI,KAAK,OAAO,CAAA;IACtC,GAAG,CAAC,EAAE,CAAC,MAAM,EAAE,MAAM,KAAK,IAAI,CAAA;CAC/B;AAWD;;GAEG;AACH,qBAAa,MAAO,YAAW,UAAU;IACvC,QAAQ,EAAE,SAAS,EAAE,CAAA;IACrB,gBAAgB,EAAE,SAAS,EAAE,CAAA;IAC7B,QAAQ,EAAE,SAAS,EAAE,CAAA;IACrB,gBAAgB,EAAE,SAAS,EAAE,CAAA;IAC7B,QAAQ,EAAE,MAAM,CAAC,QAAQ,CAAA;IACzB,MAAM,EAAE,gBAAgB,CAAA;gBAGtB,OAAO,EAAE,MAAM,EAAE,EACjB,EACE,OAAO,EACP,MAAM,EACN,KAAK,EACL,UAAU,EACV,QAA0B,GAC3B,EAAE,cAAc;IAqBnB,GAAG,CAAC,GAAG,EAAE,MAAM;IAyCf,OAAO,CAAC,CAAC,EAAE,IAAI,GAAG,OAAO;IAczB,eAAe,CAAC,CAAC,EAAE,IAAI,GAAG,OAAO;CAWlC"}

View File

@@ -0,0 +1,119 @@
"use strict";
// give it a pattern, and it'll be able to tell you if
// a given path should be ignored.
// Ignoring a path ignores its children if the pattern ends in /**
// Ignores are always parsed in dot:true mode
Object.defineProperty(exports, "__esModule", { value: true });
exports.Ignore = void 0;
const minimatch_1 = require("minimatch");
const pattern_js_1 = require("./pattern.js");
const defaultPlatform = (typeof process === 'object' &&
process &&
typeof process.platform === 'string') ?
process.platform
: 'linux';
/**
* Class used to process ignored patterns
*/
class Ignore {
relative;
relativeChildren;
absolute;
absoluteChildren;
platform;
mmopts;
constructor(ignored, { nobrace, nocase, noext, noglobstar, platform = defaultPlatform, }) {
this.relative = [];
this.absolute = [];
this.relativeChildren = [];
this.absoluteChildren = [];
this.platform = platform;
this.mmopts = {
dot: true,
nobrace,
nocase,
noext,
noglobstar,
optimizationLevel: 2,
platform,
nocomment: true,
nonegate: true,
};
for (const ign of ignored)
this.add(ign);
}
add(ign) {
// this is a little weird, but it gives us a clean set of optimized
// minimatch matchers, without getting tripped up if one of them
// ends in /** inside a brace section, and it's only inefficient at
// the start of the walk, not along it.
// It'd be nice if the Pattern class just had a .test() method, but
// handling globstars is a bit of a pita, and that code already lives
// in minimatch anyway.
// Another way would be if maybe Minimatch could take its set/globParts
// as an option, and then we could at least just use Pattern to test
// for absolute-ness.
// Yet another way, Minimatch could take an array of glob strings, and
// a cwd option, and do the right thing.
const mm = new minimatch_1.Minimatch(ign, this.mmopts);
for (let i = 0; i < mm.set.length; i++) {
const parsed = mm.set[i];
const globParts = mm.globParts[i];
/* c8 ignore start */
if (!parsed || !globParts) {
throw new Error('invalid pattern object');
}
// strip off leading ./ portions
// https://github.com/isaacs/node-glob/issues/570
while (parsed[0] === '.' && globParts[0] === '.') {
parsed.shift();
globParts.shift();
}
/* c8 ignore stop */
const p = new pattern_js_1.Pattern(parsed, globParts, 0, this.platform);
const m = new minimatch_1.Minimatch(p.globString(), this.mmopts);
const children = globParts[globParts.length - 1] === '**';
const absolute = p.isAbsolute();
if (absolute)
this.absolute.push(m);
else
this.relative.push(m);
if (children) {
if (absolute)
this.absoluteChildren.push(m);
else
this.relativeChildren.push(m);
}
}
}
ignored(p) {
const fullpath = p.fullpath();
const fullpaths = `${fullpath}/`;
const relative = p.relative() || '.';
const relatives = `${relative}/`;
for (const m of this.relative) {
if (m.match(relative) || m.match(relatives))
return true;
}
for (const m of this.absolute) {
if (m.match(fullpath) || m.match(fullpaths))
return true;
}
return false;
}
childrenIgnored(p) {
const fullpath = p.fullpath() + '/';
const relative = (p.relative() || '.') + '/';
for (const m of this.relativeChildren) {
if (m.match(relative))
return true;
}
for (const m of this.absoluteChildren) {
if (m.match(fullpath))
return true;
}
return false;
}
}
exports.Ignore = Ignore;
//# sourceMappingURL=ignore.js.map

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,97 @@
import { Minipass } from 'minipass';
import { Path } from 'path-scurry';
import type { GlobOptions, GlobOptionsWithFileTypesFalse, GlobOptionsWithFileTypesTrue, GlobOptionsWithFileTypesUnset } from './glob.js';
import { Glob } from './glob.js';
export { escape, unescape } from 'minimatch';
export type { FSOption, Path, WalkOptions, WalkOptionsWithFileTypesTrue, WalkOptionsWithFileTypesUnset, } from 'path-scurry';
export { Glob } from './glob.js';
export type { GlobOptions, GlobOptionsWithFileTypesFalse, GlobOptionsWithFileTypesTrue, GlobOptionsWithFileTypesUnset, } from './glob.js';
export { hasMagic } from './has-magic.js';
export { Ignore } from './ignore.js';
export type { IgnoreLike } from './ignore.js';
export type { MatchStream } from './walker.js';
/**
* Syncronous form of {@link globStream}. Will read all the matches as fast as
* you consume them, even all in a single tick if you consume them immediately,
* but will still respond to backpressure if they're not consumed immediately.
*/
export declare function globStreamSync(pattern: string | string[], options: GlobOptionsWithFileTypesTrue): Minipass<Path, Path>;
export declare function globStreamSync(pattern: string | string[], options: GlobOptionsWithFileTypesFalse): Minipass<string, string>;
export declare function globStreamSync(pattern: string | string[], options: GlobOptionsWithFileTypesUnset): Minipass<string, string>;
export declare function globStreamSync(pattern: string | string[], options: GlobOptions): Minipass<Path, Path> | Minipass<string, string>;
/**
* Return a stream that emits all the strings or `Path` objects and
* then emits `end` when completed.
*/
export declare function globStream(pattern: string | string[], options: GlobOptionsWithFileTypesFalse): Minipass<string, string>;
export declare function globStream(pattern: string | string[], options: GlobOptionsWithFileTypesTrue): Minipass<Path, Path>;
export declare function globStream(pattern: string | string[], options?: GlobOptionsWithFileTypesUnset | undefined): Minipass<string, string>;
export declare function globStream(pattern: string | string[], options: GlobOptions): Minipass<Path, Path> | Minipass<string, string>;
/**
* Synchronous form of {@link glob}
*/
export declare function globSync(pattern: string | string[], options: GlobOptionsWithFileTypesFalse): string[];
export declare function globSync(pattern: string | string[], options: GlobOptionsWithFileTypesTrue): Path[];
export declare function globSync(pattern: string | string[], options?: GlobOptionsWithFileTypesUnset | undefined): string[];
export declare function globSync(pattern: string | string[], options: GlobOptions): Path[] | string[];
/**
* Perform an asynchronous glob search for the pattern(s) specified. Returns
* [Path](https://isaacs.github.io/path-scurry/classes/PathBase) objects if the
* {@link withFileTypes} option is set to `true`. See {@link GlobOptions} for
* full option descriptions.
*/
declare function glob_(pattern: string | string[], options?: GlobOptionsWithFileTypesUnset | undefined): Promise<string[]>;
declare function glob_(pattern: string | string[], options: GlobOptionsWithFileTypesTrue): Promise<Path[]>;
declare function glob_(pattern: string | string[], options: GlobOptionsWithFileTypesFalse): Promise<string[]>;
declare function glob_(pattern: string | string[], options: GlobOptions): Promise<Path[] | string[]>;
/**
* Return a sync iterator for walking glob pattern matches.
*/
export declare function globIterateSync(pattern: string | string[], options?: GlobOptionsWithFileTypesUnset | undefined): Generator<string, void, void>;
export declare function globIterateSync(pattern: string | string[], options: GlobOptionsWithFileTypesTrue): Generator<Path, void, void>;
export declare function globIterateSync(pattern: string | string[], options: GlobOptionsWithFileTypesFalse): Generator<string, void, void>;
export declare function globIterateSync(pattern: string | string[], options: GlobOptions): Generator<Path, void, void> | Generator<string, void, void>;
/**
* Return an async iterator for walking glob pattern matches.
*/
export declare function globIterate(pattern: string | string[], options?: GlobOptionsWithFileTypesUnset | undefined): AsyncGenerator<string, void, void>;
export declare function globIterate(pattern: string | string[], options: GlobOptionsWithFileTypesTrue): AsyncGenerator<Path, void, void>;
export declare function globIterate(pattern: string | string[], options: GlobOptionsWithFileTypesFalse): AsyncGenerator<string, void, void>;
export declare function globIterate(pattern: string | string[], options: GlobOptions): AsyncGenerator<Path, void, void> | AsyncGenerator<string, void, void>;
export declare const streamSync: typeof globStreamSync;
export declare const stream: typeof globStream & {
sync: typeof globStreamSync;
};
export declare const iterateSync: typeof globIterateSync;
export declare const iterate: typeof globIterate & {
sync: typeof globIterateSync;
};
export declare const sync: typeof globSync & {
stream: typeof globStreamSync;
iterate: typeof globIterateSync;
};
export declare const glob: typeof glob_ & {
glob: typeof glob_;
globSync: typeof globSync;
sync: typeof globSync & {
stream: typeof globStreamSync;
iterate: typeof globIterateSync;
};
globStream: typeof globStream;
stream: typeof globStream & {
sync: typeof globStreamSync;
};
globStreamSync: typeof globStreamSync;
streamSync: typeof globStreamSync;
globIterate: typeof globIterate;
iterate: typeof globIterate & {
sync: typeof globIterateSync;
};
globIterateSync: typeof globIterateSync;
iterateSync: typeof globIterateSync;
Glob: typeof Glob;
hasMagic: (pattern: string | string[], options?: GlobOptions) => boolean;
escape: (s: string, { windowsPathsNoEscape, }?: Pick<import("minimatch").MinimatchOptions, "windowsPathsNoEscape"> | undefined) => string;
unescape: (s: string, { windowsPathsNoEscape, }?: Pick<import("minimatch").MinimatchOptions, "windowsPathsNoEscape"> | undefined) => string;
};
//# sourceMappingURL=index.d.ts.map

View File

@@ -0,0 +1 @@
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../src/index.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAA;AACnC,OAAO,EAAE,IAAI,EAAE,MAAM,aAAa,CAAA;AAClC,OAAO,KAAK,EACV,WAAW,EACX,6BAA6B,EAC7B,4BAA4B,EAC5B,6BAA6B,EAC9B,MAAM,WAAW,CAAA;AAClB,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAA;AAGhC,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,WAAW,CAAA;AAC5C,YAAY,EACV,QAAQ,EACR,IAAI,EACJ,WAAW,EACX,4BAA4B,EAC5B,6BAA6B,GAC9B,MAAM,aAAa,CAAA;AACpB,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAA;AAChC,YAAY,EACV,WAAW,EACX,6BAA6B,EAC7B,4BAA4B,EAC5B,6BAA6B,GAC9B,MAAM,WAAW,CAAA;AAClB,OAAO,EAAE,QAAQ,EAAE,MAAM,gBAAgB,CAAA;AACzC,OAAO,EAAE,MAAM,EAAE,MAAM,aAAa,CAAA;AACpC,YAAY,EAAE,UAAU,EAAE,MAAM,aAAa,CAAA;AAC7C,YAAY,EAAE,WAAW,EAAE,MAAM,aAAa,CAAA;AAE9C;;;;GAIG;AACH,wBAAgB,cAAc,CAC5B,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,4BAA4B,GACpC,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC,CAAA;AACvB,wBAAgB,cAAc,CAC5B,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,6BAA6B,GACrC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC,CAAA;AAC3B,wBAAgB,cAAc,CAC5B,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,6BAA6B,GACrC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC,CAAA;AAC3B,wBAAgB,cAAc,CAC5B,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,WAAW,GACnB,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC,GAAG,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC,CAAA;AAQlD;;;GAGG;AACH,wBAAgB,UAAU,CACxB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,6BAA6B,GACrC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC,CAAA;AAC3B,wBAAgB,UAAU,CACxB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,4BAA4B,GACpC,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC,CAAA;AACvB,wBAAgB,UAAU,CACxB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,CAAC,EAAE,6BAA6B,GAAG,SAAS,GAClD,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC,CAAA;AAC3B,wBAAgB,UAAU,CACxB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,WAAW,GACnB,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC,GAAG,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC,CAAA;AAQlD;;GAEG;AACH,wBAAgB,QAAQ,CACtB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,6BAA6B,GACrC,MAAM,EAAE,CAAA;AACX,wBAAgB,QAAQ,CACtB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,4BAA4B,GACpC,IAAI,EAAE,CAAA;AACT,wBAAgB,QAAQ,CACtB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,CAAC,EAAE,6BAA6B,GAAG,SAAS,GAClD,MAAM,EAAE,CAAA;AACX,wBAAgB,QAAQ,CACtB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,WAAW,GACnB,IAAI,EAAE,GAAG,MAAM,EAAE,CAAA;AAQpB;;;;;GAKG;AACH,iBAAe,KAAK,CAClB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,CAAC,EAAE,6BAA6B,GAAG,SAAS,GAClD,OAAO,CAAC,MAAM,EAAE,CAAC,CAAA;AACpB,iBAAe,KAAK,CAClB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,4BAA4B,GACpC,OAAO,CAAC,IAAI,EAAE,CAAC,CAAA;AAClB,iBAAe,KAAK,CAClB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,6BAA6B,GACrC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAA;AACpB,iBAAe,KAAK,CAClB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,WAAW,GACnB,OAAO,CAAC,IAAI,EAAE,GAAG,MAAM,EAAE,CAAC,CAAA;AAQ7B;;GAEG;AACH,wBAAgB,eAAe,CAC7B,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,CAAC,EAAE,6BAA6B,GAAG,SAAS,GAClD,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,CAAA;AAChC,wBAAgB,eAAe,CAC7B,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,4BAA4B,GACpC,SAAS,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,CAAA;AAC9B,wBAAgB,eAAe,CAC7B,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,6BAA6B,GACrC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,CAAA;AAChC,wBAAgB,eAAe,CAC7B,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,WAAW,GACnB,SAAS,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,GAAG,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,CAAA;AAQ9D;;GAEG;AACH,wBAAgB,WAAW,CACzB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,CAAC,EAAE,6BAA6B,GAAG,SAAS,GAClD,cAAc,CAAC,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,CAAA;AACrC,wBAAgB,WAAW,CACzB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,4BAA4B,GACpC,cAAc,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,CAAA;AACnC,wBAAgB,WAAW,CACzB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,6BAA6B,GACrC,cAAc,CAAC,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,CAAA;AACrC,wBAAgB,WAAW,CACzB,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,EAC1B,OAAO,EAAE,WAAW,GACnB,cAAc,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,GAAG,cAAc,CAAC,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,CAAA;AASxE,eAAO,MAAM,UAAU,uBAAiB,CAAA;AACxC,eAAO,MAAM,MAAM;;CAAsD,CAAA;AACzE,eAAO,MAAM,WAAW,wBAAkB,CAAA;AAC1C,eAAO,MAAM,OAAO;;CAElB,CAAA;AACF,eAAO,MAAM,IAAI;;;CAGf,CAAA;AAEF,eAAO,MAAM,IAAI;;;;;;;;;;;;;;;;;;;;;;;CAgBf,CAAA"}

View File

@@ -0,0 +1,68 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.glob = exports.sync = exports.iterate = exports.iterateSync = exports.stream = exports.streamSync = exports.globIterate = exports.globIterateSync = exports.globSync = exports.globStream = exports.globStreamSync = exports.Ignore = exports.hasMagic = exports.Glob = exports.unescape = exports.escape = void 0;
const minimatch_1 = require("minimatch");
const glob_js_1 = require("./glob.js");
const has_magic_js_1 = require("./has-magic.js");
var minimatch_2 = require("minimatch");
Object.defineProperty(exports, "escape", { enumerable: true, get: function () { return minimatch_2.escape; } });
Object.defineProperty(exports, "unescape", { enumerable: true, get: function () { return minimatch_2.unescape; } });
var glob_js_2 = require("./glob.js");
Object.defineProperty(exports, "Glob", { enumerable: true, get: function () { return glob_js_2.Glob; } });
var has_magic_js_2 = require("./has-magic.js");
Object.defineProperty(exports, "hasMagic", { enumerable: true, get: function () { return has_magic_js_2.hasMagic; } });
var ignore_js_1 = require("./ignore.js");
Object.defineProperty(exports, "Ignore", { enumerable: true, get: function () { return ignore_js_1.Ignore; } });
function globStreamSync(pattern, options = {}) {
return new glob_js_1.Glob(pattern, options).streamSync();
}
exports.globStreamSync = globStreamSync;
function globStream(pattern, options = {}) {
return new glob_js_1.Glob(pattern, options).stream();
}
exports.globStream = globStream;
function globSync(pattern, options = {}) {
return new glob_js_1.Glob(pattern, options).walkSync();
}
exports.globSync = globSync;
async function glob_(pattern, options = {}) {
return new glob_js_1.Glob(pattern, options).walk();
}
function globIterateSync(pattern, options = {}) {
return new glob_js_1.Glob(pattern, options).iterateSync();
}
exports.globIterateSync = globIterateSync;
function globIterate(pattern, options = {}) {
return new glob_js_1.Glob(pattern, options).iterate();
}
exports.globIterate = globIterate;
// aliases: glob.sync.stream() glob.stream.sync() glob.sync() etc
exports.streamSync = globStreamSync;
exports.stream = Object.assign(globStream, { sync: globStreamSync });
exports.iterateSync = globIterateSync;
exports.iterate = Object.assign(globIterate, {
sync: globIterateSync,
});
exports.sync = Object.assign(globSync, {
stream: globStreamSync,
iterate: globIterateSync,
});
exports.glob = Object.assign(glob_, {
glob: glob_,
globSync,
sync: exports.sync,
globStream,
stream: exports.stream,
globStreamSync,
streamSync: exports.streamSync,
globIterate,
iterate: exports.iterate,
globIterateSync,
iterateSync: exports.iterateSync,
Glob: glob_js_1.Glob,
hasMagic: has_magic_js_1.hasMagic,
escape: minimatch_1.escape,
unescape: minimatch_1.unescape,
});
exports.glob.glob = exports.glob;
//# sourceMappingURL=index.js.map

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,3 @@
{
"type": "commonjs"
}

View File

@@ -0,0 +1,77 @@
/// <reference types="node" />
import { GLOBSTAR } from 'minimatch';
export type MMPattern = string | RegExp | typeof GLOBSTAR;
export type PatternList = [p: MMPattern, ...rest: MMPattern[]];
export type UNCPatternList = [
p0: '',
p1: '',
p2: string,
p3: string,
...rest: MMPattern[]
];
export type DrivePatternList = [p0: string, ...rest: MMPattern[]];
export type AbsolutePatternList = [p0: '', ...rest: MMPattern[]];
export type GlobList = [p: string, ...rest: string[]];
/**
* An immutable-ish view on an array of glob parts and their parsed
* results
*/
export declare class Pattern {
#private;
readonly length: number;
constructor(patternList: MMPattern[], globList: string[], index: number, platform: NodeJS.Platform);
/**
* The first entry in the parsed list of patterns
*/
pattern(): MMPattern;
/**
* true of if pattern() returns a string
*/
isString(): boolean;
/**
* true of if pattern() returns GLOBSTAR
*/
isGlobstar(): boolean;
/**
* true if pattern() returns a regexp
*/
isRegExp(): boolean;
/**
* The /-joined set of glob parts that make up this pattern
*/
globString(): string;
/**
* true if there are more pattern parts after this one
*/
hasMore(): boolean;
/**
* The rest of the pattern after this part, or null if this is the end
*/
rest(): Pattern | null;
/**
* true if the pattern represents a //unc/path/ on windows
*/
isUNC(): boolean;
/**
* True if the pattern starts with a drive letter on Windows
*/
isDrive(): boolean;
/**
* True if the pattern is rooted on an absolute path
*/
isAbsolute(): boolean;
/**
* consume the root of the pattern, and return it
*/
root(): string;
/**
* Check to see if the current globstar pattern is allowed to follow
* a symbolic link.
*/
checkFollowGlobstar(): boolean;
/**
* Mark that the current globstar pattern is following a symbolic link
*/
markFollowGlobstar(): boolean;
}
//# sourceMappingURL=pattern.d.ts.map

View File

@@ -0,0 +1 @@
{"version":3,"file":"pattern.d.ts","sourceRoot":"","sources":["../../src/pattern.ts"],"names":[],"mappings":";AAEA,OAAO,EAAE,QAAQ,EAAE,MAAM,WAAW,CAAA;AACpC,MAAM,MAAM,SAAS,GAAG,MAAM,GAAG,MAAM,GAAG,OAAO,QAAQ,CAAA;AAGzD,MAAM,MAAM,WAAW,GAAG,CAAC,CAAC,EAAE,SAAS,EAAE,GAAG,IAAI,EAAE,SAAS,EAAE,CAAC,CAAA;AAC9D,MAAM,MAAM,cAAc,GAAG;IAC3B,EAAE,EAAE,EAAE;IACN,EAAE,EAAE,EAAE;IACN,EAAE,EAAE,MAAM;IACV,EAAE,EAAE,MAAM;IACV,GAAG,IAAI,EAAE,SAAS,EAAE;CACrB,CAAA;AACD,MAAM,MAAM,gBAAgB,GAAG,CAAC,EAAE,EAAE,MAAM,EAAE,GAAG,IAAI,EAAE,SAAS,EAAE,CAAC,CAAA;AACjE,MAAM,MAAM,mBAAmB,GAAG,CAAC,EAAE,EAAE,EAAE,EAAE,GAAG,IAAI,EAAE,SAAS,EAAE,CAAC,CAAA;AAChE,MAAM,MAAM,QAAQ,GAAG,CAAC,CAAC,EAAE,MAAM,EAAE,GAAG,IAAI,EAAE,MAAM,EAAE,CAAC,CAAA;AAMrD;;;GAGG;AACH,qBAAa,OAAO;;IAIlB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;gBAUrB,WAAW,EAAE,SAAS,EAAE,EACxB,QAAQ,EAAE,MAAM,EAAE,EAClB,KAAK,EAAE,MAAM,EACb,QAAQ,EAAE,MAAM,CAAC,QAAQ;IA6D3B;;OAEG;IACH,OAAO,IAAI,SAAS;IAIpB;;OAEG;IACH,QAAQ,IAAI,OAAO;IAGnB;;OAEG;IACH,UAAU,IAAI,OAAO;IAGrB;;OAEG;IACH,QAAQ,IAAI,OAAO;IAInB;;OAEG;IACH,UAAU,IAAI,MAAM;IAUpB;;OAEG;IACH,OAAO,IAAI,OAAO;IAIlB;;OAEG;IACH,IAAI,IAAI,OAAO,GAAG,IAAI;IAetB;;OAEG;IACH,KAAK,IAAI,OAAO;IAoBhB;;OAEG;IACH,OAAO,IAAI,OAAO;IAelB;;OAEG;IACH,UAAU,IAAI,OAAO;IAUrB;;OAEG;IACH,IAAI,IAAI,MAAM;IASd;;;OAGG;IACH,mBAAmB,IAAI,OAAO;IAQ9B;;OAEG;IACH,kBAAkB,IAAI,OAAO;CAM9B"}

View File

@@ -0,0 +1,219 @@
"use strict";
// this is just a very light wrapper around 2 arrays with an offset index
Object.defineProperty(exports, "__esModule", { value: true });
exports.Pattern = void 0;
const minimatch_1 = require("minimatch");
const isPatternList = (pl) => pl.length >= 1;
const isGlobList = (gl) => gl.length >= 1;
/**
* An immutable-ish view on an array of glob parts and their parsed
* results
*/
class Pattern {
#patternList;
#globList;
#index;
length;
#platform;
#rest;
#globString;
#isDrive;
#isUNC;
#isAbsolute;
#followGlobstar = true;
constructor(patternList, globList, index, platform) {
if (!isPatternList(patternList)) {
throw new TypeError('empty pattern list');
}
if (!isGlobList(globList)) {
throw new TypeError('empty glob list');
}
if (globList.length !== patternList.length) {
throw new TypeError('mismatched pattern list and glob list lengths');
}
this.length = patternList.length;
if (index < 0 || index >= this.length) {
throw new TypeError('index out of range');
}
this.#patternList = patternList;
this.#globList = globList;
this.#index = index;
this.#platform = platform;
// normalize root entries of absolute patterns on initial creation.
if (this.#index === 0) {
// c: => ['c:/']
// C:/ => ['C:/']
// C:/x => ['C:/', 'x']
// //host/share => ['//host/share/']
// //host/share/ => ['//host/share/']
// //host/share/x => ['//host/share/', 'x']
// /etc => ['/', 'etc']
// / => ['/']
if (this.isUNC()) {
// '' / '' / 'host' / 'share'
const [p0, p1, p2, p3, ...prest] = this.#patternList;
const [g0, g1, g2, g3, ...grest] = this.#globList;
if (prest[0] === '') {
// ends in /
prest.shift();
grest.shift();
}
const p = [p0, p1, p2, p3, ''].join('/');
const g = [g0, g1, g2, g3, ''].join('/');
this.#patternList = [p, ...prest];
this.#globList = [g, ...grest];
this.length = this.#patternList.length;
}
else if (this.isDrive() || this.isAbsolute()) {
const [p1, ...prest] = this.#patternList;
const [g1, ...grest] = this.#globList;
if (prest[0] === '') {
// ends in /
prest.shift();
grest.shift();
}
const p = p1 + '/';
const g = g1 + '/';
this.#patternList = [p, ...prest];
this.#globList = [g, ...grest];
this.length = this.#patternList.length;
}
}
}
/**
* The first entry in the parsed list of patterns
*/
pattern() {
return this.#patternList[this.#index];
}
/**
* true of if pattern() returns a string
*/
isString() {
return typeof this.#patternList[this.#index] === 'string';
}
/**
* true of if pattern() returns GLOBSTAR
*/
isGlobstar() {
return this.#patternList[this.#index] === minimatch_1.GLOBSTAR;
}
/**
* true if pattern() returns a regexp
*/
isRegExp() {
return this.#patternList[this.#index] instanceof RegExp;
}
/**
* The /-joined set of glob parts that make up this pattern
*/
globString() {
return (this.#globString =
this.#globString ||
(this.#index === 0 ?
this.isAbsolute() ?
this.#globList[0] + this.#globList.slice(1).join('/')
: this.#globList.join('/')
: this.#globList.slice(this.#index).join('/')));
}
/**
* true if there are more pattern parts after this one
*/
hasMore() {
return this.length > this.#index + 1;
}
/**
* The rest of the pattern after this part, or null if this is the end
*/
rest() {
if (this.#rest !== undefined)
return this.#rest;
if (!this.hasMore())
return (this.#rest = null);
this.#rest = new Pattern(this.#patternList, this.#globList, this.#index + 1, this.#platform);
this.#rest.#isAbsolute = this.#isAbsolute;
this.#rest.#isUNC = this.#isUNC;
this.#rest.#isDrive = this.#isDrive;
return this.#rest;
}
/**
* true if the pattern represents a //unc/path/ on windows
*/
isUNC() {
const pl = this.#patternList;
return this.#isUNC !== undefined ?
this.#isUNC
: (this.#isUNC =
this.#platform === 'win32' &&
this.#index === 0 &&
pl[0] === '' &&
pl[1] === '' &&
typeof pl[2] === 'string' &&
!!pl[2] &&
typeof pl[3] === 'string' &&
!!pl[3]);
}
// pattern like C:/...
// split = ['C:', ...]
// XXX: would be nice to handle patterns like `c:*` to test the cwd
// in c: for *, but I don't know of a way to even figure out what that
// cwd is without actually chdir'ing into it?
/**
* True if the pattern starts with a drive letter on Windows
*/
isDrive() {
const pl = this.#patternList;
return this.#isDrive !== undefined ?
this.#isDrive
: (this.#isDrive =
this.#platform === 'win32' &&
this.#index === 0 &&
this.length > 1 &&
typeof pl[0] === 'string' &&
/^[a-z]:$/i.test(pl[0]));
}
// pattern = '/' or '/...' or '/x/...'
// split = ['', ''] or ['', ...] or ['', 'x', ...]
// Drive and UNC both considered absolute on windows
/**
* True if the pattern is rooted on an absolute path
*/
isAbsolute() {
const pl = this.#patternList;
return this.#isAbsolute !== undefined ?
this.#isAbsolute
: (this.#isAbsolute =
(pl[0] === '' && pl.length > 1) ||
this.isDrive() ||
this.isUNC());
}
/**
* consume the root of the pattern, and return it
*/
root() {
const p = this.#patternList[0];
return (typeof p === 'string' && this.isAbsolute() && this.#index === 0) ?
p
: '';
}
/**
* Check to see if the current globstar pattern is allowed to follow
* a symbolic link.
*/
checkFollowGlobstar() {
return !(this.#index === 0 ||
!this.isGlobstar() ||
!this.#followGlobstar);
}
/**
* Mark that the current globstar pattern is following a symbolic link
*/
markFollowGlobstar() {
if (this.#index === 0 || !this.isGlobstar() || !this.#followGlobstar)
return false;
this.#followGlobstar = false;
return true;
}
}
exports.Pattern = Pattern;
//# sourceMappingURL=pattern.js.map

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,59 @@
import { MMRegExp } from 'minimatch';
import { Path } from 'path-scurry';
import { Pattern } from './pattern.js';
import { GlobWalkerOpts } from './walker.js';
/**
* A cache of which patterns have been processed for a given Path
*/
export declare class HasWalkedCache {
store: Map<string, Set<string>>;
constructor(store?: Map<string, Set<string>>);
copy(): HasWalkedCache;
hasWalked(target: Path, pattern: Pattern): boolean | undefined;
storeWalked(target: Path, pattern: Pattern): void;
}
/**
* A record of which paths have been matched in a given walk step,
* and whether they only are considered a match if they are a directory,
* and whether their absolute or relative path should be returned.
*/
export declare class MatchRecord {
store: Map<Path, number>;
add(target: Path, absolute: boolean, ifDir: boolean): void;
entries(): [Path, boolean, boolean][];
}
/**
* A collection of patterns that must be processed in a subsequent step
* for a given path.
*/
export declare class SubWalks {
store: Map<Path, Pattern[]>;
add(target: Path, pattern: Pattern): void;
get(target: Path): Pattern[];
entries(): [Path, Pattern[]][];
keys(): Path[];
}
/**
* The class that processes patterns for a given path.
*
* Handles child entry filtering, and determining whether a path's
* directory contents must be read.
*/
export declare class Processor {
hasWalkedCache: HasWalkedCache;
matches: MatchRecord;
subwalks: SubWalks;
patterns?: Pattern[];
follow: boolean;
dot: boolean;
opts: GlobWalkerOpts;
constructor(opts: GlobWalkerOpts, hasWalkedCache?: HasWalkedCache);
processPatterns(target: Path, patterns: Pattern[]): this;
subwalkTargets(): Path[];
child(): Processor;
filterEntries(parent: Path, entries: Path[]): Processor;
testGlobstar(e: Path, pattern: Pattern, rest: Pattern | null, absolute: boolean): void;
testRegExp(e: Path, p: MMRegExp, rest: Pattern | null, absolute: boolean): void;
testString(e: Path, p: string, rest: Pattern | null, absolute: boolean): void;
}
//# sourceMappingURL=processor.d.ts.map

View File

@@ -0,0 +1 @@
{"version":3,"file":"processor.d.ts","sourceRoot":"","sources":["../../src/processor.ts"],"names":[],"mappings":"AAEA,OAAO,EAAY,QAAQ,EAAE,MAAM,WAAW,CAAA;AAC9C,OAAO,EAAE,IAAI,EAAE,MAAM,aAAa,CAAA;AAClC,OAAO,EAAa,OAAO,EAAE,MAAM,cAAc,CAAA;AACjD,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAA;AAE5C;;GAEG;AACH,qBAAa,cAAc;IACzB,KAAK,EAAE,GAAG,CAAC,MAAM,EAAE,GAAG,CAAC,MAAM,CAAC,CAAC,CAAA;gBACnB,KAAK,GAAE,GAAG,CAAC,MAAM,EAAE,GAAG,CAAC,MAAM,CAAC,CAAa;IAGvD,IAAI;IAGJ,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,OAAO;IAGxC,WAAW,CAAC,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,OAAO;CAM3C;AAED;;;;GAIG;AACH,qBAAa,WAAW;IACtB,KAAK,EAAE,GAAG,CAAC,IAAI,EAAE,MAAM,CAAC,CAAY;IACpC,GAAG,CAAC,MAAM,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,KAAK,EAAE,OAAO;IAMnD,OAAO,IAAI,CAAC,IAAI,EAAE,OAAO,EAAE,OAAO,CAAC,EAAE;CAOtC;AAED;;;GAGG;AACH,qBAAa,QAAQ;IACnB,KAAK,EAAE,GAAG,CAAC,IAAI,EAAE,OAAO,EAAE,CAAC,CAAY;IACvC,GAAG,CAAC,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,OAAO;IAWlC,GAAG,CAAC,MAAM,EAAE,IAAI,GAAG,OAAO,EAAE;IAS5B,OAAO,IAAI,CAAC,IAAI,EAAE,OAAO,EAAE,CAAC,EAAE;IAG9B,IAAI,IAAI,IAAI,EAAE;CAGf;AAED;;;;;GAKG;AACH,qBAAa,SAAS;IACpB,cAAc,EAAE,cAAc,CAAA;IAC9B,OAAO,cAAoB;IAC3B,QAAQ,WAAiB;IACzB,QAAQ,CAAC,EAAE,OAAO,EAAE,CAAA;IACpB,MAAM,EAAE,OAAO,CAAA;IACf,GAAG,EAAE,OAAO,CAAA;IACZ,IAAI,EAAE,cAAc,CAAA;gBAER,IAAI,EAAE,cAAc,EAAE,cAAc,CAAC,EAAE,cAAc;IAQjE,eAAe,CAAC,MAAM,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE;IAmGjD,cAAc,IAAI,IAAI,EAAE;IAIxB,KAAK;IAQL,aAAa,CAAC,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI,EAAE,GAAG,SAAS;IAqBvD,YAAY,CACV,CAAC,EAAE,IAAI,EACP,OAAO,EAAE,OAAO,EAChB,IAAI,EAAE,OAAO,GAAG,IAAI,EACpB,QAAQ,EAAE,OAAO;IA8CnB,UAAU,CACR,CAAC,EAAE,IAAI,EACP,CAAC,EAAE,QAAQ,EACX,IAAI,EAAE,OAAO,GAAG,IAAI,EACpB,QAAQ,EAAE,OAAO;IAUnB,UAAU,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,GAAG,IAAI,EAAE,QAAQ,EAAE,OAAO;CASvE"}

Some files were not shown because too many files have changed in this diff Show More