# `Storehouse.Archive`
[🔗](https://github.com/hattoriai/storehouse/blob/v0.2.0/lib/storehouse/archive.ex#L1)

An archive of files, stored ones and the app's own, made as a stream: an
export of an account's files, a folder to download, a backup. Nothing is
written to the disk and no more than a chunk is in memory, however large
the archive.

    entries = [
      {"README.txt", "Your photographs, as of 23 October 2026.\n"}
      | for(photo <- photos, do: {"photos/" <> photo.name, {:stored, photo.key}})
    ]

    # Its size before it is made: "3.2 GB, 1,204 photographs".
    {:ok, bytes} = Storehouse.Archive.size(entries)

    # Into the store, out of the request, then the app's function with it.
    Storehouse.Archive.store_later(entries, "exports/42.zip",
      expires_in: 86_400,
      then: {MyApp.Exports, :ready, [account.id]}
    )

## Entries

`{name, content}`, the name a path in the archive, the content one of:

  * `{:stored, key}` - a stored file, read a chunk at a time from
    `:adapter`, by the size the store gives.
  * a binary.
  * `{:call, {module, function, args}}` - a function that gives the
    content, a binary or an enumerable of iodata, called as the archive
    is made: what the app makes for it (a JSON of its records), and what
    a job can keep (`store_later/3`).
  * `{:stream, enumerable}` - iodata, as the archive is made.

A name is a relative path of UTF-8, with no `..`, no backslash, and no
leading `/`: the archive never puts a file outside the folder it is
opened in. Two entries never have one name. A stored file must be the
size it was when the archive began: one that changed stops it.

## Formats

`:format` chooses it, else `config :storehouse, :archive_format`:

  * `:zip` (`Storehouse.Archive.Zip`, the default) - opened everywhere;
    compressed, encrypted with AES-256, past 4 GB (ZIP64).
  * `:tar` (`Storehouse.Archive.Tar`) - for Unix tools and backups, and
    gzipped (`.tar.gz`).
  * A module of the app's (`Storehouse.Archive.Format`).

## Compression

`:compression` is `:auto` (the default): each file that shrinks is
compressed (text, CSV, JSON, SVG, documents), and each that does not is
stored as it is (photographs, video, audio, other archives, PDF). Both
give the same bytes back, compression loses nothing: a photograph only
takes ten to thirty times longer to compress, to save 1 to 3 %.
`:compress` compresses every file; `:none`, none.

## Splitting

`max_bytes:` makes several archives, each a whole one under the size,
that each open alone (`export-1.zip`, `export-2.zip`, …): for a
mailbox's limit, a FAT32 drive's 4 GB. A file larger than the size alone
gives `{:error, {:too_large, name}}`.

## Out of the request

Archiving 2 GB takes as long as reading 2 GB from the store and writing
2 GB back: a minute or two, not a request's time. `store_later/3` makes it
in a job of the app's runner (`Storehouse.Runner`: Oban's, kept through
deploys and tried again), then calls the app's function with it.

## Telemetry

`store/3` is a span, `[:storehouse, :archive, ...]`, with the `:key`,
the `:format`, the number of `:entries`, and the `:bytes` made.

# `entry`

```elixir
@type entry() ::
  {String.t(),
   binary()
   | {:stored, Storehouse.Adapter.key()}
   | {:call, mfa()}
   | {:stream, Enumerable.t()}}
```

A file in the archive: its name, and its content.

# `size`

```elixir
@spec size([entry()], keyword()) ::
  {:ok, non_neg_integer()} | {:about, non_neg_integer()} | {:error, term()}
```

The archive's size, before it is made, with the stored files' sizes
asked of the store (not read):

  * `{:ok, bytes}` - exact, when nothing is compressed.
  * `{:about, bytes}` - with files compressed: the most it can be, its
    files' sizes as they are (a zip's or a tar's no larger).
  * `{:error, :unknown_size}` - with an entry of unknown size (a stream,
    a function).

Takes `:adapter`, `:format`, `:compression` and `:password`.

# `store`

```elixir
@spec store([entry()], Storehouse.Adapter.key(), keyword()) ::
  {:ok, Storehouse.File.t() | [Storehouse.File.t()]} | {:error, term()}
```

The archive, written into the store under `key` as it is made
(`c:Storehouse.Adapter.stream_open/2`: S3's multipart upload, a file on
the disk): a `Storehouse.File`, with its size and checksum. With
`:max_bytes`, a list of them, the parts' keys `key`'s with `-1`, `-2`, …
before the extension.

Takes the options of `stream/2`, and:

  * `:expires_in` - seconds after which it is deleted, under an expiring
    key (`Storehouse.expiring_key/2`): an export whose link lasts a day.
  * `:max_bytes` - the largest each archive is (see Splitting).

A store's error, or a file not the size it was, gives `{:error, reason}`,
and nothing is left in the store.

# `store_later`

```elixir
@spec store_later([entry()], Storehouse.Adapter.key(), keyword()) ::
  :ok | {:error, term()}
```

Makes the archive (`store/3`) out of the request, in a job of the app's
runner (`Storehouse.Runner`), then calls the app's `:then` with
`{:ok, file}` (or the list of parts): to email its link.

    Storehouse.Archive.store_later(entries, "exports/42.zip",
      expires_in: 86_400,
      then: {MyApp.Exports, :ready, [account.id]}
    )

A job keeps what it is given, so its entries are what can be kept:
`{:stored, key}`, `{:call, mfa}`, and binaries of up to 1 MB; not
`{:stream, enumerable}`, which `{:call, mfa}` gives instead. A
`:password` is never kept: give `{:call, mfa}`, a function that gives it
in the job. A store that fails makes the job try again.

Takes the options of `store/3`, `:then` (required), and `:runner`.

# `stream`

```elixir
@spec stream([entry()], keyword()) :: {:ok, Enumerable.t()} | {:error, term()}
```

The archive as a stream of iodata, made as it is taken. Raises
`Storehouse.ReadError` when the store fails halfway, and an
`ArgumentError` when a file is not the size it was.

Takes `:adapter`, `:format`, `:compression`, `:password` (with a format
that encrypts), `:chunk_bytes` (as for `Storehouse.stream/2`) and
`:modified_at`, the time each file is dated (the default is now).

---

*Consult [api-reference.md](api-reference.md) for complete listing*