# Storehouse v0.2.0 - Table of Contents

> Files for Phoenix: uploads checked by their bytes, stored under safe keys on the local disk or in a store of your own, and served safely.

## Pages

- [Storehouse](readme.md)
- [Stores](stores.md)
- [Serving](serving.md)
- [LiveView](liveview.md)
- [Direct uploads](direct_uploads.md)
- [Uploads from any client](any_client.md)
- [Private files](private_files.md)
- [Ecto](ecto.md)
- [After the upload](analysis.md)
- [Variants](variants.md)
- [Work out of the request](background.md)
- [Features](features.md)
- [Changelog](changelog.md)

## Modules

- [Storehouse.Metadata](Storehouse.Metadata.md): A file's own metadata (`Storehouse.File`'s `:metadata`): a map the app
keeps with the file, in its record's column and in the JSON it sends,
of what a person entered (a caption, a credit, a description for
readers who cannot see it) and what the app decided.

- Files
  - [Storehouse](Storehouse.md): Files for Phoenix: a file checked by its bytes (`Storehouse.Check`),
stored under a new, safe key through a store of the app's
(`Storehouse.Adapter`: the local disk, S3, …), and served safely
(`Storehouse.Local.Plug`).
  - [Storehouse.Check](Storehouse.Check.md): The type of a file, from its bytes.
  - [Storehouse.File](Storehouse.File.md): A stored file: what `Storehouse.store/2` gives, and what an app keeps in
its records (or a document's data) to show and serve it again.
  - [Storehouse.ReadError](Storehouse.ReadError.md): Raised while a stream of `Storehouse.stream/2` is taken, when the store
fails to give the next chunk: its `:key`, and the store's `:reason`.

- Archives
  - [Storehouse.Archive](Storehouse.Archive.md): An archive of files, stored ones and the app's own, made as a stream: an
export of an account's files, a folder to download, a backup. Nothing is
written to the disk and no more than a chunk is in memory, however large
the archive.
  - [Storehouse.Archive.Entry](Storehouse.Archive.Entry.md): A file of an archive, as a format (`Storehouse.Archive.Format`) is given
it
  - [Storehouse.Archive.Format](Storehouse.Archive.Format.md): A format of archive (`Storehouse.Archive`): zip (`Storehouse.Archive.Zip`,
the default), tar (`Storehouse.Archive.Tar`), or one of the app's.
  - [Storehouse.Archive.Tar](Storehouse.Archive.Tar.md): The tar format of `Storehouse.Archive`: for Unix tools and backups,
gzipped (`.tar.gz`) or not, made as a stream.
  - [Storehouse.Archive.Zip](Storehouse.Archive.Zip.md): The zip format of `Storehouse.Archive`, its default: opened by every
system, as it is made, with nothing on the disk.

- Stores
  - [Storehouse.Adapter](Storehouse.Adapter.md): A store of files: the local disk (`Storehouse.Local`), S3, another object
store, a database. Name it in the config
  - [Storehouse.Local](Storehouse.Local.md): A `Storehouse.Adapter` that keeps files in a directory on the local
disk.
  - [Storehouse.Local.Plug](Storehouse.Local.Plug.md): Serves the files of `Storehouse.Local`, and takes the ones a browser
uploads straight to it, when the store signed the upload.
  - [Storehouse.Mirror](Storehouse.Mirror.md): A store that keeps its files in a primary store and copies them to
others, its mirrors: to move to another provider without stopping (new
files go to both, the old ones are copied, then the app reads from the
new one), or to keep a second copy in another provider or region.
  - [Storehouse.S3](Storehouse.S3.md): A `Storehouse.Adapter` for Amazon S3, and the stores that speak its API:
Cloudflare R2, MinIO, Tigris, Backblaze B2, DigitalOcean Spaces, Google
Cloud Storage's interoperability. With no dependency: it signs its
requests itself (AWS Signature Version 4).

- Analysis
  - [Storehouse.Analysis](Storehouse.Analysis.md): What an analysis step (`Storehouse.Analyzer`) made of a file, kept in the
file's `:analysis`, by the step's name
  - [Storehouse.Analyzer](Storehouse.Analyzer.md): Work on a file after it is stored: what it is read for (an image's EXIF,
a placeholder, a video's length, a PDF's pages), each step an analyzer,
what it made of the file kept in the file's `:analysis`, by its name, as
a `Storehouse.Analysis`: its status, its result, why it failed.
  - [Storehouse.Analyzer.BlurHash](Storehouse.Analyzer.BlurHash.md): A step (`Storehouse.Analyzer`) that makes a photograph's
[BlurHash](https://blurha.sh) (`Storehouse.Image.BlurHash`), under
`"blurhash"`: a string of 20 to 30 characters, kept with the file, that
a page decodes into a blur.
  - [Storehouse.Analyzer.EditMetadata](Storehouse.Analyzer.EditMetadata.md): A step (`Storehouse.Analyzer`) that edits the metadata inside a stored
file without re-encoding it, and writes the edited file back over the
original, under `"edit_metadata"`.
  - [Storehouse.Analyzer.Placeholder](Storehouse.Analyzer.Placeholder.md): A step (`Storehouse.Analyzer`) that makes what a page shows while a
photograph loads, under `"placeholder"`
  - [Storehouse.Analyzer.Size](Storehouse.Analyzer.Size.md): A step (`Storehouse.Analyzer`) that asks the store the file's size and
type as stored, without reading it: `%{"bytes" => …, "content_type" => …}`,
under `"size"`. For a file uploaded straight to the store, whose size
was what the browser said until then.
  - [Storehouse.Analyzer.ThumbHash](Storehouse.Analyzer.ThumbHash.md): A step (`Storehouse.Analyzer`) that makes a photograph's
[ThumbHash](https://evanw.github.io/thumbhash/) (`Storehouse.Image.ThumbHash`),
under `"thumbhash"`: about 25 bytes, kept with the file, that a page
decodes into a blur of its colours, proportions and transparency.

- Images
  - [Storehouse.Image.BlurHash](Storehouse.Image.BlurHash.md): [BlurHash](https://blurha.sh): an image's placeholder as a short string,
20 to 30 characters, for a page to decode into a blur (`blurhash` on
npm, and ports in most languages). The same string as its reference
implementation, from the image's pixels: `Storehouse.Analyzer.BlurHash`
gives it from a stored file.

  - [Storehouse.Image.JPEG](Storehouse.Image.JPEG.md): Edits the metadata inside a JPEG without re-encoding it: removes where a
photo was taken, keeps only a few tags, or writes an author's name, and
never touches the image's data.
  - [Storehouse.Image.MetadataEditor](Storehouse.Image.MetadataEditor.md): What edits the metadata inside a stored file for the
`Storehouse.Analyzer.EditMetadata` step: Storehouse's own
(`Storehouse.Image.MetadataEditor.Builtin`, in Elixir, for JPEG, PNG and
WebP), or one of
the app's, around ExifTool, a NIF, a service.
  - [Storehouse.Image.MetadataEditor.Builtin](Storehouse.Image.MetadataEditor.Builtin.md): Storehouse's own metadata editor (`Storehouse.Image.MetadataEditor`), in
Elixir, with no dependency: JPEG (`Storehouse.Image.JPEG`), PNG
(`Storehouse.Image.PNG`) and WebP (`Storehouse.Image.WebP`), each reading only a
file's metadata into memory and copying its image data from file to
file. Other types (HEIC, AVIF, TIFF, a camera's raw files) are
`:unsupported`: an editor around ExifTool edits those.

  - [Storehouse.Image.PNG](Storehouse.Image.PNG.md): Edits the metadata inside a PNG without re-encoding it, as
`Storehouse.Image.JPEG` does a JPEG's: the same changes, the same tags.
  - [Storehouse.Image.ThumbHash](Storehouse.Image.ThumbHash.md): [ThumbHash](https://evanw.github.io/thumbhash/): an image's placeholder
in about 25 bytes, its colours, its proportions and its transparency, for
a page to decode into a blur (`thumbhash` on npm, and ports in most
languages). The same bytes as its reference implementation, from the
image's pixels: `Storehouse.Analyzer.ThumbHash` gives them from a stored
file.

  - [Storehouse.Image.WebP](Storehouse.Image.WebP.md): Edits the metadata inside a WebP without re-encoding it, as
`Storehouse.Image.JPEG` does a JPEG's: the same changes, the same tags.

- Variants
  - [Storehouse.Analyzer.Warm](Storehouse.Analyzer.Warm.md): A step (`Storehouse.Analyzer`) that makes an image's sizes before a
visitor asks for them (`Storehouse.Responsive.warm/3`), under `"warm"`:
for an app that runs its steps out of the request already
(`Storehouse.analyze_later/3`), on a queue of their own.
  - [Storehouse.Components](Storehouse.Components.md): Function components for HEEx, from `Storehouse.Responsive`'s data.
  - [Storehouse.Renderer](Storehouse.Renderer.md): What makes a file's variants (`Storehouse.Variants`), or the URLs of
the service that makes them. A renderer does one or both
  - [Storehouse.Renderer.Imgproxy](Storehouse.Renderer.Imgproxy.md): The URLs of variants [imgproxy](https://imgproxy.net) renders
(`Storehouse.Renderer`): signed, so only the presets' sizes are ever
rendered, and the same every time, for a CDN in front of it to keep.
The app renders nothing, and stores nothing.
  - [Storehouse.Renderer.Vix](Storehouse.Renderer.Vix.md): Renders variants in the app, with libvips through Vix
(`Storehouse.Renderer`): the default renderer when Vix is loaded.
  - [Storehouse.Responsive](Storehouse.Responsive.md): What a page needs to show a photograph at every screen's size, from a
preset of variants with a ladder of widths (`Storehouse.Variants`): its
`src`, its `srcset`, its size, its placeholder. As data, for a JSON API,
or as an `<img>` (`Storehouse.Components.img/1`).
  - [Storehouse.Variant](Storehouse.Variant.md): One size of a file, in one format, as a preset makes it: what a renderer
(`Storehouse.Renderer`) is given to render it, or to build its URL.
  - [Storehouse.Variants](Storehouse.Variants.md): A file's other sizes and formats, by preset: a photograph's thumbnail,
its sizes for a `srcset`, its WebP. Only the presets' sizes are ever
made, so no one can have the server render at a size they choose.
  - [Storehouse.Variants.Plug](Storehouse.Variants.Plug.md): Serves variants (`Storehouse.Variants`) at the URLs
`Storehouse.variant_url/3` gives, making each the first time it is
asked for: `GET <at>/<original key>/<preset>-<width>-<fingerprint>`.

- Work out of the request
  - [Storehouse.Job](Storehouse.Job.md): A piece of Storehouse's work done out of the request: a file's
analysis, a deletion, a mirror's copy. A runner keeps it and runs it
(`Storehouse.Runner`).
  - [Storehouse.Runner](Storehouse.Runner.md): What keeps and runs Storehouse's work out of the request
(`Storehouse.Job`): files analyzed later (`Storehouse.analyze_later/3`),
deleted later (`Storehouse.delete_later/2`), copied to a mirror
(`Storehouse.Mirror`). One runner does all of it, and another takes its
place with no other change.
  - [Storehouse.Runner.Memory](Storehouse.Runner.Memory.md): A runner (`Storehouse.Runner`) in a process of the app's: each job run
beside the app's requests, and when it fails, again after a while (a
second, then twice as long each time, up to five minutes), up to its
`:max_attempts`. Then it is given up (`Storehouse.Job.discarded/3`).
  - [Storehouse.Runner.Oban](Storehouse.Runner.Oban.md): A runner (`Storehouse.Runner`) in Oban: each job an Oban job
(`Storehouse.Runner.Oban.Worker`), kept in the app's database, so it
outlives a deploy, and retried with Oban's backoff, up to the job's
`:max_attempts`. A job waiting is not inserted twice (Oban's `unique`).
  - [Storehouse.Runner.Oban.Worker](Storehouse.Runner.Oban.Worker.md): The Oban worker of `Storehouse.Runner.Oban`: a job's map
(`Storehouse.Job.to_map/1`) as its arguments, run with
`Storehouse.Job.run/1`, retried by Oban when it fails.

- Cleanup
  - [Storehouse.Sweeper](Storehouse.Sweeper.md): Sweeps, every `:every` seconds, on a store with no lifecycle rule (the
local disk, MinIO)

- Events
  - [Storehouse.Notify](Storehouse.Notify.md): What happens to files, told to the app as it happens: a file stored, its
analysis done or a step of it failed, a file deleted. For a page that
shows the files to everyone who has it open, as their analysis comes
in.
  - [Storehouse.Notify.PubSub](Storehouse.Notify.PubSub.md): Broadcasts what happens to files (`Storehouse.Notify`) on the app's
`Phoenix.PubSub`, as `{:storehouse, event, info}`.

- Tests
  - [Storehouse.Memory](Storehouse.Memory.md): A `Storehouse.Adapter` that keeps files in memory, for tests: every
callback, the optional ones too, with no disk, no bucket and no
container, and each test's files its own, as Ecto's sandbox keeps each
test's rows.
  - [Storehouse.Test](Storehouse.Test.md): Assertions on the files a test stored, with `Storehouse.Memory`

- Private files
  - [Storehouse.Proxy](Storehouse.Proxy.md): A file of the store sent through the app: it checks that the person may
see it, then streams it from the store, a part at a time, so that the
bucket's address never leaves the app and a CDN in front keeps what it
may. The sibling of `Storehouse.Redirect`, which sends the person to a
signed URL instead.
  - [Storehouse.Redirect](Storehouse.Redirect.md): A lasting URL of your app for a file that is not public: it checks that
the person may see the file, then redirects to a URL the store signs
for a few minutes (`Storehouse.signed_url/2`). Keep this URL in your
records and documents, not the signed one, which expires.

- A CDN in front
  - [Storehouse.Purge](Storehouse.Purge.md): What a CDN in front forgets when a file is deleted: the file's URLs, its
variants' (`Storehouse.Variants`), and its cache tag. Keys are never
reused, so a file kept for a year never changes; it only goes, and a
CDN would go on serving it. A purger tells the CDN.
  - [Storehouse.Purge.Cloudflare](Storehouse.Purge.Cloudflare.md): A purger (`Storehouse.Purge`) for Cloudflare's cache: a zone's
`purge_cache`, with an API token that may purge it.

- Resumable uploads (tus)
  - [Storehouse.Tus](Storehouse.Tus.md): The uploads in progress of `Storehouse.Tus.Plug`: their state, kept in
ETS by this process, and those that expired, given up every
`:sweep_every` seconds (their parts deleted from the store).
  - [Storehouse.Tus.Plug](Storehouse.Tus.Plug.md): Resumable uploads through the app, by the [tus](https://tus.io) protocol
(1.0.0): for clients that speak it (Uppy's Tus plugin, tus-js-client,
the mobile and desktop clients), and for stores the browser cannot send
to straight (the local disk). A connection lost halfway carries on from
the last byte the app has.
  - [Storehouse.Tus.State](Storehouse.Tus.State.md): Where `Storehouse.Tus.Plug` keeps each upload between requests: its key,
size, offset, metadata and, on a store of multipart uploads, its parts
and the bytes not yet a part. `Storehouse.Tus.State.ETS` is the default,
one node's.
  - [Storehouse.Tus.State.ETS](Storehouse.Tus.State.ETS.md): `Storehouse.Tus.State` in ETS, in the table of the `Storehouse.Tus`
process, which the app starts: one node's.

- Any client
  - [Storehouse.Batch](Storehouse.Batch.md): Uploads straight to the store, many at a time: what a frontend that is
not a LiveView (React, Vue, a native app) asks of the app around the
browser's `PUT`s. `presign/2` signs a batch, `register/2` checks the
files once they are there. `Storehouse.Plug.Presign` and
`Storehouse.Plug.Register` answer them in JSON.
  - [Storehouse.Plug.Multipart](Storehouse.Plug.Multipart.md): Very large files sent from the browser straight to the store in parts
(`Storehouse.presign_multipart/4`), for the JavaScript client or a
frontend of the app's own: JSON endpoints beside `Storehouse.Plug.Presign`.
  - [Storehouse.Plug.Presign](Storehouse.Plug.Presign.md): Signs uploads straight to the store for a frontend that is not a
LiveView: a JSON endpoint in front of `Storehouse.Batch.presign/2`.
  - [Storehouse.Plug.Register](Storehouse.Plug.Register.md): Checks the files a browser uploaded straight to the store, signed by
`Storehouse.Plug.Presign`, and gives them to the app: a JSON endpoint in
front of `Storehouse.Batch.register/2`.

- LiveView
  - [Storehouse.Live](Storehouse.Live.md): The files of a LiveView upload, stored.
  - [Storehouse.Live.Writer](Storehouse.Live.Writer.md): A LiveView upload writer (`Phoenix.LiveView.UploadWriter`) that streams
each chunk on to the store as it comes, with no file on the app's disk:
to S3 in the parts of a multipart upload, to the local disk in its
place. `Storehouse.Live.allow/3` with `mode: :stream` gives it to the
upload.

- Ecto
  - [Storehouse.Ecto](Storehouse.Ecto.md): Files in records, with Ecto: a column of type `Storehouse.Ecto.File`
(or a list of them), files put in a changeset with `put_file/4`, made
permanent when the record is saved, and deleted once nothing has them.
  - [Storehouse.Ecto.File](Storehouse.Ecto.File.md): An Ecto type for a stored file (`Storehouse.File`) in a record's column:
a map (`jsonb` in PostgreSQL) of its key, URL, name, type, size,
checksum, width, height, metadata (`Storehouse.Metadata`) and analysis
(`Storehouse.Analysis`, each step's as `Storehouse.Analysis.to_map/1`). No table of files: the record has its file.

- A library of files
  - [Storehouse.Ecto.Blob](Storehouse.Ecto.Blob.md): A library of files: a row a file, owned by someone (an account, a site),
referenced from many records, kept when the record that first used it
is deleted. Its tables are `Storehouse.Ecto.Migrations`'s; which records
use each file is `Storehouse.Ecto.Usage`'s.
  - [Storehouse.Ecto.Blob.Schema](Storehouse.Ecto.Blob.Schema.md): A blob schema of the app's own (`Storehouse.Ecto.Blob`): its primary
key a UUID, its timestamps in microseconds, its usage table, and
`fields/0` in its `schema` block.
  - [Storehouse.Ecto.Migrations](Storehouse.Ecto.Migrations.md): The tables of a library of files (`Storehouse.Ecto.Blob`,
`Storehouse.Ecto.Usage`), for a migration of the app's
  - [Storehouse.Ecto.Usage](Storehouse.Ecto.Usage.md): Which records use each file of a library (`Storehouse.Ecto.Blob`): kept
as the records change, so the library can say what each file is used
by, and refuse to delete one that is used.

## Mix Tasks

- [mix storehouse.copy](Mix.Tasks.Storehouse.Copy.md): Copies the files of one store to another (`Storehouse.copy_prefix/2`),
with its progress: to move to another provider, fill a mirror
(`Storehouse.Mirror`), or bring a production account's files into
development.
- [mix storehouse.reprocess](Mix.Tasks.Storehouse.Reprocess.md): Runs analysis steps over the files of a field in every record of a
schema (`Storehouse.Ecto.reprocess/5`), with its progress: a step added
after files were stored, files from before analysis existed.