Stores

A store keeps the files: Storehouse.Local (the local disk, the default), Storehouse.S3 (S3 and the stores that speak its API), or an adapter of your own (Storehouse.Adapter).

The local disk

# config/config.exs
config :storehouse, adapter: Storehouse.Local

# config/runtime.exs
if config_env() == :prod do
  config :storehouse, Storehouse.Local,
    root: System.get_env("UPLOADS", "/var/lib/my_app/uploads"),
    url_prefix: "/uploads"
end

# config/dev.exs
config :storehouse, Storehouse.Local, root: Path.expand("../tmp/uploads", __DIR__)

In a release, give an absolute root outside the release directory, in the :prod block of config/runtime.exs. A relative path is relative to the working directory, and files in the release directory are lost at the next deploy. Keep the production root inside the :prod block: config/runtime.exs runs in every environment, after config/dev.exs, so outside the block it would replace the development root.

Serve the files with Storehouse.Local.Plug (see Serving).

S3, and the stores that speak its API

Storehouse.S3 keeps the files in a bucket of Amazon S3, or of a store with its API: Cloudflare R2, MinIO, Tigris, Backblaze B2, DigitalOcean Spaces, Google Cloud Storage's interoperability. It needs no other library: it signs its requests itself.

# config/runtime.exs
if config_env() == :prod do
  config :storehouse, adapter: Storehouse.S3

  config :storehouse, Storehouse.S3,
    bucket: "my-app-uploads",
    region: "eu-west-1",
    access_key_id: System.fetch_env!("AWS_ACCESS_KEY_ID"),
    secret_access_key: System.fetch_env!("AWS_SECRET_ACCESS_KEY")
end

Another store: its endpoint, whose URLs name the bucket in their path (path_style, the default with an endpoint), and the public_url of its files when it serves them from a domain of yours or a CDN:

config :storehouse, Storehouse.S3,
  bucket: "uploads",
  region: "auto",
  endpoint: "https://<account>.r2.cloudflarestorage.com",
  access_key_id: System.fetch_env!("R2_ACCESS_KEY_ID"),
  secret_access_key: System.fetch_env!("R2_SECRET_ACCESS_KEY"),
  public_url: "https://files.example.com"

A file's url is its public URL: of a public bucket, or of public_url. A private bucket serves its files at signed URLs (see Private files).

Several stores

An app often keeps files in more than one place: a public bucket behind a CDN for avatars and a private one for invoices, large media on R2, an EU customer's files in an EU bucket. Each is a module of the app's, with a config of its own:

defmodule MyApp.PrivateFiles, do: use Storehouse.S3
defmodule MyApp.Media, do: use Storehouse.S3
defmodule MyApp.Scratch, do: use Storehouse.Local

# config/runtime.exs
config :storehouse, MyApp.PrivateFiles, bucket: "acme-private", region: "eu-west-1", ...
config :storehouse, MyApp.Media, bucket: "acme-media", endpoint: "https://….r2.cloudflarestorage.com", ...
config :storehouse, MyApp.Scratch, root: "/var/lib/my_app/scratch", url_prefix: "/scratch", secret: ...

Give a store as adapter: to any call (store/2, presign_upload/4, verify_upload/2, signed_url/2, read/2, delete/2, …, Storehouse.Live.allow/3, Storehouse.Redirect), or to an Ecto field, whose files are then made permanent, deleted and analyzed there:

field :invoice, Storehouse.Ecto.File, store: MyApp.PrivateFiles

A second local store is served by a plug of its own:

plug Storehouse.Local.Plug, store: MyApp.Scratch

Storehouse.S3 and Storehouse.Local themselves stay the default store, read from config :storehouse, Storehouse.S3 (or Storehouse.Local), as before. With use Storehouse.S3, otp_app: :my_app, a store's config is config :my_app, MyApp.PrivateFiles, ….

A Storehouse.File does not record its store: the field (or the call) names it.

Moving to another store

Storehouse.copy/2 copies a file from one store to another, under the same key, and Storehouse.copy_prefix/2 every file under a prefix ("" for all of them), several at once:

Storehouse.copy_prefix("", from: MyApp.OldMedia, to: MyApp.Media, max_concurrency: 8)
#=> {:ok, %{copied: 1_204, skipped: 0, failed: []}}

or from the command line, with its progress:

mix storehouse.copy --from MyApp.OldMedia --to MyApp.Media

A file the destination has already, of the same size and checksum, is skipped, so a copy can be stopped and run again, and copies only what is missing. Two buckets the same credentials reach copy in S3 itself, with no byte through the app; between other stores, a file goes a chunk at a time, its checksum checked on the way.

To move without stopping, write to both while the old files are copied, with a mirror: a store that writes to its primary and copies each file to its mirrors, out of the request, retried until they answer (Storehouse.Mirror).

defmodule MyApp.Media do
  use Storehouse.Mirror, primary: MyApp.OldMedia, mirrors: [MyApp.NewMedia]
end

Use MyApp.Media as the store. Its copies are jobs of the app's runner (Work out of the request).
Copy what there was: mix storehouse.copy --from MyApp.OldMedia --to MyApp.NewMedia.
Swap them, primary: MyApp.NewMedia, mirrors: [MyApp.OldMedia], so the app reads the new one; then use MyApp.NewMedia alone.

The same copy brings an account's files from production into development, to look into a problem with real files:

mix storehouse.copy --from MyApp.ProductionMedia --to MyApp.Media --prefix accounts/42/

In development

Two ways, as you like it.

The local disk, with nothing else to run: Storehouse.Local in config/dev.exs, its plug in the endpoint, and a secret for direct uploads and signed URLs. The browser uploads straight to the plug as it would to a bucket, so the LiveView is the same in development and in production.

# config/dev.exs
config :storehouse, adapter: Storehouse.Local

config :storehouse, Storehouse.Local,
  root: Path.expand("../tmp/uploads", __DIR__),
  secret: "a development secret of at least 32 bytes"

LocalStack (or MinIO), S3 on your machine: Storehouse.S3, with its endpoint and any keys.

# config/dev.exs
config :storehouse, adapter: Storehouse.S3

config :storehouse, Storehouse.S3,
  bucket: "uploads",
  region: "us-east-1",
  endpoint: "http://localhost:4566",
  access_key_id: "test",
  secret_access_key: "test"

awslocal s3 mb s3://uploads
awslocal s3api put-bucket-cors --bucket uploads --cors-configuration file://cors.json

with the CORS rule of Direct uploads, for the origin of your development server (http://localhost:4000).

In tests

Storehouse.Memory keeps files in memory, each test's its own, so tests run with async: true never see each other's files:

# config/test.exs
config :storehouse, adapter: Storehouse.Memory

# test/test_helper.exs
{:ok, _pid} = Storehouse.Memory.start_link()
ExUnit.start()

# test/support/data_case.ex, in its setup
Storehouse.Memory.checkout()

A task the test starts sees its files; another process, once Storehouse.Memory.allow/2 lets it (a LiveView's process, as Ecto's sandbox does). Storehouse.Test asserts on what was stored, and Storehouse.Memory.fail_next/2 makes the next call of a callback fail:

import Storehouse.Test

test "the avatar is stored", %{user: user} do
  {:ok, user} = Accounts.update_avatar(user, upload)
  assert_stored user.avatar.key
end

test "a store that fails", %{user: user} do
  Storehouse.Memory.fail_next(:put, :unavailable)
  assert {:error, _reason} = Accounts.update_avatar(user, upload)
end

An upload straight to the store has no server to go to: make the browser's part of it with Storehouse.Memory.put_upload/2, then verify it as in production.

The local disk, in a directory of the test's own, works too, for tests that serve files through Storehouse.Local.Plug:

# config/test.exs
config :storehouse, Storehouse.Local,
  root: Path.expand("../tmp/test-uploads", __DIR__),
  secret: "a test secret of at least 32 bytes"

Another store

Write a module with the three callbacks of Storehouse.Adapter (put/3, url/1 and delete/1), and the optional ones for direct uploads and signed URLs (presign_upload/3, read_head/2, signed_url/2) and for reading files back (head/1, read_range/3), and name it in the config:

config :storehouse, adapter: MyApp.Storage

or give it per call: Storehouse.store(path, adapter: MyApp.Storage). The Storehouse.Adapter docs have an example on ExAws.S3, for an app that has it already.

Two rules for an adapter:

Store each file with its checked content type (meta.content_type), and with Content-Disposition: attachment for every type that Storehouse.Check.inline?/1 refuses. Without this, a store such as S3 serves an uploaded HTML file inline, and it can run a script on the store's origin.
The URL goes into your records and documents, so it must stay valid. Give a public or CDN URL, or a path of your app (for example "/files/" <> key) whose controller checks access and redirects to a short-lived signed URL.

Keys

Storehouse.key/2 makes yyyy/mm/<uuid>-<safe name><extension>. The :key option of Storehouse.store/2 (and of Storehouse.Live.consume/3) makes another, a function of the name and the checked content type: a prefix per account, for example.

Storehouse.store(path, key: &("#{account.slug}/uploads/" <> Storehouse.key(&1, &2)))

An adapter refuses a key that Storehouse.valid_key?/1 refuses: no .., no absolute path, nothing but ASCII letters, digits, ., -, _ and /.

The same file again

People upload the same file twice: a folder dropped again, a photo in two places. Give :existing, a function of a checksum (SHA-256, in Base64, as Storehouse.checksum/1 gives it) that finds the file your app has with it, and the same file is that file, {:existing, file}, with nothing stored again:

existing = fn checksum -> Repo.get_by(Photo, account_id: account.id, checksum: checksum) |> then(&(&1 && &1.file)) end

case Storehouse.store(path, owner: "accounts/#{account.id}", existing: existing) do
  {:ok, file} -> create_photo(account, file)
  {:existing, file} -> {:ok, find_photo(account, file)}
  {:error, reason} -> {:error, reason}
end

For an upload straight to the store, the browser reads the file's SHA-256 and sends it when it asks to upload: with :checksum and :existing, presign_upload/4 gives {:existing, file} for a file you have, and nothing is uploaded. Any other upload is signed with the checksum, so the store takes only the bytes that have it: a browser cannot claim another file's checksum and be given that file. (S3, and the local plug, check it. Allow its headers in the bucket's CORS: see Direct uploads.)

Two people may upload the same file at the same moment, before either is saved. verify_upload/2 with :existing checks again with the checksum the store keeps, and gives {:existing, file} for the second, deleting its copy.

Scope the function to the owner, as above: with :owner, Storehouse never gives back another owner's file, whatever the function answers.

Files that belong to an owner

In an app whose files belong to someone (an account, a workspace, a site), give :owner, and each file's key is under the owner's prefix:

{:ok, file} = Storehouse.store(path, owner: "accounts/#{account.id}")
file.key
#=> "accounts/42/2026/10/0b6…-palette.png"

store/2, presign_upload/4, Storehouse.Live.allow/3 and its consume/3 take it. A key a browser sends back is only a claim: check it is the owner's before you take it. verify_upload/2 and Storehouse.Ecto.put_file/4 do, with :owner, and refuse another's with :not_owner, leaving that file where it is. Storehouse.owns?/2 checks a key yourself.

:room is what the owner may still store, in bytes: a file larger than it is refused with {:error, :storage_full}. Presigning checks the size the browser declared; verify_upload/2 checks the size the store has, and deletes a file that does not fit.

room = account.storage_limit - account.storage_used
Storehouse.presign_upload(name, type, bytes, owner: "accounts/#{account.id}", room: room)

Storehouse.usage/2 counts an owner's files in the store itself, and Storehouse.delete_prefix("accounts/42/") deletes them, when the account goes.

Files from the server

A seed, an import or a file the app makes itself is stored the same way, with no browser: Storehouse.store/2 checks it, stores it and gives a Storehouse.File. The file at the path stays where it is; the store has its copy.

{:ok, file} = Storehouse.store("priv/seeds/timeline.png", name: "Timeline.png")

Reading files back

A file the app keeps is read back to work on it: its metadata read, a smaller image made of it, sent on to another service, put in a zip. Storehouse.head/2 gives what the store knows of it without reading it, and Storehouse.read/2, Storehouse.stream/2 and Storehouse.download/3 give its bytes:

{:ok, %{bytes: bytes, content_type: type}} = Storehouse.head(file)

# All at once, for a file of a known, small size.
{:ok, binary} = Storehouse.read(file, max_bytes: 50_000_000)

# A chunk at a time, into a zip or a response, with no more of it in memory than a chunk.
{:ok, stream} = Storehouse.stream(file, chunk_bytes: 8_388_608)
Enum.into(stream, File.stream!("export/" <> file.name))

# To the disk, for a tool that wants a path.
:ok = Storehouse.download(file, "/tmp/original.jpg")

The size a browser declares when it uploads straight to the store is a claim. Give :max_bytes: the store is asked the file's size first, and a larger file gives {:error, {:too_large, bytes}} before a byte of it is read. A stream reads each chunk as it is taken, and raises Storehouse.ReadError when the store fails halfway; download/3 gives that error as {:error, reason}, and leaves nothing at the path.

Archives: a zip of files

An export of an account's files, a folder to download, a backup: Storehouse.Archive makes an archive of stored files and the app's own as a stream, with nothing on the disk and no more than a chunk in memory, past 4 GB too.

entries = [
  {"README.txt", "Your photographs, as of 23 October 2026.\n"},
  {"stories.json", {:call, {MyApp.Exports, :stories_json, [account.id]}}}
  | for(photo <- photos, do: {"photos/" <> photo.name, {:stored, photo.key}})
]

# Its size before it is made: "3.2 GB, 1,204 photographs".
{:ok, bytes} = Storehouse.Archive.size(entries)

Making it takes as long as reading its files and writing it back, a minute or two for a few GB: not a request's time. store_later/3 makes it in a job of the app's runner (Oban's, kept through deploys), then calls your function with it, to send its link:

Storehouse.Archive.store_later(entries, "exports/42.zip",
  expires_in: 86_400,
  then: {MyApp.Exports, :ready, [account.id]}
)

# lib/my_app/exports.ex
def ready({:ok, zip}, account_id) do
  {:ok, url} = Storehouse.signed_url(zip, expires_in: 86_400, disposition: :attachment)
  MyApp.Mailer.deliver_export(account_id, url)
end

store/3 does the same at once, and stream/2 gives the archive as a stream, for a response sent in chunks.

What it can be:

Compressed where it shrinks: text, CSV and JSON are deflated, photographs and video, compressed already, stored as they are (compression: :auto, the default; :compress or :none for all). Compression loses nothing: unzipped, a file is the same bytes.
Encrypted with a password, AES-256: password: "…" (in a job, a function that gives it, {:call, {m, f, a}}: a job never keeps one). 7-Zip, WinZip and Keka open it; macOS's Archive Utility does not.
Split in archives of at most a size, each whole, max_bytes: 2_000_000_000: exports/42-1.zip, exports/42-2.zip.
A tar, gzipped or not, format: :tar, or a format of yours (Storehouse.Archive.Format), for all of them with config :storehouse, archive_format: :tar.

A name is a path in the archive, never outside it: .., a leading / and a backslash are refused.

Files that expire

An export whose link is emailed for a day, a PDF to download for a week, a report a job made: expires_in: stores a file that is deleted after its time.

{:ok, file} = Storehouse.store(path, expires_in: 7 * 86_400, owner: account.id)

Storehouse.expires_at(file)
#=> ~U[2026-10-30 12:00:00Z]

# Its link never outlives it; once it expired, there is none.
{:ok, url} = Storehouse.signed_url(file, expires_in: 30 * 86_400)

Its key is under "expiring/", then the days it is kept, then when it expires (Storehouse.expiring_key/2): "expiring/7d/1792843200/42/…". On S3, give the bucket one lifecycle rule for each number of days you use, on its prefix:

{"Rules": [
  {"ID": "storehouse-expiring-1d", "Status": "Enabled", "Filter": {"Prefix": "expiring/1d/"}, "Expiration": {"Days": 1}},
  {"ID": "storehouse-expiring-7d", "Status": "Enabled", "Filter": {"Prefix": "expiring/7d/"}, "Expiration": {"Days": 7}}
]}

On the local disk, and a store with no lifecycle rules, Storehouse.Sweeper deletes them once their time has come.

An archive takes expires_in: too (Storehouse.Archive.store/3), and anything else made in the store an expiring key: Storehouse.expiring_key("exports/42.csv", 86_400).

Files from an address

Storehouse.fetch/2 downloads a file and stores it: an image pasted by its address, a file an agent names. It fetches only from the hosts you give, https unless you allow http, follows a redirect only to one of them, and stops a download as soon as it is larger than :max_bytes:

Storehouse.fetch(url,
  hosts: ["images.example.com", "*.cdn.example.com"],
  accept: Storehouse.Check.images(),
  max_bytes: 10_000_000
)

Never give a host whose addresses people choose freely, and never one of your own network: the server would fetch what they ask for.

Previews

A PDF can show an image of its first page, a video a poster frame, a large photo a thumbnail. Storehouse does not make them: give :preview, a function of the stored file and the path of the file that returns the URL of the image, or nil. It runs while the file is still there. With Poppler and FFmpeg installed:

defmodule MyApp.Previews do
  # The first page of a PDF (Poppler), the first frame of a video (FFmpeg).
  def make(%Storehouse.File{content_type: "application/pdf"} = file, path),
    do: store(file, ".png", &pdftoppm(path, &1))

  def make(%Storehouse.File{content_type: "video/" <> _} = file, path),
    do: store(file, ".jpg", &ffmpeg(path, &1))

  def make(_file, _path), do: nil

  defp pdftoppm(path, image),
    do: System.cmd("pdftoppm", ["-png", "-singlefile", "-r", "72", path, Path.rootname(image)])

  defp ffmpeg(path, image),
    do: System.cmd("ffmpeg", ["-v", "error", "-i", path, "-frames:v", "1", image])

  # Renders the image to a temporary file and stores it next to the file.
  defp store(file, extension, render) do
    image = Path.join(System.tmp_dir!(), "preview-#{System.unique_integer([:positive])}#{extension}")

    try do
      with {_output, 0} <- render.(image),
           {:ok, preview} <- Storehouse.store(image, key: fn _, _ -> file.key <> extension end) do
        preview.url
      else
        _error -> nil
      end
    after
      File.rm(image)
    end
  end
end

The URL goes into the file's preview. A result that is not a safe URL, or an exception, gives no preview and logs a warning: the file is stored either way. In a LiveView, the function runs in the LiveView's process: give the tools a time limit.

Deleted files

Storehouse does not know which of your records use a file. When one goes, delete its file with Storehouse.delete/2, or with Storehouse.delete_later/2, a job of the app's runner, tried again until the store answers (Work out of the request).

Many files at once

Keys under a prefix go together: an account's ("accounts/42/", with the :key option above), every version of a file. Storehouse.list/2 gives them, a page at a time, and Storehouse.delete_prefix/2 deletes them, a thousand at a time on S3:

{:ok, files} = Storehouse.list("accounts/42/")
Enum.sum_by(files, & &1.bytes)

{:ok, count} = Storehouse.delete_prefix("accounts/42/")
:ok = Storehouse.delete_prefix_later("accounts/42/")

An empty prefix, or "/", is refused: nothing deletes a whole store.

Temporary files never saved

A bucket's lifecycle rule deletes the files under the temporary prefix (see the Ecto guide). The local disk, and a store with no lifecycle rules, has Storehouse.Sweeper: every hour, on one node of a cluster, it deletes the temporary files older than :max_age seconds.

# lib/my_app/application.ex
children = [
  MyApp.Repo,
  {Storehouse.Sweeper, max_age: 86_400},
  MyAppWeb.Endpoint
]

Storehouse.Sweeper.sweep/1 does the same at once, from a scheduled job of yours.

← Previous Page Storehouse

Next Page → Serving