Storehouse.Adapter behaviour (Storehouse v0.2.0)

Copy Markdown View Source

A store of files: the local disk (Storehouse.Local), S3, another object store, a database. Name it in the config:

config :storehouse, adapter: MyApp.Storage

or give it per call with the :adapter option of Storehouse.store/2.

It has three callbacks, Kotoba's and Hakushi's (Kotoba.Storage, Hakushi.Storage): a store written for one of them is a Storehouse adapter as it is.

An S3 adapter

With ExAws.S3:

defmodule MyApp.Storage do
  @behaviour Storehouse.Adapter

  @bucket "my-app-uploads"

  @impl true
  def put(key, path, meta) do
    disposition = if Storehouse.Check.inline?(meta.content_type), do: "inline", else: "attachment"

    path
    |> ExAws.S3.Upload.stream_file()
    |> ExAws.S3.upload(@bucket, key, content_type: meta.content_type, content_disposition: disposition)
    |> ExAws.request()
    |> case do
      {:ok, _response} -> {:ok, url(key)}
      {:error, reason} -> {:error, reason}
    end
  end

  @impl true
  def url(key), do: "https://#{@bucket}.s3.amazonaws.com/" <> key

  @impl true
  def delete(key) do
    case ExAws.request(ExAws.S3.delete_object(@bucket, key)) do
      {:ok, _response} -> :ok
      {:error, reason} -> {:error, reason}
    end
  end
end

The URL goes into the app's records and documents, so it must stay valid: a public or CDN URL, or a path of the app ("/files/" <> key) whose controller checks access and redirects to a short-lived signed URL. See the Stores guide.

Summary

Types

A file list/1 gives: its key, its size, and when it was last written.

What a store knows of a file without reading it: its size in :bytes, its :content_type as stored (or nil), its :checksum, the SHA-256 in Base64 (or nil when the store keeps none), and when it was last written, :modified_at, where the store says.

A storage key, for example "2026/10/0b6…-board.png" (Storehouse.key/2).

What Storehouse knows of the file: :name (the client's name, for display only), :content_type (checked on the server, Storehouse.Check), :bytes, :checksum (its SHA-256, in Base64), which a store that can checks as it takes the file (Storehouse.S3 does), and :cache_control, the Cache-Control a store that keeps headers serves it with.

A part of a multipart upload, as the store has it.

An upload a browser makes, straight to the store: an HTTP method, a url that carries its signature, and the headers it sends as they are.

Callbacks

The SHA-256 of the file with key, in Base64, as the store knows it, or nil when it keeps none: for a file uploaded straight to it. Optional.

Copies the file with key from to the key to, in the store, without it going through the app: to make a temporary file permanent (Storehouse.promote/2). A missing from gives {:error, :enoent}. Optional.

Deletes the file with key. A key with no file gives :ok.

Deletes the files with keys, at most a thousand, at once. A key with no file is deleted. Optional: without it, Storehouse.delete_prefix/2 deletes one at a time with delete/1.

What the store knows of the file with key, without reading it: to learn its size before a byte of it is read (Storehouse.head/2). A missing file gives {:error, :enoent}. Optional, with read_range/3.

The files whose keys start with prefix, in the order of their keys, as an enumerable that asks the store as it is taken (and raises Storehouse.ReadError when the store fails then). Optional. See Storehouse.list/2.

Gives up an upload: its parts are deleted.

Puts the parts together: the file is then under its key.

The parts the store has of an upload: {:ok, parts}, by number.

Sends part number of an upload from the app itself (Storehouse.Tus.Plug), with its SHA-256, which the store checks: {:ok, part}.

Signs the browser's PUT of part number, with its SHA-256 in Base64 (:checksum) when it is given, so that the store refuses other bytes, and its size (:bytes) when it is given, so that it refuses a larger part: {:ok, %{method: "PUT", url: …, headers: %{…}}}.

Starts a multipart upload of a file under key, of meta's type, that the browser sends in parts straight to the store (Storehouse.presign_multipart/4): {:ok, upload_id}. Optional, with the other multipart_ callbacks: Storehouse.S3 has them.

The uploads under prefix started and neither completed nor given up, with their :key, :upload_id and :started_at: for the sweeper.

An upload of the file with key that a browser makes itself, without the store's keys, valid opts[:expires_in] seconds: of meta.bytes bytes exactly, of the type meta.content_type. Optional: a store without it takes its files through the app (Storehouse.store/2). See Storehouse.presign_upload/4.

Stores the file at path under key, and returns its URL.

The first bytes bytes of the file with key (all of it, when it is shorter), and its size: to check a file uploaded straight to the store. Optional; needed with presign_upload/3.

The bytes of the file with key from first to last, both included (fewer, when the file ends sooner; "" past its end): what Storehouse.read/2, Storehouse.stream/2 and Storehouse.download/3 read the file with, a part at a time. A missing file gives {:error, :enoent}. Optional, with head/1.

A URL of the file with key that is valid opts[:expires_in] seconds, for a file that is not public. opts[:disposition] is "inline" or "attachment", and opts[:filename] the name a download gets, when given. Optional. See Storehouse.signed_url/2.

Gives up a streamed file: nothing of it stays in the store.

Ends a streamed file, which is then in the store: its URL.

Opens a file with key that its bytes are written to as they come, in stream_write/2: an upload streamed through the app, with no file of its own on the disk (Storehouse.Live.allow/3, mode: :stream). Optional, with the three other stream_ callbacks.

Writes the next bytes of a streamed file.

Told that a file the browser uploaded straight to the store (presign_upload/3) was checked and kept (Storehouse.verify_upload/2): for a store that does more with a file it takes (Storehouse.Mirror copies it to its mirrors). Optional.

Returns the URL of the file with key.

Types

entry()

@type entry() :: %{key: key(), bytes: non_neg_integer(), modified_at: DateTime.t()}

A file list/1 gives: its key, its size, and when it was last written.

head()

@type head() :: %{
  :bytes => non_neg_integer(),
  :content_type => String.t() | nil,
  :checksum => String.t() | nil,
  optional(:modified_at) => DateTime.t() | nil
}

What a store knows of a file without reading it: its size in :bytes, its :content_type as stored (or nil), its :checksum, the SHA-256 in Base64 (or nil when the store keeps none), and when it was last written, :modified_at, where the store says.

key()

@type key() :: String.t()

A storage key, for example "2026/10/0b6…-board.png" (Storehouse.key/2).

meta()

@type meta() :: %{
  optional(:name) => String.t(),
  optional(:content_type) => String.t(),
  optional(:bytes) => non_neg_integer(),
  optional(:checksum) => String.t(),
  optional(:cache_control) => String.t()
}

What Storehouse knows of the file: :name (the client's name, for display only), :content_type (checked on the server, Storehouse.Check), :bytes, :checksum (its SHA-256, in Base64), which a store that can checks as it takes the file (Storehouse.S3 does), and :cache_control, the Cache-Control a store that keeps headers serves it with.

part()

@type part() :: %{
  number: pos_integer(),
  etag: String.t(),
  bytes: non_neg_integer(),
  checksum: String.t() | nil
}

A part of a multipart upload, as the store has it.

upload()

@type upload() :: %{
  method: String.t(),
  url: String.t(),
  headers: %{required(String.t()) => String.t()}
}

An upload a browser makes, straight to the store: an HTTP method, a url that carries its signature, and the headers it sends as they are.

Callbacks

checksum(key)

(optional)
@callback checksum(key()) :: {:ok, String.t() | nil} | {:error, term()}

The SHA-256 of the file with key, in Base64, as the store knows it, or nil when it keeps none: for a file uploaded straight to it. Optional.

copy(from, to)

(optional)
@callback copy(from :: key(), to :: key()) :: :ok | {:error, term()}

Copies the file with key from to the key to, in the store, without it going through the app: to make a temporary file permanent (Storehouse.promote/2). A missing from gives {:error, :enoent}. Optional.

delete(key)

@callback delete(key()) :: :ok | {:error, term()}

Deletes the file with key. A key with no file gives :ok.

delete_many(list)

(optional)
@callback delete_many([key()]) :: :ok | {:error, term()}

Deletes the files with keys, at most a thousand, at once. A key with no file is deleted. Optional: without it, Storehouse.delete_prefix/2 deletes one at a time with delete/1.

head(key)

(optional)
@callback head(key()) :: {:ok, head()} | {:error, term()}

What the store knows of the file with key, without reading it: to learn its size before a byte of it is read (Storehouse.head/2). A missing file gives {:error, :enoent}. Optional, with read_range/3.

list(prefix)

(optional)
@callback list(prefix :: String.t()) :: {:ok, Enumerable.t()} | {:error, term()}

The files whose keys start with prefix, in the order of their keys, as an enumerable that asks the store as it is taken (and raises Storehouse.ReadError when the store fails then). Optional. See Storehouse.list/2.

multipart_abort(key, t)

(optional)
@callback multipart_abort(key(), String.t()) :: :ok | {:error, term()}

Gives up an upload: its parts are deleted.

multipart_complete(key, t, list)

(optional)
@callback multipart_complete(key(), String.t(), [part()]) :: :ok | {:error, term()}

Puts the parts together: the file is then under its key.

multipart_parts(key, t)

(optional)
@callback multipart_parts(key(), String.t()) :: {:ok, [part()]} | {:error, term()}

The parts the store has of an upload: {:ok, parts}, by number.

multipart_put(key, t, pos_integer, binary)

(optional)
@callback multipart_put(key(), String.t(), pos_integer(), binary()) ::
  {:ok, part()} | {:error, term()}

Sends part number of an upload from the app itself (Storehouse.Tus.Plug), with its SHA-256, which the store checks: {:ok, part}.

multipart_sign(key, t, pos_integer, keyword)

(optional)
@callback multipart_sign(key(), String.t(), pos_integer(), keyword()) ::
  {:ok, %{method: String.t(), url: String.t(), headers: map()}}
  | {:error, term()}

Signs the browser's PUT of part number, with its SHA-256 in Base64 (:checksum) when it is given, so that the store refuses other bytes, and its size (:bytes) when it is given, so that it refuses a larger part: {:ok, %{method: "PUT", url: …, headers: %{…}}}.

multipart_start(key, meta)

(optional)
@callback multipart_start(key(), meta()) :: {:ok, String.t()} | {:error, term()}

Starts a multipart upload of a file under key, of meta's type, that the browser sends in parts straight to the store (Storehouse.presign_multipart/4): {:ok, upload_id}. Optional, with the other multipart_ callbacks: Storehouse.S3 has them.

multipart_uploads(prefix)

(optional)
@callback multipart_uploads(prefix :: String.t()) ::
  {:ok, [%{key: key(), upload_id: String.t(), started_at: DateTime.t()}]}
  | {:error, term()}

The uploads under prefix started and neither completed nor given up, with their :key, :upload_id and :started_at: for the sweeper.

presign_upload(key, meta, opts)

(optional)
@callback presign_upload(key(), meta(), opts :: keyword()) ::
  {:ok, upload()} | {:error, term()}

An upload of the file with key that a browser makes itself, without the store's keys, valid opts[:expires_in] seconds: of meta.bytes bytes exactly, of the type meta.content_type. Optional: a store without it takes its files through the app (Storehouse.store/2). See Storehouse.presign_upload/4.

put(key, path, meta)

@callback put(key(), path :: Path.t(), meta()) ::
  {:ok, url :: String.t()} | {:error, term()}

Stores the file at path under key, and returns its URL.

The file at path is a temporary file: the adapter copies it, and keeps no reference to it.

read_head(key, bytes)

(optional)
@callback read_head(key(), bytes :: pos_integer()) ::
  {:ok, head :: binary(), size :: non_neg_integer()} | {:error, term()}

The first bytes bytes of the file with key (all of it, when it is shorter), and its size: to check a file uploaded straight to the store. Optional; needed with presign_upload/3.

read_range(key, first, last)

(optional)
@callback read_range(key(), first :: non_neg_integer(), last :: non_neg_integer()) ::
  {:ok, binary()} | {:error, term()}

The bytes of the file with key from first to last, both included (fewer, when the file ends sooner; "" past its end): what Storehouse.read/2, Storehouse.stream/2 and Storehouse.download/3 read the file with, a part at a time. A missing file gives {:error, :enoent}. Optional, with head/1.

signed_url(key, opts)

(optional)
@callback signed_url(key(), opts :: keyword()) :: {:ok, String.t()} | {:error, term()}

A URL of the file with key that is valid opts[:expires_in] seconds, for a file that is not public. opts[:disposition] is "inline" or "attachment", and opts[:filename] the name a download gets, when given. Optional. See Storehouse.signed_url/2.

stream_abort(stream)

(optional)
@callback stream_abort(stream :: term()) :: :ok

Gives up a streamed file: nothing of it stays in the store.

stream_close(stream)

(optional)
@callback stream_close(stream :: term()) :: {:ok, url :: String.t()} | {:error, term()}

Ends a streamed file, which is then in the store: its URL.

stream_open(key, meta)

(optional)
@callback stream_open(key(), meta()) :: {:ok, stream :: term()} | {:error, term()}

Opens a file with key that its bytes are written to as they come, in stream_write/2: an upload streamed through the app, with no file of its own on the disk (Storehouse.Live.allow/3, mode: :stream). Optional, with the three other stream_ callbacks.

stream_write(stream, iodata)

(optional)
@callback stream_write(stream :: term(), iodata()) ::
  {:ok, stream :: term()} | {:error, term()}

Writes the next bytes of a streamed file.

uploaded(key)

(optional)
@callback uploaded(key()) :: :ok

Told that a file the browser uploaded straight to the store (presign_upload/3) was checked and kept (Storehouse.verify_upload/2): for a store that does more with a file it takes (Storehouse.Mirror copies it to its mirrors). Optional.

url(key)

@callback url(key()) :: String.t()

Returns the URL of the file with key.