Storehouse.Mirror (Storehouse v0.2.0)

Copy Markdown View Source

A store that keeps its files in a primary store and copies them to others, its mirrors: to move to another provider without stopping (new files go to both, the old ones are copied, then the app reads from the new one), or to keep a second copy in another provider or region.

defmodule MyApp.Media do
  use Storehouse.Mirror, primary: MyApp.S3Media, mirrors: [MyApp.R2Media]
end

config :storehouse, adapter: MyApp.Media

It is a Storehouse.Adapter, used wherever a store is:

  • Writes go to the primary: a file stored, streamed, uploaded straight from a browser (Storehouse.verify_upload/2) or promoted. Once the primary has it, each mirror is given a copy of it (Storehouse.copy/2), out of the request.
  • Deletes go to the primary, then to each mirror.
  • Reads, URLs and uploads straight from a browser are the primary's.

A mirror that fails

A mirror's copy or deletion never fails the primary's: it is a job of the app's runner (Storehouse.Runner), tried again until the mirror answers. With Oban's (Storehouse.Runner.Oban), the jobs are kept in the database and outlive a deploy; with Storehouse.Runner.Memory they live in a process of the app's, and those not done when the app stops are not done. Storehouse.copy_prefix/2 (or mix storehouse.copy) copies what a mirror is missing, and only that, to catch it up.

Moving to another store

  1. Write to both: use Storehouse.Mirror, primary: Old, mirrors: [New].
  2. Copy the files there were: mix storehouse.copy --from Old --to New.
  3. Read from the new one: primary: New, mirrors: [Old], while the old one is still wanted, then New alone.

Options

  • :primary - the store files are written to and read from.
  • :mirrors - the stores given a copy of each.
  • :runner - the runner of its jobs, in place of the app's (Storehouse.Runner).