Jekyll2026-07-12T01:04:06+00:00https://sgeos.github.io/feed/elixir.xmlBrendan A R Sechter’s Development Blog | ElixirA personal technical notebook covering systems programming, systems philosophy, tooling, mathematics, and emerging software paradigms. Brendan SechterStoring Elixir Release Configuration in Environment Variables with Distillery2016-09-18T06:31:36+00:002016-09-18T06:31:36+00:00https://sgeos.github.io/phoenix/elixir/erlang/ecto/distillery/postgresql/mysql/2016/09/18/storing-elixir-release-configuration-in-environment-variables-with-distillery

This post covers storing settings like database configuration in environment variables with distillery. A prior post covered this with the Elixir Release Manager (exrm).

Software Versions

$ date -u "+%Y-%m-%d %H:%M:%S +0000"
2016-09-18 06:31:36 +0000
$ uname -vm
FreeBSD 12.0-CURRENT #0 r304324: Thu Aug 18 13:27:23 JST 2016     root@mirage.sennue.com:/usr/obj/usr/src/sys/MIRAGE_KERNEL  amd64
$ mix hex.info
Hex:    0.13.0
Elixir: 1.3.2
OTP:    19.0.7
* snip *
$ mix phoenix.new -v
Phoenix v1.2.1
$ cat mix.lock | grep distillery | cut -d" " -f 3,6 | sed 's/[",]//g'
distillery: 0.9.9

Creating a Sample Project

Create a new Phoenix project.

mix phoenix.new phoenix_environment_settings --no-brunch
cd phoenix_environment_settings
mix ecto.create

Create a simple JSON memo endpoint.

mix phoenix.gen.json Memo memos title:string body:string

Revise web/router.ex. Uncomment the “/api” scope and add the “/memos” route.

web/router.ex.partial listing

  scope "/api", PhoenixEnvironmentSettings do
    pipe_through :api

    resources "/memos", MemoController, except: [:new, :edit]
  end

Run the migration.

mix ecto.migrate

Tests should pass.

mix test

Start the server.

mix phoenix.server

POST and GET a memo to make sure the server works. A prior post covers a shell script for conveniently interacting with this sample app.

# POST new
curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "New Title", "body": "This is the new memo body."}}' http://localhost:4000/api/memos
# GET id 1
curl -H 'Content-Type: application/json' http://localhost:4000/api/memos/1

Environment Variable Definitions

Define the following environment variables. Feel free to use different values and a static value for SECRET_KEY_BASE. Note that depending on how your database is configured, DB_PASSWORD may be unnecessary for connections from localhost.

export NODE_NAME=leaf_node
export COOKIE=thin_mints
export DB_USER=phoenix_environment_settings
export DB_PASSWORD=phoenix_environment_settings_password
export DB_NAME=phoenix_environment_settings_prod
export DB_HOST=localhost
export HOST=host.example.org
export PORT=7777
export SECRET_KEY_BASE=$(elixir -e ":crypto.strong_rand_bytes(48) |> Base.encode64 |> IO.puts")

Configuring the Application

Set server to true in config/prod.exs. Also add a version entry. Replace static configuration with dynamic configuration that will be pulled in from environment variables. None of this needs to be kept out version control, so it is safe to delete config/prod.secret.exs. Those settings have been merged in.

config/prod.exs

use Mix.Config

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Endpoint,
  http: [port: {:system, "PORT"}],
  url: [host: "${HOST}", port: {:system, "PORT"}],
  cache_static_manifest: "priv/static/manifest.json",
  server: true,
  root: ".",
  version: Mix.Project.config[:version]

config :logger, level: :info

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Endpoint,
  secret_key_base: "${SECRET_KEY_BASE}"

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: "${DB_USER}",
  password: "${DB_PASSWORD}",
  database: "${DB_NAME}",
  hostname: "${DB_HOST}",
  pool_size: 20

Alternatively, keep hard coded values in config/prod.secret.exs when running local release builds and use a version with dynamic environment variable settings in a real production environment.

config/prod.secret.exs

use Mix.Config

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Endpoint,
  secret_key_base: "${SECRET_KEY_BASE}"

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: "${DB_USER}",
  password: "${DB_PASSWORD}",
  database: "${DB_NAME}",
  hostname: "${DB_HOST}",
  pool_size: 20

Optionally, add dynamic configuration with a default values to config/dev.exs. The environment variable replacement above is only suitable for releases. The interpreted solution below is only suitable for development with mix.

config/dev.exs partial listing

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Endpoint,
  http: [port: System.get_env("PORT") || 4000],

# ... at the end of the file ...

# Configure your database
config :phoenix_environment_settings, PhoenixEnvironmentSettings.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: System.get_env("DB_USER") || "postgres",
  password: System.get_env("DB_PASSWORD") || "postgres",
  database: System.get_env("DB_NAME") || "phoenix_environment_settings_dev",
  hostname: System.get_env("DB_HOST") || "localhost",
  pool_size: 10

Generating a Release

Add distillery to mix.exs as a project dependency.

mix.exs partial listing

  defp deps do
    [{:phoenix, "~> 1.2.1"},
     {:phoenix_pubsub, "~> 1.0"},
     {:phoenix_ecto, "~> 3.0"},
     {:postgrex, ">= 0.0.0"},
     {:phoenix_html, "~> 2.6"},
     {:phoenix_live_reload, "~> 1.0", only: :dev},
     {:gettext, "~> 0.11"},
     {:distillery, "~> 0.9"}, # this line is new
     {:cowboy, "~> 1.0"}]
  end

Install and initialize distillery. This will create the rel/ directory.

mix deps.get
mix deps.compile
mix release.init

Environment variables will be used as knobs to configure the app. Note that the REPLACE_OS_VARS=true environment variable needs to be defined to use environment variables for dynamic configuration.

The rel/vm.args file is primarily used to configure the erlang VM. It can also be used to define application configuration parameters. Application configuration parameters defined in this file can be passed into the program as atoms or integers. Note that the location of this file can be configured with the RELEASE_CONFIG_DIR environment variable. Add the following to rel/vm.args.

rel/vm.args

## Name of the node
-name ${NODE_NAME}

## Cookie for distributed erlang
-setcookie ${COOKIE}

## App Settings
-phoenix_environment_settings port ${PORT}

Alternatively, rel/sys.config can be used to pass in application configuration parameters. This file is written in Erlang.

rel/sys.config

[
  {phoenix_environment_settings, [
    {port, "${PORT}"}
  ]}
].

The Elixir config/config.exs file is probably a better place to define non-VM settings for an Elixir application. The distillery documentation seems to indicate that a release can either use sys.config or config.exs but not both. The exact config.exs settings for this project were covered in the Configuring the Application section above.

Adding Release Tasks

After deploying a release with distillery, mix is no longer available. Define a module for release tasks with a mix ecto.migrate equivalent. Note that this is an Erlang module written in Elixir.

lib/release_tasks.ex

defmodule :release_tasks do
  def migrate do
    {:ok, _} = Application.ensure_all_started(:phoenix_environment_settings)
    path = Application.app_dir(:phoenix_environment_settings, "priv/repo/migrations")
    Ecto.Migrator.run(PhoenixEnvironmentSettings.Repo, path, :up, all: true)
    :init.stop()
  end
end

After generating a release, this task can be run as follows.

REPLACE_OS_VARS=true rel/phoenix_environment_settings/bin/phoenix_environment_settings command release_tasks migrate

The mix ecto.create and mix ecto.drop tasks are run less frequently so it may just makes sense to just manually create or drop the database and user. PostgreSQL commands follow.

# PostgreSQL `mix ecto.create` equivalent
psql -c "CREATE USER ${DB_USER} WITH PASSWORD '${DB_PASSWORD}';"
createdb "${DB_NAME}"
psql -c "GRANT ALL PRIVILEGES ON DATABASE ${DB_NAME} to ${DB_USER};"

# PostgreSQL `mix ecto.drop` equivalent
dropdb "${DB_NAME}"
dropuser "${DB_USER}"

# PostgresSQL interactive terminal
PGPASSWORD="${DB_PASSWORD}" psql -U "${DB_USER}" "${DB_NAME}"

The MySQL commands look like this.

# MySQL `mix ecto.create` equivalent
mysql -e "CREATE USER '${DB_USER}'@'localhost' IDENTIFIED BY '${DB_PASSWORD}';"
mysql -e "CREATE DATABASE ${DB_NAME};"
mysql -e "GRANT ALL PRIVILEGES ON ${DB_NAME}.* TO '${DB_USER}'@'localhost' IDENTIFIED BY '${DB_PASSWORD}';"
mysql -e "FLUSH PRIVILEGES;"

# MySQL `mix ecto.drop` equivalent
mysql -e "DROP DATABASE ${DB_NAME};"
mysql -e "DROP USER '${DB_USER}'@'localhost';"

# MySQL interactive terminal
mysql -u"${DB_USER}" -p"${DB_PASSWORD}" "${DB_NAME}"

Note that the password security in the above commands is less than ideal. Also, an existing superuser and password may need to be explicily specified when creating the new user and database.

Adding Custom Commands for Release Tasks

Distillery supports custom commands. To wrap the above release tasks in custom commands, first create the rel/commands directory.

mkdir rel/commands

Next add a command for ecto.migrate.

rel/commands/ecto_migrate

#!/usr/bin/env sh

# mix ecto.migrate` equivalent
"${SCRIPT}" command release_tasks migrate

The PostgreSQL versions of the ecto.create and ecto.drop commands follow.

rel/commands/ecto_create

#!/usr/bin/env sh

# PostgreSQL `mix ecto.create` equivalent
psql -c "CREATE USER ${DB_USER} WITH PASSWORD '${DB_PASSWORD}';" &&
createdb "${DB_NAME}" &&
psql -c "GRANT ALL PRIVILEGES ON DATABASE ${DB_NAME} to ${DB_USER};" &&
echo "The database '${DB_NAME}' and role '${DB_USER}' have been created."

rel/commands/ecto_drop

#!/usr/bin/env sh

# PostgreSQL `mix ecto.drop` equivalent
dropdb "${DB_NAME}" &&
dropuser "${DB_USER}" &&
echo "The database '${DB_NAME}' and role '${DB_USER}' have been dropped."

Alternatively, the MySQL versions of ecto.create and ecto.drop look like this.

rel/commands/ecto_create

#!/usr/bin/env sh

# MySQL `mix ecto.create` equivalent
mysql -e "CREATE USER '${DB_USER}'@'localhost' IDENTIFIED BY '${DB_PASSWORD}';" &&
mysql -e "CREATE DATABASE ${DB_NAME};" &&
mysql -e "GRANT ALL PRIVILEGES ON ${DB_NAME}.* TO '${DB_USER}'@'localhost' IDENTIFIED BY '${DB_PASSWORD}';" &&
mysql -e "FLUSH PRIVILEGES;" &&
echo "The database '${DB_NAME}' and role '${DB_USER}' have been created."

rel/commands/ecto_drop

#!/usr/bin/env sh

# MySQL `mix ecto.drop` equivalent
mysql -e "DROP DATABASE ${DB_NAME};" &&
mysql -e "DROP USER '${DB_USER}'@'localhost';" &&
echo "The database '${DB_NAME}' and role '${DB_USER}' have been dropped."

Finally, modify rel/config.exs so that the custom commands are present. The following version is a rewrite.

rel/config.exs

use Mix.Releases.Config,
    default_release: :default,
    default_environment: :prod

environment :prod do
  set include_erts: true
  set include_src: false
  set commands: [
    "ecto.migrate": "rel/commands/ecto_migrate",
    "ecto.create": "rel/commands/ecto_create",
    "ecto.drop": "rel/commands/ecto_drop"
  ]
end

release :phoenix_environment_settings do
  set version: current_version(:phoenix_environment_settings)
end

Running the Release

Build the release with the configuration files.

# rm -rf rel/phoenix_environment_settings/ # optional, no stale release files
MIX_ENV=prod mix compile
# brunch build --production # if using brunch
MIX_ENV=prod mix phoenix.digest
MIX_ENV=prod mix release --env=prod

Run the ecto.create and ecto.migrate custom commands defined in the above sections.

REPLACE_OS_VARS=true rel/phoenix_environment_settings/bin/phoenix_environment_settings ecto.create
REPLACE_OS_VARS=true rel/phoenix_environment_settings/bin/phoenix_environment_settings ecto.migrate

Start the release.

REPLACE_OS_VARS=true rel/phoenix_environment_settings/bin/phoenix_environment_settings start

Make sure the server responds.

curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "Memo A", "body": "Alpha memo body."}}' "http://localhost:${PORT}/api/memos"
curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "Memo B", "body": "Beta memo body."}}' "http://localhost:${PORT}/api/memos"
curl "http://localhost:${PORT}/api/memos"

Stop the release.

REPLACE_OS_VARS=true rel/phoenix_environment_settings/bin/phoenix_environment_settings start

Custom Application Settings

Add this configuration gist to lib/config.ex. This configuration wrapper allows the same convenient {:system, "VARIABLE", "default"} convention to be used with both mix and releases. Note that this will not help configure things like PhoenixEnvironmentSettings.Repo because they were not written to get settings via this module. The linked to version has the typespecs and documentation.

lib/config.ex

defmodule Config do
  def get(app, key, default \\ nil) when is_atom(app) and is_atom(key) do
    case Application.get_env(app, key) do
      {:system, env_var} ->
        case System.get_env(env_var) do
          nil -> default
          val -> val
        end
      {:system, env_var, preconfigured_default} ->
        case System.get_env(env_var) do
          nil -> preconfigured_default
          val -> val
        end
      nil ->
        default
      val ->
        val
    end
  end

  def get_integer(app, key, default \\ nil) do
    case get(app, key, nil) do
      nil -> default
      n when is_integer(n) -> n
      n ->
        case Integer.parse(n) do
          {i, _} -> i
          :error -> default
        end
    end
  end
end

Add these lines to config/config.exs.

config/config.exs partial listing

config :phoenix_environment_settings,
  ecto_repos: [PhoenixEnvironmentSettings.Repo],
  welcome_message: {:system, "WELCOME_MESSAGE", "Hello, world!"},
  magic_number: {:system, "MAGIC_NUMBER", 42}

Access the settings in iex -S mix like this. You should get the default values.

iex -S mix

Config.get :phoenix_environment_settings, :welcome_message
Config.get_integer :phoenix_environment_settings, :magic_number

Define the environment variables and run the commands in iex again. You should get the environment variable values this time.

export WELCOME_MESSAGE="Welcome. Try, but there is no escape."
export MAGIC_NUMBER=-1

This is useful for defining environment variable knobs to control run time behavior. It is not a solution for problems that rely on compile time behavior, like using environment variables to specify dynamic routes.

Other Considerations

Note that when using REPLACE_OS_VARS=true, the environment variables in rel/sys.conf or config/config.exs will always be replaced with strings. The following almost certainly does not work as expected.

export DB_POOL_SIZE=20

config/prod.exs or config/prod.secret.exs partial listing

  pool_size: "${DB_POOL_SIZE}"

The following will work in development, but not production.

config/dev.exs partial listing

  pool_size: (System.get_env("DB_POOL_SIZE") || "10") |> String.to_integer

If integers or atoms need to be passed in directly, use vm.args. The author could not figure out how to pass DB_POOL_SIZE to Repo via vm.args.

Next Steps

Consider looking into conform, a library for working with init-style configuration. Also consider looking into edeliver for deployment.

References:

]]>Brendan SechterElixir/Erlang, Running OTP Observer Remotely2016-09-16T14:17:43+00:002016-09-16T14:17:43+00:00https://sgeos.github.io/elixir/erlang/observer/2016/09/16/elixir_erlang_running_otp_observer_remotely

This post covers running the OTP’s observer on a node running on a remote machine. It is a step-by-step of this gist.

Software Versions

$ date -u "+%Y-%m-%d %H:%M:%S +0000"
2016-09-16 14:17:43 +0000

# localhost
$ uname -vm
Darwin Kernel Version 15.6.0: Mon Aug 29 20:21:34 PDT 2016; root:xnu-3248.60.11~1/RELEASE_X86_64 x86_64
$ mix hex.info
Hex:    0.13.0
Elixir: 1.3.2
OTP:    19.0
* snip *

# remotehost
$ uname -vm
FreeBSD 12.0-CURRENT #0 r304324: Thu Aug 18 13:27:23 JST 2016     root@mirage.sennue.com:/usr/obj/usr/src/sys/MIRAGE_KERNEL  amd64
$ mix hex.info
Hex:    0.13.0
Elixir: 1.3.2
OTP:    19.0.7
* snip *

Instructions

On the remote host, start a node with known name and cookie values. iex or erl will be used for this example.

sh remotehost

IF=em0
HOST=$(ifconfig ${IF} | grep 'inet ' | cut -d ' ' -f 2)
NAME="wild_exhibisionist@${HOST}"
COOKIE="banana_chocolate_chip_nut_butter"
printf "NAME='${NAME}'\nCOOKIE='${COOKIE}'\n"
iex --name "${NAME}" --cookie "${COOKIE}"
# OR
erl -name "${NAME}" -setcookie "${COOKIE}"

From the localhost, ssh into the remote host and run epmd.

sh localhost

# use your own remote host
REMOTE_HOST=192.168.1.14
ssh "${REMOTE_HOST}" "epmd -names"

With the previously started node running, the output should look something like this.

sh localhost

epmd: up and running on port 4369 with data:
name wild_exhibisionist at port 48341

Note the ports that empd and the node you want to debug are using. Reconnect to the remote host with these ports forwarded.

sh localhost

# below NODE_PORT setting assumes there is only one node running
EPMD_PORT=$(ssh "${REMOTE_HOST}" "epmd -names" | grep epmd | sed 's/[^0-9]//g')
NODE_PORT=$(ssh "${REMOTE_HOST}" "epmd -names" | grep name | sed 's/[^0-9]//g')
printf "REMOTE_HOST='${REMOTE_HOST}'\nEPMD_PORT='${EPMD_PORT}'\nNODE_PORT='${NODE_PORT}'\n"
ssh -L "${EPMD_PORT}:${REMOTE_HOST}:${EPMD_PORT}" -L "${NODE_PORT}:${REMOTE_HOST}:${NODE_PORT}" "${REMOTE_HOST}"

Start a hidden node running the observer app.

sh localhost

# both interfaces probably need to be on the same network
IF=en0
HOST=$(ifconfig ${IF} | grep 'inet ' | cut -d ' ' -f 2)
NAME="observer@${HOST}"
# use the same cookie value
COOKIE="banana_chocolate_chip_nut_butter"
printf "NAME='${NAME}'\nCOOKIE='${COOKIE}'\n"
iex --name "${NAME}" --cookie "${COOKIE}" --hidden -e ":observer.start"
# OR
erl -name "${NAME}" -setcookie "${COOKIE}" -hidden -run observer

In observer, go to Nodes -> Connect Node and enter the name of the remote node.

If you are having trouble connecting, try to manually connect the nodes from the iex or erl prompt. If the nodes do not connect manually, solve that problem. After the nodes have been manually connected, go to Nodes -> Connect Node and enter the name of the remote node in observer.

iex localhost

Node.connect :"wild_exhibisionist@192.168.1.14"

iex remotehost

Node.connect :"observer@192.168.1.2"

erl localhost

net_kernel:connect('wild_exhibisionist@192.168.1.14').

erl remotehost

net_kernel:connect('observer@192.168.1.2').

Use the following to get a remote shell.

sh localhost

REMOTE_NODE="wild_exhibisionist@192.168.1.14"
iex --name "${NAME}" --cookie "${COOKIE}" --remsh "${REMOTE_NODE}"
# OR
erl -name "${NAME}" -setcookie "${COOKIE}" -remsh "${REMOTE_NODE}"

Note that :runtime_tools must be in the applications list to observe an app released with exrm.

mix.exs partial listing

  def application do
    [mod: {MyApp, []},
     applications: [:runtime_tools]] # add to this list
  end

References:

]]>Brendan SechterStoring Elixir Release Configuration in Environment Variables with exrm2016-09-11T18:09:35+00:002016-09-11T18:09:35+00:00https://sgeos.github.io/phoenix/elixir/erlang/ecto/exrm/postgresql/mysql/2016/09/11/storing-elixir-release-configuration-in-environment-variables

This post covers storing settings like database configuration in environment variables.

Software Versions

$ date -u "+%Y-%m-%d %H:%M:%S +0000"
2016-09-11 18:09:35 +0000
$ uname -vm
FreeBSD 12.0-CURRENT #0 r304324: Thu Aug 18 13:27:23 JST 2016     root@mirage.sennue.com:/usr/obj/usr/src/sys/MIRAGE_KERNEL  amd64
$ mix hex.info
Hex:    0.13.0
Elixir: 1.3.2
OTP:    19.0.5
* snip *
$ mix phoenix.new -v
Phoenix v1.2.1
$ cat mix.lock | grep exrm | cut -d" " -f 3,6 | sed 's/[",]//g'
exrm: 1.0.8

Creating a Sample Project

Create a new Phoenix project.

mix phoenix.new phoenix_environment_settings --no-brunch
cd phoenix_environment_settings
mix ecto.create

Create a simple JSON memo endpoint.

mix phoenix.gen.json Memo memos title:string body:string

Revise web/router.ex. Uncomment the “/api” scope and add the “/memos” route.

web/router.ex.partial listing

  scope "/api", PhoenixEnvironmentSettings do
    pipe_through :api

    resources "/memos", MemoController, except: [:new, :edit]
  end

Run the migration.

mix ecto.migrate

Tests should pass.

mix test

Start the server.

mix phoenix.server

POST and GET a memo to make sure the server works. A prior post covers a shell script for conveniently interacting with this sample app.

# POST new
curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "New Title", "body": "This is the new memo body."}}' http://localhost:4000/api/memos
# GET id 1
curl -H 'Content-Type: application/json' http://localhost:4000/api/memos/1

Environment Variable Definitions

Define the following environment variables. Feel free to use different values and a static value for SECRET_KEY_BASE. Note that depending on how your database is configured, DB_PASSWORD may be unnecessary for connections from localhost.

export NODE_NAME=leaf_node
export COOKIE=thin_mints
export DB_USER=phoenix_environment_settings
export DB_PASSWORD=phoenix_environment_settings_password
export DB_NAME=phoenix_environment_settings_prod
export DB_HOST=localhost
export HOST=host.example.org
export PORT=7777
export SECRET_KEY_BASE=$(elixir -e ":crypto.strong_rand_bytes(48) |> Base.encode64 |> IO.puts")

Configuring the Application

Set server to true in config/prod.exs. Replace static configuration with dynamic configuration that will be pulled in from environment variables. None of this needs to be kept out version control, so it is safe to delete config/prod.secret.exs. Those settings have been merged in.

config/prod.exs

use Mix.Config

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Endpoint,
  http: [port: {:system, "PORT"}],
  url: [host: "${HOST}", port: {:system, "PORT"}],
  cache_static_manifest: "priv/static/manifest.json",
  server: true,
  root: ".",
  version: Mix.Project.config[:version]

config :logger, level: :info

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Endpoint,
  secret_key_base: "${SECRET_KEY_BASE}"

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: "${DB_USER}",
  password: "${DB_PASSWORD}",
  database: "${DB_NAME}",
  hostname: "${DB_HOST}",
  pool_size: 20

Alternatively, keep hard coded values in config/prod.secret.exs when running local release builds and use a version with dynamic environment variable settings in a real production environment.

config/prod.secret.exs

use Mix.Config

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Endpoint,
  secret_key_base: "${SECRET_KEY_BASE}"

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: "${DB_USER}",
  password: "${DB_PASSWORD}",
  database: "${DB_NAME}",
  hostname: "${DB_HOST}",
  pool_size: 20

Optionally, add dynamic configuration with a default values to config/dev.exs. The environment variable replacement above is only suitable for releases. The interpreted solution below is only suitable for development with mix.

config/dev.exs partial listing

config :phoenix_environment_settings, PhoenixEnvironmentSettings.Endpoint,
  http: [port: System.get_env("PORT") || 4000],

# ... at the end of the file ...

# Configure your database
config :phoenix_environment_settings, PhoenixEnvironmentSettings.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: System.get_env("DB_USER") || "postgres",
  password: System.get_env("DB_PASSWORD") || "postgres",
  database: System.get_env("DB_NAME") || "phoenix_environment_settings_dev",
  hostname: System.get_env("DB_HOST") || "localhost",
  pool_size: 10

Adding Release Tasks

After deploying a release with exrm, mix is no longer available. Define a module for release tasks with a mix ecto.migrate equivalent. Note that this is an Erlang module written in Elixir.

lib/release_tasks.ex

defmodule :release_tasks do
  def migrate do
    {:ok, _} = Application.ensure_all_started(:phoenix_environment_settings)
    path = Application.app_dir(:phoenix_environment_settings, "priv/repo/migrations")
    Ecto.Migrator.run(PhoenixEnvironmentSettings.Repo, path, :up, all: true)
    :init.stop()
  end
end

After generating a release, this task can be run as follows.

RELX_REPLACE_OS_VARS=true rel/phoenix_environment_settings/bin/phoenix_environment_settings command release_tasks migrate

The mix ecto.create and mix ecto.drop tasks are run less frequently so it probably just makes sense to just manually create or drop the database and user. PostgreSQL commands follow.

# PostgreSQL `mix ecto.create` equivalent
psql -c "CREATE USER ${DB_USER} WITH PASSWORD '${DB_PASSWORD}';"
createdb "${DB_NAME}"
psql -c "GRANT ALL PRIVILEGES ON DATABASE ${DB_NAME} to ${DB_USER};"

# PostgreSQL `mix ecto.drop` equivalent
dropdb "${DB_NAME}"
dropuser "${DB_USER}"

# PostgresSQL interactive terminal
PGPASSWORD="${DB_PASSWORD}" psql -U "${DB_USER}" "${DB_NAME}"

The MySQL commands look like this.

# MySQL `mix ecto.create` equivalent
mysql -e "CREATE USER '${DB_USER}'@'localhost' IDENTIFIED BY '${DB_PASSWORD}';"
mysql -e "CREATE DATABASE ${DB_NAME};"
mysql -e "GRANT ALL PRIVILEGES ON ${DB_NAME}.* TO '${DB_USER}'@'localhost' IDENTIFIED BY '${DB_PASSWORD}';"
mysql -e "FLUSH PRIVILEGES;"

# MySQL `mix ecto.drop` equivalent
mysql -e "DROP DATABASE ${DB_NAME};"
mysql -e "DROP USER '${DB_USER}'@'localhost';"

# MySQL interactive terminal
mysql -u"${DB_USER}" -p"${DB_PASSWORD}" "${DB_NAME}"

Run the mix ecto.create equivalent now.

Note that the password security in the above commands is less than ideal. Also, an existing superuser and password may need to be explicily specified when creating the new user and database.

Generating a Release

Add the elixir release manager (exrm) to mix.exs as a project dependency.

mix.exs partial listing

  defp deps do
    [{:phoenix, "~> 1.2.1"},
     {:phoenix_pubsub, "~> 1.0"},
     {:phoenix_ecto, "~> 3.0"},
     {:postgrex, ">= 0.0.0"},
     {:phoenix_html, "~> 2.6"},
     {:phoenix_live_reload, "~> 1.0", only: :dev},
     {:gettext, "~> 0.11"},
     {:exrm, "~> 1.0.8"}, # this line is new
     {:cowboy, "~> 1.0"}]
  end

Install exrm and build a release. This will create the rel/ directory.

mix deps.get
mix deps.compile
MIX_ENV=prod mix compile
# brunch build --production # if using brunch
MIX_ENV=prod mix phoenix.digest
MIX_ENV=prod mix release

Run the migration task defined in the Adding Release Tasks section.

RELX_REPLACE_OS_VARS=true rel/phoenix_environment_settings/bin/phoenix_environment_settings command release_tasks migrate

Environment variables will be used as knobs to configure the app. Note that the RELX_REPLACE_OS_VARS=true environment variable needs to be defined to use environment variables for dynamic configuration.

The rel/vm.args file is primarily used to configure the erlang VM. It can also be used to define application configuration parameters. Application configuration parameters defined in this file can be passed into the program as atoms or integers. Note that the location of this file can be configured with the RELEASE_CONFIG_DIR environment variable. Add the following to rel/vm.args.

rel/vm.args

## Name of the node
-name ${NODE_NAME}

## Cookie for distributed erlang
-setcookie ${COOKIE}

## App Settings
-phoenix_environment_settings port ${PORT}

Alternatively, rel/sys.config can be used to pass in application configuration parameters. This file is written in Erlang.

rel/sys.config

[
  {phoenix_environment_settings, [
    {port, "${PORT}"}
  ]}
].

The Elixir config/config.exs file is probably a better place to define non-VM settings for an Elixir application. It is ultimately merged with rel/sys.config. The exact settings for this project were covered in the Configuring the Application section above.

Rebuild the release with the configuration files.

MIX_ENV=prod mix compile
# brunch build --production # if using brunch
MIX_ENV=prod mix phoenix.digest
MIX_ENV=prod mix release

Start the release in the console.

RELX_REPLACE_OS_VARS=true rel/phoenix_environment_settings/bin/phoenix_environment_settings console

Make sure the server responds.

curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "Memo A", "body": "Alpha memo body."}}' "http://localhost:${PORT}/api/memos"
curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "Memo B", "body": "Beta memo body."}}' "http://localhost:${PORT}/api/memos"
curl "http://localhost:${PORT}/api/memos"

Exit the console with ^C.

Custom Application Settings

Add this configuration gist to lib/config.ex. This configuration wrapper allows the same convenient {:system, "VARIABLE", "default"} convention to be used with both mix and releases. Note that this will not help configure things like PhoenixEnvironmentSettings.Repo because they were not written to get settings via this module. The linked to version has the typespecs and documentation.

lib/config.ex

defmodule Config do
  def get(app, key, default \\ nil) when is_atom(app) and is_atom(key) do
    case Application.get_env(app, key) do
      {:system, env_var} ->
        case System.get_env(env_var) do
          nil -> default
          val -> val
        end
      {:system, env_var, preconfigured_default} ->
        case System.get_env(env_var) do
          nil -> preconfigured_default
          val -> val
        end
      nil ->
        default
      val ->
        val
    end
  end

  def get_integer(app, key, default \\ nil) do
    case get(app, key, nil) do
      nil -> default
      n when is_integer(n) -> n
      n ->
        case Integer.parse(n) do
          {i, _} -> i
          :error -> default
        end
    end
  end
end

Add these lines to config/config.exs.

config/config.exs partial listing

config :phoenix_environment_settings,
  ecto_repos: [PhoenixEnvironmentSettings.Repo],
  welcome_message: {:system, "WELCOME_MESSAGE", "Hello, world!"},
  magic_number: {:system, "MAGIC_NUMBER", 42}

Access the settings in iex -S mix like this. You should get the default values.

iex -S mix

Config.get :phoenix_environment_settings, :welcome_message
Config.get_integer :phoenix_environment_settings, :magic_number

Define the environment variables and run the commands in iex again. You should get the environment variable values this time.

export WELCOME_MESSAGE="Welcome. Try, but there is no escape."
export MAGIC_NUMBER=-1

This is useful for defining environment variable knobs to control run time behavior. It is not a solution for problems that rely on compile time behavior, like using environment variables to specify dynamic routes.

Other Considerations

Note that when using RELX_REPLACE_OS_VARS=true, the environment variables in rel/sys.conf and config/config.exs will always be replaced with strings. The following almost certainly does not work as expected.

export DB_POOL_SIZE=20

config/prod.exs or config/prod.secret.exs partial listing

  pool_size: "${DB_POOL_SIZE}"

The following will work in development, but not production.

config/dev.exs partial listing

  pool_size: (System.get_env("DB_POOL_SIZE") || "10") |> String.to_integer

If integers or atoms need to be passed in directly, use vm.args. The author could not figure out how to pass DB_POOL_SIZE to Repo via vm.args.

Next Steps

Consider looking into distillery, the “new exrm” written in pure Elixir. Also consider looking into conform, a library for working with init-style configuration. For deployment, edeliver is worth looking at.

References:

]]>Brendan SechterPhoenix/Elixir, Adding prod.secret.exs.example to a git Repository2016-07-18T02:18:42+00:002016-07-18T02:18:42+00:00https://sgeos.github.io/phoenix/elixir/git/2016/07/18/phoenix-adding-prod-secret-exs-example-to-git-repository

prod.secret.exs is not added to the git repository by default. This is a good thing. The problem is that when a repository is cloned, prod.secret.exs is missing. The file can usually be copied over, but this is a hassle when possible. I’m sure there are people out there who can type up a new prod.secret.exs from scratch. I want an easier path.

This post covers adding a prod.secret.exs.example file to a project. This file can be used to regenerate prod.secret.exs with a new secret_key_base.

Software Versions

$ date -u "+%Y-%m-%d %H:%M:%S +0000"
2016-07-18 02:18:42 +0000
$ uname -vm
FreeBSD 11.0-ALPHA6 #0 r302384: Thu Jul  7 22:40:47 JST 2016     root@mirage.sennue.com:/usr/obj/usr/src/sys/MIRAGE_KERNEL  amd64
$ mix hex.info
Hex:    0.12.1
Elixir: 1.3.2
OTP:    18.3.4.1

Built with: Elixir 1.2.5 and OTP 18.3.3
* snip *
$ mix phoenix.new -v
Phoenix v1.1.4

Instructions

Create a new project.

mix phoenix.new memo_api
cd memo_api

Create a git repository and add the project to it.

git init
git add .
git commit -m "Initial commit."

Copy config/prod.secret.exs to config/prod.secret.exs.example.

cp config/prod.secret.exs config/prod.secret.exs.example

Change the secret_key_base to a value that is not sensitive. It should be a value that can easily be replaced by sed.

config/prod.secret.exs.example partial listing

# Regenerate config/prod.secret.exs with the following commands
#   $ SECRET_KEY_BASE=$(elixir -e ":crypto.strong_rand_bytes(48) |> Base.encode64 |> IO.puts")
#   $ sed "s|SECRET+KEY+BASE|$SECRET_KEY_BASE|" config/prod.secret.exs.example >config/prod.secret.exs
config :memo_api, MemoApi.Endpoint,
  secret_key_base: "SECRET+KEY+BASE"

Add config/prod.secret.exs.example to git.

git add config/prod.secret.exs.example
git commit -m "Added config/prod.secret.exs.example."

config/prod.secret.exs can be regenerated with the following commands

SECRET_KEY_BASE=$(elixir -e ":crypto.strong_rand_bytes(48) |> Base.encode64 |> IO.puts")
sed "s|SECRET+KEY+BASE|$SECRET_KEY_BASE|" config/prod.secret.exs.example >config/prod.secret.exs

References:

]]>Brendan SechterPhoenix as a Service on FreeBSD2016-03-21T15:48:51+00:002016-03-21T15:48:51+00:00https://sgeos.github.io/phoenix/elixir/erlang/freebsd/2016/03/21/phoenix-as-a-service-on-freebsd

In this post a Phoenix app will be installed as a service on FreeBSD. The goals are the same as the Elixir as a Service on FreeBSD post.

  • The app needs to be able to be rebuilt and the service restarted to reflect changes on a development machine.
  • The service needs to automatically start when the machine is booted.
  • The service sould work like any other service.
service phoenix_service start
service phoenix_service stop
service phoenix_service restart
service phoenix_service status

This post assumes Phoenix, Elixir and PostgreSQL are already installed on FreeBSD.

Software Versions

$ date -u "+%Y-%m-%d %H:%M:%S +0000"
2016-03-21 15:48:51 +0000
$ uname -vm
FreeBSD 11.0-CURRENT #0 r296925: Wed Mar 16 20:53:04 JST 2016     root@mirage.sennue.com:/usr/obj/usr/src/sys/MIRAGE_KERNEL  amd64
$ mix hex.info
Hex:    0.11.3
Elixir: 1.2.3
OTP:    18.2.4
* snip *
$ mix phoenix.new -v
Phoenix v1.1.4

Creating a Sample Project

Create a new Phoenix project.

mix phoenix.new phoenix_service --no-brunch
cd phoenix_service
mix ecto.create

This sample project needs to be simple. Let’s make a JSON memo service.

mix phoenix.gen.json Memo memos title:string body:string

Revise the web/router.ex file. The “/api” scope needs to be uncommented and the “/memos” route needs to be added.

web/router.ex file

defmodule PhoenixService.Router do
  use PhoenixService.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", PhoenixService do
    pipe_through :browser # Use the default browser stack
  
    get "/", PageController, :index
  end

  scope "/api", PhoenixService do
    pipe_through :api

    resources "/memos", MemoController, except: [:new, :edit]
  end
end

Run the migration.

mix ecto.migrate

Set server to true in config/prod.exs. Also make sure a dynamic port configuration is used.

config/prod.exs partial listing

config :phoenix_service, PhoenixService.Endpoint,
  http: [port: {:system, "PORT"}], # dynamic port configuration
  url: [host: "example.com", port: 80],
  cache_static_manifest: "priv/static/manifest.json", # added comma
  server: true # this line is new

Optionally, add a dynamic port configuration with a default value to config/dev.exs. The above build time solution is suitable for releases. The interpreted solution below is suitable for development.

config/dev.exs partial listing

  # http: [port: 4000], # old line 10
  http: [port: System.get_env("PORT") || Application.get_env(:phoenix_service, :port) || 4000],

Tests should pass

mix test

Start the server.

mix phoenix.server

POST and GET a memo to make sure the server works. A prior post covers a shell script for conveniently interacting with this sample app.

# POST new
curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "New Title", "body": "This is the new memo body."}}' http://localhost:4000/api/memos
# GET id 1
curl -H 'Content-Type: application/json' http://localhost:4000/api/memos/1

Generating a Release

Now that the Phoenix app is working, it is time to build a release. Add the elixir release manager (exrm) to mix.exs as a project dependency.

mix.exs partial listing

  defp deps do
    [{:phoenix, "~> 1.1.4"},
     {:postgrex, ">= 0.0.0"},
     {:phoenix_ecto, "~> 2.0"},
     {:phoenix_html, "~> 2.4"},
     {:phoenix_live_reload, "~> 1.0", only: :dev},
     {:gettext, "~> 0.9"},
     {:exrm, "~> 1.0.2"}, # this line is new
     {:cowboy, "~> 1.0"}]
  end

Install exrm and build a release. This will create the rel/ directory.

mix deps.get
mix deps.compile
MIX_ENV=prod mix ecto.create
MIX_ENV=prod mix ecto.migrate
MIX_ENV=prod mix compile
# brunch build --production # if using brunch
MIX_ENV=prod mix phoenix.digest
MIX_ENV=prod mix release

The rc script will use environment variable knobs to configure the app. Note that the RELX_REPLACE_OS_VARS=true environment variable needs to be defined to use environment variables for dynamic configuration.

The vm.args file is primarily used to configure the erlang VM. It can also be used to define application configure parameters. Application configuration parameters defined in this file can be passed into the program as atoms or integers. Note that the location of this file can be configured with the RELEASE_CONFIG_DIR environment variable. Add the following to rel/vm.args.

rel/vm.args

## Name of the node
-name ${NODE_NAME}

## Cookie for distributed erlang
-setcookie ${COOKIE}

## App Settings
-phoenix_service port ${PORT}

Alternatively, sys.config can be used to pass in application configuration parameters. In this file, application configuration parameters defined with environment variables must be strings. Pass the port setting in as above or add the following to rel/sys.config. The app module should work with either solution. Adding both files will not break anything. Note that rel/sys.config is written in Erlang.

rel/sys.config

[
  {phoenix_service, [
    {port, "${PORT}"}
  ]}
].

Rebuild the release with the configuration files.

MIX_ENV=prod mix release

Start the release in the console.

RELX_REPLACE_OS_VARS=true PORT=7777 rel/phoenix_service/bin/phoenix_service console

Make sure the server responds.

curl http://localhost:7777/api/memos

Exit the console with ^C.

Installing the Release as a Service

Now that the release is working, it is time to set it up as a service. Perform the initial install.

su
sh
PROJECT=phoenix_service
INSTALL_DIR=/usr/local/opt
VERSION=$(cat rel/${PROJECT}/releases/start_erl.data | cut -d' ' -f2)
INSTALL_TAR=`pwd`/rel/$PROJECT/releases/$VERSION/$PROJECT.tar.gz
mkdir -p $INSTALL_DIR/$PROJECT
(cd $INSTALL_DIR/$PROJECT; tar -xf $INSTALL_TAR)
pw adduser $PROJECT -d $INSTALL_DIR/$PROJECT -s /usr/sbin/nologin -c "$PROJECT system service user"
chown -R $PROJECT:$PROJECT $INSTALL_DIR/$PROJECT

An rc script defines the the service.

  • phoenix_service_run() is called from the other functions. It configures and calls the release. HOME is set to the installation directory to force the erlang cookie file to be written there regardless of the phoenix_service_user setting.
  • phoenix_service_status() echoes a user friendly message if the release can be pinged.
  • extra_commands is used to add the console and remote_console commands. console is used to start a new service and attach a console to it. remote_console() is used to connect a console to the service if it is already running.
  • Add shutdown to the keyword list if the service needs to gracefull shutdown when the machine restarts.
  • The rest is standard rc configuration.

Add following to /usr/local/etc/rc.d/phoenix_service

/usr/local/etc/rc.d/phoenix_service

#!/bin/sh
#
# PROVIDE: phoenix_service
# REQUIRE: networking
# KEYWORD:
 
. /etc/rc.subr
 
name="phoenix_service"
rcvar="${name}_enable"
install_dir="/usr/local/opt/${name}"
version=$(cat ${install_dir}/releases/start_erl.data | cut -d' ' -f2)
command="${install_dir}/bin/${name}"
 
start_cmd="${name}_start"
stop_cmd="${name}_stop"
status_cmd="${name}_status"
console_cmd="${name}_console"
remote_console_cmd="${name}_remote_console"
extra_commands="console remote_console"

load_rc_config $name
: ${phoenix_service_enable:="no"}
: ${phoenix_service_port:="4000"}
: ${phoenix_service_user:=${name}}
: ${phoenix_service_node_name:="${name}@127.0.0.1"}
: ${phoenix_service_cookie:="${name}"}
: ${phoenix_service_config_dir:="${install_dir}/releases/${version}/${name} start"}

phoenix_service_run()
{
  RELX_REPLACE_OS_VARS=true \
  HOME="${install_dir}" \
  RELEASE_CONFIG_DIR="${phoenix_service_config}" \
  NODE_NAME="${phoenix_service_node_name}" \
  COOKIE="${phoenix_service_cookie}" \
  PORT="${phoenix_service_port}" \
  su -m "$phoenix_service_user" -c "$command $1"
}

phoenix_service_start()
{
  phoenix_service_run start
}

phoenix_service_stop()
{
  phoenix_service_run stop
}

phoenix_service_status()
{
  ping_result=`phoenix_service_run ping`
  echo "${ping_result}"
  case "${ping_result}" in
    *pong*)
      echo "${name} is running."
      ;;
  esac
}

phoenix_service_console()
{
  phoenix_service_run console
}

phoenix_service_remote_console()
{
  phoenix_service_run remote_console
}

load_rc_config $name
run_rc_command "$1"

Make /usr/local/etc/rc.d/phoenix_service read only and executable.

chmod 555 /usr/local/etc/rc.d/phoenix_service

Enable and configure the service in /etc/rc.conf.

/etc/rc.conf lines to add

phoenix_service_enable="YES"
phoenix_service_port=8248
phoenix_service_node_name="suzaku"
phoenix_service_cookie="cookie-of-the-southern-flame"

The service can now be started. If the service is enabled, it will automatically start when the machine boots.

service phoenix_service start
# POST new
curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "Service Running", "body": "The Phoenix service is running on '"$(hostname)"'."}}' http://localhost:8248/api/memos
# GET id 1
curl -H 'Content-Type: application/json' http://localhost:8248/api/memos/1

Optional: Adding a Release to the Systemwide Path

Adding a release to the systemwide path is not necessary, but it can be convenient. Create a directory for the convenience pass through script.

mkdir -p $INSTALL_DIR/bin

PORT, NODE_NAME and COOKIE need default values because vm.args has no useful default fallbacks. There is no good way to automatically select a port, so 8080 is a hard coded default. Add the following script to /usr/local/opt/bin/phoenix_service.

/usr/local/opt/bin/phoenix_service

#!/bin/sh

SCRIPT=$(realpath $0)
BASENAME=$(basename $SCRIPT)
BASEDIR=$(dirname $SCRIPT)
COMMAND=$(realpath $BASEDIR/../$BASENAME/bin/$BASENAME)

: ${NODE_NAME:=${BASENAME}}
: ${COOKIE:=${BASENAME}}
: ${PORT:="8080"}

NODE_NAME=${NODE_NAME} \
COOKIE=${COOKIE} \
PORT=${PORT} \
RELX_REPLACE_OS_VARS=true \
$COMMAND "$@"

Make the script executable.

chmod +x $INSTALL_DIR/bin/$PROJECT

Add /usr/local/opt/bin to the global path in /etc/profile for sh, and /etc/csh.cshrc for csh. Consider updating the root path in /root/.cshrc.

/usr/local/opt/bin partial listing

PATH=/usr/local/opt/bin:$PATH
export PATH

/etc/csh.cshrc partial listing

set path=(/usr/local/opt/bin $path)

/root/.cshrc partial listing

set path = (/usr/local/opt/bin /sbin /bin /usr/sbin /usr/bin /usr/local/sbin /usr/local/bin $HOME/bin)

Update the path in the current shell if necessary.

# sh bash
source /etc/csh.cshrc
# csh tcsh
. /etc/profile

Fix permissions if you want to be able to run as any user. This may have security implications.

chmod 755 $INSTALL_DIR/$PROJECT/bin/$PROJECT
chmod 755 $INSTALL_DIR/$PROJECT/bin/nodetool
chmod 755 $INSTALL_DIR/$PROJECT/releases/$VERSION/$PROJECT.sh
mkdir -p $INSTALL_DIR/$PROJECT/log
chmod 777 $INSTALL_DIR/$PROJECT/log
chmod 777 $INSTALL_DIR/$PROJECT/log/*.*
mkdir -p $INSTALL_DIR/$PROJECT/tmp/erl_pipes/$PROJECT
chmod 777 $INSTALL_DIR/$PROJECT/tmp/erl_pipes/$PROJECT
mkdir -p $INSTALL_DIR/$PROJECT/running-config/
chmod 777 $INSTALL_DIR/$PROJECT/running-config/
chmod 666 $INSTALL_DIR/$PROJECT/running-config/*.*
chown -R $PROJECT:$PROJECT $INSTALL_DIR/$PROJECT

The release can now be conveniently controlled.

# start service
NODE_NAME=rudra COOKIE=treasure PORT=5678 phoenix_service start
# POST new
curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "Service Running", "body": "The Phoenix service is running on '"$(hostname)"'."}}' http://localhost:5678/api/memos
# GET id 1
curl -H 'Content-Type: application/json' http://localhost:5678/api/memos/1
# stop service
NODE_NAME=rudra COOKIE=treasure phoenix_service stop

Setup complete. Switch from root to a normal user.

exit # sh
exit # su

Updating

Casual updates on a development machine can be performed as follows.

# brunch build --production # if using brunch
MIX_ENV=prod mix phoenix.digest # if static assets could have changed
MIX_ENV=prod mix release
su
sh
PROJECT=phoenix_service
INSTALL_DIR=/usr/local/opt
VERSION=$(cat rel/${PROJECT}/releases/start_erl.data | cut -d' ' -f2)
INSTALL_TAR=`pwd`/rel/$PROJECT/releases/$VERSION/$PROJECT.tar.gz
(cd $INSTALL_DIR/$PROJECT; \
tar -xf $INSTALL_TAR)
chown -R $PROJECT:$PROJECT $INSTALL_DIR/$PROJECT
service $PROJECT restart
exit # sh
exit # su

Note that the path permissions will need to be fixed again if you added the release to the systemwide path and want to be able to run it as any user.

Troubleshooting

Make sure to set server to true in config/prod.exs. If the server is mysteriously not working, start the release with console to see error messages.

What Next?

Consider looking into edeliver for deployment. “edeliver is based on deliver and provides a bash script to build and deploy Elixir and Erlang applications and perform hot-code upgrades.”

References:

]]>Brendan SechterInstalling Phoenix, Elixir and PostgreSQL on FreeBSD2016-03-19T07:35:02+00:002016-03-19T07:35:02+00:00https://sgeos.github.io/phoenix/elixir/postgresql/freebsd/2016/03/19/installing-phoenix-elixir-and-postgresql-on-freebsd

This post covers installing Phoenix, Elixir and PostgreSQL on a clean installation of FreeBSD. The post assumes FreeBSD is installed and you have created an unprivileged user. FreeBSD installation instructions can be found here.

Software Versions

$ date -u "+%Y-%m-%d %H:%M:%S +0000"
2016-03-19 07:35:02 +0000
$ uname -vm
FreeBSD 11.0-CURRENT #0 r296925: Wed Mar 16 20:53:04 JST 2016     root@mirage.sennue.com:/usr/obj/usr/src/sys/MIRAGE_KERNEL  amd64
$ psql --version
psql (PostgreSQL) 9.4.6
$ mix hex.info
Hex:    0.11.3
Elixir: 1.2.3
OTP:    18.2.4
* snip *
$ mix phoenix.new -v
Phoenix v1.1.4

Bootstrapping the system

Login as root and install portmaster. If you are using packages instead of ports, you will need to install packages instead of using portmaster.

portsnap fetch extract
(cd /usr/ports/ports-mgmt/portmaster && make config-recursive && make config-recursive && make install clean)
portmaster security/sudo

The following utilities may also be useful. Note that ftp/curl is used at the end of the post.

portmaster ports-mgmt/portupgrade devel/git shells/bash editors/vim ftp/curl ftp/wget

Installing PostgreSQL

Install databases/postgresql94-server as root. If you would rather install a different version, do that instead.

portmaster databases/postgresql94-server

Initialize the database.

service postgresql initdb

If you want to enable remote connections, add the following line to /usr/local/pgsql/data/postgresql.conf.

listen_addresses = '*'

If you want to change the port PostgreSQL listens on, add the following line to /usr/local/pgsql/data/postgresql.conf. Replace 5432 with the port you want to use.

port = 5432                            # (change requires restart)

To use password hash authentication, add the following line to /usr/local/pgsql/data/pg_hba.conf. Replace 192.168.0.0/16 with your own network. Consider changing “trust” to “md5” for local and loopback IP address connections.

host    all             all             192.168.0.0/16          md5

Enable PostgreSQL in /etc/rc.conf.

echo 'postgresql_enable="YES"' >> /etc/rc.conf

Start PostgreSQL.

service postgresql start

Add a PostgreSQL super-user with database and role creation privileges. Replace username with your unprivileged FreeBSD user login.

su pgsql
createuser -sdrP username

You should now be able to start psql as your non-root user.

psql

Installing Elixir and Phoenix

Install Elixir and Phoenix.

sudo portmaster lang/elixir
mix local.hex
mix archive.install https://github.com/phoenixframework/archives/raw/master/phoenix_new.ez

Optionally, install npm and brunch.

sudo portmaster www/node www/npm
sudo npm install -g brunch

Creating a Sample Project

Create a new Phoenix project.

mix phoenix.new phoenix_service --no-brunch
cd phoenix_service
mix ecto.create

Create a simple JSON memo service.

mix phoenix.gen.json Memo memos title:string body:string

Revise the web/router.ex file. The “/api” scope needs to be uncommented and the “/memos” route needs to be added.

web/router.ex file

defmodule PhoenixService.Router do
  use PhoenixService.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", PhoenixService do
    pipe_through :browser # Use the default browser stack
  
    get "/", PageController, :index
  end

  scope "/api", PhoenixService do
    pipe_through :api

    resources "/memos", MemoController, except: [:new, :edit]
  end
end

Run the migration.

mix ecto.migrate

Tests should pass.

mix test

Running the Sample Project

Start the server.

mix phoenix.server

The following curl commands can be used to interact with the running JSON API. My previous post covers a shell script for interacting with this sample app.

# POST new
curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "New Title", "body": "This is the new memo body."}}' http://localhost:4000/api/memos
# PATCH id 1
curl -H 'Content-Type: application/json' -X PATCH -d '{"memo": {"title": "Patched Title"}}' http://localhost:4000/api/memos/1
curl -H 'Content-Type: application/json' -X PATCH -d '{"memo": {"body": "Patched memo body."}}' http://localhost:4000/api/memos/1
# PUT id 1
curl -H 'Content-Type: application/json' -X PUT -d '{"memo": {"title": "Updated Title", "body": "Updated memo body."}}' http://localhost:4000/api/memos/1
# GET all
curl -H 'Content-Type: application/json' http://localhost:4000/api/memos
# GET id 1
curl -H 'Content-Type: application/json' http://localhost:4000/api/memos/1
# DELETE id 1
curl -H 'Content-Type: application/json' -X DELETE http://localhost:4000/api/memos/1

References:

]]>Brendan SechterA Shell Script for Working with Phoenix JSON APIs2016-03-19T00:55:41+00:002016-03-19T00:55:41+00:00https://sgeos.github.io/phoenix/elixir/sh/2016/03/19/a-shell-script-for-working-with-phoenix-json-apis

I started writing another post, but wound up writing a non-trivial script for working with Phoenix JSON APIs. That script is covered in this post. It is written for the FreeBSD Bourne shell, but it should be compatible with bash.

This post also serves as a getopts example. getopts was used instead of getopt because getopt does not support command line arguments that contain spaces.

Software Versions

$ date -u "+%Y-%m-%d %H:%M:%S +0000"
2016-03-19 00:55:41 +0000
$ uname -vm
FreeBSD 11.0-CURRENT #0 r296925: Wed Mar 16 20:53:04 JST 2016     root@mirage.sennue.com:/usr/obj/usr/src/sys/MIRAGE_KERNEL  amd64
$ mix hex.info
Hex:    0.11.3
Elixir: 1.2.3
OTP:    18.2.4
* snip *
$ mix phoenix.new -v
Phoenix v1.1.4

Instructions

Create a new Phoenix project.

mix phoenix.new phoenix_service --no-brunch
cd phoenix_service
mix ecto.create

This example will use a JSON memo service.

mix phoenix.gen.json Memo memos title:string body:string

Revise the web/router.ex file. The “/api” scope needs to be uncommented and the “/memos” route needs to be added.

web/router.ex file

defmodule PhoenixService.Router do
  use PhoenixService.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", PhoenixService do
    pipe_through :browser # Use the default browser stack
  
    get "/", PageController, :index
  end

  scope "/api", PhoenixService do
    pipe_through :api

    resources "/memos", MemoController, except: [:new, :edit]
  end
end

Run the migration.

mix ecto.migrate

Tests should pass.

mix test

Running the Sample Project

Start the server.

mix phoenix.server

The following curl commands can be used to interact with the JSON API.

# POST new
curl -H 'Content-Type: application/json' -X POST -d '{"memo": {"title": "New Title", "body": "This is the new memo body."}}' http://localhost:4000/api/memos
# PATCH id 1
curl -H 'Content-Type: application/json' -X PATCH -d '{"memo": {"title": "Patched Title"}}' http://localhost:4000/api/memos/1
curl -H 'Content-Type: application/json' -X PATCH -d '{"memo": {"body": "Patched memo body."}}' http://localhost:4000/api/memos/1
# PUT id 1
curl -H 'Content-Type: application/json' -X PUT -d '{"memo": {"title": "Updated Title", "body": "Updated memo body."}}' http://localhost:4000/api/memos/1
# GET all
curl -H 'Content-Type: application/json' http://localhost:4000/api/memos
# GET id 1
curl -H 'Content-Type: application/json' http://localhost:4000/api/memos/1
# DELETE id 1
curl -H 'Content-Type: application/json' -X DELETE http://localhost:4000/api/memos/1

A script is easier to use if you need to make more than a couple of curl calls. It can also be extended to support authentication and everything else your API may need.

Create a script to GET, POST, PATCH, PUT and DELETE memos. Note that a Bourne Shell tutorial can be found here.

memo_api.sh

#!/bin/sh

reset() {
  HOST=http://localhost:4000
  SCOPE=api
  ROUTE=memos
  METHOD="GET"
  HEADERS="Content-Type: application/json"
  ID=""
  TITLE=""
  BODY=""
}

usage() {
  reset
  echo "Usage:  ${0} [options]"
  echo "Options:"
  echo "  -o HOST : set URL host, defaults to \"${HOST}\""
  echo "  -s SCOPE : set URL scope, defaults to \"${SCOPE}\""
  echo "  -r ROUTE : set URL route, defaults to \"${ROUTE}\""
  echo "  -X METHOD : set HTTP method, defaults to \"${METHOD}\""
  echo "  -H HEADERS : set HTTP headers, defaults to \"${HEADERS}\""
  echo "  -i ID : set memo id, defaults to \"${ID}\""
  echo "  -t TITLE : set memo title, defaults to \"${TITLE}\""
  echo "  -b BODY : set memo body, defaults to \"${BODY}\""
  echo "  -h : display this help"
  echo "Examples:"
  echo "  ${0} -X GET"
  echo "  ${0} -X GET -i 7"
  echo "  ${0} -X POST -t \"Memo Title\" -b \"Memo body here.\""
  echo "  ${0} -X PATCH -t \"Patched title.\" -i 7"
  echo "  ${0} -X PATCH -b \"Patched body.\" -i 7"
  echo "  ${0} -X PUT -t \"New Title\" -b \"New body.\" -i 7"
  echo "  ${0} -X DELETE -i 7"
  exit ${1}
}

reset
while getopts "o:s:r:X:H:i:t:b:h" opt
do
  case "${opt}" in
    o) HOST="${OPTARG}" ;;
    s) SCOPE="${OPTARG}" ;;
    r) ROUTE="${OPTARG}" ;;
    X) METHOD="${OPTARG}" ;;
    H) HEADERS="${OPTARG}" ;;
    i) ID="${OPTARG}" ;;
    t) TITLE="${OPTARG}" ;;
    b) BODY="${OPTARG}" ;;
    h) usage 1 ;;
    \?) usage 2 ;;
  esac
done
shift $(expr ${OPTIND} - 1)

case "${METHOD}" in
  GET)
    curl -H "${HEADERS}" -X ${METHOD} "${HOST}/${SCOPE}/${ROUTE}${ID:+"/${ID}"}"
    ;;
  POST)
    PAYLOAD='{"memo": {"title": "'"${TITLE:-(no title)}"'", "body": "'"${BODY:-(no body)}"'"}}'
    curl -H "${HEADERS}" -X ${METHOD} -d "${PAYLOAD}" "${HOST}/${SCOPE}/${ROUTE}" 
    ;;
  PUT)
    PAYLOAD='{"memo": {"title": "'"${TITLE:-(no title)}"'", "body": "'"${BODY:-(no body)}"'"}}'
    curl -H "${HEADERS}" -X ${METHOD} -d "${PAYLOAD}" "${HOST}/${SCOPE}/${ROUTE}/${ID:?'No ID specified.'}" 
    ;;
  PATCH)
    # if defined replace individual fields with
    # JSON fragments followed by a comma and space
    TITLE=${TITLE:+"\"title\": \"${TITLE}\", "}
    BODY=${BODY:+"\"body\": \"${BODY}\", "}
    # strip trailing comma and space
    PAYLOAD="$(echo "${TITLE}${BODY}" | sed 's/, $//g')"
    # complete JSON payload
    PAYLOAD="{\"memo\": {${PAYLOAD}}}"
    curl -H "${HEADERS}" -X ${METHOD} -d "${PAYLOAD}" "${HOST}/${SCOPE}/${ROUTE}/${ID:?'No ID specified.'}" 
    ;;
  DELETE)
    curl -H "${HEADERS}" -X ${METHOD} "${HOST}/${SCOPE}/${ROUTE}/${ID:?'No ID specified.'}"
    ;;
  *)
    usage 2
    ;;
esac
echo ""

The reset function resets the variables to their initial state. It is called before parsing command line arguments and displaying the usage.

The usage function displays usage, options and examples. It also terminates the script.

The getopts loop parses command line arguments into variables.

The case statement at the end uses the HTTP method to select a curl call.

  • The GET method works with or without an id.
  • The POST method creates a payload and posts it.
  • PUT is like post except the verb is different and it requires an id.
  • PATCH is the most complicated. It builds a payload and strips the trailing comma. Then it sends the payload to a URL with an id.
  • DELETE requires an id.

The script can be used as follows. Note that trying to delete a memo that does not exist will fail with a wall of HTML.

chmod +x memo_api.sh
./memo_api.sh -X POST -t "Memo Title" -b "Memo body here."
./memo_api.sh -X GET -i 1
./memo_api.sh -X POST -t "Another Memo" -b "This memo's body."
./memo_api.sh -X PATCH -t "Patched title." -i 2
./memo_api.sh -X PATCH -b "Patched body." -i 1
./memo_api.sh -X PUT -t "New Title" -b "New body." -i 2
./memo_api.sh -X GET
./memo_api.sh -X DELETE -i 1

This script could be customized to work with other APIs.

References:

]]>Brendan SechterWorking with mix project iex Code Snippets Stored in Files2016-03-14T17:08:09+00:002016-03-14T17:08:09+00:00https://sgeos.github.io/elixir/iex/2016/03/14/working_with_mix_project_iex_code_snippets_stored_in_files

iex can be used for interactively experimenting with elixir. If this experimentation becomes elaborate, it can be useful to store the commands in a file. This post covers working with files that store elixir code snippets. Attention is paid to code that depends on mix project modules.

Software Versions

$ date -u "+%Y-%m-%d %H:%M:%S +0000"
2016-03-14 17:08:09 +0000
$ uname -vm
FreeBSD 11.0-CURRENT #0 r296709: Sat Mar 12 21:18:38 JST 2016     root@mirage.sennue.com:/usr/obj/usr/src/sys/MIRAGE_KERNEL  amd64
$ elixir --version
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false]

Elixir 1.2.3

Instructions

A single line can be run from the command line.

elixir -e 'IO.puts("Hello, mix run!")' # general elixir code
mix run -e 'IO.puts("Hello, mix run!")' # project specific code

Something more elaborate probably wants to live in a file.

snippets/hello.exs

#!/usr/bin/env elixir

IO.puts "Hello, Elixir!"

The above file be run from the command line as is, but it will not have acces to modules in the mix project.

chmod +x snippets/hello.exs
./snippets/hello.exs

Run the file with mix run if any mix project modules are referenced.

mix run snippets/hello.exs

Note that c or Code.load_file can be used to load the file in iex.

c "snippets/hello.exs"
Code.load_file "snippets/hello.exs"

Code.load_file can be used to load one file from another file. This is fragile if using relative paths.

snippets/indirect.exs

#!/usr/bin/env elixir

Code.load_file "snippets/hello.exs"

To run mix code snippets directly from the command line, add the following file to some place on your path. For example, $(which mix)-snippet.

mix-snippet

#!/bin/sh

mix run "$1"

Make mix-snippet executable.

chmod +x mix-snippet # use the actual path

The mix-snippet shebang can now be used.

snippets/hello-mix-snippet.exs

#!/usr/bin/env mix-snippet

IO.puts "Hello, mix-snippet!"

Code snippets that depend on the mix project can now be executed from the command line.

chmod +x snippets/mix-snippet-hello.exs
./snippets/mix-snippet-hello.exs

References:

]]>Brendan SechterElixir, Executing AST2016-01-31T19:49:00+00:002016-01-31T19:49:00+00:00https://sgeos.github.io/elixir/metaprogramming/ast/2016/01/31/elixir-execute-ast

Inspecting AST is a good way to learn about elixir macros. Executing AST can also be instructional, but it is a little more nuanced than directly executing elixir code. This post covers executing AST with Code.eval_quoted(). Note that outside of experimentation, executing AST with Code.eval_quoted() is probably the wrong thing to do.

Thanks to everyone in #elixir-lang on Freenode who answered my questions about executing AST.

Software Versions

$ date 
February  1, 2016 at 04:49:00 AM JST
$ uname -vm
FreeBSD 11.0-CURRENT #0 r287598: Thu Sep 10 14:45:48 JST 2015     root@:/usr/obj/usr/src/sys/MIRAGE_KERNEL  amd64
$ elixir --version
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false]
Elixir 1.2.2

Instructions

This is a single file example of a naive solution that does not work. Put the following in ast.exs.

#!/usr/bin/env elixir

defmodule MacroLibrary do
  # a simple macro that delegates work to a complex function
  defmacro process_macro(input) do
    quote do
      process_function(unquote(input))
    end
  end

  # the complex function called by the macro
  def process_function(input) do
    input
    |> Enum.each(&IO.puts/1)
  end
end

defmodule Script do
  import MacroLibrary # important

  def main(args) do
    # call the macro directly
    process_macro(["Args:" | args])
    |> IO.puts

    # macro AST
    ast = quote do
      process_macro(["Args:" | args]) # BROKEN line 28
    end

    # display the AST
    ast
    |> Macro.to_string
    |> IO.puts

    # display the expanded AST
    ast
    |> Macro.expand(__ENV__)
    |> Macro.to_string
    |> IO.puts

    # execute the AST
    {result, _binding} = ast
    |> Code.eval_quoted # BROKEN line 44

    # display the result
    result
    |> IO.puts
  end
end

# run the script
Script.main(System.argv)

Make the script executable.

$ chmod +x ast.exs

This is the expected outputed.

$ ./ast.exs a b c
Args:
a
b
c
ok
process_macro(["Args:" | args])
process_function(["Args:" | args])
Args:
a
b
c
ok

This is the actual output.

$ ./ast.exs a b c
Args:
a
b
c
ok
process_macro(["Args:" | args])
process_function(["Args:" | args])
** (CompileError) nofile:1: undefined function process_function/1
    expanding macro: MacroLibrary.process_macro/1
    nofile:1: (file)

What happened?

Directly invoking the macro works as expected. The naive expectation is that the AST should just execute because the code is the same as the direct invocation.

The problem is that context is missing. The AST evaluates as process_macro([“Args:” | args]) which expands to process_function([“Args:” | args]). process_function is defined in the MacroLibrary module. The macro library is included in the Script module. That information is stored in the current context. Without the context, Code.eval_quoted does not know where to find process_function.

Change line 44 to the following:

    |> Code.eval_quoted([], __ENV__) # BROKEN line 44

There is another problem.

$ ./ast.exs a b c
Args:
a
b
c
ok
process_macro(["Args:" | args])
process_function(["Args:" | args])
** (CompileError) ast.exs:44: undefined function args/0
    expanding macro: MacroLibrary.process_macro/1
    ast.exs:44: Script (module)

When executing the macro directly, args resolves properly. Bindings are not part of the environment context and need to be passed in separately.

Change line 44 to the following:

    |> Code.eval_quoted([args: args], __ENV__) # CORRECT line 44

This did not fix the problem. The macro can be directly invoked with process_macro([“Args:” | args]), but executed AST needs to use var!() to access variables. This means that the direct macro call and the executable AST need to be slightly different.

Change line 28 to the following:

      process_macro(["Args:" | var!(args)]) # CORRECT line 28

The example finally works.

$ ./ast.exs a b c
Args:
a
b
c
ok
process_macro(["Args:" | var!(args)])
process_function(["Args:" | var!(args)])
Args:
a
b
c
ok

AST needs to be defined and executed as below.

# direct invocation
result = process_macro(["Args:" | args])

# indirect invocation
ast = quote do
  process_macro(["Args:" | var!(args)])
end

{result, _binding} = ast
|> Code.eval_quoted([args: args], __ENV__)

References:

]]>Brendan SechterElixir as a Service on FreeBSD2016-01-16T22:36:31+00:002016-01-16T22:36:31+00:00https://sgeos.github.io/elixir/erlang/2016/01/16/elixir-as-a-service_on_freebsd

Ease of deployment is one of the things I seriously consider when evaluating technology solutions. In this post, a modified version of the Elixir getting started echo server will be installed as a FreeBSD service. There are three goals.

  • The app needs to be able to be rebuilt and the service restarted to reflect changes on a development machine.
  • The service needs to automatically start when the machine is booted.
  • The service sould work like any other service.
service elixir_echo start
service elixir_echo stop
service elixir_echo restart
service elixir_echo status

The steps are a little involved, but ultimately straightforward.

Software Versions

$ date
January 17, 2016 at 07:37:31 PM JST
$ uname -a
FreeBSD mirage.sennue.com 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r287598: Thu Sep 10 14:45:48 JST 2015     root@:/usr/obj/usr/src/sys/MIRAGE_KERNEL  amd64
$ elixir --version
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false]
Elixir 1.2.0

Instructions

elixir needs to be installed. The tooling to install dependencies for elixir apps comes with the port.

sudo portmaster lang/elixir

First, start a new project for the echo service.

mix new --sup elixir_echo
cd elixir_echo

Next, add a modified version of the echo server from the Elixir getting started section. This version loads the port from environment variables or the app configuration environment. The default fallback is port 4040. It also gracefully handles closed sockets. The code goes in lib/echo_elixir.ex.

defmodule ElixirEcho do
  use Application
  require Logger

  @default_port 4040

  @doc false
  def start(_type, _args) do
    import Supervisor.Spec

    port = "PORT" |> get_env(@default_port) |> to_integer(@default_port)

    children = [
      supervisor(Task.Supervisor, [[name: ElixirEcho.TaskSupervisor]]),
      worker(Task, [ElixirEcho, :accept, [port]])
    ]

    opts = [strategy: :one_for_one, name: ElixirEcho.Supervisor]
    Supervisor.start_link(children, opts)
  end

  @doc """
  Starts accepting connections on the given `port`.
  """
  def accept(port) do
    {:ok, socket} = :gen_tcp.listen(port,
                      [:binary, packet: :line, active: false, reuseaddr: true])
    Logger.info "Accepting connections on port #{port}"
    loop_acceptor(socket)
  end

  defp loop_acceptor(socket) do
    {:ok, client} = :gen_tcp.accept(socket)
    {:ok, pid} = Task.Supervisor.start_child(ElixirEcho.TaskSupervisor, fn -> serve(client) end)
    :ok = :gen_tcp.controlling_process(client, pid)
    Logger.info "Opened connection"
    loop_acceptor(socket)
  end

  defp serve(socket) do
    status = socket
    |> read_line()
    |> write_line(socket)

    case status do
      :ok ->
        serve(socket)
      :closed ->
        Logger.info "Closed connection"
        :gen_tcp.close(socket)
        :ok
    end
  end

  defp read_line(socket) do
    case :gen_tcp.recv(socket, 0) do
      {:ok, data} ->
        data
      {:error, :closed} ->
        :closed
    end
  end

  defp write_line(:closed, _socket), do: :closed
  defp write_line(line, socket) do
    :gen_tcp.send(socket, line)
    Logger.info "Echo: #{line |> String.strip}"
    :ok
  end

  # Utility Functions

  def get_env(name, default) do
    case (
      System.get_env(name) ||
      Application.get_env(:elixir_echo, name |> String.downcase |> String.to_atom)
    ) |> to_string
    do
      "" -> default |> to_string
      result -> result
    end
  end

  def to_integer(x, _default) when is_integer(x), do: x
  def to_integer(x, default) when is_binary(x) do
    case x |> Integer.parse do
      {result, _} -> result
      :error -> default
    end
  end
  def to_integer(_, default), do: default
end

Add the elixir release manager (exrm) to mix.exs as a project dependency.

  defp deps do
    [{:exrm, "~> 1.0.0-rc7"}]
  end

Install exrm and build a release. This will create the rel/ directory.

mix deps.get
mix deps.compile
mix release

The rc script will use environment variable knobs to configure the app. Note that the RELX_REPLACE_OS_VARS=true environment variable needs to be defined to use environment variables for dynamic configuration.

The vm.args file is primarily used to configure the erlang VM. It can also be used to define application configure parameters. Application configuration parameters defined in this file can be passed into the program as atoms or integers. Add the following to rel/vm.args.

## Name of the node
-name ${NODE_NAME}

## Cookie for distributed erlang
-setcookie ${COOKIE}

## App Settings
-elixir_echo port ${PORT}

Alternatively, sys.config can be used to pass in application configuration parameters. In this file, application configuration parameters defined with environment variables must be strings. Pass the port setting in as above or add the following to rel/sys.config. The app module was written to work with either solution. Adding both files will not break anything.

[
  {elixir_echo, [
    {port, "${PORT}"}
  ]}
].

Build a release with the configuration files and do the initial install.

mix release
su
sh
PROJECT=elixir_echo
INSTALL_DIR=/usr/local/opt
VERSION=$(cat rel/${PROJECT}/releases/start_erl.data | cut -d' ' -f2)
INSTALL_TAR=`pwd`/rel/$PROJECT/releases/$VERSION/$PROJECT.tar.gz
mkdir -p $INSTALL_DIR/$PROJECT
(cd $INSTALL_DIR/$PROJECT; \
tar -xf $INSTALL_TAR)
pw adduser $PROJECT -d $INSTALL_DIR/$PROJECT -s /usr/sbin/nologin -c "$PROJECT system service user"
chown -R $PROJECT:$PROJECT $INSTALL_DIR/$PROJECT

An rc script defines the the service. elixir_echo_run() is called from the other functions. It configures and calls the release. HOME is set to the installation directory to force the erlang cookie file to be written there regardless of elixir_echo_user setting. elixir_echo_status() echoes a user friendly message if the release can be pinged. Add shutdown to the keyword list if the service needs to gracefull shutdown when the machine restarts. The rest is standard rc configuration. Add the rc script to /usr/local/etc/rc.d/elixir_echo

#!/bin/sh
#
# PROVIDE: elixir_echo
# REQUIRE: networking
# KEYWORD:
 
. /etc/rc.subr
 
name="elixir_echo"
rcvar="${name}_enable"
install_dir="/usr/local/opt/${name}"
version=$(cat ${install_dir}/releases/start_erl.data | cut -d' ' -f2)
command="${install_dir}/bin/${name}"
 
start_cmd="${name}_start"
stop_cmd="${name}_stop"
status_cmd="${name}_status"

load_rc_config $name
: ${elixir_echo_enable:="no"}
: ${elixir_echo_port:="4040"}
: ${elixir_echo_user:=${name}}
: ${elixir_echo_node_name:="${name}@127.0.0.1"}
: ${elixir_echo_cookie:="${name}"}
: ${elixir_echo_config_dir:="${install_dir}/releases/${version}/${name} start"}

elixir_echo_run()
{
  RELX_REPLACE_OS_VARS=true \
  HOME="${install_dir}" \
  RELEASE_CONFIG_DIR="${elixir_echo_config}" \
  NODE_NAME="${elixir_echo_node_name}" \
  COOKIE="${elixir_echo_cookie}" \
  PORT="${elixir_echo_port}" \
  su -m "$elixir_echo_user" -c "$command $1"
}

elixir_echo_start()
{
  elixir_echo_run start
}

elixir_echo_stop()
{
  elixir_echo_run stop
}

elixir_echo_status()
{
  ping_result=`elixir_echo_run ping`
  echo "${ping_result}"
  case "${ping_result}" in
    *pong*)
      echo "${name} is running."
      ;;
  esac
}

load_rc_config $name
run_rc_command "$1"

Enable and configure the service in /etc/rc.conf

elixir_echo_enable="YES"
elixir_echo_port=8255
elixir_echo_node_name="parrot"
elixir_echo_cookie="cracker"

The service can now be started. If the service is enabled, it will automatically start when the machine boots.

service elixir_echo start
telnet 127.0.0.1 8255 # make sure it works

Optional: Adding a Release to the Systemwide Path

Adding a release to the systemwide path is not necessary, but it can be convenient. The pass through script can be pointed at the development install instead of the service install if you want to build with exrm mix release –dev.

Create a directory for the convenience pass through script.

mkdir -p $INSTALL_DIR/bin

NODE_NAME and COOKIE need default values because vm.args has no useful default fallbacks. PORT has a fallback. Add the script to /usr/local/opt/bin/elixir_echo

#!/bin/sh
SCRIPT=$(realpath $0)
BASENAME=$(basename $SCRIPT)
BASEDIR=$(dirname $SCRIPT)
COMMAND=$(realpath $BASEDIR/../$BASENAME/bin/$BASENAME)

: ${NODE_NAME:=${BASENAME}}
: ${COOKIE:=${BASENAME}}
NODE_NAME=$NODE_NAME COOKIE=$COOKIE RELX_REPLACE_OS_VARS=true $COMMAND "$@"

Make the script executable.

chmod +x $INSTALL_DIR/bin/$PROJECT

Add /usr/local/opt/bin to the global path in /etc/profile for sh.

PATH=/usr/local/opt/bin:$PATH
export PATH

Add /usr/local/opt/bin to the global path in /etc/csh.cshrc for csh. Consider updating the root path in /root/.cshrc.

set path=(/usr/local/opt/bin $path)

Update the path in the current shell if necessary.

# sh bash
source /etc/csh.cshrc
# csh tcsh
. /etc/profile

Fix permissions if you want to be able to run as any user. This has security implications. It may make more sense to point the script at the development release if it is being used as a development convenience.

chmod 755 $INSTALL_DIR/$PROJECT/bin/$PROJECT
chmod 755 $INSTALL_DIR/$PROJECT/bin/nodetool
chmod 755 $INSTALL_DIR/$PROJECT/releases/$VERSION/$PROJECT.sh
mkdir -p $INSTALL_DIR/$PROJECT/log
chmod 777 $INSTALL_DIR/$PROJECT/log
chmod 777 $INSTALL_DIR/$PROJECT/log/*.*
mkdir -p $INSTALL_DIR/$PROJECT/tmp/erl_pipes/$PROJECT
chmod 777 $INSTALL_DIR/$PROJECT/tmp/erl_pipes/$PROJECT
mkdir -p $INSTALL_DIR/$PROJECT/running-config/
chmod 777 $INSTALL_DIR/$PROJECT/running-config/
chmod 666 $INSTALL_DIR/$PROJECT/running-config/*.*
chown -R $PROJECT:$PROJECT $INSTALL_DIR/$PROJECT

The release can now be conveniently controlled.

NODE_NAME=canary COOKIE=sesame PORT=5678 elixir_echo start
telnet 127.0.0.1 5678 # make sure it works

Setup Complete

Switch from root to a normal user.

exit # sh
exit # su

Updating

Casual updates on a development machine can be performed as follows.

mix release
su
sh
PROJECT=elixir_echo
INSTALL_DIR=/usr/local/opt
VERSION=$(cat rel/${PROJECT}/releases/start_erl.data | cut -d' ' -f2)
INSTALL_TAR=`pwd`/rel/$PROJECT/releases/$VERSION/$PROJECT.tar.gz
(cd $INSTALL_DIR/$PROJECT; \
tar -xf $INSTALL_TAR)
chown -R $PROJECT:$PROJECT $INSTALL_DIR/$PROJECT
service $PROJECT restart
exit # sh
exit # su

Troubleshooting

If configuration looks like it should be working, but nothing changes, try deleting the running-config directory. Sometimes rebooting fixes problems, like killing zombie nodes with strange default names and cookie values.

What Next?

Consider looking into edeliver for deployment. “edeliver is based on deliver and provides a bash script to build and deploy Elixir and Erlang applications and perform hot-code upgrades.”

References:

]]>Brendan Sechter