linux对erlang的otp工程进行打包和部署

By AverageJoeWang
 标签:

这个是一个典型的erlangOTP工程,以下来解释相关目录和打包文件,本文以uv_collector项目名称作为示例

典型的erlangOTP工程

一.OTP目录解释

  • .idea和.rebar不是OTP的,本人用的是idea编译器,此处用于存放编译器相关语法字典等;erlang是用rebar进行编译的,此处存放rebar的erlcinfo信息

  • deps是存放该工程的依赖项

  • doc是用于存储该工程的document文档
  • ebin是该工程src目录下的代码编译后的beam文件的存放位置
  • log是用于存放编译运行日志信息
  • priv存储该工程的private内容,比如erlang需要调用python接口等等,那么python就放在priv目录下
  • src目录是我们的工作目录,存放项目的代码

二.工程打包

在打包的时候本人一般是在linux环境下进行打包的,那么linux的的环境很简单,就只需要安装rebar和erlang就行了

1.linux环境相关

  • rebar安装,按照一下链接下载和安装,注意是在root权限下进行配置

    https://github.com/AverageJoeWang/rebar
    
  • erlang,如果是ubuntu16.04LTS版本按照以下命令直接安装就行了(原谅本人在学习erlang的时候这个新版本已经出来,我就没去折腾其他版本的环境了)

sudo apt-get update
sudo apt-get install erlang

2.打包文件

在打包的时候会涉及到一下一些文件

Makefile

这个文件相信大家不会陌生,就是linux下进行自动化打包的文件

node_name := uv_collector

release:
    rm -rf test/logs
    rm -rf build
    rm -f test/*.beam
    rebar clean
    rebar compile
    mkdir -p ./build/$(node_name);cp -rf ./ebin ./build/$(node_name)/;cp -rf ./priv ./build/$(node_name)/;cp -rf ./include ./build/$(node_name)/
    mkdir -p ./rel;cd ./rel;rm -rf *;rebar create-node nodeid=$(node_name);cp ../reltool.config ./;rebar generate;tar -cjf $(node_name).tar.bz2 $(node_name)
    mkdir -p ../build_linux
    mv rel/$(node_name).tar.bz2 ../build_linux

app.cinfig

看名字后缀就知道是配置文件,该文件的容是改项目的配置项

[{uv_collector,
  [    {http_port, 8099},
    {oss_domain, "oss-cn-shenzhen.aliyuncs.com"},
    {oss_bucket, "q4-sz-record"},
    {oss_access_key_id, "btX9Qks0VQvmAo3Z"},
    {oss_access_key_secret, "C16VRLu2Ydse1f702Hz3aPy42nbRhX"},
    {cdn_list, [qukan, netcenter, dnion]},
    {erl_odbc_connect_str,"DSN=qukan4_c"}
  ]},
  {lager, [
    %% Note: application:start(lager) overwrites previously defined environment variables
    %%       thus declaration of default handlers is done at lager_app.erl

    %% What colors to use with what log levels
    {colored, false},
    {colors, [
      {debug, "\e[0;38m"},
      {info, "\e[1;37m"},
      {notice, "\e[1;36m"},
      {warning, "\e[1;33m"},
      {error, "\e[1;31m"},
      {critical, "\e[1;35m"},
      {alert, "\e[1;44m"},
      {emergency, "\e[1;41m"}

    ]},
    {handlers,
      [{lager_console_backend, debug},
        {lager_file_backend,
          [{file, "log/error.log"},
            {level, error},
            {size, 10485760},
            {date, "$D0"},
            {count, 5}
          ]},
        {lager_file_backend,
          [{file, "log/console.log"},
            {level, debug},
            {size, 10485760},
            {date, "$D0"},
            {count, 5}]}
      ]},
    %% Whether to write a crash log, and where. Undefined means no crash logger.
    {crash_log, "log/crash.log"},
    %% Maximum size in bytes of events in the crash log - defaults to 65536
    {crash_log_msg_size, 65536},
    %% Maximum size of the crash log in bytes, before its rotated, set
    %% to 0 to disable rotation - default is 0
    {crash_log_size, 10485760},
    %% What time to rotate the crash log - default is no time
    %% rotation. See the README for a description of this format.
    {crash_log_date, "$D0"},
    %% Number of rotated crash logs to keep, 0 means keep only the
    %% current one - default is 0
    {crash_log_count, 5},
    %% Whether to redirect error_logger messages into lager - defaults to true
    {error_logger_redirect, true},
    %% How many messages per second to allow from error_logger before we start dropping them
    {error_logger_hwm, 50},
    %% How big the gen_event mailbox can get before it is switched into sync mode
    {async_threshold, 20},
    %% Switch back to async mode, when gen_event mailbox size decrease from `async_threshold'
    %% to async_threshold - async_threshold_window
    {async_threshold_window, 5}
  ]}
].

rebar.config

这个文件也是一个配置文件,是用来从svn或者git相关的版本管理中获取deps的依赖工程,因为公司用的是svn,这里仅仅使用内部的链接

%% Erlang compiler options
%{erl_opts, [no_debug_info]}.
{erl_opts, [{parse_transform, lager_transform}]}.

%% Whether to enable coverage reporting. Default is `false'
{cover_enabled, true}.

%% Whether to print coverage report to console. Default is `false'
{cover_print_enabled, true}.

%% Whether to export coverage report to file. Default is `false'
{cover_export_enabled, true}.

%% Override the default "logs" directory in which SUITEs are logged
{ct_log_dir, "test/logs"}.

%% Option to pass extra parameters when launching Common Test
%{ct_extra_params, "-boot start_sasl -s myapp"}.

%% Where to put any downloaded dependencies. Default is "deps"
{deps_dir, "deps"}.

%% What dependencies we have, dependencies can be of 3 forms, an application
%% name as an atom, eg. mochiweb, a name and a version (from the .app file), or
%% an application name, a version and the SCM details on how to fetch it (SCM
%% type, location and revision).
%% Rebar currently supports git, hg, bzr, svn, and rsync.
{deps, [
  %% Dependencies can be marked as 'raw'. Rebar does not require
  %% such dependencies to have a standard Erlang/OTP layout
  %% which assumes the presence of either
  %% "src/dependency_name.app.src" or "ebin/dependency_name.app"
  %% files.
  %%
  %% 'raw' dependencies can still contain 'rebar.config' and
  %% even can have the proper OTP directory layout, but they
  %% won't be compiled.
  %%
  %% Only a subset of rebar commands will be executed on the
  %% 'raw' subdirectories: get-deps, update-deps, check-deps,
  %% list-deps and delete-deps.
  %%{protobuffs, "", {git, "https://github.com/basho/erlang_protobuffs.git", {branch, "master"}}},
  {lager, "", {svn, "http://192.168.1.85:9880/VideoServer/trunk/opensource/lager", 'HEAD'}},
  {meck, "", {svn, "http://192.168.1.85:9880/VideoServer/trunk/opensource/meck", 'HEAD'}},
  {qukan_lib, "", {svn, "http://192.168.1.85:9880/VideoServer/trunk/qk_common/qukan_lib", 'HEAD'}},
  {qk_odbc_pool, "", {svn, "http://192.168.1.85:9880/VideoServer/trunk/qk_common/qk_odbc_pool", 'HEAD'}},
  {qk_json, "", {svn, "http://192.168.1.85:9880/VideoServer/trunk/qk_common/qk_json", 'HEAD'}},
  {qk_ipdb, "", {svn, "http://192.168.1.85:9880/VideoServer/trunk/qk_common/qk_ipdb", 'HEAD'}},
  {rfc4627_jsonrpc, "", {svn, "http://192.168.1.85:9880/VideoServer/trunk/opensource/rfc4627_jsonrpc", 'HEAD'}},
  {goldrush, "", {svn, "http://192.168.1.85:9880/VideoServer/trunk/opensource/goldrush", 'HEAD'}}
%%  {mysql, "", {svn, "http://192.168.1.85:9880/VideoServer/trunk/opensource/mysql", 'HEAD'}}
]}.

reltool.config

用来配置工程需要用到的相关sys信息,同时定义在rebar.config在获取了deps依赖项之后实际在打包过程中需要include的工程

%% -*- mode: erlang -*-
%% ex: ft=erlang
{sys, [
  {lib_dirs, ["../deps", "../build"]},
  {erts, [{mod_cond, derived}, {app_file, strip}]},
  {app_file, strip},
  {rel, "uv_collector", "1",
    [
      kernel,
      stdlib,
      lager,
      meck,
      inets,
      qk_odbc_pool,
      qk_ipdb,
      uv_collector
    ]},
  {rel, "start_clean", "",
    [
      kernel,
      stdlib
    ]},
  {boot_rel, "uv_collector"},
  {profile, embedded},
  {incl_cond, derived},
  {excl_archive_filters, [".*"]}, %% Do not archive built libs
  {excl_sys_filters, ["^bin/(?!start_clean.boot)",
    "^erts.*/bin/(dialyzer|typer)",
    "^erts.*/(doc|info|include|lib|man|src)"]},
  {excl_app_filters, ["\.gitignore"]},
  {app, kernel, [{incl_cond, include}]},
  {app, stdlib, [{incl_cond, include}]},
  {app, inets, [{incl_cond, include}]},
  {app, goldrush, [{incl_cond, include}]},
  {app, lager, [{incl_cond, include}]},
  {app, mysql, [{incl_cond, exclude}]},
  {app, qukan_lib, [{incl_cond, include}]},
  {app, hipe, [{incl_cond, exclude}]},
  {app, qk_odbc_pool, [{incl_cond, include}]},
  {app, rfc4627_jsonrpc, [{incl_cond, include}]},
  {app, qk_ipdb, [{incl_cond, include}]},
  {app, qk_json, [{incl_cond, include}]},
  {app, uv_collector, [{mod_cond, app}, {incl_cond, include}]}
]}.

{target_dir, "uv_collector"}.

{overlay, [
  {mkdir, "log/sasl"},
  {copy, "files/erl", "\{\{erts_vsn\}\}/bin/erl"},
  {copy, "files/nodetool", "\{\{erts_vsn\}\}/bin/nodetool"},
  {copy, "uv_collector/bin/start_clean.boot",
    "\{\{erts_vsn\}\}/bin/start_clean.boot"},
  {copy, "files/uv_collector", "bin/uv_collector"},
  {copy, "../uv_collector.cmd", "bin/uv_collector.cmd"},
  {copy, "files/start_erl.cmd", "bin/start_erl.cmd"},
  {copy, "files/install_upgrade.escript", "bin/install_upgrade.escript"},
  {copy, "../app.config", "etc/app.config"},
  {copy, "../vm.args", "etc/vm.args"}
]}.

uv_collector.cmd

@setlocal

@set node_name=uv_collector

@rem Get the absolute path to the parent directory,
@rem which is assumed to be the node root.
@for /F "delims=" %%I in ("%~dp0..") do @set node_root=%%~fI

@set releases_dir=%node_root%\releases

@rem Parse ERTS version and release version from start_erl.data
@for /F "usebackq tokens=1,2" %%I in ("%releases_dir%\start_erl.data") do @(
    @call :set_trim erts_version %%I
    @call :set_trim release_version %%J
)

@set vm_args=%node_root%\etc\vm.args
@set sys_config=%node_root%\etc\app.config
@set node_boot_script=%releases_dir%\%release_version%\%node_name%
@set clean_boot_script=%releases_dir%\%release_version%\start_clean

@rem extract erlang cookie from vm.args
@for /f "usebackq tokens=1-2" %%I in (`findstr /b \-setcookie "%vm_args%"`) do @set erlang_cookie=%%J

@set erts_bin=%node_root%\erts-%erts_version%\bin

@set service_name=%node_name%_%release_version%

@set erlsrv="%erts_bin%\erlsrv.exe"
@set epmd="%erts_bin%\epmd.exe"
@set escript="%erts_bin%\escript.exe"
@set werl="%erts_bin%\werl.exe"
@set nodetool="%erts_bin%\nodetool"

@if "%1"=="usage" @goto usage
@if "%1"=="install" @goto install
@if "%1"=="uninstall" @goto uninstall
@if "%1"=="start" @goto start
@if "%1"=="stop" @goto stop
@if "%1"=="restart" @call :stop && @goto start
@if "%1"=="console" @goto console
@if "%1"=="ping" @goto ping
@if "%1"=="query" @goto query
@if "%1"=="attach" @goto attach
@if "%1"=="upgrade" @goto upgrade
@echo Unknown command: "%1"

:usage
@echo Usage: %~n0 [install^|uninstall^|start^|stop^|restart^|console^|ping^|query^|attach^|upgrade]
@goto :EOF

:install
@set description=Erlang node %node_name% in %node_root%
@set start_erl=%node_root%\bin\start_erl.cmd
@set args= ++ %node_name% ++ %node_root%
@%erlsrv% add %service_name% -c "%description%" -sname %node_name% -w "%node_root%" -m "%start_erl%" -args "%args%" -stopaction "init:stop()."
@goto :EOF

:uninstall
@%erlsrv% remove %service_name%
@%epmd% -kill
@goto :EOF

:start
@%erlsrv% start %service_name%
@goto :EOF

:stop
@%erlsrv% stop %service_name%
@goto :EOF

:console
@start "%node_name% console" %werl% -boot "%node_boot_script%" -config "%sys_config%" -args_file "%vm_args%" -sname %node_name%
@goto :EOF

:ping
@%escript% %nodetool% ping -sname "%node_name%" -setcookie "%erlang_cookie%"
@exit %ERRORLEVEL%
@goto :EOF

:query
@%erlsrv% list %service_name%
@exit %ERRORLEVEL%
@goto :EOF

:attach
@for /f "usebackq" %%I in (`hostname`) do @set hostname=%%I
start "%node_name% attach" %werl% -boot "%clean_boot_script%" -remsh %node_name%@%hostname% -sname console -setcookie %erlang_cookie%
@goto :EOF

:upgrade
@if "%2"=="" (
    @echo Missing upgrade package argument
    @echo Usage: %~n0 upgrade {package base name}
    @echo NOTE {package base name} MUST NOT include the .tar.gz suffix
    @goto :EOF
)
@%escript% %node_root%\bin\install_upgrade.escript %node_name% %erlang_cookie% %2
@goto :EOF

:set_trim
@set %1=%2
@goto :EOF

vm.args

这个文件在打包之后会在最后打包解压之后的etc目录下,用于设置工程在运行时候的附加功能,-heart如果不注释的话则在linux的进程中使用kill是关闭不了相应的服务的,而只能去这个项目下的bin目录的可执行文件+stop才能停止

## Name of the node
-name uv_collector@127.0.0.1

## Cookie for distributed erlang
-setcookie uv_collector

## Heartbeat management; auto-restarts VM if it dies or becomes unresponsive
## (Disabled by default..use with caution!)
##-heart

## Enable kernel poll and a few async threads
##+K true
##+A 5

## Increase number of concurrent ports/sockets
##-env ERL_MAX_PORTS 4096

## Tweak GC to run more often
##-env ERL_FULLSWEEP_AFTER 10

-hidden

三.打包

将windows下的工程放到linux下,进入这个工程

cd uv_colletcor/
mkdir priv include
make

之后在uv_collector同级的目录的build_linux目录下回有一个uv_collector的tar文件,那么你可以利用scp -p uv_collector.tar.bz2 root@ip:目录进行复制到远程服务器,然后利用tar -vxf uv_collector.tar.bz2进行解压

四.解压之后目录分析

解压之后会有bin,erts-7.3,etc,lib,log,releases这6个目录

  • bin是相当于调用目录

进入bin目录之后会有可执行文件,相关参数如下

/uv_collector 
Usage: uv_collector {start|start_boot <file>|foreground|stop|restart|reboot|ping|console|getpid|console_clean|console_boot <file>|attach|eval|remote_console|upgrade}
  • erts-7.3是存放项目的beam等相关文件脚本,erlang自带的库等

  • etc是配置文件存放目录,存放app.config,vm.args文件

  • lib存放需要用到的依赖库

  • log日志目录

  • releases版本信息和scripts脚本相关