├── .gitignore ├── LICENSE ├── README.md ├── include └── fuge.hrl ├── rebar.config ├── rebar.lock ├── src ├── fuge.app.src ├── fuge.erl ├── fuge_app.erl ├── fuge_server.erl ├── fuge_subscriber.erl ├── fuge_subscriber_logger.erl └── fuge_sup.erl └── test └── fuge_test.erl /.gitignore: -------------------------------------------------------------------------------- 1 | .rebar3 2 | _* 3 | .eunit 4 | *.o 5 | *.beam 6 | *.plt 7 | *.swp 8 | *.swo 9 | .erlang.cookie 10 | ebin 11 | log 12 | erl_crash.dump 13 | .rebar 14 | logs 15 | _build 16 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016, Sukumar Yethadka. All rights reserved. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # fuge 2 | 3 | fuge is an Erlang library for carefully refactoring critical paths. Fuge 4 | allows you to implement [Branch by 5 | Abstraction](http://martinfowler.com/bliki/BranchByAbstraction.html) in 6 | production. 7 | 8 | ## Inspiration 9 | 10 | See the excellent [GitHub Scientist 11 | Post](http://githubengineering.com/scientist/) on why this pattern is useful. 12 | 13 | [Github Scientist Ruby library](https://github.com/github/scientist) 14 | 15 | ## Features 16 | 17 | Current set of features supported by fuge: 18 | 19 | * Run old code alongside one or multiple versions of new code. fuge runs all 20 | the versions of the code provided and provides results in a stable order. 21 | Only the result of the control code is returned. 22 | * Compare the results of each version. The output of each of version executed 23 | is available for comparison. 24 | * Get run time for each version. Run time allows us to compare and see which 25 | version of the code is faster. 26 | * The different code versions are run in a random order. This allows us to 27 | average out the advantage/disadvantage of the code's run order. 28 | * Subscriber model for publishing information. fuge is agnostic to how results 29 | are used afterwards and is extensible using subscribers. fuge also provides 30 | a way to maintain state for each subscriber, removing the necessity of 31 | making the subscriber a process with state. 32 | * Allows context information for every run. Context information can allow 33 | debugging the performance difference and it is passed on to the subscribers 34 | as is. 35 | * Control the frequency of how often the experiment is run. e.g. run the 36 | experiment 42% of the time. This allows us to run the experiment less 37 | frequently in cases when the overhead of running multiple versions is large. 38 | 39 | ## Warning 40 | 41 | Please keep the following in mind before using fuge in production! 42 | 43 | * fuge is currently in alpha stage and in heavy development. 44 | * fuge is to be run only on code without side effects and code that is 45 | idempotent. 46 | * While fuge itself has a very low overhead (one ETS read, timing information, 47 | one gen_server cast), running multiple versions of the code in production 48 | will have a performance hit. 49 | 50 | ## Usage 51 | 52 | We try to find the sum of consecutive numbers for illustration. 53 | 54 | ```erlang 55 | % Start fuge application 56 | 1> ok = application:start(fuge). 57 | ok 58 | % Create a new fuge 59 | 2> ok = fuge:new(my_fuge). 60 | ok 61 | % Control code 62 | 3> Control = fun () -> lists:sum(lists:seq(1, 1000)) end. 63 | #Fun 64 | % Candidate code 65 | 4> Candidate = fun() -> 1000 * (1000 + 1) div 2 end. 66 | #Fun 67 | % Run the experiment 68 | 5> fuge:run(my_fuge, Control, Candidate). 69 | 500500 70 | ``` 71 | 72 | Output with the default logging subscriber: 73 | 74 | ```erlang 75 | =INFO REPORT==== 18-Feb-2016::07:55:22 === 76 | Fuge: {fuge,my_fuge,[fuge_subscriber_logger],[]} 77 | =INFO REPORT==== 18-Feb-2016::07:55:22 === 78 | Result: {fuge_result,undefined, 79 | {fuge_data,81,500500}, 80 | [{fuge_data,52,500500}], 81 | [0,1]} 82 | ``` 83 | 84 | In the "Result" portion, we can see that the first version of the code ran in 85 | 81 microseconds vs the candidate's 52 microseconds. We can also see that the 86 | value "500500" returned by both the versions is the same. 87 | 88 | ## Planned Features 89 | 90 | * Subscribers for different use cases (e.g. graphite) 91 | 92 | ## Roadmap 93 | 94 | * Tests for edge cases 95 | * Benchmark 96 | * Run subscribers in separate process to avoid building a queue on fuge_server. 97 | * Maybe create an example application for clear understanding of usage. 98 | * Add options to fine tune the run. 99 | -------------------------------------------------------------------------------- /include/fuge.hrl: -------------------------------------------------------------------------------- 1 | -record(fuge, { 2 | % Name of the experiment 3 | name :: string() | binary(), 4 | % List of middlewares to be run for this experiment 5 | subscribers :: list(fuge_subscriber()), 6 | % Options for the experiment 7 | options :: options() 8 | }). 9 | 10 | -record(fuge_data, { 11 | %% Time taken to run the code (in microseconds) 12 | duration :: integer(), 13 | %% Final value of the code 14 | value :: any() 15 | }). 16 | 17 | -record(fuge_result, { 18 | % Meta information about the experiment for context (optional) 19 | context :: term(), 20 | % Data about control 21 | control :: fuge_data(), 22 | % Data about candidate 23 | candidates :: list(fuge_data()), 24 | % Order of execution 25 | execution_order :: list() 26 | }). 27 | 28 | -type fuge_subscriber() :: atom(). 29 | -type fuge() :: #fuge{}. 30 | -type fuge_data() :: #fuge_data{}. 31 | -type result() :: #fuge_result{}. 32 | -type subscriber_state() :: any(). 33 | -type name() :: term(). 34 | -type error() :: {error, term()}. 35 | -type options() :: [{frequency, pos_integer()}]. 36 | -type subscriber() :: atom(). 37 | -------------------------------------------------------------------------------- /rebar.config: -------------------------------------------------------------------------------- 1 | {erl_opts, [debug_info]}. 2 | {deps, []}. -------------------------------------------------------------------------------- /rebar.lock: -------------------------------------------------------------------------------- 1 | []. 2 | -------------------------------------------------------------------------------- /src/fuge.app.src: -------------------------------------------------------------------------------- 1 | {application, fuge, 2 | [{description, "An Erlang library for refactoring critical paths"}, 3 | {vsn, "0.1.0"}, 4 | {registered, []}, 5 | {mod, { fuge_app, []}}, 6 | {applications, 7 | [kernel, 8 | stdlib 9 | ]}, 10 | {env,[]}, 11 | {modules, []}, 12 | {maintainers, []}, 13 | {licenses, []}, 14 | {links, []} 15 | ]}. 16 | -------------------------------------------------------------------------------- /src/fuge.erl: -------------------------------------------------------------------------------- 1 | %%%------------------------------------------------------------------- 2 | %% @doc fuge public interface 3 | %% @end 4 | %%%------------------------------------------------------------------- 5 | 6 | -module(fuge). 7 | 8 | -include("fuge.hrl"). 9 | 10 | -export([new/1, new/2, new/3, 11 | run/3, run/4, run/5]). 12 | 13 | -export_type([fuge/0, 14 | name/0, 15 | error/0, 16 | result/0, 17 | subscriber_state/0 18 | ]). 19 | 20 | -define(DEFAULT_SUBSCRIBERS, [fuge_subscriber_logger]). 21 | -define(DEFAULT_OPTIONS, []). 22 | 23 | -spec new(name()) -> ok | error() | list(error()). 24 | new(Name) -> 25 | new(Name, ?DEFAULT_SUBSCRIBERS). 26 | 27 | -spec new(name(), list(subscriber())) -> ok | error() | list(error()). 28 | new(Name, Subscribers) -> 29 | new(Name, Subscribers, ?DEFAULT_OPTIONS). 30 | 31 | -spec new(name(), list(subscriber()), options()) -> ok | error() | list(error()). 32 | new(Name, Subscribers, Options) -> 33 | case validate_options(Options) of 34 | [] -> 35 | fuge_server:new(Name, Subscribers, Options); 36 | Error -> 37 | Error 38 | end. 39 | 40 | -spec run(name(), fun(), fun() | list(fun())) -> any(). 41 | run(Name, Control, Candidates) when is_list(Candidates) -> 42 | run(Name, Control, Candidates, undefined, []); 43 | run(Name, Control, Candidate) -> 44 | run(Name, Control, [Candidate], undefined, []). 45 | 46 | -spec run(name(), fun(), fun() | list(fun()), any()) -> any(). 47 | run(Name, Control, Candidates, Context) when is_list(Candidates) -> 48 | run(Name, Control, Candidates, Context, []); 49 | run(Name, Control, Candidate, Context) -> 50 | run(Name, Control, [Candidate], Context, []). 51 | 52 | -spec run(name(), fun(), fun() | list(fun()), any(), list()) -> any(). 53 | run(Name, Control, Candidates, Context, Options) when is_list(Candidates) -> 54 | case fuge_server:get(Name) of 55 | {ok, Fuge} -> 56 | fuge_run(Fuge, Control, Candidates, Context, Options); 57 | {error, not_found} -> 58 | Control() 59 | end; 60 | run(Name, Control, Candidate, Context, Options) -> 61 | run(Name, Control, [Candidate], Context, Options). 62 | 63 | %%------------------------------------------------------------------- 64 | %% Internal functions 65 | %%------------------------------------------------------------------- 66 | 67 | %% Check all the fuge options before the actual run 68 | fuge_run(Fuge, Control, Candidates, Context, Options) -> 69 | case proplists:get_value(frequency, Fuge#fuge.options) of 70 | undefined -> 71 | do_run(Fuge, Control, Candidates, Context, Options); 72 | Value -> 73 | case Value >= random:uniform(100) of 74 | true -> 75 | do_run(Fuge, Control, Candidates, Context, Options); 76 | false -> 77 | Control() 78 | end 79 | end. 80 | 81 | do_run(Fuge, Control, Candidates, Context, Options) -> 82 | NumberedCandidates = lists:zip(lists:seq(1, length(Candidates)), Candidates), 83 | Experiments = shuffle([{0, Control} | NumberedCandidates]), 84 | {Order, _} = lists:unzip(Experiments), 85 | RunExperiment = fun ({Id, Fun}) -> {Id, do_run_experiment(Fun, Options)} end, 86 | {_, [ControlR | CandidatesR]} = 87 | lists:unzip(lists:keysort(1, lists:map(RunExperiment, Experiments))), 88 | Result = #fuge_result{context = Context, 89 | control = ControlR, 90 | candidates = CandidatesR, 91 | execution_order = Order}, 92 | fuge_server:experiment(Fuge, Result), 93 | ControlR#fuge_data.value. 94 | 95 | % TODO think and use sane options 96 | do_run_experiment(Fun, _Options) -> 97 | Start = os:timestamp(), 98 | Result = Fun(), 99 | End = os:timestamp(), 100 | #fuge_data{duration = timer:now_diff(End, Start), 101 | value = Result}. 102 | 103 | shuffle(List) -> 104 | [X || {_, X} <- lists:sort([{random:uniform(), E} || E <- List])]. 105 | 106 | %% Validate all options and return ones that failed 107 | validate_options(Options) -> 108 | [Return || Return <- 109 | [validate_option(Option, Value) || {Option, Value} <- Options], 110 | Return =/= ok]. 111 | 112 | validate_option(frequency, Value) 113 | when is_integer(Value), 114 | Value > 0, 115 | Value =< 100 -> 116 | ok; 117 | validate_option(Option, Value) -> 118 | {error, {invalid_option, {Option, Value}}}. 119 | -------------------------------------------------------------------------------- /src/fuge_app.erl: -------------------------------------------------------------------------------- 1 | %%%------------------------------------------------------------------- 2 | %% @doc fuge public API 3 | %% @end 4 | %%%------------------------------------------------------------------- 5 | 6 | -module(fuge_app). 7 | 8 | -behaviour(application). 9 | 10 | %% Application callbacks 11 | -export([start/2, 12 | stop/1]). 13 | 14 | %%------------------------------------------------------------------- 15 | %% API 16 | %%------------------------------------------------------------------- 17 | 18 | start(_StartType, _StartArgs) -> 19 | fuge_sup:start_link(). 20 | 21 | stop(_State) -> 22 | ok. 23 | -------------------------------------------------------------------------------- /src/fuge_server.erl: -------------------------------------------------------------------------------- 1 | %%%------------------------------------------------------------------- 2 | %% @doc fuge server 3 | %% @end 4 | %%%------------------------------------------------------------------- 5 | 6 | -module(fuge_server). 7 | 8 | -behaviour(gen_server). 9 | 10 | -include("fuge.hrl"). 11 | 12 | -export([start_link/0, 13 | new/3, 14 | get/1, 15 | experiment/2 16 | ]). 17 | 18 | %% gen_server callbacks 19 | -export([init/1, 20 | handle_call/3, 21 | handle_cast/2, 22 | handle_info/2, 23 | terminate/2, 24 | code_change/3]). 25 | 26 | -record(state, {}). 27 | 28 | -type startlink_ret() :: {ok, pid()} | ignore | {error, term()}. 29 | 30 | %% ETS table used for storing all experiments 31 | -define(STORE, fuge_server_store). 32 | 33 | %%------------------------------------------------------------------- 34 | %% API 35 | %%------------------------------------------------------------------- 36 | 37 | -spec start_link() -> startlink_ret(). 38 | start_link() -> 39 | gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). 40 | 41 | -spec new(name(), list(), list()) -> ok | error(). 42 | new(Name, Subscribers, Options) -> 43 | gen_server:call(?MODULE, {new, Name, Subscribers, Options}). 44 | 45 | -spec get(name()) -> {ok, fuge()} | error(). 46 | get(Name) -> 47 | case ets:lookup(?STORE, Name) of 48 | [] -> 49 | {error, not_found}; 50 | [{Name, Fuge}] -> 51 | {ok, Fuge} 52 | end. 53 | 54 | -spec experiment(fuge(), result()) -> ok. 55 | experiment(Fuge, Result) -> 56 | gen_server:cast(?MODULE, {result, Fuge, Result}). 57 | 58 | %%------------------------------------------------------------------- 59 | %% gen_server callback implementation 60 | %%------------------------------------------------------------------- 61 | 62 | init([]) -> 63 | ets:new(?STORE, [named_table, {read_concurrency, true}]), 64 | {ok, #state{}}. 65 | 66 | handle_call({new, Name, Subscribers, Options}, _From, State) -> 67 | case ?MODULE:get(Name) of 68 | {error, not_found} -> 69 | Fuge = create_fuge(Name, Subscribers, Options), 70 | true = ets:insert_new(?STORE, {Name, Fuge}), 71 | lists:foreach( 72 | fun (Subscriber) -> 73 | true = ets:insert_new(?STORE, 74 | {subscriber_key(Name, Subscriber), 75 | subscriber_value(Fuge, Subscriber)}) 76 | end, Subscribers), 77 | {reply, ok, State}; 78 | {ok, _Fuge} -> 79 | {reply, {error, already_exists}, State} 80 | end; 81 | handle_call(_Request, _From, State) -> 82 | {reply, ignored, State}. 83 | 84 | handle_cast({result, Fuge, Result}, State) -> 85 | lists:foreach(fun (SubscriberName) -> 86 | handle_subscriber(Fuge, SubscriberName, Result) 87 | end, Fuge#fuge.subscribers), 88 | {noreply, State}; 89 | handle_cast(_Msg, State) -> 90 | {noreply, State}. 91 | 92 | handle_info(_Info, State) -> 93 | {noreply, State}. 94 | 95 | terminate(_Reason, _State) -> 96 | ok. 97 | 98 | code_change(_OldVsn, State, _Extra) -> 99 | {ok, State}. 100 | 101 | %%------------------------------------------------------------------- 102 | %% Internal functions 103 | %%------------------------------------------------------------------- 104 | 105 | create_fuge(Name, Subscribers, Options) -> 106 | #fuge{name = Name, 107 | subscribers = Subscribers, 108 | options = Options}. 109 | 110 | subscriber_key(Name, {SubscriberName, _SubscriberState}) -> 111 | {Name, SubscriberName}; 112 | subscriber_key(Name, SubscriberName) -> 113 | {Name, SubscriberName}. 114 | 115 | subscriber_value(Fuge, {SubscriberName, SubscriberState}) -> 116 | SubscriberName:init(Fuge, SubscriberState); 117 | subscriber_value(Fuge, SubscriberName) -> 118 | subscriber_value(Fuge, {SubscriberName, undefined}). 119 | 120 | get_subscriber(Name, SubscriberName) -> 121 | case ets:lookup(?STORE, subscriber_key(Name, SubscriberName)) of 122 | [] -> 123 | {error, not_found}; 124 | [{_SubscriberName, SubscriberState}] -> 125 | SubscriberState 126 | end. 127 | 128 | handle_subscriber(Fuge, SubscriberName, Result) -> 129 | SubscriberState = get_subscriber(Fuge#fuge.name, SubscriberName), 130 | SubscriberName:handle_result(Fuge, SubscriberState, Result). 131 | -------------------------------------------------------------------------------- /src/fuge_subscriber.erl: -------------------------------------------------------------------------------- 1 | %%%------------------------------------------------------------------- 2 | %% @doc fuge subscriber behaviour 3 | %% @end 4 | %%%------------------------------------------------------------------- 5 | 6 | -module(fuge_subscriber). 7 | 8 | -include("fuge.hrl"). 9 | 10 | -callback init(Fuge :: fuge(), 11 | State :: subscriber_state()) -> 12 | NewState :: subscriber_state(). 13 | 14 | -callback handle_result(Fuge :: fuge(), 15 | State :: subscriber_state(), 16 | Result :: result()) -> 17 | ok. 18 | -------------------------------------------------------------------------------- /src/fuge_subscriber_logger.erl: -------------------------------------------------------------------------------- 1 | -module(fuge_subscriber_logger). 2 | 3 | -behaviour(fuge_subscriber). 4 | 5 | -include("fuge.hrl"). 6 | 7 | -export([init/2, 8 | handle_result/3]). 9 | 10 | init(_Fuge, State) -> 11 | State. 12 | 13 | handle_result(Fuge, _State, Result) -> 14 | error_logger:info_msg("Fuge: ~p", [Fuge]), 15 | error_logger:info_msg("Result: ~p", [Result]). 16 | -------------------------------------------------------------------------------- /src/fuge_sup.erl: -------------------------------------------------------------------------------- 1 | %%%------------------------------------------------------------------- 2 | %% @doc fuge top level supervisor. 3 | %% @end 4 | %%%------------------------------------------------------------------- 5 | 6 | -module(fuge_sup). 7 | 8 | -behaviour(supervisor). 9 | 10 | %% API 11 | -export([start_link/0]). 12 | 13 | %% Supervisor callbacks 14 | -export([init/1]). 15 | 16 | -define(SERVER, ?MODULE). 17 | 18 | %%------------------------------------------------------------------- 19 | %% API functions 20 | %%------------------------------------------------------------------- 21 | 22 | start_link() -> 23 | supervisor:start_link({local, ?SERVER}, ?MODULE, []). 24 | 25 | %%------------------------------------------------------------------- 26 | %% Supervisor callbacks 27 | %%------------------------------------------------------------------- 28 | 29 | init([]) -> 30 | {ok, {{one_for_all, 0, 1}, [server()]}}. 31 | 32 | %%------------------------------------------------------------------- 33 | %% Internal functions 34 | %%------------------------------------------------------------------- 35 | 36 | server() -> 37 | {fuge_server, {fuge_server, start_link, []}, 38 | temporary, 5000, worker, []}. 39 | -------------------------------------------------------------------------------- /test/fuge_test.erl: -------------------------------------------------------------------------------- 1 | -module(fuge_test). 2 | 3 | -include_lib("eunit/include/eunit.hrl"). 4 | 5 | -export([control/0, 6 | candidate1/0, 7 | candidate2/0 8 | ]). 9 | 10 | fuge_test_() -> 11 | {setup, fun setup/0, fun teardown/1, 12 | [ 13 | {"Simple run", fun simple_run/0}, 14 | {"Frequency option", fun frequency_option/0} 15 | ]}. 16 | 17 | setup() -> 18 | application:start(fuge). 19 | 20 | teardown(_) -> 21 | application:stop(fuge). 22 | 23 | simple_run() -> 24 | ok = fuge:new(simple_run), 25 | fuge:run(test, fun ?MODULE:control/0, fun ?MODULE:candidate1/0), 26 | fuge:run(test, fun ?MODULE:control/0, [fun ?MODULE:candidate1/0, 27 | fun ?MODULE:candidate2/0]), 28 | timer:sleep(100). 29 | 30 | frequency_option() -> 31 | Subscribers = [], 32 | Frequency = 42, 33 | Options = [{frequency, Frequency}], 34 | ErrorMargin = 0.1, 35 | NumRuns = 1000, 36 | 37 | ok = fuge:new(frequency_option, Subscribers, Options), 38 | 39 | ControlTable = ets:new(control, []), 40 | ets:insert(ControlTable, {count, 0}), 41 | CandidateTable = ets:new(candidate, []), 42 | ets:insert(CandidateTable, {count, 0}), 43 | Control = fun() -> ets:update_counter(ControlTable, count, 1) end, 44 | Candidate = fun() -> ets:update_counter(CandidateTable, count, 1) end, 45 | 46 | lists:foreach(fun (_) -> 47 | fuge:run(frequency_option, Control, Candidate) 48 | end, lists:seq(1, NumRuns)), 49 | 50 | ?assert(((NumRuns * ((Frequency - Frequency * ErrorMargin) / 100)) < 51 | ets:lookup_element(CandidateTable, count, 2))), 52 | 53 | ?assert(ets:lookup_element(CandidateTable, count, 2) < 54 | (NumRuns * ((Frequency + Frequency * ErrorMargin) / 100))). 55 | 56 | %%------------------------------------------------------------------- 57 | %% Internal functions 58 | %%------------------------------------------------------------------- 59 | 60 | %% Control function calculates sum of all numbers between 1, 1000 using 61 | %% functions from standard library 62 | control() -> 63 | lists:sum(lists:seq(1, 1000)). 64 | 65 | %% Candidate function calculates sum of all numbers between 1, 1000 using 66 | %% custom function code 67 | candidate1() -> 68 | sum(1, 1000). 69 | 70 | %% Candidate function calculates sum of all numbers between 1, 1000 using 71 | %% the formula -> n * (n + 1) / 2 72 | candidate2() -> 73 | 1000 * (1000 + 1) div 2. 74 | 75 | sum(Start, End) -> 76 | sum(Start, End, 0). 77 | 78 | sum(End, End, Sum) -> 79 | End + Sum; 80 | sum(Start, End, Sum) -> 81 | sum(Start+1, End, Start + Sum). 82 | --------------------------------------------------------------------------------