Erlang Quickcheck Design Patterns – Black Box

Name
Black Box

Also Known As
ETS Driven Testing

Intent
When writing Quickcheck unit tests for an Erlang application, it usually happens that you require to stub some external application, since you’re not interested in System Tests, but you want to test your application as a mere black box.

Motivation
You want complete control on the results provided by your stubs, so that you can successfully check the properties you need for your application.

Sample Code

In your quickcheck module, just define in your state the results you need, together with their default value:

-record(state, {stub_result = true}).

Define the initial state as follows:

initial_state() ->
    init.

initial_state_data() ->
    #state{}.

In your property, create an ETS table containing the results (don’t forget to delete it after running the tests) and call the run_commands function by passing an extra parameter:

prop_commands() ->
    ?FORALL({Cmds, State}, custom_commands(),
            ?TRAPEXIT(begin
                          results = ets:new(results, [named_table, set, public]),
                          ets:insert(results, {stub_result, State#state.stub_result}),
                          {H,S,Res} = eqc_fsm:run_commands(?MODULE, Cmds, [{var, stub_result}]),
                          true = ets:delete(results),
                          ...

The custom_commands function is the place where you set the list of available results:

custom_commands() ->
    ?LET(State,
         #state{stub_result = oneof([true, false])},
         {more_commands(100, eqc_fsm:commands(?MODULE, {init, State})), State}).

You can check the result in your postconditions, as follows:

postcondition(_From, _To, S, {call, _, _, _}, R) ->
    S#state.stub_result == R.

Obviously, your stubs need to contain something like:

stubbed_function() ->
    ets:lookup_element(results, stub_result, 2).

Consequences
Your tests will be able to cover a bigger number of code lines (lemmings).

Thanks
Thanks to Master Magnus Henoch for his help.