Stubbing Binaries for Testing

Being a Dev Ops guy, I write gems, scripts and hacks that tend to hook in to system binaries. I know, it would probably be better to write C bindings, but heretofore that hasn't been something I really want to tackle. Additionally, I almost always write tests for my stuff. I've come across the problem where my tests pass locally, but not on Travis-CI. Which annoys me to no end... hences this tweet:

Yes, I noticed the typo... s/ and / an /... better?

The way I've solved this is by building a method to define the binary location in my utilities and stubbing the binaries in a very simple method. Simply put, I have a shell script named for the binary that echo's the standard output and ends with exit 0.

Now, for this to work most effiecntly, you have to also stub the parsed output in your tests, so you can control that output in a more testable way.


These examples come from HTTPerf.rb, whose source can be found [here](].

Binary Stub - httperf

echo "httperf --client=0/1 --server=localhost --port=80 --uri=/ --send-buffer=4096 --recv-buffer=16384 --num-conns=1 --num-calls=1
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
Maximum connect burst length: 0

Total: connections 1 requests 1 replies 1 test-duration 0.000 s

Connection rate: 4524.6 conn/s (0.2 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 0.2 avg 0.2 max 0.2 median 0.5 stddev 0.0
Connection time [ms]: connect 0.1
Connection length [replies/conn]: 1.000

Request rate: 4524.6 req/s (0.2 ms/req)
Request size [B]: 62.0

Reply rate [replies/s]: min 0.0 avg 0.0 max 0.0 stddev 0.0 (0 samples)
Reply time [ms]: response 0.1 transfer 0.0
Reply size [B]: header 154.0 content 168.0 footer 0.0 (total 322.0)
Reply status: 1xx=0 2xx=0 3xx=0 4xx=1 5xx=0

CPU time [s]: user 0.00 system 0.00 (user 0.0% system 0.0% total 0.0%)
Net I/O: 1696.7 KB/s (13.9*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0
exit 0
# vim: filetype=sh:

view on github

Result Stubbing - httperf

I read in sample data from text files and make them globally accessable for all tests.

# file: spec_helper.rb
# ...
$results_raw ="./spec/dummy_results.txt","r").read
$verbose_raw ="./spec/dummy_verbose_results.txt","r").read
# ...

view on github

You can see the text files here:

I'm sure there are other ways. I'm sure there are better ways. This way has worked for me thus far and fixes the issue with httperf not being available on Travis-CI.