pinpill
Alternate test orchestrator and results aggregator, designed to run over the bp binary in https://github.com/MobileNativeFoundation/bluepill
Why does this exist?
bluepill
is the upstream test orchestrator and results aggregator. bp
is the binary that actually runs the tests on simulators.
At the moment, bluepill
collects the requests tests, splits the jobs into shards over the simulators. bp
then works its way through
all of the tests in the given shard. After all tests are done, bluepill
collects the results and joins them into a top level XML file.
pinpill
follows a job queue pattern instead of the shards used by bluepill
. It maintains a queue of all the jobs that need to be
done, and assign them to any idle simulator. This means that we do not enforce which job runs on which simulator. It is simply a first
come first served basis. It assigns individual jobs to bp
, opposed to an entire shard at once.
One critical benefit of this paradigm is that it flexibly supports non-deterministic changes to the tests being run. For example,
if a test fails, it will be automatically retried X times. In the bluepill
pattern, all of these retries must run on the simulator
that the original attempt was assigned to, resulting in serial retries. In the pinpill
pattern, these retries are treated as new
jobs, assigned to any idle simulator, and run with improved concurrency.
Dependency on bluepill
pinpill
does NOT implement logic to actually run tests. It is simply an interface that wraps around the bp
binary. pinpill
assigns
jobs to the bp
binary and interprets the results. It only handles the orchestration and results aggregation side of things.
Building release
xcodebuild -project pinpill.xcodeproj -scheme pinpill -configuration Release -derivedDataPath build
This will produce a binary located at
build/Build/Products/Release/pinpill