Nimble
Use Nimble to express the expected outcomes of Swift
or Objective-C expressions. Inspired by
Cedar.
How to Use Nimble
Some Background: Expressing Outcomes Using Assertions in XCTest
Apple's Xcode includes the XCTest framework, which provides
assertion macros to test whether code behaves properly.
For example, to assert that 1 + 1 = 2
, XCTest has you write:
Or, in Objective-C:
XCTest assertions have a couple of drawbacks:
- Not enough macros. There's no easy way to assert that a string
contains a particular substring, or that a number is less than or
equal to another. - It's hard to write asynchronous tests. XCTest forces you to write
a lot of boilerplate code.
Nimble addresses these concerns.
Nimble: Expectations Using expect(...).to
Nimble allows you to express expectations using a natural,
easily understood language:
The
expect
function autocompletes to includefile:
andline:
,
but these parameters are optional. Use the default values to have
Xcode highlight the correct line when an expectation is not met.
To perform the opposite expectation--to assert something is not
equal--use toNot
or notTo
:
Custom Failure Messages
Would you like to add more information to the test's failure messages? Use the description
optional argument to add your own text:
Or the *WithDescription version in Objective-C:
Type Safety
Nimble makes sure you don't compare two types that don't match:
Nimble uses generics--only available in Swift--to ensure
type correctness. That means type checking is
not available when using Nimble in Objective-C. :sob:
Operator Overloads
Tired of so much typing? With Nimble, you can use overloaded operators
like ==
for equivalence, or >
for comparisons:
Operator overloads are only available in Swift, so you won't be able
to use this syntax in Objective-C. :broken_heart:
Lazily Computed Values
The expect
function doesn't evaluate the value it's given until it's
time to match. So Nimble can test whether an expression raises an
exception once evaluated:
Objective-C works the same way, but you must use the expectAction
macro when making an expectation on an expression that has no return
value:
C Primitives
Some testing frameworks make it hard to test primitive C values.
In Nimble, it just works:
In fact, Nimble uses type inference, so you can write the above
without explicitly specifying both types:
In Objective-C, Nimble only supports Objective-C objects. To
make expectations on primitive C values, wrap then in an object
literal:
Asynchronous Expectations
In Nimble, it's easy to make expectations on values that are updated
asynchronously. Just use toEventually
or toEventuallyNot
:
Note: toEventually triggers its polls on the main thread. Blocking the main
thread will cause Nimble to stop the run loop. This can cause test pollution
for whatever incomplete code that was running on the main thread. Blocking the
main thread can be caused by blocking IO, calls to sleep(), deadlocks, and
synchronous IPC.
In the above example, ocean
is constantly re-evaluated. If it ever
contains dolphins and whales, the expectation passes. If ocean
still
doesn't contain them, even after being continuously re-evaluated for one
whole second, the expectation fails.
Sometimes it takes more than a second for a value to update. In those
cases, use the timeout
parameter:
You can also provide a callback by using the waitUntil
function:
waitUntil
also optionally takes a timeout parameter:
Note: waitUntil
triggers its timeout code on the main thread. Blocking the main
thread will cause Nimble to stop the run loop to continue. This can cause test
pollution for whatever incomplete code that was running on the main thread.
Blocking the main thread can be caused by blocking IO, calls to sleep(),
deadlocks, and synchronous IPC.
In some cases (e.g. when running on slower machines) it can be useful to modify
the default timeout and poll interval values. This can be done as follows:
Objective-C Support
Nimble has full support for Objective-C. However, there are two things
to keep in mind when using Nimble in Objective-C:
-
All parameters passed to the
expect
function, as well as matcher
functions likeequal
, must be Objective-C objects or can be converted into
anNSObject
equivalent: -
To make an expectation on an expression that does not return a value,
such as-[NSException raise]
, useexpectAction
instead of
expect
:
The following types are currently converted to an NSObject
type:
- C Numeric types are converted to
NSNumber *
NSRange
is converted toNSValue *
char *
is converted toNSString *
For the following matchers:
equal
beGreaterThan
beGreaterThanOrEqual
beLessThan
beLessThanOrEqual
beCloseTo
beTrue
beFalse
beTruthy
beFalsy
haveCount
If you would like to see more, file an issue.
Disabling Objective-C Shorthand
Nimble provides a shorthand for expressing expectations using the
expect
function. To disable this shorthand in Objective-C, define the
NIMBLE_DISABLE_SHORT_SYNTAX
macro somewhere in your code before
importing Nimble:
Disabling the shorthand is useful if you're testing functions with
names that conflict with Nimble functions, such asexpect
or
equal
. If that's not the case, there's no point in disabling the
shorthand.
Built-in Matcher Functions
Nimble includes a wide variety of matcher functions.
Type Checking
Nimble supports checking the type membership of any kind of object, whether
Objective-C conformant or not:
Objects can be tested for their exact types using the beAnInstanceOf
matcher:
Equivalence
Values must be Equatable
, Comparable
, or subclasses of NSObject
.
equal
will always fail when used to compare one or more nil
values.
Identity
It is important to remember that beIdenticalTo
only makes sense when comparing
types with reference semantics, which have a notion of identity. In Swift,
that means types that are defined as a class
.
This matcher will not work when comparing types with value semantics such as
those defined as a struct
or enum
. If you need to compare two value types,
consider what it means for instances of your type to be identical. This may mean
comparing individual properties or, if it makes sense to do so, conforming your type
to Equatable
and using Nimble's equivalence matchers instead.
Comparisons
Values given to the comparison matchers above must implement
Comparable
.
Because of how computers represent floating point numbers, assertions
that two floating point numbers be equal will sometimes fail. To express
that two numbers should be close to one another within a certain margin
of error, use beCloseTo
:
For example, to assert that 10.01
is close to 10
, you can write:
There is also an operator shortcut available in Swift:
(Type option+x to get ≈
on a U.S. keyboard)
The former version uses the default delta of 0.0001. Here is yet another way to do this:
(Type option+shift+= to get ±
on a U.S. keyboard)
If you are comparing arrays of floating point numbers, you'll find the following useful:
Values given to the
beCloseTo
matcher must be coercable into a
Double
.
Types/Classes
Instances must be Objective-C objects: subclasses of
NSObject
,
or Swift objects bridged to Objective-C with the@objc
prefix.
For example, to assert that dolphin
is a kind of Mammal
:
beAnInstanceOf
uses the-[NSObject isMemberOfClass:]
method to
test membership.beAKindOf
uses-[NSObject isKindOfClass:]
.
Truthiness
Swift Assertions
If you're using Swift, you can use the throwAssertion
matcher to check if an assertion is thrown (e.g. fatalError()
). This is made possible by @mattgallagher's CwlPreconditionTesting library.
Notes:
- This feature is only available in Swift.
- The tvOS simulator is supported, but using a different mechanism, requiring you to turn off the
Debug executable
scheme setting for your tvOS scheme's Test configuration.
Swift Error Handling
You can use the throwError
matcher to check if an error is thrown.
When working directly with Error
values, using the matchError
matcher
allows you to perform certain checks on the error itself without having to
explicitly cast the error.
The matchError
matcher allows you to check whether or not the error:
- is the same type of error you are expecting.
- represents a particular error value that you are expecting.
This can be useful when using Result
or Promise
types, for example.
Note: This feature is only available in Swift.
Exceptions
Note: Swift currently doesn't have exceptions (see #220).
Only Objective-C code can raise exceptions that Nimble will catch.
Collection Membership
In Swift
contain
takes any number of arguments. The expectation
passes if all of them are members of the collection. In Objective-C,
contain
only takes one argument for now.
For example, to assert that a list of sea creature names contains
"dolphin" and "starfish":
contain
andbeEmpty
expect collections to be instances of
NSArray
,NSSet
, or a Swift collection composed ofEquatable
elements.
To test whether a set of elements is present at the beginning or end of
an ordered collection, use beginWith
and endWith
:
beginWith
andendWith
expect collections to be instances of
NSArray
, or ordered Swift collections composed ofEquatable
elements.
Like contain
, in Objective-C beginWith
and endWith
only support
a single argument for now.
For code that returns collections of complex objects without a strict
ordering, there is the containElementSatisfying
matcher:
Strings
Collection Elements
Nimble provides a means to check that all elements of a collection pass a given expectation.
Swift
In Swift, the collection must be an instance of a type conforming to
Sequence
.
Objective-C
In Objective-C, the collection must be an instance of a type which implements
the NSFastEnumeration
protocol, and whose elements are instances of a type
which subclasses NSObject
.
Additionally, unlike in Swift, there is no override to specify a custom
matcher function.
Collection Count
For Swift, the actual value must be an instance of a type conforming to Collection
.
For example, instances of Array
, Dictionary
, or Set
.
For Objective-C, the actual value must be one of the following classes, or their subclasses:
NSArray
,NSDictionary
,NSSet
, orNSHashTable
.
Notifications
This matcher is only available in Swift.
Result
This matcher is only available in Swift.
Matching a value to any of a group of matchers
Note: This matcher allows you to chain any number of matchers together. This provides flexibility,
but if you find yourself chaining many matchers together in one test, consider whether you
could instead refactor that single test into multiple, more precisely focused tests for
better coverage.
Custom Validation
The String
provided with .failed()
is shown when the test fails.
When using toEventually()
be careful not to make state changes or run process intensive code since this closure will be ran many times.
Writing Your Own Matchers
In Nimble, matchers are Swift functions that take an expected
value and return a Predicate
closure. Take equal
, for example:
The return value of a Predicate
closure is a PredicateResult
that indicates
whether the actual value matches the expectation and what error message to
display on failure.
The actual
equal
matcher function does not match when
expected
are nil; the example above has been edited for brevity.
Since matchers are just Swift functions, you can define them anywhere:
at the top of your test file, in a file shared by all of your tests, or
in an Xcode project you distribute to others.
If you write a matcher you think everyone can use, consider adding it
to Nimble's built-in set of matchers by sending a pull request! Or
distribute it yourself via GitHub.
For examples of how to write your own matchers, just check out the
Matchers
directory
to see how Nimble's built-in set of matchers are implemented. You can
also check out the tips below.
PredicateResult
PredicateResult
is the return struct that Predicate
return to indicate
success and failure. A PredicateResult
is made up of two values:
PredicateStatus
and ExpectationMessage
.
Instead of a boolean, PredicateStatus
captures a trinary set of values:
Meanwhile, ExpectationMessage
provides messaging semantics for error reporting.
Predicates should usually depend on either .expectedActualValueTo(..)
or
.fail(..)
when reporting errors. Special cases can be used for the other enum
cases.
Finally, if your Predicate utilizes other Predicates, you can utilize
.appended(details:)
and .appended(message:)
methods to annotate an existing
error with more details.
A common message to append is failing on nils. For that, .appendedBeNilHint()
can be used.
Lazy Evaluation
actualExpression
is a lazy, memoized closure around the value provided to the
expect
function. The expression can either be a closure or a value directly
passed to expect(...)
. In order to determine whether that value matches,
custom matchers should call actualExpression.evaluate()
:
In the above example, actualExpression
is not nil
-- it is a closure
that returns a value. The value it returns, which is accessed via the
evaluate()
method, may be nil
. If that value is nil
, the beNil
matcher function returns true
, indicating that the expectation passed.
Type Checking via Swift Generics
Using Swift's generics, matchers can constrain the type of the actual value
passed to the expect
function by modifying the return type.
For example, the following matcher, haveDescription
, only accepts actual
values that implement the Printable
protocol. It checks their description
against the one provided to the matcher function, and passes if they are the same:
Customizing Failure Messages
When using Predicate.simple(..)
or Predicate.simpleNilable(..)
, Nimble
outputs the following failure message when an expectation fails:
You can customize this message by modifying the way you create a Predicate
.
Basic Customization
For slightly more complex error messaging, receive the created failure message
with Predicate.define(..)
:
In the example above, msg
is defined based on the string given to
Predicate.define
. The code looks akin to:
Full Customization
To fully customize the behavior of the Predicate, use the overload that expects
a PredicateResult
to be returned.
Along with PredicateResult
, there are other ExpectationMessage
enum values you can use:
For matchers that compose other matchers, there are a handful of helper
functions to annotate messages.
appended(message: String)
is used to append to the original failure message:
For a more comprehensive message that spans multiple lines, use
appended(details: String)
instead:
Supporting Objective-C
To use a custom matcher written in Swift from Objective-C, you'll have
to extend the NMBPredicate
class, adding a new class method for your
custom matcher. The example below defines the class method
+[NMBPredicate beNilMatcher]
:
The above allows you to use the matcher from Objective-C:
To make the syntax easier to use, define a C function that calls the
class method:
Properly Handling nil
in Objective-C Matchers
When supporting Objective-C, make sure you handle nil
appropriately.
Like Cedar,
most matchers do not match with nil. This is to bring prevent test
writers from being surprised by nil
values where they did not expect
them.
Nimble provides the beNil
matcher function for test writer that want
to make expectations on nil
objects:
If your matcher does not want to match with nil, you use Predicate.define
or Predicate.simple
.
Using those factory methods will automatically generate expected value failure messages when they're nil.
Migrating from the Old Matcher API
Previously (<7.0.0
), Nimble supported matchers via the following types:
Matcher
NonNilMatcherFunc
MatcherFunc
All of those types have been replaced by Predicate
. The old API has been
removed completely in Nimble v10.
Installing Nimble
Nimble can be used on its own, or in conjunction with its sister
project, Quick. To install both
Quick and Nimble, follow the installation instructions in the Quick
Documentation.
Nimble can currently be installed in one of two ways: using CocoaPods, or with
git submodules.
Installing Nimble as a Submodule
To use Nimble as a submodule to test your macOS, iOS or tvOS applications, follow
these 4 easy steps:
- Clone the Nimble repository
- Add Nimble.xcodeproj to the Xcode workspace for your project
- Link Nimble.framework to your test target
- Start writing expectations!
For more detailed instructions on each of these steps,
read How to Install Quick.
Ignore the steps involving adding Quick to your project in order to
install just Nimble.
Installing Nimble via CocoaPods
To use Nimble in CocoaPods to test your macOS, iOS or tvOS applications, add
Nimble to your podfile and add the use_frameworks!
line to enable Swift
support for CocoaPods.
Finally run pod install
.
Using Nimble without XCTest
Nimble is integrated with XCTest to allow it work well when used in Xcode test
bundles, however it can also be used in a standalone app. After installing
Nimble using one of the above methods, there are two additional steps required
to make this work.
- Create a custom assertion handler and assign an instance of it to the
globalNimbleAssertionHandler
variable. For example:
- Add a post-build action to fix an issue with the Swift XCTest support
library being unnecessarily copied into your app
- Edit your scheme in Xcode, and navigate to Build -> Post-actions
- Click the "+" icon and select "New Run Script Action"
- Open the "Provide build settings from" dropdown and select your target
- Enter the following script contents:
rm "${SWIFT_STDLIB_TOOL_DESTINATION_DIR}/libswiftXCTest.dylib"
You can now use Nimble assertions in your code and handle failures as you see
fit.