As I’ve said before, for my small home project, where the most interesting work lies at the intersection of AppKit and a cross-platform C++ engine, I can’t write code the “unit test” way – writing the tests first – because those intersections are precisely what can’t be isolated from the entire application.
But I have changed some habits.
Now, I move the little snippets of logic I’ve written from their organically-grown locations to separate stand-alone functions.
These functions are grouped by purpose in their own files, such as
UTF8 Functions.c or
Geometry Translation Functions.m.
Because they aren’t classes, they need all their context passed in as parameters. This can be a pain in the ass, but it also prevents me from grouping too much functionality together; it would be too tedious to add all the necessary parameters.
It also means that I can access the logic directly from unit tests. I can try out in isolation all the little variations I need to really ensure the code is sound.
This works well for a small, home project. Splitting functionality out into lots of additional files may work less well in large-scale endeavors, where there are already hundreds of files in a project.
Then again, in such huge projects, the data model may be elaborate enough that there are more classes that, without modification, can be tested in isolation from AppKit (or each other).
I was getting a strange smearing effect when I tried to scroll in the main view of my application’s window.
I knew my content-drawing routines were being invoked correctly. I knew that the effect seemed to be happening somewhere in the depths of Cocoa, not in my code. But that’s all I knew.
Turns out, it’s because I was setting my view frames to fractional values (they’re floats, after all), but my content engine was still drawing things per integer coordinates.
This is the kind of thing that unit testing won’t help.
My goal was “make resizing the window and then scrolling through the main view work.” Can’t write a test for that because “work” means “look right,” and you can’t test for that when you don’t know how to make it look right yet.
It’s one part experimenting and one part perseverance and one part lightbulb going off after things go wrong.
And I find myself working through these kinds of issues a lot.
Anyone who says “Always write your tests first” can stick that in their pipe and smoke it.
Doctor: So don’t laugh.
“Legacy” in this case means “anything without unit tests,” which means just about all code out there. Including, I’m ashamed to admit, most of the codebase of the Neutrino project I’m currently working on.
Why, after all that effort I put into adding unit test capabilities to my project, have I mostly failed to follow through? Feathers’ book describes it well. Currently, neither programming languages nor external libraries and frameworks make it easy to separate the unique logic of your code from all the dependencies that it requires to actually get its work done.
My codebase, for example, does a lot of custom drawing to a view, and does that drawing in response to calls from the cross-platform engine that actually drives the process. So I’ve got two huge dependencies: the engine, and the Cocoa framework. How do you get around that?
But it’s not just Neutrino. I prefer to work on code that’s in the middle of complicated systems, doing interesting and complicated things.
The classes in Kent Beck’s Test Driven Development: by Example were small data model classes that dealt only with each other. Well, sure, that would be easy! Real world code is rarely so straightforward. Beck describes a world so far removed from mine it might as well be in another galaxy.
Feathers, on the other hand, gets dirty in the trenches. But he doesn’t reveal any magic formulas. Instead, over and over again, he says: stop laughing. Change the code.
Does the class you want to test rely on a well-protected singleton? Change the singleton so it isn’t, even though that allows others to use it incorrectly.
Does the class you want to test have some private methods? Make them public.
Does the class you want to test rely on other classes from a library or framework? Change your code not to refer to such classes directly, even at the cost of added complexity and overhead, so you can break the dependency and use dummy objects instead for your tests.
All these are ugly solutions, design-wise. And Feathers knows it; he makes multiple apologies for his techniques. But he stresses that they work.
Well-designed code, code that only allows itself to be used in the way it needs to be for production scenarios, turns out to be anathema to unit testing.
When I programmed primarily in C++, I looked askance at the typeless nature of Objective-C. How will the compiler check that you’re doing the right thing? What I’ve found since is that very few errors result from this extra freedom from type that you have in Objective-C. Maybe all those safeguards and extra complexity in C++ weren’t necessary after all?
Similarly, I wonder if the next major programming language might be optimized for unit tests. It will have even fewer compile-time restraints, even fewer security restraints, but many mechanisms to make it easier to test what you’ve just written – maybe even require such tests, or auto-generate such tests.
Until then, I’m going to see what the results are when I make my Neutrino code a lot less jolly.
1. Open Xcode 2.1
2. Make a Cocoa application project. I actually have the OCUnit stuff installed from a previous Xcode version, so my list of new projects included “Cocoa Application + Test”, which I chose (but which you shouldn’t, see below). I called my project “Testy”.
3. Add a C++ class with a member function that’s defined in the .cpp file. I called my class “Testy” as well, and the member function “test()”.
4. In the build settings for the application target, turn off ZeroLink for all configurations: choose “All Configurations” as your configuration in the target’s Info window, select the ZeroLink setting, and hit the delete key. This should turn it from bold to non-bold, and the checkbox to unchecked.
5. Build and run the application, be sure it runs OK, then quit it.
6. Add a Unit Test Bundle target. Add your app target as a direct dependency. Set the target’s Bundle Loader and Test Host settings as specified in Chris Hanson’s wonderful Unit Testing Cocoa Applications post.
7. Add a new Objective-C test case class file. Be sure it has an .mm suffix. In that file, instantiate your C++ class, and call the member function from that instance.
8. Build the Unit Test Bundle target.
It should work, right? You’re using a default project, default targets, following the instructions carefully. But it doesn’t work. The unit test bundle can’t find the C++ symbols from the application.
And the reason why is an additional little wrinkle I discovered today.
9. In your application target’s Info window build settings pane, search on “symbols hidden”. Turn it off. Rebuild your application and your unit test bundle.
Now it works.
Step 9 isn’t needed for applications with Objective-C classes and Objective-C unit test bundles. But it is needed for applications with C++ classes and Objective-C unit test bundles.
So if you have that combination, as I do, don’t forget step 9!
P.S. Second lesson: if you’ve got the old OCUnit stuff on your system, get rid of it before using Xcode 2.1-style unit tests. They don’t go well together. Because I didn’t, I had a lot of “scratch head, fiddle with it some more” sub-steps in the above that you should be spared.
I’d like to explain the "complicated and fragile" comment I made in my last post.
What was complicated and fragile about it? Let me count the ways:
In order to make the tests take place when the application is launched, I had to put the
-SenTest All argument on the project’s MyDocApp executable. I didn’t find a way to conditionalize that to a target: When I instead added
$(MY_BUILD_SETTING) as the argument, and then set
-SenTest All for my Tests aggregrate target, no argument was passed in to the application.
This constraint means that I can’t, for instance, put any tests in MainMenu.nib, because if I did they would always be invoked, even if my Tests framework was not present. Not so cool.
Anybody know a way conditionalize such executable arguments?
Why not just add a second custom executable to the project? It could have a different set of arguments. Use one executable for Tests, and one for No-Tests.
Unfortunately, such a custom executable reference in the target has to be specified by full path, which is a deal-breaker. When I tried to set the path for such a custom executable relative to the shared build directory, I got circular dependency errors, probably because the executable in question was already a product of one of the project’s targets.
Both the copying of the Tests framework to the application and deleting them from the application only work if you specify the application name in build phase locations directly by string. This means duplication, and breakage if you only change the name in one place and not the others.
The to location is the Subpath section of the Copy Files build phase of the Tests target. I can’t use
$(PRODUCT_NAME).$(WRAPPER_EXTENSION) there because those build setting values aren’t set for the Tests target, where the copying has to take place. They’re set for the MyDocApp target, which can be used by either Tests or No-Tests.
The from location is in the script text in the Shell Script Files build phase of the No-Tests target. If you look at that text, you’ll see that I actually delete two things: the framework in
$(TARGET_BUILD_DIRECTORY), which is where it was originally built, and the framework inside the MyDocApp application bundle inside
$(TARGET_BUILD_DIRECTORY), which is where the framework was copied to.
The Simple Life
The first two points describe the fragility of the setup: the changes I can make are more constricted or error-prone than I would like.
But the general complexity of the setup also needs to be noted. My setup has two entirely new embedded frameworks that the OCUnit template does not have. It has five targets instead of two. There are 22 steps in the README to change a generic Cocoa project into this.
Is it worth it? I’ll let you know.
You may have received the impression from my last post, in my ongoing saga to get a OCUnit testing setup I’m happy with, that I had everything built and working correctly.
You would be wrong.
What I had working correctly was the modification of my – rather complex – project file to use two embedded frameworks for, respectively, most of the app’s source code and its tests.
But my testing framework was still strongly linked to the application.
What I’ve done since then is gotten my project to work with a weakly-linked testing framework, and made aggregate targets that can, without any recompilation, either run the application with the tests or run the application without the tests, merely by copying in or removing the testing framework.
It’s still far more complicated and fragile than I would like, but it works! And in addition, in order to placate my demanding readers, I have made a far simpler sample Cocoa document-based application, called MyDocApp, which has all the same changes I made to my complicated project, and a README file which explains what those changes were. It’s a long list!
MyDocApp is available at http://umbar.com/macdev/MyDocApp.zip.
…he was riding into the sunset with the cry, “It’s time to try it out for real!”
The good news: after spending lots of time converting my application to the two-framework setup I talked about in my I’ve Been Framed! post, I have it working again.
The bad news: I continued having problems even after I solved the ones I talked about in my Looks Can Be Deceiving post.
Now, a lot of this is because I decided, as long as I was embedding my own frameworks, in addition I would try to embed the Omni frameworks I was also using. Not only embed those frameworks, but also put dependencies on those framework projects in my own project, so everything would always get built correctly.
Using that setup, I kept getting strange compilation errors that would go away or change after each clean rebuild. Undefined symbol errors, code logic compilation errors that only showed up after the third rebuild, inability to find the embedded frameworks. You name it.
In general, my advice is: keep fiddling with it, you’ll get it to work eventually. But don’t expect a cakewalk.
Specifically, about the app framework setup:
main.c*/m*must be in the app itself. All other code can be in the app framework.
- MainMenu.nib must be in the app itself. All other nibs can be in the app framework.
- The Finder icons for the application and any of its documents must be in the app itself.
2. means that you may need to put a few tests in the app itself, not just in the tests framework, if you want to test something in MainMenu.nib.
And specifically, about the Omni frameworks:
- Add the Omni projects to your project and use the framework references within those projects – don’t also add the frameworks to your project under the Frameworks group.
- I found no way around modifying the Omni projects themselves to have an
INSTALL_PATHof “@executable_path/../Frameworks”. It would be nice if you didn’t have to touch those projects at all for this purpose, but it looks like you do.
- You must link the Omni frameworks to both your app framework and the app itself, or you may get linker errors.
- As I warn in my Don’t Use the Brown Project File post, at this time, you really really should use the
*.pbprojversions of the Omni projects, not the