Using “Refactoring” as an Excuse to Skip Writing Tests

I caught myself falling into a bad practice this week. I was starting to use the “Refactor” phase of the TDD loop (Write a test, Write code to pass test, Refactor) to implement new code. In the Who’s Working On What code, at the end of May 27th, commit fb9990…, I had just about everything working with just a single project displayed for each engineer.

So the next thing I wanted to implement was to display more than 1 project for (some) engineers. I’d setup the simple structures to allow 2 to be displayed, knowing that at some point I’d need to make this a variable number using an array of projects.

So I erroneously thought to myself, “let’s just refactor the code to use an array instead of fixed properties. I already have a bazillion tests (about 300), so they should catch any issues.

This might have been an ok approach (but not a great one), except it became hard to tell when I started adding new functionality. Luckily, I had chosen to work off of a branch. When it became apparent that what I was doing was wrong, I simply killed the branch and started over. The problem with that approach is that it allows new code to be written (under the guise of “refactoring”) that might not be actually tested. And one of the chief benefits of TDD is that you can be sure that all the code you write is tested.

So, another lesson learned (or “relearned”?). Only write code that the tests compel you to write.

“Duplicate Classes with Same Name” Bug

I ran into the most puzzling issue this past week. While working with the new MMDrawerController package on Github (written by some friends here at Mutual Mobile), I ran into a situation where the test runtime believes that there are 2 separate classes, both named “MMDrawerBarButtonItem”.

The failing source code for the project is on Github in WhosWorkingOnWhat at commit c19a6a7de5cda609c75b70d93aed91687fdfcf00. You can uncomment the code in that commit to see that the log shows that there are 2 subclasses of UIBarButtonItem with the same name “MMDrawerBarButtonItem”.

What’s happening in testNavBarLeftButton is that the code under test assigning an instance of one class named “MMDrawerBarButtonItem” to the leftBarButtonItem. The test is then trying to confirm that an instance of the correct class has been assigned to the left button,  but when the test compares it with [MMDrawerBarButtonItem class], it is getting the second class with that name, and therefore failing the assertion!

This is a most peculiar bug, and so far has me completely stumped. Any ideas on how my tests are managing to create 2 classes with the same name?

Update 5/28/13:

Last week one of the senior iOS engineers ran into a similar problem on his project, and determined the cause and a workaround. The problem is caused by the combination of CocoaPods and OCUnit Application tests.

CocoaPods was used to add a 3rd party library. In my case, it is MMDrawerController, in his it was something else. Then in order to test code that interacts with that 3rd party library, the Podfile was updated to add the same library to the unit test target. And voila, when tests are run, 2 instances of the classes in the 3rd party library will exist in the combined product code + unit test code test target.

Normally the compiler and linker would detect this duplication, but no errors are generated  when the duplicates exists separately in the product target and the unit test target, which then get combined when running the unit tests.

My workaround was to simply stop using CocoaPods. My friends solution involved putting dependencies into a separate static library. There’s probably a better solution, and I’ll update this when it is identified.

OCUnit Logic Tests Are Dead

… long live OCUnit Application tests.

I’ve been espousing the use of logic tests for quite awhile now. But recently, I’ve had a chance to work with some projects that have a fairly large number of OCUnit application unit tests. To my surprise, these tests run very fast, roughly sub-second per hundreds o tests.

Of course, test execution times will vary depending on the individual project and tests, but it appears that in general, there’s no need to jump through hoops using OCUnit Logic tests, since simply using OCUnit Application tests should work just fine.

This also applies to GHUnit. I’ve seen a project with over 600 tests execute in under 2 seconds.

Anyone have observations that disagree with this?


I Should Have Unit Tested

Ok, it’s confession time. A couple weeks ago I started a working on what I thought would be a very simple app. It’s essentially just a tableview with sort/filter options to keep track of what the 60 or so engineers in the department are working on. It would be stand alone, with the data entered and updated manually. I expected to be the only one using this app, so I chose to just throw it together without unit tests.

What? I know, right? What was I thinking? I was thinking that TDD didn’t make sense for what amounted to a simple prototype. I was thinking that TDD didn’t make sense if it was a personal use app. I think I wasn’t thinking clearly.

So tableviews are fairly simple creatures, with the SDK doing all of the heavy lifting. At the same time though, creating unit tests for it is fairly simple also, since it’s behavior is so well defined. But I needed more than just a tableview. I need a data entry screen to manually enter and edit the 60 engineer records. And that’s where the problems started occurring.

Keyboard dodging is a bit tricky (scrolling an entry field so it isn’t under the keyboard when it appears). I spent a lot of time in trial-and-error getting it working, and it still isn’t working correctly in all situations.

And data needs to be converted and formatting to and from dates and percentages. And that starts to get tricky. Again, I spent a lot of trial-and-error time getting that working.

But then I hit a strange bug, and have spent the past couple days trying to debug it. And then it struck me: the time that I had spent debugging this code could have been used to create unit tests first.

So consider this my formal apology to everyone. I’ve been espousing TDD for a long time, and using this exact situation as one of the compelling reasons. Sorry, I guess I fell of the TDD wagon, but I’m back on now.  I’m restarting this project, and you can be sure that I’ll be using TDD for everything going forward.

HowsMyFuel Step 2: Time Display

Ok, so having created the basic app with a map view using TDD, the next step is to add a time display. Creating a time display is really easy in iOS. Doing it using TDD and a well planned architecture maybe not so easy.

Time events and information may be needed in several places in the app. In addition to simply displaying the current time, I may need time information to determine if/when I’ve stopped at a gas station. So I probably need a time service to provide time notifications than can be observed wherever needed. I could sprinkle NSTimer and NSDate code around the app wherever time info is needed, but that would be ugly, and violate the DRY principle (“Don’t Repeat Yourself”). In a TDD/unit tested environment, having a single copy of well tested code becomes even more of a benefit.

So I’m going to create a model object (eg MVC) to generate time notifications every second.

Next question: I’m going to need time formatted into a text string for display. Where should we put the text string formatting code? We’re trying to keep the ViewController thin, so that means moving it up into the Presenter (since it is UI related), or over into the Time model. So I ask myself the question: “In the future, when I’ve long forgotten how I implemented this code, where will I look for it?” In this case, there isn’t a clear answer. The Time model will generate notifications which the Presenter will observe, and the Presenter will then send a message to the ViewController to display/update it’s time display. The Time model needs to understand the NSTimer and NSDate classes, so maybe if I do the formatting in the Time model, I can relieve other objects from having to. So I’m going to include a formatted time string in the notifications that the Time model broadcasts. If more control, or multiple formats are need in the future, then I can easily extend the Time model to do so. For now, we’ll stick with the TDD concept of doing the minimum thing.

So to follow along in the code, refer to the Commit log on Github starting April 27, 2013.



First Observations on TDD with VIPER

In a previous post, I mentioned that I was rewriting WhatsMySpeed (now called HowsMyFuel and posted publicly on Github) using TDD and the VIP architecture. I’m committing changes at each step of the TDD cycle. I’m exploring to see if commit comments on Git can be used blog style. Let me know if this works for you.

One of the reasons in VIP of separating the ViewController into 3 components (ViewController+Presenter+Interactor) is to eliminate the Massive View Controller, so common in iOS apps. I’m discovering that one of the up sides to doing this is that I can move many of the unit tests from application to logic tests.

View controller tests typically require running in OCUnit application tests. OCUnit application tests are slower, since the app bundle is loaded. I don’t know how much slower yet, but I hope to test this later, as the number of unit tests gets larger.

So what goes into the View Controller vs the Presenter or Interactor? Normally the View Controller does nearly all the work. That’s why iOS view controllers tend to be huge. And huge typically means hard to read, even harder to unit test. However, I’m attempting to reduce the size and complexity of the view controller by removing all business logic. By that, I mean the logic that decides what an event does.

ViewController should contain basic iOS glue stuff; outlets, actions, etc.

Presenter should contain display-specific business logic, without interacting with the models.

Interactor should implement the business logic, aka use cases. It maps model data to the form needed by the Presenter and/or ViewController.

For example, the HowsMyFuel code as of April 25, 2013 contains code to register a tap gesture recognizer, and handle the tap event to recenter the map display when it is tapped. As is often the case, the event handling code is about 5 times the size of the business logic, which obscures the business logic, or intended behavior. To fix that, I’m going to elevate the business logic into the Presenter, where the expected behavior will be made crystal clear (when map is tapped, recenter the display on the user). Since this behavior is related directly to the UI display, I’ll put it in the Presenter. If it was dealing with Model data and/or not UI display specific, then I’d elevate it even higher to the Interactor.

One Unit Test Framework to Rule Them All

I kid myself about changing my mind daily regarding the “best” unit testing frameworks. I’ve been advocating for Kiwi for several months now. Well, here goes again.


Pros: I absolutely love Kiwi. It allows test to be well organized (RSpec style), has excellent mock support (including Class mocks), is fast, and is easy to use (runs on top of OCUnit logic tests), and has asynchronous support.

Cons: Debugging tests is difficult. Whenever I try to debug tests, I typically get dumped into assembler code. Yech!

Maybe I just don’t understand how to debug block code.

I’ve also been considering whether having disorganized tests is actually a symptom (smell) of having classes that are too big. I’ve been reviewing an associate’s code that uses OCHamcrest and Mockito, and I must say that I find it quite readable.

So, the current HowsMyFuel TDD project (April 22, commit 068c19d75e) fails unit tests, and it isn’t clear why. I could probably guess, and use trial-and-error to fix it, but I’d prefer to have a clear diagnostic approach to debugging and fixing tests.

So I’m going to rewrite the tests using OCUnit, OCHamcrest, and OCMockito and see if the error occurs, and if so, how tough it is to debug and fix.

Moving OCUnit tests between Logic and Application tests

Sometimes logic tests will crash because the code being exercised is trying to access things that don’t exist unless the application is loaded. For example, simply trying to instantiate an instance of UIFont will work in an OCUnit Application test, but crash in an OCUnit Logic test.

Another example I recently ran into was during testing of the AppDelegate. The application:didFinishLaunchingWithOptions created by the Empty Application project template can be run and tested with logic tests, but the Single View project template’s version will crash, requiring it to be run in an application test.

The difference between a logic test and an application test is that the application test target is configured to execute in the context of the loaded application. This is done by setting the bundle loader and test host build settings. The Apple developer documentation Setting Up Application Unit Tests describes in detail how to do this.

It is interesting to note, that it was the call to makeKeyAndVisible that crashed, and that call is present in both versions. It wasn’t until after I created a view and made it the rootViewController prior to calling makeKeyAndVisible that it started crashing.

This creates somewhat of a problem for TDD. The basic philosophy behind TDD is to create new tests, watch the tests fail, then implement code to make the tests pass. There is an expectation that new code added will not break previously passing tests unless a mistake is made. But this is exactly what happened.

So I’m currently working on a workflow where I create all tests as Logic tests, then move them to the Application test target if/when the application/bundle is needed. Both targets can be added to the Cmd+U Test action, keeping things fairly simple.

For an example of this code, refer to my HowsMyFuel project on Github. Take a look at the commit comments to see how the code progressed from Logic tests only, to crashing Logic tests, to finally adding Application tests and moving the failing test over.

Update 6/5/13: I now use Application tests almost exclusively. Once the situation described above is understood, then the change in behavior can be acknowledged, and tests updated appropriately. This may even result in an increased understanding of the run time behavior, which is a good thing.

VIP Architecture

One of the architects at work has started exploring using an architecture he is calling VIP (View-Interactor-Presenter?). He started thinking about this idea from reading an Uncle Bob blog entry on Yesterday he presented his ideas to the group during a brown bag session, and it got me thinking about the implications of such an architecture to unit testing.

The main idea is to abstract and extract use cases up out of the other components, primarily the view-controller, into a separate, simple NSObject called the Interactor. This object then deals with implementing use cases (aka business logic).

In a sense, the traditional MVC model becomes the Presenter component, used similarly as in an MVP architecture.

What about the Model? This is still a bit fuzzy in my thinking, but there can actually be two different types of model, and places where they exist. The Interactor interfaces with the traditional model, which could be CoreData, a RESTful API extraction layer, local persistence, etc. A key concept though, is that this Model uses fully abstracted, reusable objects, not customized for a particular presentation method or style.

Within the Presenter component, there may exist another presentation-specific model object that converts the reusable Model object into a form needed for presentation. Like I said, still too fuzzy of a definition, but it feels like a good starting point.

What does this mean, really? I think it means that business logic gets put into a single place, instead of being scattered around in various other places. The really big implication here is that the high level behavior of the system is defined in abstract terms in the Interactor! I think this makes it possible to do TDD starting with tests built from Use Cases, which I believe is the definition of Acceptance Test Driven Development (ATDD).

I’m going to start redoing my What’s My Speed sample app this weekend to see how well this concept works, hopefully firming up some of the fuzzy areas. In order to really test the ATDD aspects, I’ll need to start from scratch, creating and working from User Stories. I’ll be posting my progress as I go.

Update: I’m renaming the new code HowsMyFuel to avoid confusion with the older code. HowsMyFuel will use OCUnit, OCHamcreset, and OCMockito for unit testing. The code is posted publicly on Github.

Kiwi + Application Tests = ATDD?

I’ve been playing with various ways that Kiwi can be used for testing code. It certainly has powerful capability for performing low level unit testing in a very organized way. I’ve been doing this using strictly OCUnit logic tests.

I don’t tend to use OCUnit application tests much, since they tend to be slower. And frankly, it didn’t even occur to me to try running Kiwi under application tests until last week. I ran into a situation where the code that I’ve been writing is 100% unit tested, and passing all tests, but the app doesn’t do what I want it to do.

Upon thinking about how this happened, I realized that my TDD focus has caused me to think at too low a level. I had skipped too quickly through the much needed high level design in order to quickly get to writing tests.

So I began thinking about how I could write higher level tests first, akin to acceptance tests, and use those to drive TDD at a lower level. This prompted me to think about whether Kiwi would run on Application tests, which I now know works.

So I’m heading down a direction where I’ll be using Kiwi in OCUnit application tests to define expected behavior (story acceptance tests), and then Kiwi in OCUnit logic tests to implement those behaviors via TDD. I’m exciting about this approach, and I’ll be updating this post going forward regarding how it is working.