Saturday, April 16, 2005

Unit Testing Structure Via Reflection

I love testing my code. I'm still not to writing the test first though. I tend to write a small bit of code and then, write the test. Bad monkey, I know! But, I do testing and coding in small steps at least. Writing the test first gets the protocol to feel right from the get go. Testing is great to ensure my code works correctly but, what if I want to test the structure of my code? You might ask why anyone would want to do that. For one, I found it great to make sure code is used correctly. For instance, I've been writing a tolerant XML parser (that also parses HTML) for use in some of my projects. In my parser, I use a temporary output stream that I keep around for purely performance reasons. The only problem is that if two nested calls try to use it, there is a clash and weird results happen. So, I restricted the use to one method. But, what if I forget about this method and use the direct accessors? Even in languages that provide constricted access, this would be a problem since access is local to the object and "private" would still make it accessible. So, I wrote a test to tell me when I have done something wrong and tell me! First, let me show you the one method that I want the internal methods in my class to call:
    useOutputDuring: aOneArgBlock

    self output position > 0 ifTrue: [self warning: 'Output is being used'].
    [aOneArgBlock value: self output.
    ^self output contents]
    ensure: [self output resetToStart]
I basically send in a block that takes the stream as an argument. It then returns the contents of the stream and resets the stream for the next user. I also added a check in the beginning to warn me if it gets invoked from nested calls. Now, here's the test method:

    | accessors localCalls onlyCall |
    accessors := OrderedCollection new.
    self readerClass withAllSubAndSuperclassesDo: [:class |
    accessors addAll: (class whichSelectorsAccess: 'output')].
    self assert: (accessors size = 2).
    self assert: (accessors allSatisfy: [:each | each = 'output' or: [each = 'output:']]).
    localCalls := self readerClass allLocalCallsOn: #output.
    self assert: (localCalls size = 1).
    onlyCall := localCalls anyOne readStream upTo: Character space; upToEnd.
    self assert: onlyCall = 'useOutputDuring:'
The first two asserts make sure that only one setter and getter access the instance variable, output. I like accessors, so I doubt I will ever violate those, but you never know when a brain fart might occur. The last two asserts are to make sure that there is only one sender of the "output" method and that it is the "useOutputDuring:" method. This test is super easy with Smalltalk's metaclass facilities where not only can I query a class's method and instance variables, but I can also ask questions of the code itself. Smalltalk is super nice in the fact that I can questions like "Who accesses this variable?" and "Who sends this method locally?" Very powerful stuff to use ensure code is used correctly or at least warn a developer about it.

There's a lot of possibilities to explore here. One use could be to make sure access to certain methods is caught. It might be fine to call the method, you just might want to make someone think before they use it. Think of some of the lint checks for "become:". And speaking of lint, you could have lint tests like this to make sure that are no non-referenced instance variables in your classes or senders of "halt". Just another testament to the power that we enjoy in Smalltalk.


Ryan Davis said...

Hey Blaine,

You might want to check out ZenTest, a package I wrote for Ruby that uses reflection to audit and generate both test and code. It uses naming convention and a simple 1:m mapping rule to see what is missing on either side then generates appropriate test and code (I need a better name for the non-test method) methods that raise/flunk.

I've been using it for quite some time now and really love it. I'll write tests first and generate the implementation, and I'll use it to catch up on legacy code that I inherit.

Blaine said...

COOL! I checked it out. I can't wait to start using it myself. Thanks for the info!