Rails Test Prescriptions Blog

Keeping Your Application Healthy Since 2008

Category Archives: Rails

Setting Up Fast No-Rails Tests

The key to fast tests is simple: don’t do slow things.

Warning: this post is a kind of long examination of a problem, namely, how to integrate fast non-Rails tests and slow Rails tests in the same test suite. This may be a problem nobody is having. But having seen a sample of how this might work, I was compelled to try and make it work in my toy app. You’ve been warned, hope you like it.

In a Rails app, “don’t do slow things” largely means “don’t load Rails”. Which means that the application logic that you are testing should be separable from Rails implementation details like, say, ActiveRecord. One way to do that is to start putting application logic in domain objects that use ActiveRecord as an implementation detail for persistence.

By one of those coincidences that aren’t really coincidences, not only does separating logic from persistence give you fast tests, it also gives you more modular, easier to maintain code.

To put that another way, in a truly test-driven process, if the tests are hard to write, that is assumed to be evidence that the code design is flawed. For years, most of the Rails testing community, myself included, have been ignoring the advice of people like Jay Fields and Michael Feathers, who told us that true unit tests don’t touch the database, and we said, “but it is so easy to write a model test in Rails that hits the database, we are sure it will be fine.” And we’ve all, myself included, been stuck with test suites that take way too long to run, wondering how we got there.

Well, if the tests get hard to write or run, we’re supposed to consider the possibility that the code is the issue. In this case, that our code is too entangled with ActiveRecord. Hence, fast tests. And better code.

Anyway, I built a toy app placing logic in domain objects for the Mountain West workshop. In building this, I wanted to try a whole bunch of domain patterns at once, fast tests, DCI, presenters, dependency injection. There are a lot of things that I have to say about messing around with some of the domain object patterns floating around, but first…

Oh. My. God. It is great to be back in a code base where the tests ran so fast that I didn’t have time to lose focus while the tests ran. It occurred to me that it is really impossible to truly do TDD if the tests don’t run fast, and that means we probably have a whole generation of Rails programmers who have never done TDD, who only know tests as the multi-minute slog they need to get through to check in their code, and don’t know how much fun fast TDD is.

Okay, at some unspecified future point, I’ll talk about some of the other patterns. Right now, I want to talk about fast tests, and some ideas about how to make them run. While the basic idea of “don’t do slow things” is not hard, there are some logistical issues about managing Rails-stack and non-Rails stack tests in the same code base that are non obvious. Or at least they weren’t obvious to me.

One issue is file logistics. Basically, in order to run tests without Rails, you just don’t load Rails. In a typical Rails/RSpec setup, that means not requiring spec_helper into the test file. However, even without spec_helper, you still need some of the same functionality.

For instance, you still need to load code into your tests. This is easy enough, where spec_helper loaded Rails and triggered the Rails auto load, you just need to explicitly require the files that you need for each spec file. If your classes are really distributing responsibility, you should only need to require the actual class under test and maybe one or two others. I also create a fast_spec_helper.rb file, which starts like this:

$: << File.expand_path("app")
require 'pry'
require 'awesome_print'

Pry and Awesome Print are there because they are useful in troubleshooting, the addition to the load path is purely a convenience when requiring my domain classes.

There is another problem, which is that your domain classes still need to reference Rails and ActiveRecord classes. This is a little messier.
I hope it’s clear why this is a problem – even if you are separating domain logic from Rails, the two layers still need to interact, even if it’s just of the load/save variety. So your non-Rails tests and the code they call may still reference ActiveRecord objects, and you need to not have your tests blow up when that happens. Ideally, you also don’t want the tests to load Rails, either, since that defeats the purpose of the fast test.

Okay, so you need a structure for fast tests that allows you to load the code you need, and reference the names of ActiveRecord objects without loading Rails itself.

Very broadly speaking, there are two strategies for structuring fast tests. You can put your domain tests in a new top-level directory – Corey Haines used spec-no-rails in his shopping cart reference application. Alternately, you can put domain tests with everything else in the spec directory, with subdirectories like spec/presenters and the like, just have those files load your fast_spec_helper. About a month ago, Corey mentioned on Twitter and GitHub that he had moved his code in this direction.

There are tradeoffs. The separate top-level approach enforces a much stricter split between Rails tests and domain tests – in particular, it makes it easier to run just the domain tests without loading Rails. On the other hand, the directory structure is non-standard, there is a whole ecosystem of testing tools that basically assumes that you have one test directory.
It’s not hard to support multiple spec directories with a few custom rake tasks, though it is a little awkward. Since your Rails objects are never loaded in the domain object test suite, though, it’s very easy to stub them out with dummy classes that are only used by the domain object tests.

As I mentioned, Corey has also shown an example with all the tests under single directory and some namespacing magic. I’m not 100% sure if I like the single top-level better. But I can explain how he got it to work.

With everything being under the same top level directory, it’s easier to run the whole suite, but harder to just run the fast tests (not very hard, just harder). Where it gets weird is when your domain objects reference Rails objects. As mentioned before, even though your domain objects shouldn’t need ActiveRecord features, they may need to reference the name of an ActiveRecord class, often just to call find or save methods. Often, “fast” tests get around this by creating a dummy class with the same name as the ActiveRecord class.

Anyway, if you are running your fast and slow tests together, you’re not really controlling the order of test runs. Specifically, you don’t know if the ActiveRecord version of your class is available when your fast test just wants the dummy version. So you need dummy versions of your ActiveRecord classes that are only available from the fast tests, while the real ActiveRecord objects are always visible from the rest of the test suite.

I think I’m not explaining this well. Let’s say I have an ActiveRecord object called Trip. I’ve taken the logic for purchasing a trip and placed it in a domain object, called PurchaseTripContext. All that’s fine, and I can test PurchaseTripContext in a domain object test without Rails right up until the point where it actually needs to reference the Trip class because it needs to create one.

The thing is, you don’t actually need the entire Trip class to test the PurchaseTripContext logic, you just need something named Trip that you can create, set some attributes on, and save. It’s kind of a fancy mock. And if you just require the existing Trip, then ActiveRecord loads Rails, which is what we are trying to avoid.

There are a few ways to solve this access problem:

If you have a separate spec_fast directory that only runs on its own, then this is easy. You can create just a dummy class called Trip – I make the dummy class a subclass of OpenStruct, which works tolerably well. class Trip < OpenStruct; end.

You could also use regular stub, but there are, I think, two reasons why I found that less helpful. First is that the stubs kind of need to be recreated for each test, whereas a dummy class basically gets declared once. Second, OpenStruct lets you hold on to a little state, which – for me – makes these tests easier to write.

Anyway, if your domain logic tests are mixed into the single spec directory, then the completely separate dummy class doesn’t work – the ActiveRecord class might already be loaded. Worse, you you can’t depend on the ActiveRecord class being there because you’d like to run your domain test standalone without running Rails. You can still create your own dummy Trip class, but it requires a little bit of Ruby module munging, more on that in a second.

If you want to get fancy, you can use some form of dependency injection to make the relationship between TripPurchaseContext and Trip dynamic, and use any old dummy class you want. One warning – it’s common when using low-ceremony dependency injection to make the injected class a parameter of the constructor with a default, as in def initialize(user, trip_class = Trip). That’s fine, but it doesn’t completely solve our testing problem because the use of Trip in the parameter list needs to be resolved at load time, so the constant Trip still needs some value.

Or, you could bite the bullet and bring the Rails stack in to test because of the dependency. For the moment, I reject this out of hand.

This isn’t an exhaustive list, there are any number of increasingly insane inheritance or metaprogramming things on the table. Or under the table.

So, if we choose a more complicated test setup with multiple directories, we get an easy way to specify these dummy classes. If we want the easier single-directory test setup, then we need to do something fancier to make the dummy classes work for the fast tests but be ignored by the Rails-specific tests.

At this point, I’m hoping this makes sense. Okay, the problem is that we want a class to basically have selective visibility. Here’s the solution I’m trying – this is based on a gist that Corey Haines posted a while back. I think I’m filling in the gaps to make this a full solution.

For this to work, we take advantage of a quirk in they way Ruby looks up class and module names. Ruby class and module names are just like any other Ruby constant. When you refer to a constant that does not have any scope information, like, say, the class name Trip, Ruby first looks in the current module, but if the current module doesn’t contain the class, then Ruby looks in the global scope. (That’s why sometimes you see a constant prefixed with ::, as in ::Trip, the :: forces a global lookup first).

That’s perfect for us, as it allows us to put a Trip class in a module and have it shadow the ActiveRecord Trip class in the global scope. There’s one catch, though – the spec, the domain class, and the dummy object all have to be part of the same local module for them all to use the same dummy class.

After some trial and error (lots of error, actually), here’s a way that I found which works with both the fast tests and the naming conventions of Rails autoload. I’m not convinced this is the best way, so I’m open to suggestions.

So, after 2000 words of prologue, here is a way to make fast tests run in the same spec directory in the same spec run as your Rails tests.

Step 1: Place all your domain-specific logic classes in sub modules.

I have sub directories app/travel/presenters, and app/travel/roles and the like, where travel is the name of the Rails application. I’m not in love with the convention of putting all the domain specific directories at a separate level, but it’s what you need to do in Rails to allow autoloaded classes to be inside a module.

So, my PurchaseTripContext class, for example, lives at app/travel/contexts/purchase_trip_context.rb, and starts out:

module Contexts
  class PurchaseTripContext
    # stuff
  end
end

Step 2: Place your specs in the same module

The spec for this lives at spec/contexts/purchase_trip_context_spec.rb (yes, that’s an inconsistency in the directory structure between the spec and app directories.) The spec also goes inside the module:

module Contexts
  describe PurchaseTripContext do
    it "creates a purchase" do
      #stuff
    end
  end
end

Step 3: Dummy objects

The domain objects are in a module, the specs are in a module, now for the dummy classes. Basically, I just put something like this in my fast_spec_helper.rb file:

module Contexts
  class Trip < OpenStruct; end
  class User < OpenStruct; end
end

This solves the problem, for some definition of “solves” and “problem”. The fast tests see the dummy class, the Rails tests see the Rails class. The tests can be run all together or in any smaller combination. The cost is a little module overhead that’s only slightly off-putting in terms of finding classes. I’m willing to pay that for fast tests. One place this falls down, though, is if more than one of my sub-modules need dummy classes – each sub-module then needs its own set, which does get a little ugly. I suspect there’s a way to clean that up that I haven’t found yet.

In fact, I wonder if there’s a way to clean up the whole thing. I half expect to post this and have somebody smart come along and tell me I’m over complicating everything – wouldn’t be the first time.

Next up, I’ll talk a little bit about how some of the OO patterns for domain objects work, and how they interact with testing.

Cucumber Rails 0.4: The De-Web-Step-ining

Consider this part of an occasional series where I attempt to revisit tools discussed in Rails Test Prescriptions that have undergone some revision. (NOTE: Most of this was written before the DHH Twitter-storm about testing this week. For the purposes of this post, I’m choosing to pretend the whole thing didn’t happen.)

The cucumber-rails gem released version 0.4 last week, which had some significant changes, and intensified what we might call the opinionated nature of Cucumber over what a Cucumber scenario should look like.

If you update cucumber-rails, you need to re-run the rails generate cucumber:install to see the new stuff.

There are a couple of minor changes — the default env.rb file is much simpler, the capybara date selector steps now work with Rails 3, that kind of thing. The biggest change, though is conceptual, and comes in two parts.

Part one is best laid out by the new first line of the web_steps.rb file:

# TL;DR: YOU SHOULD DELETE THIS FILE

The header goes on to say that if you make use of these steps you will end up with verbose and brittle cucumber features. Also, your hair will fall out, and you will have seven years bad luck. The last may be more implied than stated.

Why would they do such a thing? And what’s the “official” preferred way to use Cucumber now?

Well, it’s not like the Cucumber dev team has me on speed-dial or anything like that, but since they subtly included in the web_steps.rb file links to, count ’em, three separate blog posts explaining how to best use Cucumber, I will follow that subtle, yet blazing, trail and try to put it together in some coherent way so that I can understand it.

(Note to Cucumber dev team: if you feel the need to link to this post in future versions of Cucumber, you should consider yourself as having permission to do so….)

Anyway, the Cucumber team is making a very opinionated statement about how to use Cucumber “with the grain”, and I actually don’t think that statement is “don’t use the web_steps” file — I think that some parts of the web_steps file have a place in the Cucumber world.

Here’s the statement as I see it:

  • A Cucumber scenario is an acceptance test.
  • As such, the scenario should completely be in the domain of the user.
  • A Cucumber scenario should not have any reference to implementation details.
  • Implementation details include, but are not limited to: CSS selectors, class names, attribute names, and HTML display text.

As a good rule of thumb, if you are putting something in your Cucumber steps in quotation marks, you should at least think about whether your Cucumber scenario is at a high enough level. In the Cucumber world, the place for implementation-specific details is in the step definition files. If the acceptance criteria changes, the scenario should change, but if the implementation changes, only the step definitions should change.

This sharp separation between the acceptance test and the implementation is a feature, not a bug, in Cucumber (By the way, you do not want bugs in your cucumbers. Yuck.) The separation is what makes Cucumber a true black-box test of your application, and not a black box riddled with holes.

That said, full-stack testing that is based on knowing implementation details — which is “integration testing” rather than “acceptance testing” — is a perfectly valid thing to do, especially in a case where there isn’t an external customer that needs or wants to see the acceptance testing. But, if you are actually doing integration testing, then you don’t need the extra level of indirection that Cucumber offers — you should drop down to Steak, or Rails integration tests, or the new Capybara acceptance test DSL or something.

Okay, so. Acceptance testing is not integration testing, and if you are trying to do integration testing via Cucumber, you will be frustrated, because that’s not what Cucumber is good at. To me, there’s a value in acceptance testing, or in this case, acceptance test driven development, because it’s helpful to try and describe the desired system behavior without any implementation details confusing the issue.

Which brings us back to the question of how you actually replace the web steps in your Cucumber scenarios. Essentially the idea is to replace implementation-based steps with steps that describe behavior more generically. You might have something like this:

Scenario: Updating a user profile
  Given a user named "Noel" with a preference for "Cool Stuff"
  When I go to the edit profile page
  And I fill in "bananas" for "Favorite Food"
  And I select "Comic Books" from "Preferences"
  And I press "Submit"
  Then I should see "Bananas" 
  And I should see "Comic Books"

That’s not horrible, because it doesn’t have any explicit CSS or code in it, but it’s still very much about implementation details, such as the exact starting state of the user, the labels in the form, and the details of the output. On the plus side, the only step definition you’d need to write for this is for the first step, every other step is covered by an existing web step. But… I’ve written my share of Cucumber scenarios that look like this, and it’s not the best way to go. It’s hard to tell from this step what the most important parts are and what system behavior is actually being described.

The implicit version of the scenario looks more like this:

Scenario: Updating a user profile
  Given I am an existing user with a partially completed profile
  When I go to edit my profile
  And I fill in new preferences
  Then I see my new preferences on my profile page

Two questions to answer: why is this better, and how does it work?

The second question first. We need to write step definitions for all these steps. Normally, I write these in terms of the underlying Capybara or Webrat API rather than calling web steps. The second step doesn’t need a full definition, it just needs an entry for /edit my profile/ in the paths.rb file (right now, it seems like that’s about the only step in the web steps file that the Cucumber team is willing to use), but the other three steps need definitions — here’s what they might look like, this might have a typo or syntax jumble, it’s just the basic idea.

  
Given /^I am an existing user with a partially completed profile$/ do
  @user = Factory(:user)
  @user.profile = Factory(:profile, :preference => "Cool Stuff",
     :favorite_food => nil)
end 

When /^I fill in new preferences$/ do
  fill_in("Favorite Food", :with => "Bananas")
  select("Comic Books", :from => "Preferences")
  click_button("Submit")
end

Then /^I see my new preferences on my profile page$/
  with_scope("preference listing") do
    page.should have_selector(selector_for("bananas are my favorite food"))
    page.should have_selector(selector_for("comic books are my preference"))
  end
end
  

If you are used to Cucumber but haven’t used the 0.4 Rails version yet, the last step will look unfamiliar. Bear with me for a second.

Why is the second version better? It’s not because it’s shorter — it’s a bit longer, although only a bit (the first version would need a step definition for the user step as well). However, the length is split into more manageable chunks. The Cucumber scenario is shorter, and more to the point, each step is more descriptive in terms of what it does and how it fits into the overall scenario. The new step definitions you need to write add a little complexity, but not very much, and my Cucumber experience is that the at this size, the complexity of the step definitions is rarely the bottleneck. (For the record, the bottleneck is usually getting the object environment set up, followed by the inevitable point of intersection with implementation details, which is why I’m so keen to try and minimize intersection with the implementation.)

Yes, the scenario is something you could show a non-developer member of the team, but I also think it’s easier for coders to comprehend, at least in terms of getting across the goals of the system. And this is supposed to be an acceptance test — making the goals of the system explicit is the whole point.

Okay, either you believe me at this point or you don’t. I suspect that some of you look at the step definitions and say “hey, I could string those seven lines of code together and call it a test all by itself”. Again, if that’s what works for you, fine. Any full-stack testing is probably better than no full-task setting. Try it once, though. For me.

Back to the step definitions, the last one uses the selector_for method — and I hope I’m using it right here because I haven’t gotten a chance to work with it yet, and the docs aren’t totally clear to me. The idea behind selector_for is to be analogous to the path_to method, but instead of being a big long case statement that turns a natural language phrase into a path, it’s a big long case statement that turns a natural language phrase into a CSS selector. The big lng case statement is in the support folder in a selectors.rb file. The with_scope method uses the same big case statement to narrow the statements inside the block to DOM elements within the snippet.

As with the paths, the idea is to take information that is implementation specific and likely to be duplicated and quarantine it into one particular location. As I said, I haven’t really incorporated this into my Cucumber routine yet, my first thought is that it’ll be nice to hide some of the complex CSS selectors I use in view testing, but I worry that the selectors.rb file will become a mess and that there’s less probability of duplicating a snippet.

I sure wish I had a rousing conclusion to reward you as this post nears the 1750 word mark. I like the direction that these changes are taking Cucumber, they are in line with what I’ve found to be the best use of the tool. Take a chance and try writing tests as implicitly as you can, as an exercise and see if it works for you.

Creating, Sending, and Verifying CSV files using Comma

Here’s something I haven’t done in a while — a genuine code blog entry. I needed to add a simple CSV file output, here’s how I did it, tests and all.

I used two gems, FasterCSV, which I assume that most of you are familiar with, and Comma, which is a nice little DSL for specifying CSV formats. (Thanks again to Jason Pearl for reminding me what the gem was called…).

My Cucumber test for the feature looked like this (details have been changed to protect the innocent…)

  Background:
    Given I have valid adminstrative credentials
    And I have a user named "Charlie"
    And I have a user named "Bravo"
  Scenario: CSV Downloads
    When I go to the admin user page
    And I follow the link "CSV Report"
    Then I should see the table data:
        | Name       | Assigned  Number  | Email                |
        | Charlie    | 123435            | charlie@brandeis.edu |
        | Bravo      | 132134            | bravo@brandeis.edu   |

Just one note on this is that both the assigned number (which you can imagine as an id given to to the user by some other entity, such as a social security number) and the email, are deterministic and generated by the step definition called by the background task.

The downside of this Cuke test is that it’s a little more explicit than I normally like, since the numbers and the email are kind of magical here. That said, the ability to test the tabular CSV data from a Cuke table is powerful enough that this still seemed like the best way. Other opinions welcome.

The only step here that is undefined and interesting is the last one. And you might think it’d be complicated, but in fact it’s super easy:

Then /^I should see the table data:$/ do |table|
  actual_table = FasterCSV.parse(page.body)
  table.diff!(actual_table)
end

This does some complicated things compactly. The result of the eventual controller action is the text of the CSV file, and goes in page.body (since I’m using Capybara). FasterCSV parses that back into an array of arrays, and then I used Cucumber’s table compare method table#diff to compare the parsed CSV table to the table I passed in from the feature file. In theory, Cucumber provides a pretty output if the tables don’t match, but in practice I find I’m currently getting an error in the formatter.

This is nice, with the minor downside that the tables need to be an exact match. If that’s not feasible, then you can do more complex logic there to compare tables.

Next up was the model test, which I wrote more or less like this.

it "should produce expected csv" do
  @user_1 = Factory.build(:user, :name => "Fred", :assigned_number => 1, 
      :email => "fred@fred.com")
  @user_2 = Factory.build(:user, :name => "Barney", :assigned_number => 2, 
      :email => "b@b.com")
  @user_1.to_comma(:admin).should == ["Fred", "1", "fred@fred.com"]
  User.users_to_display.to_comma(:admin).should ==
      "Name,Assigned Number,Email\nBarney,2,b@b.com\,Fred,1,fred@fred.com\n"
  end

This test uses factory_girl to create two users, then uses to_comma to invoke the Comma gem, first on a single user, then on a list of users. As you can see, Comma is smart enough to return an array of data for a single object, but a CSV formatted string when given a list.

Technically, I think the last line is unnecessary, because assuming the first part of the test passes and I know that the list generating function works then I’m really just testing Comma. Of course, since I’m using Comma for the first time, testing it once seems appropriate.

It’s also worth noting that I’m showing the final form of these tests, I did a little test based exploration to see what the format was going to be. So I ran the test and just outputted the result of the to_comma, then wrote the test to match the format that I saw. Obviously, in that case you need to make sure that the format is what you want it to be, not just what the code outputs.

The Comma gem makes the model code super easy.

  comma(:admin) do
    name
    assigned_number
    email
  end

The comma method sets up a comma format, the optional symbol names the format relative to any other comma formats specified for the class. Within the block, the basic idea is that each expression specifies a column of the CSV file, header and all. In this case, I’m using the basic form, which uses attributes of the current instance. Although I’m not doing it here, Comma lets you specify a custom header name, and also allows you to call arbitrary methods of associated objects. I like it, it’s short, easy to read, and to the point.

Comma also lets you easily use the CSV format in a controller. Here’s what the controller method looks like:

  def index
    @users = User.users_to_display
    respond_to do |format|
      format.html #default
      format.csv do
        send_data(@users.to_comma(:admin),
            :filename => "users.csv", :type => "text/csv")
      end
    end
  end 

I didn’t add a controller test, on the grounds that there is no now controller logic and controller behavior is covered by the Cucumber test and the actual output is also covered by the model test (a previous controller test verifies that users_to_display is called correctly.

And that’s it. Turned out to be pretty easy.

July 26, 2010: A Hammer, A Nail, and A Giant Squid

Book Status

Beta 5 should be out early this week, featuring a mostly new chapter on testing legacy projects, and also updating the code setup and the initial walkthrough chapters to Rails 3. Over the next couple betas any remaining Rails 3 incompatibilities will also be fixed.

Book Reviews

Something new for you on a Monday, a couple of novels that I liked in the last couple of weeks.

Kraken, by China Mieville. I’m a huge Mieville fan, so I was excited for this one.

The story starts when the preserved remains of a giant squid are stolen from a London museum, and the curator of the museum is dragged into a world where an apocalyptic squid cult is one of the least weird things going on. It’s much more loose and jokey than Mieville’s other stuff, something like a cross between Mieville’s (oustanding) YA novel Un Lun Dun, Gaiman’s Neverwhere, with the magical sub-world, and a Tim Powers novel a la Last Call or Expiration Date, filled with supernatural creatures who obey obscure and and convoluted supernatural worlds.

Overall the book is a lot of fun — Mieville freely calls it a “shaggy god” story, which should give you an idea of the tone. It’s not perfect; it takes forever to get started, and the flip side of loose and jokey is that sometimes Mieveille spends a lot of time on characters or conceits that don’t tie in. But there are at least five really outstanding, audacious ideas or moments in the book, and for all that it’s loose verbally, the plot comes together nicely at the end. Plus Mieville unironically uses the phrase “squid pro quo”. So how can you go wrong?

Go Mutants!, by Larry Doyle. Doyle is a former Simpsons writer who has written a satirical mash up of pretty much every 50’s B movie. I mean all of them. You’ve got your alien wanting to take over the user, a radioactive ape, a woman who’s head is on a pan, atomic cars, teenage angst, flying saucers. There’s a lot going on, and if you don’t like one of the jokes, wait about a paragraph and there’s another one coming.

Most of the jokes land, and liked the book a lot more than I thought I would once it became clear where it was going. It’s got some heart, and some satirical bite. It’s a little too willing to compromise tone for a quick joke to be truly great, but it’s fun, and if you’ve sat through enough MST3K to see a lot of the referenced movies, you’ll probably like it. It’s got some clever alternate history bits too, for example, the initial alien contact takes place at the Polo Grounds, interrupting the famous “The Giants Win The Pennant” moment.

Links

The Rails Best Practices web site opened up, which is affiliated with the rails-bestpractices gem. Vote for best practices, and they might wind up incorporated in the gem. As I write this, leading the pack is “N+1 Queries” — presumably avoiding them. Don’t see testing yet, though, he said, banging that thing that looks like a nail with that hammer…

Speaking of tools, the Software Craftsmanship North America 2010 conference is October 15 and 16 at the Mariott Chicago O’Hare. Speakers include Dave Astels, Michael Feathers, and “Uncle Bob” Martin.

Aaron Sumner at Everyday Rails has a meta-tutorial of resources for various Rails command line tools, including the Rails command, generators, Rake, and the Unix command line. Take a look if you’d like to get better at navigating from the console.

Tim Bray from OSCON has a true essay on what all of us owe to Perl and to Desperate Perl Hackers. He’s right about what Perl has brought, although it’d still be about my eleventeenth choice of language to use.

I have kind of mixed feelings about this Emma Lindsay post at Thoughtbot’s Giant Robot’s blog. It’s about taking time in a TDD process to consider the overall design of the code. Which you should do, but which I think is already part of the TDD process. I completely agree with the workflow laid out in the post.

Two quibbles. I disagree with the sentence “Test Driven Development rests on the assumption that you basically know the optimal way to make your tests pass in advance.” — I’ve used TDD many times when I had no idea how the tests were going to pass, it works great in that case. Where TDD falls down is when you don’t know the output of the code — if you know the output, but don’t know the algorithm then TDD works fine. Also, the list of TDD steps doesn’t include a refactoring step, which is where, in practice, most of these design decisions would take place. All that said, writing a quick spike is sometimes needed to figure out the problem you want to solve and the tests you need to write.

July 21, 2010: This Code Belongs In A Museum

Status

Not much to tell, really. Spent some time getting RVM set up, since I think I’ll need it to manage simultaneously building the Rails 2 and Rails 3. Now I have a working version of the startup appendix that uses Rails 3 and Devise. I can’t speak to using Devise in practice yet, but the immediate goal of making the setup less complicated was definitely accomplished. The big question now is how much to support Rails 2 in the walk-through chapters. I’m leaning toward just having the walk-through chapters be Rails 3 with only occasional notes on how Rails 2 differs.

Links

Apple has allowed the code for the 1984-era MacPaint and QuickDraw to be released to a museum where anybody can download and peruse. QuickDraw is all in assembler, and MacPaint is a mix of Pascal and Assembler. At a quick glance, the striking thing about the MacPaint Pascal is that it actually seems quite readable. The layout is consistent, it’s commented, variable names have meaning, that kind of thing. Certainly miles ahead of any Pascal code I wrote back when I wrote Pascal code…

This is kind of a cute way to find hidden puts statements by raising exceptions. The project I’m on actually monkey patches puts to add a line to the output saying what file and what line of code the put statement is on. Surprisingly useful.

Claudio Maral has a couple of examples of cleaning up controller code by moving logic to the model.

Oh — Droptext got updated. It’s a universal iOS app for 99 cents that lets you edit files on Dropbox. This release, I think, makes the app functional. Some crash bugs were squashed, and you can now specify alternative text extensions. It’s still pretty simple (I’d love to be able to change the display font, and I’d really love TextExpander integration), but it works and I’ll definitely be giving it more of a spin.

The Ruby Hoedown has announced its speaker list for September 3 and 4. I’ve never been, but I’ve heard good things, and you won’t find a major conference that’s less costly.

July 6, 2010: Opinions are Bad For Business

The title, by the way, is from a favorite scene in a favorite movie.

Status

Now definitely working on the “dealing with legacy code” chapter, which I am hoping will be substantially more useful than the Lulu version, in that it will cover a few more techniques.

I should know later today what the timeline is for beta 4.

Links

Let’s see what we’ve got today.

Nick Quaranto over at Thoughtbot wants you to stop leaving time bombs in your tests. The answer is to use Timecop, or set dates in your tests implicitly. Be sure and check out the comments for some other good ideas.

Kent Beck has the results of a poll on how often teams deploy to production and how often teams run their functional tests. The deploy results are pretty interesting in that more teams deploy far less often than I might have expected.

So I’m still finishing up a post on pairing, but I did want to point out what Will Read suggests, which is that in a pairing environment, developers have more choice over what they do and who they work with. Hmm… that reflects well on Pivotal. My experience is being able to choose is somewhat team dependent. I’ve certainly seen paring environments where the developers didn’t feel like they had much choice in day-to-day tasks.

Ruby 1.9.2 release candidate 1 came out over the weekend, and we’re getting to a point where we’re not going to be able to ignore it any more. And by “we”, I mean “me”. Apparently they keep sneaking in new methods into Enumerable, ranging from the nifty to the seemingly insane. I won’t say which is which, though, go guess for yourself.

Back over at Everyday Rails, Aaron Sumner has a nice set of sensible advice for going from beginner projects to real projects. I’d be linking to it anyway — the fact that one of the pieces of advice is to buy my book is just gravy.

I think this post by Xavier Shay is mis-titled. It’s called, “The Perils of Opinionated Software (like Rails)”, but in argument, it’s more like “Rails Doesn’t Share My Opinions About Database Constraints”. As it happens, I don’t agree with Shay, but then I’m probably not sufficiently enterprise-y. Or, to be more specific, I do agree that database constraints can do what he says and that is sometimes useful, but I’d argue that there is also a cost involved with relying on database features (infrastructure lock-in, more complicated to set up test data, business logic distributed across levels) that I’ve generally found kind of high. This would make a very interesting debate with somebody who is heavily into a NoSQL database.

June 21, 2010: Double Double Splat Splat

Link post today. Turns out I built up more links than I thought.

Book Status

Somehow I wound up writing and editing the Rcov chapter, which, among other things, is the first time I’ve had to wrestle with RSpec 2 vs. RSpec 1 behavior, when writing about how RSpec and Rcov get along. Now I need to figure out how to write about that more coherently. Actually, I need to decide if I’m going to acknowledge RSpec 1 at all.

The book is still on sale, of course. I’ve gotten nice feedback so far, but not much of it, I’d love to hear from you. And if you like the book, and wanted to tell your friends, or the Internet at large, that’d be great, tool. (Oh, look, I’m turning into that guy…)

Links

Rebecca Murphey has written “JQuery Fundamentals” a new Creative Commons book on JQuery. Looks useful, though I’d also love it if an epub version was made available. I bet I’ll be referring here a bunch, though.

Looks like there will be a JavaScript native implementation of LessCS. Interesting. I’m still wondering how the Less/Sass thing plays out.

The previous two links are via Larkware’s Daily Shots.

Here’s a big chunk of code from Brian Cardarella that allows you to do user-selected subdomains using the Rails 3 router.

Via Everyday Rails, here’s http://www.youvegotrails.com/, which generates a Rails template for you, after you select some parameters. Pretty neat. I’d imagine it’ll grow more parameters over time.

I think the lesson of this article by Patrick McKenzie about human names is that no matter how far you go in creating a database schema, there’s always somebody who will go farther.

The Phusion team continues to tease about the impending awesomeness of Passenger 3.

Thoughtbot, in the person of Nick Quarantino, has a crazy detailed post on using Hudson for continuous integration with RVM. I’m going to go out on a limb and say this could be made easier.

I don’t read the Japanese, but supposedly Matz is blogging about possible Ruby 2.0 features. If I’m interpreting this correctly, it looks like Python-style double-splats are in play, which I’d like. (“Python Style Double Splats” is the name of my new Eric Idle cover band. Sorry.)

June 8, 2010: iPhone, iPhone, it’s off to work iPhone

Okay, There’s a New iPhone

Don’t really have a whole lot to say beyond what’s already been said. It looks very slick, and if anybody can actually pull off getting people to use video chat, it’s Apple. The form factor of video chat from a phone seems at first glance to be significantly better than from either a laptop or the iPad, in that it seems easier to hold the phone in a position to get a good angle. And as much as everybody is kicking around Google vs. Apple, it sure seems like the company that lost big was Flip. (Oh, and I’m not the first person to say it, but there’s now flash on the iPhone… well, an LCD flash for the camera, at least.)

And a new Safari

Safari 5 was released with a lot less fanfare. Big new features include an extensions system similar to Chrome, which will fully launch later this summer. There’s also a nice Reader function which is similar to the Readability bookmark. So far, I find it very pretty, a touch flakey about what sites it decides to pop up on, and downright magical at automatically following next links to stitch together a multi-page article. It also adds support for a bunch of HTML 5 features and a new JavaScript engine.

Links

All the RailsConf keynotes are being live streamed at http://en.oreilly.com/rails2010/public/content/livestream.

Feature Flipper is a simple little gem to allow you to semantically tie certain features to certain environments, making it easy to have a specific feature live only in development until it’s ready.

Oddly, this came up in conversation just yesterday. Chris Lowis points to a number of full Rails apps that are open source and can be used to learn how Rails apps are put together.

Quick snippet — an Array#only method for when you know an array only has one element. I think I would actually use this.

Finally

Neil deGrasse Tyson, speaking off the cuff on a rant subject near and dear to my heart, namely how it’s socially acceptable for otherwise lovely people to say “I’m no good at math”.

June 3, 2010: Get your Kicks on Route resources :user

Routes

Geoffrey Grossenbach at Peepcode posted a typically beautiful post/rant about Rails routing.

DHH responded in the comments of the article on YCombinator.

Grossenbach argues that Routes are unnecessary configuration and offers a couple of options for moving the routing into the controllers, as Sinatra does. DHH responds that GG’s schemes would be challenging for large projects, and that the seven default action names are an important constraint.

Everybody’s making good points here, so lets have the debate. Here are some of my points:

  • In any medium sized or large Rails project I’ve ever worked on, the routes file becomes a mess, and it’s hard to tell what’s going on. You could argue this is because the programmers have not been strictly RESTful, but I’d be more likely to say that it’s because RESTful is too inflexible for complex projects. I like the Rails 3 router changes, but I don’t see them changing this particular issue.
  • There’s no question that the RESTful route stuff is significantly less accessible to new Rails programmers than basic controller/action URL’s are. It’s a lot of magic, it’s opaque, and every time I teach Rails it’s a stumbling block.
  • I think I’d like to see the Sinatra style declarations available in the controller, as the Astaire plugin seems to offer. I’d be interested to see whether it winds up in a bigger mess, or whether the controller actions become clearer. I’d definitely try Sinatra routes on a small project.
  • I’m somewhat ambivalent on the issue of whether scaffold code “should” be generated, or whether the fact that it is generated means that it should be abstracted further. I agree with the theory that controllers should be abstracted, but in practice I’ve found that working with tools that do that, like resource_controller, tend to lead to more hassle than not when you try to override behavior (again, I think that that amount of controller magic can also be a block for new users). I am interested to see whether the Rails 3 responder objects change this at all, and I’d like to see responders be part of the default scaffold — last time I checked, they weren’t.

Bandwidth

I know I’m supposed to be outraged at AT&T changing their data plan pricing to remove the “unlimited” option. And I’d probably be kind of ticked if I was planning on a 3G iPad. Though I’d rather they kept an unlimited plan as an expensive option, the new setup is kind of preferable to the semi-unlimited where it’s unlimited until it isn’t. At least the exact costs are laid out in the new structure. And it seems like AT&T is going to make it pretty easy to switch plans back-and-forth and even add tethering on a month-to-month basis.

Looking at my own usage, I seem to have never even approached the 200MB limit of the cheaper plan, which means the change might save me money.

Book Status

Now definitely working on the style chapters that are listed in the current beta Table of Contents as “Fix Slow Tests”, “Help! My Test is Failing”, “Testing a Legacy Application”, “Testing Style and Structure”. These chapters are all currently kind of short, so they will probably be combined. The info from my Chicago Ruby talk the other day will be incorporated.

And since I haven’t put the link up in a few days, the book is still on sale.

May 28, 2010: Friday Friday Friday

Short today, but preparing some longer, more rant-y bits for the future…

Book status

Not much forward motion for the next few days, as I have a lot of other stuff to do, including preparing for Chicago Ruby on June 1 and doing a bit of touch-up on Obtiva’s 4-Day Ruby on Rails/TDD boot camp. All fun, but time consuming.

Some Links

A quick tutorial by Peter Cooper on setting up JQuery in Rails 3.

This isn’t the first time I’ve seen something like this, but this article by Alan Skorkin on Ruby Procs and Lambdas is well done and it’s worth refreshing from time to time.

We always say that Ruby methods can’t have spaces in them, but technically that’s a lie, as shown in Joon You’s screencast.

Rails Dispatch this week is by José Valim. It’s kind of rambling, but a very interesting look at plugins that mess with Rails 3 features like Responders and Generator.

May 26, 2010: Answer The Question Please

Top Story

Got interviewed yesterday by Miles Forrest and Curtis McHale of the coderpath podcast. Presumably it’ll be out and public in the next couple of weeks. You’ll be the first person I tell.

Couple things

  • It’s been a while since I did anything like this, and I always forget how much it’s a genuine skill to be able to give a clear answer without doing a lot of ums and ahs.
  • First rule of public speaking. You are always talking faster than you think you are. Slow down.
  • Miles and Curtis set it up so that you can see the questions as they are being asked, which helps a surprising amount in remembering what the heck the actual question was.
  • I thought it went okay. You always wish you’d been more clever, but it was fun and I’d do it again.

Book Status

Beta 3 did get released yesterday. There may be an issue with the epub version, it’s being looked into.

And then…

New Relic did an overview of usage patterns from their users, Ruby versions, installed gems, and the like. I love this stuff. What stood out for me:

  • Ruby 1.9 has a 2.7% share. Ruby 1.8.7 is only 39.5%.
  • Nearly half of all Rails apps are on 2.3.5, which is an impressive amount of keeping up to date. Rails 3 beta is at 1.3%
  • Assuming that there’s no overlap between the gem and the plugin, about 1/3 of Rails apps use Haml.
  • Authlogic is the top authentication gem at about 20%, restful_authentication is a little lower.
  • RSpec is at 7% — can that be accurate? Seems low.

Demitry Belitsky interviewed a number of Ruby community members about how they became successful Ruby developers. The list includes Ryan Bates, Yehuda Katz, Jamis Buck, and Geoffrey Grosenbach. Seems worth your time.

Speaking of Ryan Bates, he has a new challenge up at Ruby Learning. There are prizes.

May 25, 2010: Betas

Top Story

Beta 3 should be out today. The main change in this Beta is the inclusion of the Cucumber chapter, which has been updated both in terms of new tech, and also in terms of conclusions about how to use Cucumber.

And In Rails news…

When last we met, Rails 2.3.6 had just been released. Well, in the intervening 24 hours, we’ve bumped all the way to 2.3.8, with two bug fix releases, explained by Jeremy Kemper here and here. I don’t know why the patch releases weren’t numbered 2.3.6.1 and 2.3.6.2 — they’ve gone that route before. Anyway, enjoy.

History

This seems a good time to point to Mike Gunderloy’s historical list of Rails release dates. Confirming that Rails was pre 1.0 when I first saw it, was in the transition to 2.0 when I went to Pathfinder, and was in 2.0.2 when the Wrox book came out.

And then

I’m scheduled to be interviewed for the CoderPath podcast today, currently trying to confirm the time.

Here’s a call for combining RDoc and RSpec. Hmm… having used Python’s doctest, which is similar in goal, I’m skeptical. My sense was that the doctest things wound up being gunky enough that the file got hard to read.

Want to read Scott Chacon’s Pro Git book for free on your iPad? Sure you do.

This is a couple days old, but here’s Avdi Grimm with a nice little piece on Ruby’s Tempfile class.

The projects for the Ruby Summer of Code were announced. Looks like a good range of useful stuff.

May 24, 2010: Martin Gardner

Top Story

Rails 2.3.6 released, with more changes than I expected, taking it just a little bit beyond a bug-fix release. Follow the link for the full release notes.

Things that caught my eye:

  • Looks like they made an official plugin to give Rails 2.3.6 the default HTML safe behavior of Rails 3. Although the note is vague and I could be misunderstanding.
  • Alert and notice are now separate methods rather than just being keys on the flash.
  • You can now set the length of indexes in MySQL, and place a new column at an arbitrary point in the table.
  • Added test functions assert_present and assert_blank. Looks like I need to change a book chapter…

A matter of Ruby style

James Edward Gray had a couple of tweets on Friday praising a piece of Ruby that I was honestly unfamiliar with, namely using a regular expression inside brackets to reference a string

irb(main):001:0> "hello dolly"[/ll./]
=> "llo"
irb(main):002:0> "hello dolly"[/ll(.)/, 1]
=> "o"
irb(main):003:0> "hello dolly"[/xx/]
=> nil

Did not know you can do that. The semantics are that the regex by itself returns nil or the entire matching substring, but if you have a second integer argument in the bracket then it returns the matched group corresponding to that index.

Nifty. Undeniably compact. Keeps me from having to remember which character in the matching operator comes first.

What’s the verdict — please do chime in — cool feature, or unreadable hack?

Book status

I expect Beta 3 to be out sometime in the next 24 hours. Not completely sure what to do next, but I think I’ll probably look over some of the material on test quality in conjunction with my upcoming Chicago Ruby talk.

Not least, read this

Martin Gardner passed away over the weekend. It occurs to me that I have no idea how well-known Gardner is among those of you who read this. Gardner wrote the “Mathematical Games” column in Scientific American for 25 years ending in 1981. More to the point, at least for me, if you were a young math nerd roaming the stacks of the Wilmette Library in the early 80s, you would have found many, many collections of Gardner’s work.

Gardner’s books were among the first places I ever saw somebody having fun with the kinds of math and logic problems that are pretty much what I do for a living now. Among other things, his books were the first place I encountered Escher’s art, and I distinctly remember being fascinated by a what was basically a mechanical game playing AI using matchboxes and marbles. I was too young or impatient or dumb to actually try to solve most of the problems in Gardner’s columns, but the sense of joy that he showed in a particularly elegant puzzle and solution was infectious.

Funny footnote — I thought for a second that I’d use an anagram of “Rest In Peace Martin Gardner” as the headline, so I went to the anagram server, and it’s funny how many anagrams pop up using the word “reincarnated”, given that Gardner was such a famous skeptic.

May 20, 2010: Fontastic

Book Status

Starting to sound repetitive. Still working on the Cuke chapter, this time focusing on cleaning up the parts where I recommend ways to use Cucumber. Still hoping for a beta early next week.

Other things

This week in Yehuda, there’s a very long article about text encodings and what problems they have, and in particular how Ruby’s implementation is shaped by the complicated relationship between Unicode and Japanese.

I’m not completely sure I endorse this mechanism for using models in migrations, but I’ll mention it in case it solves a problem for you.

Jake Scruggs, blogging up a storm, today on using code in interviews. This is something that seems to have come on very quickly as a best practice. In my (admittedly quick) job search in 2007, I was never asked to do this. By 2009, it was pretty common, and I had to do several code samples, either before or during interviews. (For Obtiva, I had to pair program with Dave Hoover. In Python. Which I hadn’t used seriously for about three years.)

Quick note on how to stub paperclip during testing to avoid dependencies on ImageMagick, which seems a noble goal.

Thoughtbot has a very nice article showing the implementation of search functionality.

Last but not least

Google WebFonts. Which seems to be a new, free, set of fonts that you can link to from your app, and just use. Not a huge selection at the moment, hopefully more coming.

May 4, 2010: MacRuby and more

Top Story

MacRuby 0.6 is out. Big new features include a debugger, a new interface to Cocoa’s Grand Central Dispatch, and a rewrite of the internals of basic Ruby classes.

In a related story, the early text of Matt Aimonetti’s MacRuby book from O’Reilly is available for free online. Nice job all around.

I’m not sure if I’ve mentioned it here, but I use MacRuby and its ability to manipulate AppleScriptable programs to power my crazy-obsessive iTunes random playlist generator, so speed improvements are hoped for.

Book Update

Still in the Cucumber chapter. Had to go back to the beginning to make sure everything still made sense with Cuke 0.7. The end of proofing what I had is near, but I also need to write about newer features, most notably tags, and I need to make sure I still agree with all the ideas about when to use Cucumber that I had a year ago.

I changed the Cucumber in the chapter to use Capybara instead of Webrat, but it broke on test (having to do with a checkbox being checked). Not sure if it’s an issue in Capybara, the Cucumber web steps, or me.

The book, of course, is still on sale.

Lulu raffle tomorrow, so get those address changes in if you need to.

And Then…

Big day for O’Reilly releasing stuff. Obtiva’s Own Dave Hoover’s book Apprenticeship Patterns was also released for free — the whole book this time.

Gregg Pollack put together a big list of links on the official Ruby on Rails blog. All of these were originally covered on Gregg’s podcast, Ruby5.

Brian Hogan has a set of matchers for using Cucumber with Watir, looks like it gives you in browser tests using a syntax similar to the existing Cucumber web steps, but more detailed.

Finally

Dropbox for iPad is out as of last night. If you care about that, it’s really nice.

April 19, 2010: The Week Begins

Top Story/Book Status

This is the week — Rails Test Prescriptions should go on beta sale on Wednesday.

In a related story, railsrx.com now points to here, railsprescriptions.com also will shortly. I’ll be adding some basic about information and static pages here. At some point, I’ll probably bring over any blog content from the previous site that still seems relevant. I’m not sure if the original free version of Rails Test prescriptions will still be available (it’s becoming out of date, and there will be free samples available at Pragmatic), but I will make it available if anybody is still interested.

Tab Dump

Reg “raganwald” Braithwaite has a brief article on why Ruby needs blocks separately from lambdas, how blocks differ, and when that difference is useful.

A double dose from Yehuda Katz: A slideshow titled “Making your OSS project more like Rails“, with some interesting insight on what makes Rails work as a project. And another Bundler article addressing the issue of why Bundler appears to work differently based on the ordering of gems within it. (Short answer: it’s exposing dependency issues in the gems themselves.)

Over at Envylabs, they announced a new gem called Census, which allows you to gather demographic-style data on your users and then search for data based on their answers.

Another Rails 3 intro, this one at IBM Developer Works. I’ve written Rails stuff for them in the past, but I didn’t write this one.

Here’s a nice slideshow in HTML5 that shows off the new features of HTML5.

Finally

NetNewsWire has quickly become one of my indispensable iPad apps. The developer, Brent Simmons, in an attempt to discuss software development, has posted a number of pictures from various stages in the design and implementation of NNW-iPad. Thanks!

April 16, 2010: The cover is uncovered, at least for me…

Top Story and Book Update

I have sample cover designs. I don’t think I can show them yet, but I’ve got ’em, I like them, and hopefully we’ll have picked one to show shortly.

Tab dump

Not a whole lot today:

Here’s a look at the current status of using a non-blocking MySQL driver with Rails. The promise here is for significantly faster database access.

Two stories that are related to projects that I worked on back in my EdTech days:

Version 1.0 of Pharo has been released. Pharo is a fork of Squeak smalltalk aimed at, as far as I can tell, a more modern UI and setup. Pharo is also the reference implementation for the Seaside web framework. A lot of Ruby and Agile ideas came out of the Smalltalk community. I’d definitely recommend trying Smalltalk if you are looking for a new language. For years, I’ve given impromptu Squeak demos to show off what’s cool and what’s hard about Smalltalk. Check it out.

On a less encouraging note: Scratch, which is the current MIT media lab project to teach programming ideas to kids, had their iPhone app pulled from the app store yesterday. (The link is to Mark Guzdial’s compututing education blog — Hi Mark!) This is probably collateral damage in Apple’s war on interpreters, but there’s no way to put a good face on — it’s a disappointing decision and I’m sad I didn’t know Scratch was there to download before it was pulled.

Finally

Part Two of the 8-Bit Dr. Horrible is up — when you start the flash player, you have the option of clicking into act two, which I mention because it took me too long to figure out on my own.