reinh.com

Building a Turing Machine Simulator With Ruby (Part 1)

2011-09-03T17:41:00-07:00

To better understand Turing's machine and its contributions to mathematics and computer science, we will build a simulator in Ruby. This simulator will take a description of the machine's configuration, create a model of the machine, and run through its steps. We will be able to observe the action of the head and changes to the tape at each step. Hopefully this will serve to illuminate some aspects of Turing's machine that might otherwise be less accessible.

Although the Turing machine is often thought of as prefiguring later computer science innovations such as the state machine, its original purpose was quite different. Turing created his machine in an attempt to understand the limits of mechanical computation. That is to say, the limits of computability. The paper that introduced the Turing machine, published in 1936, refers to "computable numbers" and their use in solving David Hilbert's Entscheidungsproblem, or "decision problem".

Turing's Marvelous Machine

Informally, a Turing machine is a mathematical model of a machine that mechanically operates on a tape. This tape contains squares where the machine can read or print a symbol using a tape head. The machine can also move left and right over the tape, one square at a time. The machine's operation is fully determined by a list of elementary instructions such as "in state 42, if the symbol seen is 0, print a 1; if the symbol seen is 1, shift to the right and change into state 17; in state 17, if the symbol seen is 0, print a 1 and change into state 6;" etc. Turing called these instructions m-configurations. Modern computer scientists typically refer to them as states (as the Turing machine is a type of Finite-state machine). I will refer to them interchangeably as "instructions" or "states". Turing, in his paper, labeled these instructions using gothic script lower-case letters. We will simply use lower-case letters.

A bit more formally, a Turing machine can be specified as the 3-tuple $M= \langle Q, \Gamma, I \ \rangle$ . $Q$ is a finite, non-empty set of states where $q_0 \in Q$ is the first state; $\Gamma$ is a finite, non-empty set of tape alphabet/symbols with $S_0 \in \Gamma$ representing a blank, also called $None$ (the only symbol allowed to occur on the tape infinitely often at any step during the computation); and $I$ is a finite, non-emtpy set of instructions. An instruction $i \in I$ can be defined with the 5-tuple, or quint, $\langle q_i, S_j, S_k/N/E, L/R, q_m \rangle$ , which consists of the following:

$q_i \in Q$ , the current state.
$S_j \in \Gamma$ , the symbol scanned.
The symbol to be printed
- $S_k \in \Gamma$ , print the symbol $S_k$
- $N$ , equivalent to $S_j$ , indicates a "noop" (alternatively, print the current symbol again).
- $E$ , equivalent to $S_0$ , print a blank (erasure).
A head movement instruction:
- $L$ , move the head left.
- $R$ , move the head right.
$q_m \in Q$ , the new state.

This is a bit less formal than the definition used by Hopcroft and Ullman, for instance, but should suffice for our purposes.

Our First Turing Machine

With the formal definition out of the way, let's turn to Turing's first machine, which computes the sequence 0 1 0 1 0 1…. This machine's 3-tuple looks like this:

Q, the states: { b, c, e, f }
Γ, the symbols: { 0, 1}
I, the instructions: $\{ \langle b, None, 0, R, c \rangle, \langle c, None, N, R, e \rangle, \langle e, None, 1, R, f \ \rangle, \langle f, None, N, R, b \rangle \}$

In Turing's original table form, these same instructions are represented as:

Configuration		Behavior
m-configuration	Tape symbol	Tape operations	Final m-configuration
b	None	P0, R	c
c	None	R	e
e	None	P1, R	f
f	None	R	b

Turing describes the above table as follows:

This [example] table (and all succeeding tables of the same kind) is to be understood to mean that for a configuration described in the first two columns the operations in the third column are carried out successively, and the machine then goes over into the m-configuration in the final column

Once Again, With ASCII

For our Turing machine simulator, I decided to use an ASCII representation of the Turing machine's instructions. In this format, the instructions look like this:

Turing's First Exampleon github.com

b, None, P0R, c
c, None, R,   e
e, None, P1R, f
f, None, R,   b

The m-configurations (instructions) in my machines will begin with b, just as Turing's did. (I believe he used b for begin.)

The BNF grammar for this representation is:

<instruction> ::= <ident> "," <symbol> "," <operation> "," <ident> <EOL>
      <ident> ::= <char>
      <symbol ::= <char> | "None"
  <operation> ::= <movement> | <print> <movement>
      <print> ::= 'P' <char>
    <movement ::= 'R' | 'L'

Representing An Instruction

To begin modelling our simulated Turing machine, we need some way to represent instructions internally. We'll use Ruby arrays as tuples and Ruby symbols as, well, symbols (with the exception of None, 0 and 1, which will be be represented as nil and the integers 0 and 1, respectively). Tape operations will be represented as another array of tuples, to wit: Pn as [ :print, n ], E as [ :erase ], L as [ :left ], and R as [ :right ].

Here is our Ruby representation of the above instruction list:

Turing's First Example (Ruby)

[
  [ :b, nil, [[ :print, 0 ], [ :right ]], :c ],
  [ :c, nil, [[ :right ]],                :e ],
  [ :e, nil, [[ :print, 1 ], [ :right ]], :f ],
  [ :f, nil, [[ :right ]],                :b ]
]

Parsing An Instruction

Writing a parser for the ASCII instruction format is a fun little exercise. As a Turing machine is a finite-state machine, it should be no surprise that this specification grammar forms a regular language (which can be accepted with a finite-state machine). This means that we can parse it with regular expressions---without having two problems! (Actually, we'll use the comma separation to make tokenization a bit easier with String#split, but we're getting ahead of ourselves.)

We'll use a Turing::Parser class to parse the entire specification. It will split lines and delegate to Turing::Parser::Line class to parse individual instruction lines. This in turn will need to parse (lex) individual tokens into a Ruby representation. We'll start there:

parse_token()

class Turing::Parser::Line
  def parse_token(token)
    case token
    when "None" then nil
    when "0"    then 0
    when "1"    then 1
    else token.intern
    end
end

This will take a token like "None" and return its Ruby representation, in this case nil. Bits will be converted to Ruby Integers (close enough). Characters will be interned (converted to Ruby Symbols).

Next, we'll deal with the operations. We can write a simple string scanner that will lex the operation string and turn it into a list of operation tuples (as described above).

parse_operations()

class Turing::Parser::Line
  def parse_operations(actions)
    actions.scan(/P.|R|L|E/).map do |action|
      case action
      when /P(.)/ then [:print, parse_token($1)]
      when "R" then [:right]
      when "L" then [:left]
      when "E" then [:empty]
      end
    end
  end
end

Now we can parse each instruction line by converting tokens with parse_token and converting the operations with parse_operations (which, of course, uses parse_token itself to lex the tokens it scans):

parse_instruction()

class Turing::Parser::Line
  def initialize(line)
    @line = line
  end

  def parse_instruction
    state, symbol, operations, end_state = @line.split(",").map{|e| e.strip}

    state      = parse_token(state)
    symbol     = parse_token(symbol)
    operations = parse_operations(operations)
    end_state  = parse_token(end_state)

    [state, symbol, operations, end_state]
  end
end

Using A Struct For Instructions

We can add a bit of semantic value to the tuples we use to represent machine instructions by wrapping them in a struct. In this way, the struct is functioning like an intermediate data type. The struct will also make it easier to unpack specific values out of the tuple. Structs also have the useful property of duck typing as arrays (through the use of to_ary). You can even splat them, which will come in handy later.

Turing::Machine::Instruction

Turing::Machine::Instruction = Struct.new(:state, :symbol, :operations, :end_state)

class Turing::Parser::Line
  def parse_instruction
    state, symbol, operations, end_state = @line.split(",").map{|e| e.strip}

    state      = parse_token(state)
    symbol     = parse_token(symbol)
    operations = parse_operations(operations)
    end_state  = parse_token(end_state)

    Turing::Machine::Instruction.new state, symbol, operations, end_state
  end

Parsing The Machine Specification

Now that we can parse in individual line, parsing the entire specification is just a matter of mapping lines to their parsed version:

parse_specification()

class Turing::Parser
  def initialize(machine_spec)
    @machine_spec = machine_spec
  end

  def parse_specification
    @machine_spec.split("\n").map { |line| Line.new(line).parse_instruction }
  end
end

That's (Not) All, Folks

In upcoming parts, we'll use these instructions to form the basis of our Turing machine's configuration system; we'll implement the machine's tape and tape head, which will allow us to step through its execution; and we'll add a simple renderer that will print out the machine's operations so we can follow along.

Ch-Ch-Ch-Changes

2009-09-21T00:00:00-07:00

A couple months ago, I resigned my position at Hashrocket. This was not an easy decision for me. Working with the rocketeers has been a very fulfilling and edifying experience. The people at Hashrocket are passionate about what they do and extremely competent. I was given challenging and rewarding responsibilities. Hashrocket was a great job and I would recommend it to anyone. Why, then, did I chose to leave?

In the end, it came down to a need for personal growth. At Hashrocket, I have had the opportunity to work with interesting clients on interesting projects, to lead a number of talented development teams and to consult on Rails development and Agile methodologies to large businesses. Thanks to Hashrocket, I have expanded my core competencies and matured as a developer. Certainly I could have stayed with Hashrocket and continued this process. There was more to do and more to learn. In the end, though, I felt that it was time to for me to take on a new set of challenges. My friend and former boss, Obie Fernandez, has been wonderfully supportive of me in this and I owe him a great debt of gratitude. I am happy to say that my departure from Hashrocket has been entirely amicable. They are a great bunch, one and all, and I am fortunate to have worked with them.

As a consultant for the past eight years, both freelance and with Hashrocket, I have worked with many clients. This work has been more or less successful, my clients have been more or less happy, and the work has been more or less satisfying, but there is a common factor in its transient nature. I work on a project for weeks or months, provide as much value as my abilities allow, and then move on to another project once my role is complete. Consultants rarely experience the satisfaction and sense of accomplishment of a job well done, the rewards of the successful completion of a long and often arduous project life cycle.

So, to make a long story short, I wanted to find a project that I could call home. I wanted to feel a sense of ownership in the project and its outcome. I also wanted to learn new skills and develop new competencies. The problem is that I didn't yet know where home was, so I decided to start by taking some time off and then sort of figure things out from there. This was risky, especially in our current economic climate. Luckily, I've never been shy about making such decisions. After all, I love a good adventure, and adventures always entail a certain amount of risk.

During my vacation, I spent a lot of time playing piano and writing music. It felt good to give my creative muscles some much-needed exercise. I assessed my career and my goals. I read a lot. I played some video games. I quickly found, however, that living without a "dedicated purpose" was less than satisfying. I was not happy leaving my tools in the corner to gather dust: they needed to be used. So, how best to use them?

I decided, after some consideration, that what I wanted was the opportunity to shape the destiny of a project that I believed in. I wanted to do more than sit behind a desk and code. Luckily, I was presented with just such an opportunity only a few weeks after leaving Hashrocket. Rick Bradley, a good friend of mine and an excellent developer, had a client who was looking for a full-time hire. I was reluctant to become Yet Another Developer but I decided to look into it. I'm sure glad I did: the position turned out to be just what I was looking for.

The client was Reductive Labs, the company behind Puppet, the Ruby-based configuration management tool. This in and of itself was interesting. The Puppet codebase covers a large area both topographically and conceptually. It has a significant installation base and a large open source community. It would be a challenging project to work on and a welcome test of my abilities.

I don't have a Computer Science background. I've never taken any courses in programming. I am entirely self-taught. Puppet, on the other hand, has a lot of Serious Business going on: a parser-compiler, a graphing library, client-server systems, and so on. I have experience in some of these areas, but in others I have only a basic theoretical knowledge gleaned by way of an intense curiosity and a love of reading. I had read the Dragon Book, for instance, but I had no experiential background in language design or writing parsers and compilers. Luckily, my lack of a formal education was not a barrier to entry (nor should it be if you have the skills needed to execute or the ability to acquire them, but that's a topic for another post).

I spent a week in Portland getting to know the code and the company. While Puppet is older than Rails, the code has until recently been under the sole stewardship of Reductive founder and CEO Luke Kanies. As a company, Reductive is small and young, but the technology is relatively mature. This is an interesting combination of both the pioneering spirit of a startup and the stability of an established open source project.

During the day, I spent my time pairing with Luke and talking with the founders about their plans for Reductive Labs and Puppet. Luke has a piercing intelligence and a compelling vision for the future of both Puppet and Reductive Labs itself. I quickly saw that I would not just be yet another a developer, rather that I would also have the opportunity to lead teams, collaborate with a large and active open source community, and even more importantly, to help Luke and the rest of the Reductive team make design and strategy decisions that would inform the growth of both Puppet and the company. Plus, I would be "forced" to relocate to Portland, one of my favorite cities. Bonus.

Apparently my visit was a success: I'm pleased to say that Luke offered me a position immediately after my trip and I accepted just as quickly. As I grow into my new responsibilities at Reductive, I find that they scale well with my abilities. Luke seems happy to let me do what I'm good at and I enjoy the opportunity to carve out my own role within a growing company. Most importantly, I feel like I have the opportunity to accomplish something significant. Nothing matters more to my sense of fulfillment and satisfaction.

My faithful readers have most likely been lamenting this blog's recent lack of content. Hopefully my little story helps explain this conspicuous absence. Expect more posts in the near future. I've got plenty to talk about.

Simplify Your Dev Environment With Passenger Pane

2009-04-17T00:00:00-07:00

On OS X? Develop web applications with Ruby? Want drag-and-drop and point-and-click development server management? Then you need Passenger Pane. We'll walk you through the installation process and show you how to get a simple Rack application up and running. Thanks to Jason Noble for his help getting everything working.

Passenger Pane is an OS X preference pane designed to work in concert with Phusion Passenger and your OS X Leopard's default Apache2 installation (the same one that serves your Web Sharing). Setup is pretty simple and will probably take about 10 minutes.

Here's what we're going to do:

Install Phusion Passenger
Install Passenger Pane
Serve a "Hello World!" Rack endpoint
Profit!

Install Phusion Passenger

Phusion Passenger is the (not so) new hotness in the Rails deployment world. Phusion is great for serving Rails applications via Apache in production (especially in concert with Rails Enterprise Edition, which provides a 33% memory savings over standard MRI Ruby), but it's also great for simplifying your development environment.

The Phusion team have done a great job on the passenger install. Here are the official installation instructions. We'll repeat them here for the sake of convenience.

Open a Terminal window.
Install the passenger gem:
```
sudo gem install passenger
```
Install the passenger apache2 module:
```
sudo passenger-install-apache2-module
```
Include the module and some supporting configuration settings into an apache conf file. The code will be provided by the passenger install script.
Finally, add a Directory directive to your apache2 config that allows access to the directory where your app source codes are located. You can put this at the bottom of your passenger.conf file (replace my source directory with yours):
```
<Directory /Users/reinh/code/>
  Order Allow,Deny
  Allow from all
</Directory>
```

Note: We put the passenger apache configuration in /etc/apach2/other/passenger.conf (a file we created). The default apache2 httpd.conf imports all .conf files in /etc/apache2/other.

Install Passenger Pane

Passenger Pane gives you a Preference Pane that lets you add, remove and manage apps deployed on Phusion Passenger. Drag-and-drop to serve a new app and restart with a single click — it even provides host entries for each app so you don't have to mess with virtual hosts or your hosts file. This is great for serving multiple applications simultaneously or just generally being awesome.

Installation is simple. Download the preference pane from their homepage and double-click to install. If you installed Passenger correctly (as above), this should just work. If not, look for errors in Console.app or a helpful notice in the preference pane and ask for help on the Passenger forums or in the #passenger channel on irc.freenode.net.

After installing Passenger Pane, you will need to restart Apache. The simplest way to do this on OS X is to open the Sharing preference pane and uncheck and recheck Web Sharing.

Serve A "Hello World!" Rack Endpoint

Let's test that everything is working by creating a simple Rack "Hello World!" endpoint.

Install rack:
```
sudo gem install rack
```
Create a new directory called rack-hello-world.

Add a file inside it called config.ru with these contents:

``` ruby

run lambda{|env| [200, {"Content-Type" => "text/plain"}, ["Hello World!"]]}

```

Create a public directory inside rack-hello-world (you'll see why later).
Drag the rack-hello-world folder into your Passenger Pane panel (if you're using TextMate, you can drag it from the project panel). You will probably need to click the lock first to allow modifications.
Browse to http://rack-hello-world.local and behold the awesome!
Profit!!!

If you've done this correctly, your Passenger Pane should look similar to this:

Your browser should be showing you your glorious "Hello World!" homepage.

Now you can drag in the app folders of any Rails, Merb, Sinatra or Ramaze applications and have them instantly served by Passenger. In fact, any Ruby web application that can run on Rack can be run in Passenger Pane. How's that for a painless local development environment?

Note: Why (I hear you asking) did we create a public directory? The answer is that Apache expects to serve a public directory and will fail if one is not found under the root of the app you're serving. If you had not created the directory, you would have to look at your apache error log, which is usually located at /var/log/apache2/error_log, to find out what went wrong.

A Git Workflow for Agile Teams

2009-03-02T00:00:00-08:00

An efficient workflow for developers in Agile teams that handles features and bugs while keeping a clean and sane history.

At Hashrocket we use git both internally and in our Agile mentoring and training. Git gives us the flexibility to design a version control workflow that meets the needs of either a fully Agile team or a team that is transitioning towards an Agile process.

There are many usable git workflows. Indeed, git is really "a tool for designing VCS workflows" rather than a Version Control System itself. Or, as Linus would say, git is just a stupid content tracker.

This is by no means a normative document or my attempt to define The One True Workflow. I have found this workflow to be productive and relatively painless, especially for teams that are still learning and transitioning towards a more Agile process. I invite you to comment below and describe the git workflow that works best for you.

Credits

Anyone interested in using git in an Agile environment should read Josh Susser's Agile Git and the Story Branch Pattern. This workflow is based largely on his.

The process of squashing commits into more atomic and incremental units has been described by Justin Reagor in A "Squash" WorkFlow in Git. Justin's post also references a git rebase -i walkthrough by MadCoder that provides a good explanation the interactive rebase process.

Thanks also to Rick Bradley of OG Consulting and Josh Susser of Pivotal Labs for many fruitful and often hilarious discussions about git, Agile and ponies.

Feature Development

Our git feature development workflow consists of these steps:

Pull to update your local master
Check out a feature branch
Do work in your feature branch, committing early and often
Rebase frequently to incorporate upstream changes
Interactive rebase (squash) your commits
Merge your changes with master
Push your changes to the upstream

First, and while in your master branch (git checkout master), pull in the most recent changes:

git pull origin master

This should never create a merge commit because we are never working directly in master.

NB: Whenever you perform a pull, merge or rebase, make sure that you run tests directly afterwards. Git may not show any conflicts but that doesn't mean that two changes are compatible. Also run tests before you commit (of course).

We begin with the topmost available story in Pivotal Tracker. Let's say that it's #3275: User Can Add A Comment To a Post. First, check out a feature branch named with the story id and a short, descriptive title:

git checkout -b 3275-add-commenting

The id allows us to easily track this branch back to the story that spawned it. The title is there to give us humans a little hint as to what's in it. Do some work on this branch, committing early and often (for instance, whenever your tests pass). Rebase against the upstream frequently to prevent your branch from diverging significantly:

git fetch origin master
git rebase origin/master

NB: This is often done by checking master out and pulling, but this method requires extra steps:

git checkout master
git pull
git checkout 3275-add-commenting
git rebase master

Once work on the feature is complete, you will have a branch with a lot of small commits like "adding a model and a migration", "adding a controller and some views", "oh crap - adding tests" and so on. This is useful while developing but larger, incremental commits are more easier to maintain. We will use an interactive rebase to squash them together. Also, squashing these commits together will allow us to pretend that we wrote the tests first…

We want the rebase to affect only the commits we've made to this branch, not the commits that exist on the upstream. To ensure that we only deal with the "local" commits, use:

git rebase -i origin/master

Git will display an editor window with a list of the commits to be modified, something like:

pick 3dcd585 Adding Comment model, migrations, spec
pick 9f5c362 Adding Comment controller, helper, spec
pick dcd4813 Adding Comment relationship with Post
pick 977a754 Comment belongs to a User
pick 9ea48e3 Comment form on Post show page

Now we tell git what we to do. Change these lines to:

pick 3dcd585 Adding Comment model, migrations, spec
squash 9f5c362 Adding Comment controller, helper, spec
squash dcd4813 Adding Comment relationship with Post
squash 977a754 Comment belongs to a User
squash 9ea48e3 Comment form on Post show page

Save and close the file. This will squash these commits together into one commit and present us with a new editor window where we can give the new commit a message. We'll use the story id and title for the subject and list the original commit messages in the body:

[#3275] User Can Add A Comment To a Post

* Adding Comment model, migrations, spec
* Adding Comment controller, helper, spec
* Adding Comment relationship with Post
* Comment belongs to a User
* Comment form on Post show page

This also follows Tim Pope's git commit message best practices. Now, save and close your editor. This commit is now ready to be merged back into master. First rebase against any recent changes in the upstream. Then merge your changes back into master:

git checkout master
git merge 3275-add-commenting

And finally, push your changes to the upstream:

git push origin master

Bug Fixes

Bugfixes will use the same workflow as feature work. Name your bugfix branch after the bug id and give it a descriptive name. Prefix the branch name with "bug" to help you keep track of them, for instance: bug-3845-empty-comments-allowed.

Do work in your bugfix branch, committing early and often. Rebase frequently against the upstream and again when you are finished. Then, use an interactive rebase to squash all the commits together. Use the bug id and title for the subject of the new commit. Include "BUG" or "BUGFIX" to make these commits easier to identify. For instance:

[#3278] BUGFIX: Empty Comments Should Not Be Allowed

NB: With a bugfix, squash the commits down into one and exactly one commit that completely represents that bugfix. Half of a bugfix is useless.

QA Branch Management

In a truly Agile process, as Josh Susser explained to me, you simply deploy directly from your latest "green" build. While this may seem impossible, there are real world examples of the viability of Continuous Deployment.

For most of the teams we train, this sort of extreme agility is a goal rather than a way of life. They are transitioning towards a fully Agile process from a more rigorous one and often have a significant investment in Quality Assurance. For these teams, I recommend a compromise solution: Use a remote QA branch that is fast-forward merged from the latest "green" (all CI tests pass) master build for QA deployments.

If you don't already have a QA branch, create one from the current master (assuming it's green) and then push it to your git host:

git checkout -b qa
git push origin qa:refs/heads/qa

To update an existing QA branch with the latest changes from master, do this:

git checkout qa
git merge master
git push origin qa

Ensure that the master branch is green when you merge. Also ensure that this merge is always a Fast Forward merge. If it is not, it means that commits have been made in the QA branch that are not in the master branch. QA should be merged into from master only, never worked on directly.

Production Tagging

While an Agile team will be deploying continuously, most teams will require a more rigorous vetting process before a build is deployed to production. Tags provide a simple way for QA to vet a particular build.

Once QA has signed off on a build as ready for production, a tag should be made and deployed. In git, a tag can be created with git tag <name>. It is important that tag naming is consistent and it is useful if they sort in a meaningful way (latest last, for instance). Two good options are a timestamp or product version. Use whichever is more meaningful to your team.

The process of creating a timestamped production tag can be automated with a simple script or a git alias like this one:

git config alias.datetag '!git tag `date "+%Y%m%d%H%M"`'

With this alias, git datetag will create a new tag from HEAD with the current timestamp. You will want to be in the QA branch. If the QA branch has moved beyond the last commit vetted by your QA team, be sure to checkout that commit first.

Story Driven Development With Rails

2008-09-12T00:00:00-07:00

As a follow up on writing incremental stories, we're going to take the first story and walk through a behavior driven development process to implement it in a simple Rails application.

We will focus on making small, iterative changes and following a strict test-first philosophy where we write granular unit tests and implement them with just enough code to make them pass.

Let's review our first story:

Story #1

As A User I Want To View A List Of Projects
So that I can find a project that interests me

Acceptance:

* All projects shown in list
* List is paginated
* List is sorted by age

Cost: 1 Point

At this point, let's assume that this story is on top of our current iteration. First, we'll need to review the story for any missing information and communicate with the client to clear up any questions. Keep in mind that a story is just a way to capture a conversation about a feature. It is not set in stone. After talking to the client, we find that we will need to display a project's author name and title and that the title will need to link to that project's page. Let's update the story appropriately.

Story #1

As A User I Want To View A List Of Projects
So that I can find a project that interests me

Acceptance:

* All projects shown in list
* Show title for each project
* Show author name for each project
* Project title links to the project's page
* List is paginated
* List is sorted by project age

Cost: 1 Point

Now that the story is complete, deliverable and acceptable, we can begin work on the new feature. A implementation plan should be forming in your head. Now is the time to divide the work into testable units. In our case we already have a Project model with the requisite fields (let's say) so our work will focus on the controller and view.

Test All The F---ing Time

Client sign-off on well written acceptance tests means that the specifications you write and the feature that is implemented as a result will be more closely in line with the client's expectations. This minimizes the kind of impedance mismatch between expectation and execution that so often plagues a project with poor client communication and a disorganized process.

Now it's time to take our acceptance tests and use them to drive our iterative, test-driven development process. Let's take it from the top.

All Projects Shown in List

Let's start with the controller. A list of projects needs an index action. Starting at the top, we will need to load all of the projects. Let's write a test for this:

spec/controllers/projects_controller_spec.rb

describe ProjectsController do
  describe "getting a list of products" do
    it "loads all the projects" do
      projects = [mock_model(Project)]
      Project.stub!(:all).and_return(projects)

      get 'index'

      assigns(:projects).should == projects
    end
  end
end

Note: Stubbing the call to Project.all has the immediate benefit of eliminating the database from our test but is potentially more brittle since we cannot be sure that this interface point to our Project model will not need to change in the future.

On a side note, I tend to view controller tests as integration-level tests rather than unit tests. As such, I usually do write tests that touch the database since these are often less brittle. If you write tests that touch the database, ActiveRecord factories such as Factory Girl or object daddy are useful for populating the database with valid records in known states.

Now we can write the implementation:

app/controllers/projects_controller.rb

class ProjectsController < ApplicationController
  def index
    @projects = Project.all
  end
end

Then we have to display a list of projects. We'll write a view test to cover this:

spec/views/projects/index.html.erb_spec.rb

describe "/projects/index" do
  before(:each) do
    @project = mock_model(Project)
    @projects = [@project]
    assigns[:projects] = @projects
    render 'projects/index'
  end

  it "should include a list of projects" do
    response.should have_tag('li.project', :count => @projects.size)
  end
end

And the implementation:

app/views/projects/index.html.erb

<ul>
  <%% @projects.each do |project| %>
    <li class="project"></li>
  <%% end -%>
</ul>

Note: View tests can often be brittle. They can be made less brittle by testing only for semantically appropriate tags, classes and ids whenever possible. Using semantically rich markup in your views will make it much easier to write robust view tests -- and is also a great practice for its own sake. {:.note}

Show Title for Each Project

spec/views/projects/index.html.erb_spec.rb

describe "/projects/index" do
  # SNIP

  it "should show the title for each project" do
    response.should have_tag('li.project .title', @project.title)
  end
end

app/views/projects/index.html.erb

<ul>
  <%% @projects.each do |project| %>
    <li class="project">
      <h2 class="title"><%%= project.title %></h2>
    </li>
  <%% end -%>
</ul>

Show Author Name for Each Project

spec/views/projects/index.html.erb_spec.rb

describe "/projects/index" do
  # SNIP

  it "should show the author name for each project" do
    response.should have_tag('li.project .author_name', @project.author_name)
  end
end

app/views/projects/index.html.erb

<ul>
  <%% @projects.each do |project| %>
    <li class="project">
      <h2 class="title"><%%= project.title %></h2>
      <p class="author_name"><%%= project.author_name %></p>
    </li>
  <%% end -%>
</ul>

Note: We are using an accessor on our project model, Project#author_name. There's a good chance that this name will be taken from an associated User or Author model in any non-trivial Rails application. From an object oriented standpoint, however, having the author name hang directly from the Project model improves encapsulation.

The benefits of this were already seen in the test, where we were able to stub author_name directly on the Project mock. Without the accessor, we would be forced to stub #author on the Project mock to return an Author mock that then stubs #name just so that we could properly test the method chain project.author.name that is used in the view. Violating the Law of Demeter makes testing harder.

Project Title Links to the Project's Page

spec/views/projects/index.html.erb_spec.rb

describe "/projects/index" do
  # SNIP

  it "should have project titles that link to the project page" do
    response.should have_tag( 'li.project .title a[href=?]', project_path(@project),
      :text => @project.title)
  end
end

app/views/projects/index.html.erb

<ul>
  <%% @projects.each do |project| %>
    <li class="project">
      <h2 class="title"><%%= link_to project.title, project %></h2>
      <p class="author_name"><%%= project.author_name %></p>
    </li>
  <%% end -%>
</ul>

The story is now about half complete. I'll leave pagination and default sort order as an exercise for the user. In fact, these could also have been broken out into a secondary story or stories given that what we have done so far is an incremental unit of work.

I hope this rather contrived example shows how stories with well written acceptance tests inform a test- or behavior- driven development process and help bridge the gap between what the client expects and what the developement team delivers.

Hack && Ship

2008-08-27T00:00:00-07:00

When the OG Consulting guys were down at Hashrocket working on our latest 3-2-1, they introduced us to a pair of bash scripts called hack and ship that they use to streamline their everyday git workflow. They're so useful that we adopted them immediately and we've been using them religiously ever since. I estimate that these little scripts save me about an hour a day and, what's more, they make it easy to follow the commit early, commit often mindset that's so useful to the agile process.

Simple Software Process

Here's our typical workflow (before hack and ship):

Start a feature in "Pivotal Tracker":https://www.pivotaltracker.com
Checkout a working branch for this feature
Write a test
Write an implementation to make the test pass
Repeat 3 and 4 until the feature is complete
Commit changes to git with a "useful commit message":http://www.tpope.net/node/106
Checkout the master branch
Update the master branch
Checkout the working branch
Rebase the master branch into the working branch
Checkout the master branch
Merge the working branch into the master branch
Push the changes to the origin repository (usually on "github":https://github.com/)
Repeat.

This is pretty simple (if a bit longwinded). We each probably do this dozens of times a day. Could it be simpler? You bet!

Simplified Software Process

I'd like to introduce you to the Simplified Software Process. While it may not be a very good process, it does have one thing going for it: it's simple. We like simple. So when we saw Rick Bradley's super simple bash scripts for automating these common git tasks, we jumped on them. These scripts are designed to work with the basic git workflow we outlined above. And they have awesome names.

Here's hack.

And ship.

Now our process looks like this:

Start a feature in Pivotal Tracker
Checkout a working branch for this feature
Write a test
Write an implementation to make the test pass
Repeat 3 and 4 until the feature is complete
Commit changes to git with a useful commit message
hack
ship
Repeat.

We even wrote ourselves a little alias to make this even easier:

Which brings the workflow down to three easy steps:

Test
Implement
ssp

Combine this with a Continuous Integration build like cruisecontrol.rb and a deployment to a staging server that takes less than a minute - we love EngineYard - and you have the perfect recipe for agile, iterative, test-driven, micro-release oriented development. I've never been happier with my development process.

Rocking the Hoedown

2008-08-10T00:00:00-07:00

We had a great time at the Ruby Hoedown this weekend. The Hashrocket crew was out in force (and there was much debauchery in the Hashrocket RV). We got to meet or get better acquainted with a lot of great rubyists and hear a lot of entertaining, informative talks.

Jim and Joe's Mock Dialog

Jim Weirich and Joe O'Brien presented their talk about mocks and stubs in the form of a short play in three acts. This was very timely for me since mocking properly in tests is area we have been discussing quite a bit at Hashrocket.

One thing they said that struck a chord was that complex mocks are a code smell that can clue you in to problems or hot spots for refactoring in the code under test.

The presentation style was fresh and the tips they provided should be very valuable to anyone using mocks in testing (which should be everyone).

Ruby Best Practice Patterns

I'm afraid I played a little trick on poor Jeremy and my dear audience. Instead of spending an hour giving examples of patterns and practices, I thought it might be fun to take a more satirical slant on the material. The resulting talk, Unfactoring From Patterns: Job Security Through Code Obscurity, turned out to be a modest success (if I do say so myself) and I think rather more entertaining to boot.

I did have time to show a few actual patterns but the dialog during the question and answer period was especially valuable. David Black, Yehuda Katz and others made some great points and I really enjoyed getting a chance to discuss one of my favorite topics with such a receptive and informed audience.

We Ain't Got No Keynote

Chris decided to forgo the slides to tell us a more personal story about his growth from a lowly PHP hacker to the successful rubyist, entreprenuer and open-source champion that he is today. Chris is an exemplar for me of the right way to go about creating a personal and company brand.

Chris's years of passion for tinkering, experimenting and exploring really have payed off in a big way. I think his story could rightly be considered inspirational. I'm sure a lot of people left his talk and started side projects this very weekend. I really enjoyed his talk (even if he did tell people not to buy my book).

Flog Your Tests, Test Your Flog

Testing is such a crucial part of my process that I was very excited to hear Rick Bradley's account of his rather epic battle to add tests to flog. The irony here, of course, is that flog itself is a testing tool.

This talk combined two of my favorite topics: testing and refactoring. Rick's war stories had some great takeaways: the importance of integration tests to characterize the behavior of the existing system, the ways in which code that is written test first differs from code that is written without tests, the utter importance of testing all the f--king time. Bryal Liles will have more to say on that last topic in a bit.

I really can't say enough about Rick's talk. Hashrocket often takes on applications with little or no testing for our Rescue Missions and Rick's deliberate, careful, comprehensive methodology is exactly the process we try to follow when we refactor and resuscitate our clients' code back to health. Rick could very well write the Hashrocket Rescue Mission manual.

MIDI Machinations and Hungry Hungry Hippos

Giles Bowkett is a mad genius. Perhaps more importantly, he's an artist who can ship. He lured us all to a talk about archaeopteryx, his amazing MIDI generator, and then proceeded to smack us all with the enlightenment stick.

I won't spoil his talk for people who haven't seen it. Suffice it to say that it was far more than I expected. Giles's opinions on software development are unique and somewhat iconoclastic. Be careful, the code he writes might blow some people's minds. Giles's presentation style is engaging and he can be down right hilarious at times. Also, hippos are scary!

Wrap It Up!

Taking a road trip down to the Hoedown with the Hashrocket crew was one of the best conference experiences I've had so far. The talks were great, but I think the after-conference festivities were the real hilight. Watching Obie and Jason play beer pong on a table constructed of pool noodles (watch the video), playing werewolf with a crew of drunken rubyists and spending time with some of my favorite conference friends. Who could ask for more? Also, Jeremy has big plans for next year's Hoedown. They involve words like "free" and "Nashville". I would be inclined to add "awesome" as well.

Adding Epicycles: Copernicus On Software Development

2008-07-20T00:00:00-07:00

Most software projects suffer from increasing complexity over their lifetime. One of the most common ways that a project grows in complexity is known as "Adding Epicycles".

The geocentric model of the solar system is a very old one. First postulated by Plato and Aristotle in the 6th century BC, the idea that the Earth is at the center of the universe was later refined and standardized in Claudius Ptolemy's main astronomical book, Almagest. It is also an excellent example of a very common design failure in software development.

Ptolemy's Faulty Assumption

It seems like a pretty obvious idea: sun, stars, planets… everything looks as if it goes around the Earth. Indeed, it was almost two thousand years before Copernicus was able to provide a convincing alternative.

Ptolemy's Great Design Failure

As people began to make better and better observations of the motions of the various heavenly bodies, they began to notice that their movements were often far more complex than the Ptolemaic system could allow. The planets were particularly troublesome in this regard.

The logical step when a hypothesis no longer fits with observation is to reconsider that hypothesis and possibly discard it in place of a more fitting candidate. This is precisely what did not happen when Ptolemy set out to catalog the arrangements and movements of the celestial spheres.

What he did instead was add smaller circles that the planets move in, and then have these circles move on top of their orbital circles. These circles on top of circles - or epicycles - form the basis of Ptolemy's new and "improved" geocentric model.

Ptolemy's greatest mistake was not made in adopting the geocentric model in the first place. It was made in not abandoning it when it no longer fit observed fact. Rather than throw out his bad design, he added more and more layers of bad design to try to fit the new observations.

Copernicus's Great Refactor

This process of adding more and more complexity onto complexity continued through iteration after iteration for almost two thousand years until Copernicus stepped in. He fixed the bad design at the root of the ridiculously complex and unwieldy Ptolemaic system by putting the Sun at the center. This simple change had a profoundly simplifying effect on our understanding of the motion of the planets and other heavenly bodies.

Copernicus's heliocentric model is in retrospect a rather simple and obvious paradigm shift. Nevertheless, it marked the start of the Scientific Revolution and is often regarded as the starting point of modern astronomy.

Copernicus on Software Development

What does all of this mean for software developers? It means that if your system is built on a bad design, trying to accomodate that problem will only lead you to add more and more levels of bad design. Eventually the whole system will come crashing down under the weight of this accumulated technical debt.

This process of iteratively adding layers of complexity and bad design to attempt to shore up a faulty assumption or hopelessly flawed abstraction is called Adding Epicycles in honor of Ptolemy and his great design failure. The right thing to do is to follow Copernicus's lead and correct the faulty assumption or poor abstraction that is at the heart of your design failure. The sooner you do this, the sooner you can start simplifying your code.

Correcting a critical design flaw will often cause these layers of circle upon circle upon preposterous circle of complexty to vanish, leaving your code clean and clear and simple. Just try not to anger the Pope.

Ruby Patterns: Query Method

2008-07-17T00:00:00-07:00

Today, we will learn how the Query Method pattern handles a set of tensions we often face while designed object oriented systems.

What are patterns? Patterns are learnable and reusable answers to common programming questions. They are decisions made in response to a number of different concerns (or tensions). A well written pattern resolves each of the tensions harmoniously and provides a set of repeatable steps for its reapplication.

Let's look at a situation where various tensions pull us towards the need to make a harmonious design decision and attempt to derive a pattern for future use. As we do so, we will try to keep in mind the tensions at play and come up with steps that can be used to reapply the pattern in the future.

The Query Method pattern answers a particularly common question, "How can I ask an object for information about itself?" The tensions at play include a desire to maintain proper encapsulation, a desire to keep code DRY(Don't Repeat Yourself) and a desire to maintain or improve the readability of the resulting solution.

If you don't really care about the discovery process and you want to skip straight to the pattern itself, it's at the bottom.

Discovering A Pattern

In a recent blog post on the value of learning encapsulation before learning rails, Stephen Chu gives an excellent example of a fragment of ruby code that could be improved by application of the Query Method pattern. I'll use a slightly different example:

``` ruby

puts "#{@post.title} is recent." if @post.published_at < 2.days.ago

```

Let's look at this code in light of some of the different tensions:

Encapsulation

In his post, Stephen points out that this code breaks encapsulation by yanking state information out of the object for comparison. The comparison is being made on post information but it is being made outside the context of the post. We can change the code a bit to make this more clear:

``` ruby

class Blog
  def recent_post_list
    @posts.each do |post|
      puts "#{post.title} is recent." if post.published_at < 2.days.ago
    end
  end
end

```

The blog that is creating this recent post list shouldn't and needn't know how to determine if a post is recent. This is the post's responsibility. The blog should simply ask the post if it is recent and let the post make the calculation. This desire for encapsulation is the first tension.

Don't Repeat Yourself

This code seems simple enough. It does one thing and appears to do it efficiently. There is no violation of DRY yet, but it isn't to difficult to imagine other situations where we might need to know if a post is recent.

Let's say we are generating an HTML page and we want to apply a special class to recent posts. We will need to duplicate this logic inside that method as well. What if we later decide that we want posts in the last three days instead of two? Now we have to find each instance of this code fragment so that we can change it appropriately.

What a pain! Clearly this code is not DRY and its maintainability is compromised as a result. This is the second tension.

Readability

This code looks pretty readable. published_at and 2.days.ago reveal their intention quite clearly. If any improvement is to be made, you may think, it will be incremental and relatively small. Nevertheless, improved readability means improved maintainability, so it is always worth consideration. This is the third tension.

Composing A Pattern

This list of tensions isn't exhaustive but it should be sufficient for us to begin considering a solution. How can we improve this code in a way that most effectively addresses all of them?

This is a case where the tensions are not in direct opposition. It should be pretty easy to come up with a solution. Let's try to formulate such a pattern, including the steps necessary to reapply it again in the future.

Encapsulation and DRY

First, we'll start by encapsulating the comparison inside the post object:

``` ruby

class Post
  def is_published_at_less_than_two_days_ago?
    published_at < 2.days.ago
  end
end

puts "#{@post.title} is recent." if @post.is_published_at_less_than_two_days_ago?

```

This resolves the encapsulation tension. By factoring the behavior to a single location it also resolves the DRY tension. It doesn't do much for our readability tension, though. Let's see what we can do about that.

Readability

Patterns can often be improved by the inclusion of smaller, more granular patterns. In this case, the readability can be improved by providing the method with an Intention Revealing Selector. An intention revealing selector is, as the name suggests, one that informs to the user of the intention of the method rather than its implementation.

Our current selector - and the code it is based on - informs the user of the method's implementation. It answers the "how" question. An Intention Revealing Selector answers the more useful "why" question, as in: why does this method exist, why would I want to use it?

So: why does this method exist? It exists so that the post can be asked if it is recent or not. An intention revealing name for this selector would use the word "recent" to answer the "why" question.

The Ruby idiom for a method that is intended to return a boolean value is to end it with a "?". Let's apply the Intention Revealing Selector pattern by giving the method an intention revealing name and ending it with a "?".

``` ruby

class Post
  def recent?
    published_at < 2.days.ago
  end
end

puts "#{post.title} is recent." if post.recent?

```

I think you'll agree that this new method resolves all of the tensions harmoniously. The behavior is properly encapsulated, it is DRY because it is defined Once And Only Once, and it is significantly more readable than the original because it reveals its intention rather than its implementation.

Thus, we can formulate the Query Method Pattern as such:

Formulating The Query Method Pattern

When you want to query an object about itself in a way that is properly encapsulated, DRY and more readable, follow these steps:

Write a method that performs the query.
Put it inside the object that holds the information being queried.
Give it an Intention Revealing Selector.

The next time you're faced with this design question, you can apply the Query Method pattern.

Hopefully this process will help you begin to identify patterns of your own. If you're interested in learning more about patterns (and you should be), I highly recommend Smalltalk Best Practice Patterns by Kent Beck (non-affiliate link). It's worth learning Smalltalk just to be able to understand the code examples.

A Thinking Man's Sphinx

2008-07-14T00:00:00-07:00

Today, we'll explore the differences between UltraSphinx and ThinkingSphinx and why we chose to switch to ThinkingSphinx.

We've recently switched a number of projects to ThinkingSphinx here at Hashrocket. These projects were originally using SOLR or UltraSphinx. UltraSphinx is written by Evan Weaver. ThinkingSphinx is written by Pat Allan. They have some similarities: both use Sphinx (obviously); both are based on the underlying Ruby API for Sphinx, Riddle (also by Pat Allan); both have excellent documentation and well-written tutorials. The similarities pretty much end there, however, and the differences are far more interesting.

Basic Sphinx Configuration

Both plugins help you generate a sphinx.conf file for your each of your rails environments, but they do it in drastically different ways. ThinkingSphinx lets you use a configuration format you are already used to at the expense of reduced configuration options. UltraSphinx is more flexible but less Rubyish.

UltraSphinx

UltraSphinx generates the sphinx.conf file from a base configuration file. This base file uses the sphinx configuration syntax, passing it through ERB for some DRYness. A base file can be specified per-environment. It puts all of its configuration information in RAILS_ROOT/config/ultrasphinx/. This provides fine-grained - if rather tediously verbose - control over the multitude of Sphinx configuration options.

ThinkingSphinx

ThinkingSphinx uses a YAML configuration file that it locates at RAILS_ROOT/config/sphinx.yml. It accepts a YAML hash of configuration settings. These settings allow you to specify most of the basic Sphinx configuration options with ease but you may be out of luck if the option you need isn't available.

Basic Index Configuration

Let's start with a basic example of a sphinx index declaration. Keep in mind that your indexes will likely be significantly more complex in the real world.

UltraSphinx

UltraSphinx uses a declarative is_indexed statement in the model that feels vaguely similar in style to an association or named scope declaration. This is the usage example given in the README:

``` ruby

class Post
  is_indexed :fields => ['created_at', 'title', 'body']
end

```

This seems simple enough for such a simple case. We'll see how it looks for less trivial cases.

ThinkingSphinx

ThinkingSphinx, on the other hand, uses a define_index block in the model to allow the individual index configuration options to be stated declaratively. The canonical example from UltraSphinx would look like this in ThinkingSphinx:

``` ruby

class Post
  define_index do
    indexes created_at, title, body
  end
end

```

The first thing you may notice is that the same index configuration is three lines in ThinkingSphinx instead of one in UltraSphinx. If you look closely, you'll also see that the field names are not symbols as you might expect but method calls. We'll get into why this is in a moment.

Real World Index Configuration

Your real world applications are likely to require a significantly more complex index declaration to meet the search needs of your users. Let's look at an example of such a real world Sphinx index declaration.

UltraSphinx

Here's an example of a more realistic UltraSphinx index configuration. This is the type of configuration you're likely to use on any non-trivial project.

``` ruby

class Post < ActiveRecord::Base
  belongs_to :blog
  belongs_to :category

  is_indexed :conditions => "posts.state = 'published'",
             :fields     => [{:field => 'title', :sortable => true},
                             {:field => 'body'},
                             {:field => 'cached_tag_list'}],
             :include    => [{:association_name => "blog",
                              :field            => "title",
                              :as               => "blog",
                              :sortable         =>  true},
                             {:association_name => "blog",
                              :field            => "description",
                              :as               => "blog_description"},
                             {:association_name => "category",
                              :field            => "title",
                              :as               => "category",
                              :sortable         =>  true}]
end

```

This is about as pretty as it's going to get - and that's not very pretty. Large, deeply nested hashes of arrays of hashes are not easily scannable and will become exponentially difficult to maintain as their size and complexity increases.

ThinkingSphinx

Let's look at that same example translated to ThinkingSphinx.

``` ruby

class Post < ActiveRecord::Base
  belongs_to :blog
  belongs_to :category

  define_index do
    indexes title, :sortable => true
    indexes body, cached_tag_list

    indexes blog.description, :as => :blog_description
    indexes blog.title,       :as => :blog,     :sortable => true
    indexes category.title,   :as => :category, :sortable => true

    where "posts.state = 'published'"
  end
end

```

Not only did the number of lines decrease, the readability is far greater. I know which one I'd rather write. More importantly, I know which one I'd rather have to maintain weeks or months downline when it needs to be modified.

Notice that the declarations use methods rather than symbols. ThinkingSphinx uses some interesting metaprogramming to allow this. Notice also that indexed fields on associations are specified in the same way you would access that field. Simple.

Sphinx Rake Tasks

Both UltraSphinx and ThinkingSphinx provide a number of rake tasks for common sphinx tasks such as generating the configuration file; generating the index; and starting, stopping, and restarting the searchd daemon. Both provide abbreviations for the more common task, such as ts:in for thinking_sphinx:index or us:conf for ultrasphinx:configure.

Deployment and Configuration Management

Both UltraSphinx and ThinkingSphinx are pretty simple to deploy. You should symlink your configuration file from a shared location into your app's path after deployment, just as you probably do for your database.yml file. You will probably want to run the configuration task after you update the code. Here, for instance, is a Capistrano task to run your ThinkingSphinx configuration task:

``` ruby

namespace :sphinx do
  desc "Generate the ThinkingSphinx configuration file"
  task :configure do
    run "cd #{release_path} && rake thinking_sphinx:configure"
  end
end

```

You'll want to have this task run after each deployment:

``` ruby

after "deploy:update_code", "sphinx:configure"

```

You can create other tasks relatively easily for reindexing and managing the searchd daemon. I found a good guide to deploying a rails app with ThinkingSphinx linked from Pat Allan's blog. I found a useful set of UltraSphinx capistrano tasks in Ruberion's server tools plugin on Github. If you chose to host with EngineYard, they can manage either configuration for you with their pre-baked builds and deploy tasks.

Real World Experience

We ran into a number of issues when setting up UltraSphinx:

UltraSphinx loads your models without loading the full rails environment. This means that if your models depend on any of your lib files or any gems frozen in vendor/gems, you will have to require all of these files explicitly in each model. This is a pain.
The fundamentally sound design and code of UltraSphinx are somewhat undermined by poorly implemented exception handling. This means that while most of the time things work swimmingly, when they fail you're really sunk! The errors that you receive are often useless in diagnosing the actual problem.
We had bugs in our index that only existed on our staging and production slices. These caused page counts to be incorrect and nil records to be returned in certain cases. In certain cases it also caused 5xx errors.

Moving To ThinkingSphinx

After another Hashrocket team had success moving their project from SOLR to ThinkingSphinx, I decided to move our project as well. Moving to ThinkingSphinx proved to be a relatively painless experience. The process was essentially five-fold:

Uninstall UltraSphinx and install ThinkingSphinx.
Translate your is_indexed declaration into a @define_index@ block and change your search actions to use the ThinkingSphinx API.
Rewrite your deployment tasks to run the ThinkingSphinx rake tasks.
Stop searchd and then run your new configure, index and startd start tasks.
PROFIT!

Git Push: Just The Tip

2008-04-18T00:00:00-07:00

Today we delve into the world of git push, one of the most often used git tools. git push is typically used to update a remote ref and associated objects based on a local ref -- in other words, to push your local changes to an upstream repository -- but you can also use it to create or delete remote branches and ~~much, much more!~~ actually, that's about it.

Just The Tip

The most common use of git push is to push your local changes to your public upstream repository. Assuming that the upstream is a remote named "origin" (the default remote name if your repository is a clone) and the branch to be updated to/from is named "master" (the default branch name), this is done with:

git push origin master

Get used to this. You'll be doing it a lot.

A Bit Deeper

Git uses the term "refspec" when describing the usage of some of its commands. A refspec is essentially a name that git can resolve to a commit object. This can be the name of a branch or an arbitrary "SHA1 expression" such as master~4, among others. Git gives you a lot of ways to refer to a commit but for most purposes you'll just use the name of a branch with git push.

The kernel.org manpage for git pull will rather cryptically tell you that "The canonical format of a <refspec> parameter is +?<src>:<dst>". Most of the time this translates to <branch to push from>:<branch to push to>. The branch to push from and the colon are optional. If left out, git will push from the local branch to the remote branch of the same name. If no refspec is used at all, git will push all "heads" (f.e. branches) on the local to matching heads that exist on the remote.

In practice, this means that:

git push origin will push changes from all local branches to matching branches the origin remote.
git push origin master will push changes from the local master branch to the remote master branch.
git push origin master:staging will push changes from the local master branch to the remote staging branch if it exists.

Tips and Tricks

Create a Remote Branch

git push origin master:refs/heads/staging will create the branch staging in the origin by copying the local @master@ branch

Delete a Remote Branch

git push origin :staging will delete the branch staging from the origin repository.

Set Up A Branch's Default Remote

You can use git config to assign a default remote to a given branch. This default remote will be used to push that branch unless otherwise specified.

This is already done for you when you use git clone, allowing you to use git push without any arguments to push the local master branch to update the origin repository's master branch.

git config branch.<name>.remote <remote> can be used to specify this manually.

Custom Textile Tags For Great Justice

2008-04-09T00:00:00-07:00

Taking some inspiration (and code) from the quirky but lovable why's original post on adding yer custom blocks to RedCloth and the inimitable Geoff Grosenbach's foray into the world of custom textile figure tags, we're going to implement a customer textile tag for our blog's thumbnail images.

RedCloth's textile implementation uses some simple metaprogramming to create its own tags, which allows you to (more or less) easily create new tags to suit your own needs. The basic formula is: write a textile_#{ tag } method, where tag is the name of the tag you want to create. This method takes four arguments, tag, atts, cite and content, which are parsed from the textile by the RedCloth engine.

For our purposes, we're only concerned with content and atts, so I'll leave the rest as an exercise for the gentle reader. The html we're trying to create looks like this:

``` ruby

<div class="img" id="figure-1-1">
  <a class="fig" href="http://reinh.com/images/image.jpb">
    <img src="http://reinh.com/images/thumbs/image.jpg" alt="Figure 1.1" />
  </a>
  <p>Figure 1.1</p>
</div>

```

The tag we want to use to create it looks like this:

fig. 1.1 | image.jpg

That's quite a bit shorter and more elegant. Jumping right in to the good stuff, the method definition for our new tag looks like this:

``` ruby

def textile_fig(tag, atts, cite, content)
  span_class = "img "
  if atts =~ /class="([^\"]+)"/
    span_class += $1
  end
  (figure_number, img_url) = content.split("|").map { |w| w.strip }
  figure_name = "Figure #{figure_number}"
  figure_id = "figure-#{figure_number}".tr(".", "-")

  <<-TAG
  <div class="#{span_class}" id="#{figure_id}">
    <a class="fig" href="http://reinh.com/images/#{img_url}">
      <img src="http://reinh.com/images/thumbs/#{img_url}" alt="#{figure_name}" />
    </a>
    <p>#{figure_name}</p>
  </div>
  TAG
end

```

Let's break that down.

``` ruby

span_class = "img "
if atts =~ /class="([^\"]+)"/
  span_class += $1
end

```

This adds any classes in atts (in the form of class="foo") to the base class, img.

``` ruby

(figure_number, img_url) = content.split("|").map { |w| w.strip }
figure_name = "Figure #{figure_number}"
figure_id = "figure-#{figure_number}".tr(".", "-")

```

This breaks fig 1.1 | image.jpg down into two parts by splitting on the | and then normalizes them a bit to be used later.

Finally, the relevant parts are jammed into the html prototype and spit back as the method's return value for insertion into your textile document (and for great justice, of course).

Simple Git Continuous Integration

2008-02-12T00:00:00-08:00

The easiest way to prevent a broken build is to run your tests before you commit. This can be done with a simple git hook. Preventing your developers (and yourself) from breaking the build is as simple as putting this in your .git/hooks/pre-commit and making it executable: chmod +x .git/hooks/pre-commit.

``` sh

#!/bin/sh
rake spec 2> /dev/null

```

This will stop the commit if the specs don’t pass.

This isn’t a replacement for a more robust CI system but it does make it a lot harder to do something stupid. Redirecting STDERR to /dev/null is optional but recommended since the STDERR output of failing specs isn’t useful. It you use Test::Unit instead of RSpec (for shame), use rake test instead. Likewise, anything that returns proper error codes (0 for success, > 0 for failure) can be used.

This is mainly useful if your specs take under a minute to run, otherwise it becomes tedious. If you have long-running specs, I suggest using a special task that runs an abridged set of core specs instead.