git commit –amend: Have Your Cake and Eat it Too

For a lot of us, version control does two things which are contradictory:

  • it keeps checkpoints as we work so we can easily roll back, and
  • wraps up our work into a tidy package for others to use.

git commit --amend is one tool that solves this. It will “open up” the previous commit, adding staged files and allowing you to re-edit the commit message.

  1. Make your “checkpoint” commit, e.g. with a failing spec.
  2. Get your specs working again.
  3. Stage your changes like you normally would using git add.
  4. Instead of git commit, do git commit --amend.
  5. Edit the message to describe the feature. This will be the commit’s new message, and it will contain all the changes.

A related command: git rebase -i  (“interactive”). If you didn’t use --amend, you can go back and condense commits together as if you had. With this command, you can interactively rebase, using the “squash” command to combine multiple commits together.

A Benefit of Rails: Surrogate Keys

Today I helped a client wrestle with a database task which concluded:

. . . We’ll have to watch out inventing new “fake” customers because one day there may be a real customer with our made up ID :-(

This is a problem in lots of apps, but not in Rails.

Rails enforces the “best practice” of giving every table a primary key named id, which is an auto-incrementing integer. But many non-Rails apps do not do this. Instead, they use some real-world data as the primary key. This is called a natural primary key. E.g., a customer’s social security number. There are arguments for and against this. This is a good little wikipedia page about the differences.

One big problem with natural primary keys is that they often turn out to not be as unique as we think. Plus, then we’re put in the position of “faking” the SS number when we want to manually add customers who don’t have one, e.g. “999-….”. (I remember a college I attended which assigned 999- SS numbers to international students.) And this is the hack that the task description refers to.

In contrast to natural keys, the id field is a so-called surrogate key. It intentionally has no meaning in the real world, and so it has no reason to ever change. It doesn’t matter at all what an object’s id is, just so long as it’s unique. And since it gets set up as an auto-incrementing integer, that will be the case.

So, Rails is “opinionated software” and has made the decision for us. Every table will have;

  1. a surrogate key,
  2. named id,
  3. which is an auto-incrementing integer.

And in one swoop, a whole category of potential problems is eliminated. Secondly, this makes it very easy to get up to speed on a new Rails app.

Self-validating plain old Ruby objects with ActiveModel Validations

I’m importing lots of CSV restaurant inspection data with Ruby, and I need to make sure the cleaned up data matches the spec. For example, a violation must have a business_id and date. It can optionally have a code and description. My goal was to be able to write a class like this:

class Violation < ValidatedObject
  attr_accessor :business_id, :date, :code, :description

  validates :business_id, presence: true
  validates :date, presence: true, type: Date
end

…and it would just work, Rails-style:

Violation.new do |v|
  v.business_id = '1234'
  v.date = '2015-01-15'
end
# => ArgumentError: date must be of the class Date

Violation.new do |v|
 v.date = Date.new(2015, 1, 25)
end
# => ArgumentError: business_id is required

Violation.new do |v|
 v.business_id = '1234'
 v.date = Date.new(2015, 1, 25)
end
# => new instance Violation<...>

ActiveModel::Validations was refactored out of ActiveRecord to enable just this sort of use case. This is because the awesome list of built-in validations (here and here) and methods like #valid? are useful in a variety of contexts, not just in Rails.

It turned out that it was easy to write a small base class with the ActiveModel::Validations mixin to make the checking automatic upon initialization:

require 'active_model'

class ValidatedObject
  include ActiveModel::Validations

  def initialize(&block)
    block.call(self)
    check_validations!
  end

  def check_validations!
    fail ArgumentError, errors.messages.inspect if invalid?
  end
end

For my importing and parsing purposes, I created a small custom TypeValidator:

 # Ensure an object is a certain class. This is an example of a custom
 # validator. It's here as a nested class for easy access by subclasses.
 #
 # @example
 #   class Dog < ValidatedObject
 #     attr_accessor :weight
 #     validates :weight, type: Float
 #   end
 class TypeValidator < ActiveModel::EachValidator
   def validate_each(record, attribute, value)
     return if value.class == options[:with]

     message = options[:message] || "is not of class #{options[:with]}"
     record.errors.add attribute, message
   end
 end

Here’s the finished ValidatedObject, and an example subclass: information about a CSV feed. This set up is working great; the only change I can see making soon is changing ValidatedObject to be mixed in via include rather than subclassing.

Thanks to:

The problem with packaging in Python

Excellent article. We got to this situation because of the changing nature of writing software, and how we deploy. Our standards have risen, and packaging (and tooling in general) takes on ever greater importance.

But we don’t have to theorize and invent paradigms from scratch. A few other languages have solved this. Ruby is the best example. Let’s learn from its user experience and the use cases it serves.

Python project organization, circa early 2015

The best, current guides I’ve found

It’s hard to find current best practices for organizing a reusable python package. I’m starting a new project to get the world’s restaurant health scores online, and so searching for some kind of guide, these are the best I found.

Degrees of freedom: Four valid ways to write degrees Fahrenheit in HTML

I was working on my startup’s food safety score web app, writing code to clean up imported text like,

DELI DISPLAY COOLER HOLDING CREAM CAKE AT 44.6F BUT LOWER CORNER OF UNIT HOLDING FOODS @ 39F

We decided to make this as readable as possible without changing the content. And so, the finished text should have improved capitalization, punctuation, and typography:

Screenshot 2015-01-19 17.02.00

The interesting part was the temperatures. First I looked for a style guide because I realized I didn’t even know how a temperature should be punctuated; e.g. should there be any spaces within it? The National Geographic Society’s style manual was a good match for us. It prescribes no space between the final digit and the degree symbol:

F is the abbreviation for Fahrenheit: 32°F (no spaces, no period)

I then discovered four different, valid ways of writing this in HTML, each producing unique results:

Strategy Code Screenshot
HTML entity for degrees Fahrenheit #&8457; 1
HTML entity for degree symbol with a normal F &deg;F 2
Unicode degrees Fahrenheit character 3
Unicode degree symbol with a normal F °F 4

Notice how the fourth option both looks the best and it’s the easiest to read in the HTML source. That’s what I went with. Clearly there’s an interaction between the browser, the font, and the implementations of these protocols.

Test Driven Devops

Illustration: 500 - Internal server errorI like to apply Test Driven Development to my sysadmin work. For example, every time I add a new redirect to a web server configuration I want to make sure I haven’t broken anything else. Further, I want my SSL configurations proactively checked daily for any possible error. I use Ruby RSpec and write tests like these:

describe 'My app' do
  context 'www.myapp.com' do
    it { should be_up }
    it { should have_a_valid_cert }
  end

  it 'serves the &quot;about&quot; page without redirecting' do
    expect('http://www.myapp.com/about').to be_status 200
  end

  it 'only serves via www' do
    expect('http://myapp.com').to redirect_permanently_to 'http://www.myapp.com/'
  end

  it 'forces visitors to use https' do
    expect('myapp.com').to enforce_https_everywhere
  end
end

When I want to make a configuration change, I first write a test for the desired outcome. Naturally, it fails while the old tests pass. I then work on the config change, re-running all the tests as I go, and am finished when they all pass. I also run these automatically from a cron job to get pro-active notification of new problems.

The phrases such as have_a_valid_cert are custom RSpec matchers; they’re added into the RSpec environment by an open source library I’m writing and just made available on Github.  I’ve begun to refactor all the custom matchers out of my internal weblaws.org code into a new rspec-webservice_matchers gem which makes this easy to install.

See also

Linode vs. DigitalOcean: back to Linode for me

DigitalOcean doesn’t give their first-line support people the necessary tools to diagnose their own system problems. And so a two-day outage becomes possible.

The night before Christmas, I was scrambling to get my site back up

'top' showing load of 8 and huge wait time

‘top’ showing load of 8 and huge wait time

On the afternoon of the 24th, My New Relic monitor told me that weblaws.org was offline. Ugh. I ssh’d in to the 4GB RAM DigitalOcean Droplet and saw that the load was up to 8 (FYI, this is crazy high), although no processes were active, and it had a huge unexplainable “wait” time.

Continue reading