The problem with packaging in Python

Excellent article. We got to this situation because of the changing nature of writing software, and how we deploy. Our standards have risen, and packaging (and tooling in general) takes on ever greater importance.

But we don’t have to theorize and invent paradigms from scratch. A few other languages have solved this. Ruby is the best example. Let’s learn from its user experience and the use cases it serves.

Python project organization, circa early 2015

The best, current guides I’ve found

It’s hard to find current best practices for organizing a reusable python package. I’m starting a new project to get the world’s restaurant health scores online, and so searching for some kind of guide, these are the best I found.

Degrees of freedom: Four valid ways to write degrees Fahrenheit in HTML

I was working on my startup’s food safety score web app, writing code to clean up imported text like,


We decided to make this as readable as possible without changing the content. And so, the finished text should have improved capitalization, punctuation, and typography:

Screenshot 2015-01-19 17.02.00

The interesting part was the temperatures. First I looked for a style guide because I realized I didn’t even know how a temperature should be punctuated; e.g. should there be any spaces within it? The National Geographic Society’s style manual was a good match for us. It prescribes no space between the final digit and the degree symbol:

F is the abbreviation for Fahrenheit: 32°F (no spaces, no period)

I then discovered four different, valid ways of writing this in HTML, each producing unique results:

Strategy Code Screenshot
HTML entity for degrees Fahrenheit #&8457; 1
HTML entity for degree symbol with a normal F °F 2
Unicode degrees Fahrenheit character 3
Unicode degree symbol with a normal F °F 4

Notice how the fourth option both looks the best and it’s the easiest to read in the HTML source. That’s what I went with. Clearly there’s an interaction between the browser, the font, and the implementations of these protocols.

Test Driven Devops

Illustration: 500 - Internal server errorI like to apply Test Driven Development to my sysadmin work. For example, every time I add a new redirect to a web server configuration I want to make sure I haven’t broken anything else. Further, I want my SSL configurations proactively checked daily for any possible error. I use Ruby RSpec and write tests like these:

describe 'My app' do
  context '' do
    it { should be_up }
    it { should have_a_valid_cert }

  it 'serves the "about" page without redirecting' do
    expect('').to be_status 200

  it 'only serves via www' do
    expect('').to redirect_permanently_to ''

  it 'forces visitors to use https' do
    expect('').to enforce_https_everywhere

When I want to make a configuration change, I first write a test for the desired outcome. Naturally, it fails while the old tests pass. I then work on the config change, re-running all the tests as I go, and am finished when they all pass. I also run these automatically from a cron job to get pro-active notification of new problems.

The phrases such as have_a_valid_cert are custom RSpec matchers; they’re added into the RSpec environment by an open source library I’m writing and just made available on Github.  I’ve begun to refactor all the custom matchers out of my internal code into a new rspec-webservice_matchers gem which makes this easy to install.

Update: I’m making an online service based on this code, and am looking for beta testers.

See also

Linode vs. DigitalOcean: back to Linode for me

DigitalOcean doesn’t give their first-line support people the necessary tools to diagnose their own system problems. And so a two-day outage becomes possible.

The night before Christmas, I was scrambling to get my site back up

'top' showing load of 8 and huge wait time

‘top’ showing load of 8 and huge wait time

On the afternoon of the 24th, My New Relic monitor told me that was offline. Ugh. I ssh’d in to the 4GB RAM DigitalOcean Droplet and saw that the load was up to 8 (FYI, this is crazy high), although no processes were active, and it had a huge unexplainable “wait” time.

Continue reading