skwpspace it’s pronounced ’scoop’. yan pritzker’s home on the web

Does Mysql really need occasional restarts?

You may have seen my blog occasionally go down due to database problems. I have been constantly on email and phone with bluehost support to try to figure this out. The responses I’ve gotten always seem completely ignorant of bigger issues. Ranging from “we didn’t see a problem” to “we killed the threads so everything should be fine now”. The last time it happened, I was online at the time and literally saw some reaper scripts fire up and nuke the Mysql server and cause it to restart. So I decided to investigate why this was happening with support.

After about a round of ten emails, I got escalated to level 3 support and got the best “you’re in a helicopter” response ever: I asked why they have scripts that run that kill and restart mysql after I caught this happening during a time when my blog was down. The answer: “When there are problems with mysql the server will try to restart it…”. Uh, really?

Now maybe I’m crazy here, but can I get a show of hands of people who have such problems with their Mysql that they have to regularly restart it? In my experience, it’s possible to run a properly configured database instance with nearly infinite uptime. So, are they royally screwing something up? Or is this really just a downside of running a database for tons of clients who may be doing bad things to it?

Update: bluehost has admitted the problem is not from my account. They have a connection limit of 20 connections for Mysql for all accounts on the server. That it took me ten emails to have them admit they’re overselling capacity is downright unacceptable. And I’ve been fighting this battle with them for more than two months now. I asked them time and time again to admit they’re overselling and to fix the problem by reducing load on the server, but no one would so much as even answer my question, every time implying that it was my php scripts (a standard wordpress installation, using caching, and on low traffic with less than 5 php fastcgi instances) causing the problem. Now they’re offering to migrate me to another server, clearly again band-aiding the problem instead of addressing the core of the problem.

From bluehost support:

“When you get the error that mysql is out of connections that means mysql connections on your server have backed up to the point to where the connections are maxed. Not necessarily from your account, but the server as a whole. We don’t think a restart is the answer to the problem, however often times stopping mysql, running a repair to fix any corrupt tables then restarting will often times clear out the cause of the problem.”

We cannot receive your email at this time

I submitted an inquiry to B&H Photo, an online Photo Store that has a cute habit of ‘closing’. Evidently these people didn’t get the message that online stores don’t have hours or off days. But here’s the kicker, I emailed their customer service for an inquiry, and this is the response:


We regret that we cannot receive your e-mail at this time. We will reopen on Monday, April 28th at 9:00am. Please Re-submit your inquiry after that date.


Newsflash guys: this is the Internet. Just because your store is ‘closed’ doesn’t mean you can’t accept email. Asking your customers to re-submit an inquiry is plain silly. How about a nice friendly “We’ve received your email and we will get to it as soon as we get back.” What is the world coming to?

Elastic Server now with Passenger (mod_rails)

We’ve added Passenger support to our Rails 2.0 Elastic Server Portal, so that you can now create both Mongrel and Passenger servers and compare them. If you’re happy with what you’ve built, create it in EC2 format and deploy it to the cloud! Read more at the Elastic Server Blog

Useful git commands and quirks

Things I’ve learned from my first git experiences. First of all, git is inconsistent as hell. Every command has its own quirks and syntax, so I’m attempting to catalog some of them here. We’ve been using git svn at Planypus as a way to maintain local developer branches and still push to our svn as a central place which then triggers our integration server rebuild, cruise control, and so on.

First, make it look nice

$ git config --global color.diff auto
$ git config --global color.status auto
$ git config --global color.branch auto
$ git config --global color.interactive auto

via tpope

Onwards to the commands…

If you’re using git svn and you want to see changes that you have comitted locally but have not yet dcomitted

git svn dcommit -n

This might give you output that looks like this
diff-tree e784cfa... e784cfa...

BUT, if you want to actually see the changes, what you want is apparently the ‘patch’ output:

git diff-tree -p e784cfa... e784cfa...

If you have uncomitted changes, it’s easy to see them

git diff

BUT, if you’ve already created your commit set (git add) but not comitted, you how have to use

git diff --cached

If you’ve accidentally added some files to your commit set and want to remove them from the index (untrack them), but leave the changes in your working tree

git reset HEAD [file/dir]

To undo local changes (revert in the svn sense)

git checkout [files]

Note that this works as long as the file is not yet in your index. If you’re already tracking it for commit, you’ll need to untrack it first using git reset HEAD [file] before you’re able to check it out.

You can also use this command to check out files from the past by specifying any commit hash:

git checkout [hash] [files]

It’s just that when you don’t specify the hash, it assumes HEAD.

If you’ve got changes you want to just throw away permanently, you can just reset:

git reset --hard

But reset is a powerful command that can also take you back in time. Let’s say you messed up the last commit, you can take yourself back to the way things were right before you comitted:

git reset HEAD^

Notice I didn’t use --hard because that throws away the changes. Instead, this form of the command throws away the commit by taking you to HEAD^ (one commit before HEAD), but it leaves the changes in your working tree. If you want the changes also to be left in your index (ready to be comitted) you can use

git reset --soft HEAD^

If you’re working on something and you decide you want to scrap it or work on it later

git stash

There are lots of fun things you can do with stashes but they’ve been covered on other blogs.

on merging…

If you’ve done a git merge and have some conflicts, launch git mergetool which will take you through the conflicted files one by one using your default merging tool (FileMerge, usually, on OSX) and let you merge them. However be careful. Either because of a misconfig on my end or some other quirk, FileMerge shows the file I pulled from svn as “LOCAL” while my file with changes is “REMOTE” which is counterintuitive to me and caused me to fubar a merge the first time I tried.

Take note that after the merge, you will have files ending with .orig left on your system so you’ll have to manually delete them, unless you have the config mergetool..keepBackup set, but I haven’t tried this yet. Also, after the merge I was getting error messages about having a .git/MERGE_HEAD file, which I couldn’t find enough docs online for, so I just deleted it..and hopefully that did not fubar me.

Well that’s all for now. More might come as I learn to stop worrying and love the git.

P.P.S Many of these commands are not necessary if you use The Git Textmate Bundle and GitNub which integrate together and work very well!

The security of working at a startup

I recently had an interesting conversation with my grandfather, who was concerned about the economic downturn (recession?) and the impact it may have on my having a job. As I tried to explain to him why I choose to work in startups over big companies, I came to realize that I feel there’s a certain job security in working in a startup, perhaps even better than that of a big company.

In a startup you know when your money is going to run out. In a startup, you are responsible for the existence of own job position. By this, I mean that in a team of ten, you know that slacking off is not an option. You know that the work you do will directly result in business for your company, investment, and hence the ability of the company to pay you. Your own contribution carries a high proportional weight, which means you have a very real effect on whether your position is still available tomorrow.

A big company is an entity with a life of its own. The average employee does not have a sense of their own impact on company growth or business. Hiring and firing decisions are made on a level far enough removed from the average worker that they feel such decisions are arbitrary and unjustified. Employees talk in hushed whispers about looming layoffs without understanding why they are necessary. After all, they are just doing their jobs!

In a bigger company, layoffs due to a downturn in business or the economy are not only possible but probable. In a startup, they are unaffordable; A team of ten that loses one person loses 10% of their workforce, and faces a huge challenge in bringing a replacement up to speed in such a fast paced environment. A company of thousands can lay off handfuls of people here and there in a heartbeat in order to course-correct.

So - which is more secure? Sure you can look at a big blue chip company and say - they haven’t laid anyone off in ten years. But do you know for a fact that they won’t lay you off tomorrow because of factors you can’t even control? Think about that when deciding on your next job, and choose to work in a startup, where you are responsible for your own success (and failure).

del.icio.us is social search

I find myself more often turning to tag search on del.icio.us when I’m looking for ‘best of’ type searches. For example, if I’m shopping for a camera strap or looking for free icons, I turn to del.icio.us because I don’t want to sort through thousands of results based on relevancy determined by an algorithm; I want the results that people think are the best. And with the amount of data del.icio.us now has, it has become a great place to find things that are popular.

What’s interesting here is that social search is an emergent behavior of del.icio.us, which was designed primarily as a remote bookmarking tool. In the del.icio.us lesson on bokardo, Josh points out that “personal value precedes network value”. By optimizing for the bookmarking (personal) experience, yet making the early decision of keeping everything public by default, del.icio.us surfaces a gold mine of the “best of the best on the web”, and thus becomes exceptionally useful as a filtered search engine.

Because most of the data is put on del.icio.us for selfish reasons, it is by and large very high quality source of information. Del.icio.us is not easily ‘gamed’ by things like SEO because in order for your link to be popular, other people have to actually want to bookmark it as well. Thus I am inclined to trust a highly bookmarked result on del.icio.us over a highly ranking one on Google.

If Yahoo was smart, they would be integrating del.icio.us search results into their main search engine. Of course I would rather see Google do this but it so happens that Yahoo owns del.icio.us so a Google deal is less likely (yet not impossible in this mashed-up world). While Google continues to be a great place to start a random day, del.icio.us is becoming more and more my place to go when looking for a specific type of result.

Rails moves to Lighthouse

In a series of dev infrastructure changes, the Rails team is moving to Lighthouse as its bugtracking system. I wrote about Lighthouse more than a year ago as an application to watch in the bugtracking space, and now they’re coming into the limelight with official support from the Rails team.

I’m really happy to see that the Rails team and 37 Signals are making this move because it will force Lighthouse to become an even better application as it meets demands of many projects that are sure to follow Rails.

Too many teams put up with products that have terrible usability like Bugzilla because many programmers tend not to think about usability as an important factor. We are used to command line interfaces and white text on black backgrounds. But believe it or not, using something like Lighthouse over say, Bugzilla, boosts your productivity because it is well designed and lets you get to the information you need more quickly and with less frustration. That said, Trac is still many levels above Bugzilla in usability and design so if you’re using an in-house product, I don’t know of many better ones.

Rails moves to GitHub; Git is the new black

Rails has officially moved to GitHub. The GitHub guys have done an awesome job capitalizing on the growing trend of git usage, especially in the Rails community, and of course getting ‘official blessing’ from the Rails core team themselves means that many more projects are sure to follow.

I’ve been using git-svn to work on a local branch of our svn when I work on Planypus and have enjoyed the ability to do local commits on the go. I can’t say I’ve quite mastered all the ins and outs of it yet but working with git via git-svn is pretty smooth and it’s fun to see all the output that shows your changes being played back when you do ‘git svn rebase’. Other than that, working with git does require a bit of a paradigm shift because the primary entity is the changeset rather than the file. I hope to get a better handle on things like reverting in the near future and am watching the progress of GitHub and friends with great enthusiasm.

fix for TextMate Ruby tests broken with stack level too deep

If you experience problems running single unit tests in TextMate with Rails 2, it may be due to a bug. The problem has something to due with conflicting builder.rb instances in Rails and TextMate. The simplest and least intrusive fix is to add this to the top of your test_helper.rb


$LOAD_PATH.reject! { |e| e.include? 'TextMate' }


I’m back from Mt. Blanc

I’m back from Mont Blanc - unfortunately due to extreme weather (it dumped more than 3 feet of snow one night), the normal routes on the Blanc were not available to us. Nonetheless we did some nice climbing and skiing so it was still a good time.

The full photoset is at fotki because I didn’t want to ‘pollute’ my flickr stream which I am keeping for more artistic photography. Unfortunately until flickr allows to have multiple streams, I will keep my fotki account for a mass dumping ground of photos.

Unfortunately flickr doesn’t give you control over what appears on your ‘home’ page. Otherwise I would tell it to display only a given tag, set, or something. As it stands I have to be careful about what I upload because it changes what my flickr account looks like. Damn it, flickr!