Store git activity in MySQL with PHP

Git hooks are saving me so much time and providing me with interesting solutions to problems I didn’t even know I had. I can’t be the only person who this would be useful to, so give it a go.

As I said, I work on loads of sites, and keeping track of what’s been done and where is sometimes a bit of a pain. I keep a todo list, but if I get an emergency email from someone, chances are that won’t go through my todos. It will, however, be put into version control.

So this morning I had the bright idea to write a git hook that pushes relevant information to MySQL so that I can run activity reports later. All my bare git repositories are stored in a directory on our dedi, so it’s just a matter of making sure each repository has the post-receive hook in. I do this by keeping the actual hook in the same directory as all my repositories, then symlinking the hook into the appropriate place with the following little script. Obviously, this assumes that your post-receive hook is in the same place as your repositories, and that you want this hook everywhere. But that’s all true, so we’re all good. Once you’ve run the linked script, you’ll only have one hook to maintain and every time you create a new repository, you can just run the script again and everything will all be up-to-date.

Now for the hook. It’s not beautiful PHP, but little scripts like this rarely are, in my experience.

See the table create in the comment:

So basically we’re extracting the log data we need, doing some funky stuff to handle multi-line commit messages (I like to store lots of details as my subject messages tend to be a bit vague!). Other than that, if you’re familiar with PHP, the above should be pretty self-explanatory. If it’s not, hit the comments and I’ll explain things.

I’ve only been using this a little while, but it seems to work very well. If you use it and stumble across any bugs, I’d love to know about them!

Update: I’ve today realised that git log only logs the currently-selected branch, or master on a bare repo so I’ve added the –all switch to git log so I can get the logs for every branch. Most of it’s just “Merged blah” but that means it can be filtered easily and I’d rather have everything and need to filter than be missing something important.

git gtd mysql php

code

1Managing multiple working copies with git

We have a CMS at Buffalo, which we have deployed to several servers for a few of our clients. Until recently, it was only two separate servers and was relatively easy to manage with Subversion (though slightly cumbersome – svn ci -m “blah”; ssh server; cd /site; svn up; exit; ssh server2; cd /site; svn up;) but now that we’re deploying the same CMS to 8 servers, simple changes and bug fixes are becoming a pain in the ass to deploy.

I’ve been tentatively looking into git for quite some time now. I’m incredibly cautious of the bleeding edge when it comes to how I make money because I prefer to have things that I can rely on and will serve me well than keep up with the latest fads. That’s not to say that I’m not interested in all the cool new jazz, and I keep up as much as family life permits, but I don’t dive in and adopt without learning and understanding everything I need first. Sensible? Yes. I’m surprised at how many people flaunt this ethos.

That being said, git has proved itself invaluable to me in the last 4 months. The way it encourages you to work is great. I do all changes on branches then merge them into master and remove branches to keep everything clean. I also have $deploy-staging and $deploy-live (where $deploy is a working copy) so that I can manage configuration for each working copy. This probably isn’t git best practice, but I’ve found it to be incredibly convenient. I work on up to 20 different sites in any week, so being able to merge changes and conflicts for live stuff locally saves me headaches galore. No-brainer. My git workflow goes something like this:

git checkout -b hotfix-phperror
# do some work
git commit -am "Hotfix for PHP Error fixed"
git checkout master
git merge hotfix-phperror
# resolve any conflicts
git checkout staging
git merge master
# test on staging - everything OK
git checkout live
git merge master

That might seem like quite a bit of work, but it keeps everything tidy. Unfortunately, it does mean that if things fail on staging, I still have the history of it in master. I suppose it would make more sense to merge the hotfix into staging, then merge that into live, then merge everything into master and tag once it’s verified working to keep it clean, but either way, this tip will work.

I’m under the impression that git frowns upon what I’m about to recommend, but it works so well that I can hardly ignore it! One proviso for this is that all the working copies you push to have to be the same. Lucky for us, the configuration for our CMS is held by the live site (separate repository) rather than the CMS itself, so each working copy is the same.

With the above in mind, I’m going to assume the following:

• You’ve got a central, bare repository to push to
• Your live branch (identical for all working copies) is tracked from said central repository

First off, you have to make sure that your local working copy has all remotes available. Obviously your bare repository will be available because that’s how you’re doing things anyway, but we also need working copies. You can check that everything’s there by running a quick git remote. If you’re all good, then we can start pushing code around.

As I said before, git doesn’t seem to like this sort of thing. If you were to try and push to one of your working copies where the checked out branch is the one you’re pushing to, git will whine at you and preserve the changes you push so that you can stash any local differences. You can force this by changing to the directory and running git reset --hard or git stash, but that defeats the purpose of this so we need a workaround.

Luckily, git gives you all the hooks you need and more for this. We’ll be creating a post-update hook that will force-update your push. With that in mind, cd to the root of your working copy on one of your remotes and do something like:

echo "#!/bin/sh
cd ..
env -i git reset --hard" > .git/hooks/post-update;
chmod +x .git/hooks/post-update;

Now, when you push to the repository git will moan at you, but this hook will run and force all your changes. Awesome.

In the warning message git spits out, it threatens that new versions of git will auto-reject pushes to checked out branches, so a quick run of:

git config --set receive.denyCurrentBranch "ignore"

Will shut git up and have it doing what you tell it to. I suppose the reason it does this is it assumes that someone is working in that working copy and they don’t want you frivolously overwriting their code. Luckily, we know what we’re doing is safe so there’s no harm in doing this. Do remember, though, that all changes made in the working copy will be overwritten when you push using this method, so don’t work on live working copies. Don’t do that anyway, but definitely don’t do it here!

Now, you can do the following:

git remote add remote-alias ssh://root@blah/path/to/working/copy
git fetch
git push remote-alias live

In all the feedback, you’ll see something like “HEAD is now at 0d5431b HELLO!!!” (the hash and first line of your last commit message) which lets you know that things have worked.

Now you’re able to push to remotes without complaints, from the comfort of your local working copy, with a bit of scripting you can deploy your local live branch to all its remote locations with a little bit of scripting:

for remote in `git remote`; do git push $remote branchname; done;

For future reference, we’ll assume that this isn’t the only time you’ll do this, so save the following as a script in your path somewhere (I call it git-rpush):

for remote in `git remote`; do git push $remote $@; done;

Which you can call with git rpush live. Assuming that all your working copies have the config set and the hook installed, you can now push a change to all of your local repositories at once without having to remember which ones you actually need to push to, then spending half an hour SSH-ing all over the place to do it.

I don’t know about you, but that’s just saved me a crap-load of time. Hope it helps someone!

Coincidentally, if you do need this to be locally configurable, you can easily check out your branch and create a patch for configuration, then add the patch removal and application into hooks. I would do something like the following (assuming you already have your patch):

echo "#!/bin/sh
cd ..
env -i git apply -R config.patch" > .git/hooks/pre-update;
chmod +x .git/hooks/pre-update

This will remove your config so that no conflicts occur during the update. All you need is to re-apply the patch in the post-update, after you’ve done the reset. Insert the following:

git apply config.patch

Your config will be preserved. You can even keep these patches in version control, just change the name for all your remotes and update your hooks accordingly. Then enter my competition on how easy you can make your life by scripting it!

Any and all questions and improvements are welcome and appreciated.

I would like to thank the following post for hand-holding through the intricacies of what I was trying to achieve with hooks. So thanks!

git gtd script life

code

2trying an un-annoying lightbox

Lightboxes are really irritating. No users I ever speak to like them, but all website owners love them – communication problems there, maybe?

Here’s why I don’t like lightboxes:

  • They usually override keyboard functionality. I use my escape key to clear fields and I use my arrow keys to navigate pages – don’t overwrite that. I know best, not you.
  • They’re unnecessarily schmancy and animated – if I opened a lightbox, chances are I want to look at a picture and nothing else. Leave me alone otherwise. I don’t want to see your great “close” icon or any of that crap so just drop it!
  • Some are nearly impossible to close

There are probably a bunch more reasons, but you get the idea.

My lightbox stays out of your way. It doesn’t override any standard keyboard behaviour and it doesn’t use visual fluff to irritate you. What it does do is open a big version of the picture you clicked on so you can see it in more detail. When you’re done, it closes. Here’s how you use it:

  • Click a flickr image to open the lightbox (this might be the last time you need your mouse/trackpad)
  • j = next
  • k = previous
  • o = open the flickr page for this image
  • q = close the lightbox
  • You can also click anywhere to close the light when it’s open

If you’re wondering why j,k,q; use Vim for a day. I would’ve used e to open, but who’s going to remember that?

I’m probably going to implement a resizing thing for people with small monitors so that screens don’t get flooded – if this affects you, please tell me whether you’d like the process to automatically detect your browser size, or whether you’d prefer to handle it yourself. I’m aware that this will affect basically all portrait images – this will probably make me make it automatic. Pictures are pretty pointless if you have to scroll, but I’m tired now and can’t be bothered.

If you want the code for this, view source and find the javascript yourself. There’s a PHP file to make Flickr API calls, but that encapsulates my API credentials. If you can’t figure out what to do here, holla so we can all have a good laugh.

api flickr javascript lightbox mootools photos

code

that’s a big one

Transmitting file data …………………………………………………………………………………….
…………………………………………………………………………………….
…………………………………………………………………………………….
…………………………………………………………………………………….
………………………………………….

huge commit svn

code

6I must write out 1000 times…

Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions
Copying directories resets permissions

Not 1000, but you get the idea. Fuck.

code serveradmin unix

code

5impress yourself

Just recently, I realised how important impressing yourself is when trying to maintain enthusiasm with your job. Doesn’t really matter what you do (as long as you care!), but things can go stale if you keep doing the same thing over and over, and never really provoking an “AWESOME! I did that!” reaction. Even if it’s only something small (at 8 hours a day, 5 days a week, little things can be fairly impressive!), it can give you a needed boost to productivity (ugh), enthusiasm and general satisfaction with work, life and all that!

Personally, I get my kick from cleaning really crappy data. Lucky for me, I’ve had a lot of crappy data to clean recently, so I’ve been able to flex my XPath, Regex, semi-complex MySQL and image manipulation muscles relatively frequently. If you’re as much of a fanatical cleaner and hoarder as I am, cleaning data is probably for you. Grab a 50MB CSV and try to programmatically make sense of it. It’s fun! I promise.

code productivity work

code

3Unit testing

This is something that’s been in my peripheral vision for some time now. I’ve periodically been struggling to see the need for unit testing in general, without any justification. I can obviously see the benefit of making sure that your code all works and, seeing as I frequently work on ecommerce sites, I can see it causing far fewer headaches when making changes to purchase processes – simply being able to run a script to see if anything I changed works right.

However, one drawback of unit testing as a concept is that computers are, as a rule, infallible when it comes to this sort of thing. Humans, on the other hand, are completely fallible. If I make a change to some code in my checkout process that has a knock-on effect I didn’t count on in my unit testing (like changing the way delivery is calculated/stored/whatever), I wouldn’t necessarily foresee there being any problems in the email that gets sent after a transaction is completed (that may seem pretty esoteric, but it happened today). Part of this issue arose because I didn’t write the original site, and there was no way for me to detect that this could’ve caused a problem without going through every file on the site and manually checking that there was a problem, but another part could have been solved if a good unit testing framework was in place.

Now, when I say “good”, what I really mean is “comprehensive”. Anyone can write a good unit testing framework and here’s how, in my opinion. You test /every single dependency/. Everything. Bar nothing. If unit testing could’ve solved my problem today, it would’ve been done a little something like this (not exactly like this because the code on this site is largely a mish-mash of procedural and I-don’t-quite-understand-OOP-so-I’ll-give-it-my-best-shot – I assume it would’ve been very difficult to express this as a unit test).

Say I have a method that outputs this email that gets sent. I would write a unit test to capture the output of that method (when supplied with sample data from my database table, for a little extra reliability). I would then take this method and write my own version of it inside my unit test, so that I know it’s correct and compare the output of the two methods. If they differ, I would (PHPUnit) assertSame(legacy_method_output,my_method_output), and the test would pass if I hadn’t fucked it up. Now, the perceptive among you will have spotted something wrong here. In order to write this unit test, I would first have had to know about this email’s pre-disposition to causing problems in the first place. This, in this case, is the entire battle. Furthermore, I would have had to completely rewrite the method, solely to test that the old method hadn’t been broken in some way. For future testing purposes, this is really useful but, chances are I’d just rewrite the method and move on.

All of this isn’t to say that unit testing is fundamentally flawed. Far from it, in fact. I can see that it’s an incredibly useful tool for documentation and future-updates. When I come back to code that already has a unit testing framework, I’d simply make my alterations, run my unit tests until they all pass, then write some more to cater for my currently augmented feature-set. I would then know (with only a shadow of a doubt) that everything is brilliant and nothing is going to get fucked up when I put my code live. That’s got to be worth it! Not only the above, but unit tests seem to me to serve as excellent developer documentation. If you want to know how something’s supposed to work, never mind asking the project manager or client (who probably don’t know, and don’t really care!), you can just look and the guy who wrote it will have told you! Brilliant.

The only (and sadly, this is a pretty substantial “only”) problem is time (as with everything, I guess). You get all of this great peace of mind, but you have to put time into it. And it’s not just a small amount of time either, it’s a lot of time. Once you’ve written all your code (maybe doing it as you go along – 30 minutes a day or something) you then have to write tests to try and break it, and cater for every potential outcome that could arise from user or developer input of the code you wrote. That’s a large investment of time. I can’t quite decide if the net gain is quite worth the input, but I’m tipping towards a yes. I spend a lot of time a bit stressed about whether a modification I’ve made is going to cause unexpected problems, and I’d really like to be able to be more confident with this. I’m also one of those people who likes doing little scripty things, and gets a great sense of satisfaction from ticking boxes and watching “make test” throw out 100% success. Maybe this is for me, after all!

code phpunit programming unit testing

code

2git or Versions.app with MacVim diff

I’ve seen on the Versions Google group, there’s a python script for integrating Versions with MacVim diff, but because Versions doesn’t appear to be completely contextual when it comes to executing the script, I was getting the GUI load with none of my .vimrc stuff, so it was no good really.

I tried to write a PHP script to do this, as that’s what I’m most familiar with, but I also hit this hurdle so it was time to dust off my (formerly almost nonexistant) bash knowledge to try and get this done properly. I believe I’ve come up with a fix.

When writing this, I came across two problems. One being that Versions doesn’t escape space characters in the arguments is passes to your script, so it’s not just a simple case of export HOME=”/your/home”; mvim -d $@; that gives you at least 3 files to play with, so I’ve /tried/ to be a bit clever and parse what I think is a file into an array and use it as arguments. It’s not terribly elegant, but it works and I’ve been using it a couple of days.

The only issue is that you’ll have to change the path to your home directory in the script, because I don’t know of a way to detect it. If there is one, please let me know!

Download the attached zip, chmod +x and put it in your ~/Library/Application Support/Versions/Compare Scripts/ folder, restart Versions, then you should see it as an option in your File Comparison dropdown.

To use this with git, I kept the file the same, and then git config –global diff.tool /Users/Me/Library/Application Support/Versions/Compare Scripts/mvimd.sh and it works great with that too. Go me.

bash git macvim script everything svn versions vim

code

an open letter to Sony

Dear Sony,

if ((int) $month === 2){
    if ($year % 4 === 0) $days_in_month = 29;
    else $days_in_month = 28;
}

apocalyPS3 date february time

code

steal buffalo’s planner slider in mootools and css

On BuiltByBuffalo‘s Proposal Planner, we use a much-revered slider thingy that we’ve had a lot of compliments on. We recently received an email asking for a little tutorial on how to put this all together. The main brief is actually pretty esoteric, and you probably won’t be working with designers of the calibre of Jason Reynolds, but this should show you how to put such a thing together.

First off, you need 3 elements. I’ve gone for divs because they’re fairly non-descript and semantically pretty viable for use here. I’ve gone for the following structure:

<div id="scale" class="m1" rel="6">
  <div id="full">
    <div id="marker"></div>
  </div>
</div>

The rel on this div refers to the maximum number of positions on the slider. Unless you shamelessly steal all my source images, you might need to change this! The class refers to a CSS class that denotes the width of the guage element (#full), defined thus:

#scale.m1 #full {
  width: 43px;
}

Because the marker element (#marker) is floated right, it will always stay to the right side of the guage div, so there’s no need for any more CSS than this.

Now that we have our layout and have defined our images (I won’t insult you by teaching you how to set a background image. If you really don’t know that, the declarations are in the source!), we need a method of getting between them. In the actual planner, we do a bunch of ajax-y stuff between stages that’s not relevant here, so here’s a simple breakdown. We’ve got 2 links, next and previous. We, therefore, need 2 event methods to handle input.

In our Marker class, the initialize method deals with identifying the container, marker and guage elements so that we don’t need to keep traversing the DOM to get them. It also sets the current state of the marker and gleans the capacity from the rel attribute (even though it shouldn’t really – you can hardcode it if you want!). We then have our next and previous methods, which just check that their respective states are valid (we don’t want the marker trying to go to 0 or 7, so we stop it from happening).

The method do_morph is where all the legwork is done. We construct the class to Fx.Morph to (mootools requires the whole CSS declaration, not just the element) then define some other options for the morph (namely the duration and a method to run when it’s completed). The completed function calculates the background position needed on the marker image so that we display the correct state.

I’m not great at explaining things, but hopefully that’s done a little to make it clearer for the person who asked for this post, and easy to steal for the people who are that way inclined!

Enjoy the demo:

Oh crap, I accidentally deleted the demo. I’ll put it back when I’ve got some time. In the mean-time; check out the real thing

buffalo craptorial css html javascript mootools slider

code