Chomper Stomping jQuery/JavaScript/CSS 3/HTML 5, Java/PHP/Python/ActionScript, Git, Chrome/Firefox Extensions, Wordpress/Game/iPhone App Development and other random techie tidbits I've collected

18Nov/1010

Making Git show post-receive e-mails as an HTML color formatted diff

octocat


I wanted git to send me an e-mail with a color formatted HTML diff view of pushes/commits whenever the remote server received a push. After some digging I found that I could accomplish this using the post-receive email hooks in combination with Pygment's command line tools (using Python).

Prerequisites:

I will assume for this post that you have Git, Python and Pygments installed. It's been months since I installed Pygments, and I honestly can't remember *anything* about installing it, which means it was probably pretty straightforward and easy. You probably can just do this from your shell:

sudo easy_install Pygments

Making your Git repo send colored diff e-mails:

1. Setting up the post-receive hook

a. post-receive

The first thing you will need to do is configure your post-receive hook:

/git/your-repo.git/hooks/post-receive

In the hooks directory (cd /git/your-repo.git/hooks/) there will probably already be a "post-receive.example" file. Just copy it and rename it post-receive.

This file is basically a shell script. It is run after your repo gets a push and is updated. It is passed in 3 arguments: old revision hash, new revision hash, refname

All you really need to put in this file is the following line:

. ~/git-core/contrib/hooks/post-receive-email

What this does is calls another shell script "post-receive-email" and passes along the arguments (I think).

I'm pretty sure you could specify other shell scripts in the same way at this point if you wanted to.

b. post-receive-email

Now make the file you referenced from the post-receive file:

Move to your global hooks directory (the other one you were just in was specific to just that one repository, this one is the global hooks dir):

cd ~/git-core/contrib/hooks/

Create a file called post-receive-email. This is a shell script, but DON'T put ".sh" on the end.

Now, place the following code in that file (or, better yet, here's the file):

I'll try and point out some of the relevant portions of this script:

generate_email_header() (line 198):

This is the header of the e-mail, including the subject as well as the first few lines of the e-mail.

You may notice some variables here. ${emailprefix}, $projectdesc, etc. I'll talk a little about these later on.

Basically, what you need to know here is that if you want to change the e-mail subject or the beginning content of the e-mail, you do that here. You *could* hard code the recipients here, but don't. We'll talk more about recipients later.

generate_email_footer() (line 219):

This is the footer of the e-mail.

Notice that I placed a line here that tells you which files are causing these e-mails to be sent. I highly recommend keeping this line so that whenever you want to change anything about the e-mail, the e-mail itself tells you everything you need to know about doing so.

show_new_revisions() (line 605):

This is triggered if someone pushes up new revisions that have not ever been pushed up before. This way each code change will only be shown in one e-mail ever. This is the part that generates the actual diff itself and this is probably the single more important part oft his blog post. Lines 635 through 638:

git show -C --pretty=format:"committer: %cn%ncomments: %N%n%s%n%b" $onerev > gitlog.txt
export PYTHONPATH='/usr/local/lib/python2.4/site-packages'
pygmentize -O noclasses=True -f html -l diff gitlog.txt
pygmentize -O noclasses=True -f html -l diff -o gitlogemail.html gitlog.txt

Let's break it down line by line:

1. This is the actual git diff. If you just plop this down in your command line right now (minus the > gitlog.txt part) and replace $onerev with a sha1 hash from your log, you'll see a "preview" of how your e-mail will theoretically look. Here's some code for you to try to see what I'm talking about, do the following in your shell:

git log

Copy one of the sha1 hashes shown the log, then...

git show -C --pretty=format:"committer: %cn%ncomments: %N%n%s%n%b"

(so, for example, and don't try using this exact line because you won't have my sha1 hash in your repo so it won't work):

git show -C --pretty=format:"committer: %cn%ncomments: %N%n%s%n%b" 583038808efbde6fef4eb2e92dd2a920ba714eed

(hit "q" to get out of the diff view this throws you into)

If you don't like anything about this diff view, you can change it by editing the format:"...." part. See the git-show page in the git manual for details.

LOGBEGIN (line 656):

This is a constant (it's counterpart is LOGEND on the next line). The contents here will be used in the script in various places. It's content is placed before the diff to help separate the diff from the rest of the e-mail.

recipients (line 676):

You *could* hard-code e-mail recipients here, but DON'T, we'll once again talk about this in just a few moments...

emailprefix (line679):

emailprefix=$(git config hooks.emailprefix || echo '[SCM] ')

This specifies the text you want showing up right before the subject of the e-mail. It pulls in a setting from your gitconfig (again, we'll talk about this in a moment) but it also specifies a default in case you don't have one set. I make all my e-mails subject lines prepend with [GIT] so I know it's an automated git e-mail at a glance.

custom_showrev (line 680):

This line currently doesn't do what it looks like it does. At first line it appears that you can set your diff settings here, but this is actually unused at the moment. I was trying to make it so you could set your git-show settings in the git config, but I was having trouble getting it to work and gave up because I had bigger better fish to fry... So, don't be fooled into thinking you can change this line and have your e-mails be effected.

2. Setting your git config options

In the post-receive-email file, there were several variables that I kept saying I would talk about later. Now's the time!

We are going to set the variables in your git-config. I'm not 100% sure, but I suspect that this will allow you to have different settings per repository.

The way you do this is:

Move to your repository's home directory:

cd /git/your-repo.git/hooks/

At this point, you can dive right in and make new config settings, or you can list current settings. List current settings like this:

git config --list

To add new config settings:

git config --add blah.blah true

or

git config --add blah.blah "that is soo true dude"

or, better yet, just pop open the .git/config file in a text editor (git config -e) and add lines like this:


[blah]
blah = that is soo true dude

hooks.mailinglist

Whatever you put here will go in the "To:" field. (This is who the e-mail will be sent to).

git config --add hooks.mailinglist "ralphie@gmail.com, frank@gmail.com, steve@gmail.com"

Don't be fooled by the similarly named "hooks.announcelist" which, as far as I can tell, does nothing (theres a few lines of code that may use this if it isn't empty, but I haven't looked into this very far. Don't set it and it won't negatively effect anything. If you do set it, you may get unexpected results).

hooks.emailprefix

This is the text that will be inserted before the subject line. I like to set mine to [GIT] so I know it's an auto generated git e-mail by just glancing at the subject. If you don't set it, it defaults to [SCM].

git config --add hooks.emailprefix=[GIT]

Testing

Ok, at this point, you should be all good to go. Make a code changes, commit, push to your remote and sit back and wait on the e-mail.

Problems? I probably won't be able to help, but I'll try. Better to try asking on StackOverflow.com.

Good luck!

4Aug/100

POOF – Recursive Directory Listing in Python

I've begun working on a new project, POOF (Project Orphaned Object Finder). It searches through a directory and all sub-directories to detect files that are not referenced by any other files, or not referenced by a file you target or any of that file's targets etc. Basically, looking for unused or orphaned resources in a project of any language (but JS/HTML/CSS to start).

Tonight I worked on getting the directories listing recursively.

I'm sure there's some sort of built-in function or UNIX function I could call or something like that, but half of my goal with this project is learning Python, so, yeah. I wanted to do this one by hand.

General overview of my approach:

I made a function "listDirectories" which you pass three arguments to; directory, tabStops, path.

The directory is the directory you want to go to.
The tabStops is how many tabStops you want to display before the directory listing.
The path is the path you are coming from.

The function calls itself each time it encounters a subdirectory. You start the whole thing off with a hard-coded seed. The next step is to un-hard-code this seed so you can pass the seed values in from the command line, or just call the program from your current directory (which is how you will kick off the whole thing when this project is complete). Something like:

POOF.py -d=directory
or
POOF.py
or
POOF.py -d=directory -t=\t\t -p=C:\Projects\POOF

This will of course need to be ironed out because that third one looks UGLY.

Here's the current iteration of the code. Keep in mind this is my first real Python program:

15Sep/0812

SPE (Python IDE) for Windows


I've been playing with Python this weekend. Currently I'm rebuilding "Clink" in Python instead of finishing it in Flash (It really needs AS 3.0 to do properly, and I only have access to 2.0).

The book I am reading (Game Programming: The L Line, The Express Line to Learning) recommeds the SPE (Stani's Python Editor) IDE for creating Python programs. However, the book doesn't really say how to get the program. Stani (the IDE's author) doesn't appear to have a web page with a "download" link and instructions; so here's a download link with instructions from me instead (this is for installing on Windows):

 

Step 1: Download & Install wxPython (NOT wxWidgets!!!!)

The book states that you should download and install wxWidgets. It turns out, you need wxPython instead, which you can get here:

http://www.wxpython.org/download.php

Scroll down and get the version that matches your version of Python. Just double click the downloaded file, and it basically installs itself. I won't give you any advice on what boxes to check at the end of the install, because I don't remember what they said, and to be honest, I arbitrarily made my choices having no clue as to why I was choosing what I chose (I unchecked the top one, and left the bottom two checked).

Step 2: Download and Install SPE

Go to http://developer.berlios.de/project/showfiles.php?group_id=4161

Get this file: SPE-0.8.4.c-wx2.6.1.0-no_setup.zip

Unzip it to somewhere you will remember (I put mine in Program Files) but don't rename the folder anything other than "_spe".

Go inside the _spe directory you just created and double click SPE.py

At this point your shiny new Python IDE should start up. If not, sorry... I don't know why it didn't work. Make a shortcut to the editor on your desktop for easy opening (or start menu, or quicklaunch toolbar).

Have fun!

13Sep/0822

Installing Python (mod_python) on XAMPP (on Windows)

So you want to get Python working with XAMPP eh? Me too! So, apparently, do a lot of people; and pretty much no one who knows how is saying how from what I've found online in my cursory 5 minute search. Let's just jump right in, shall we?

Resources:

Step 0: Install XAMPP & Python

You should have installed XAMPP already, but if you are a brand new "b" to the Server Side Dev ring, and you have stumbled upon this post looking for the catch all solution to getting your feet wet with Server Side Programming and have picked Python as your poison, go get XAMPP! It basically installs and configures itself, and there is an excellent tutorial on the apachefriends website to get you through this process:

http://www.apachefriends.org/en/xampp-windows.html#641

You might not have installed Python already (although, if not, why did you choose Python over PHP or Ruby?). If you haven't installed it already, go download and install it now:

http://www.python.org/download/

Step 1: Make a Backup

If you are like me, and have a few years worth of stuff in your XAMPP directory that you don't want to 'splode when you screw up the first time you try to get Python working with XAMPP, go ahead and just zip c:\xampp to x:\xampp-backup.zip now...

Step 2: Download mod_python

Download mod_python now...

http://httpd.apache.org/modules/python-download.cgi

Wait, which one do I download?

Good question! Check your version of Apache and Python to determine which one you need

  1. Enter the text (leave out the space between phpinfo and ()):
    <? phpinfo (); ?>
    into a new text document in notepad.
  2. Save the document as "test.php" (or something) in the htdocs directory of your xampp install.
  3. Start Apache (open the xampp control panel, c:\xampp\xampp-control.exe, and click "Start" next to Apache).
    NOTE: If you have IIS running, stop it. (Control Panel, Administrative Tools, Services, IIS Admin, Stop (say yes to stopping those other services too). If you scroll up to "Apache" at this point and click on it, you will see which version you are running and can skip to the next step in the mod_python install).
  4. Browse to the document you saved in step 2 through your web browser (localhost\test.php). You can NOT just double click the file and expect it to be parsed by apache and run...
  5. Scroll down and view the output in the "apache2handler" section. The first box in the table should be "Apache Version" and should tell you which version of Apache you are running (mine is 2.2.4, so I downloaded the very last ".exe" file so that I would have Python 2.5 w/ Apache 2.2)

Step 3: install mod_python

  1. Double click the exe file
  2. Click "Next"
  3. It should find your Python install. If you have more than one version, select the one you want to use. Click "Next"
  4. Click "Next"
  5. There should be a pop-up asking where Apache is installed. If you installed xampp to c:\, then this will be c:\xampp\apache

Step 4: Configure Apache

  1. Open c:\xampp\apache\conf\httpd.conf
  2. Scroll down to the section with all the "LoadModule" lines (about line #67) and add this line:
    LoadModule python_module modules/mod_python.so

    Note: If you are unable to start Apache after this, go back and type the line in the conf file by hand instead of copying/pasting.

  3. Scroll down to the section with the <Directory "/xampp/htdocs"> tag ends (about line #232) and add these lines following the closing </Directory> tag:
    <Directory "/xampp/htdocs/test">
            AddHandler mod_python .py
            PythonHandler mptest
            PythonDebug On
    </Directory>
  4. Save and close the file.
  5. Restart Apache
  6. Create a new file (and folder) c:\xampp\htdocs\test\mptest.py
  7. Place the following text in the file, and save it (watch your formatting, Python is whitespace sensitive!):
    from mod_python import apache
    
    def handler(req):
    	req.content_type = 'text/plain'
    	req.write("Hello World!")
    	return apache.OK

Step 5: Testing Py!

Point your browser to http://localhost/test/mptest.py

You should see "Hello World!". If you don't, check this page out (and this one and this one). Also, make sure you got the path correct in the <Directory /> tag, and saved your mptest.py file to the right place.

What's next? Why not get a Python book? This one is really good:
Game Programming: The L Line, The Express Line to Learning (The L Line: The Express Line To Learning). That's right! Game programming in Python.

Or, why not check out some tutorials, starting with this one! (and check out this presentation on django)

Edit:

Some user comments:

<Directory /test> ... </Directory> <-- This is wrong!

But, the following works:

<Directory "C:/xampp/htdocs/test"> ... </Directory>

instead on adding the directory block, which turns on mod-python for a single folder, add the following at the end of the file (assumes you want to use the Publisher handler):

<IfModule python_module>
AddHandler mod_python .py
PythonHandler mod_python.publisher
PythonDebug On
</IfModule>

If you have any other comments that require code. Just send me an e-mail (cmcculloh-at-gmail-dot-com) and I'll post the code (so you don't have to worry about it getting mangled).