Today both courses will be officially over. In fact, both the final exam of AI and final review questions of ML are due tonight. It’s been 10 weeks of thrill, and I’ve enjoyed both courses tremendously. The dedication, generosity, and enthusiasm displayed by all the three professors has been inspiring, and from here I want to give my most heartfelt thanks to everyone involved in making the courses a reality.

Concretely (-grin-), I want to express the utmost respect for the professors involved. I tip my hat to you, Sirs. To anyone interested in knowing about the subject, I strongly recommend taking a look at the links provided for the courses. In fact, the Machine Learning course is scheduled to be repeated in January, and I have no doubt that the Artificial Intelligence one will also have some form of continuation eventually. In my opinion, this  Stanford School of Engineering Initiative makes the world a better place. In particular, the final session of Office Hours has made me realize that I’ve come to feel fully involved with the courses and instructors. Really, quite moving. I did have some good professors back in the old days, both in Med School and Computer Science, but rarely have I grown so fond of  an instructor. I’d really love the chance to meet them face to face simply to thank them in person.

I’ve learnt a lot of stuff, I’ve had a lot of fun, and I’m already enrolled in several of  the new courses that will start in 2012. There’s a lot to choose from: Natural Language Processing, Probabilistic Graphical Models, and a long list of endless instructional joy. Additionally, a bunch of earlier (and equally interesting) courses are available through Stanford Engineering Everywhere and MIT Open Courseware. If you feel like learning new stuff, I wholeheartedly recommend them. If the experience is anywhere similar to the one I’ve had, you’ll be amazed!

UPDATE: I sent an appreciation mail to each of the professors involved, and it turns out professor Norvig actually took the time to reply. Nothing short of astonishing given the huge amount of such emails they must have received. As I said, amazing experience, amazing courses, and amazing professors. Totally recommended.

It has been a while since I last announced one of our releases, but it has been a long time since the last one, so this one deserves some credit. We’ve been really busy implementing a lot of stuff that is really going to make life easier for all the guys out there that are using web servers. We are not done yet, and this one is a maintenance release that doesn’t really show most of what we’ve been doing. It doesn’t make sense to do so until we’re ready. Nevertheless, it adds lots and lots of improvements, and some bug fixes.

As always, a lot of development effort is being invested in our flag product, and this is something that doesn’t go by unnoticed. And just in case you haven’t noticed, take a look at  Cherokee 1.0.9 (and be amazed) ;-)

I hope you enjoy it. Feedback and feature requests are more than welcome at the mailing lists. Here are links to download the tarball and the online documentation.

I have named it episort. Not very original, but it has been a lifesaver so far.
It is much faster and accurate than before, by the way.

The problem:

I have a directory full of unclassified media files, some are duplicates, some aren’t, and each one follows a different naming convention.

I even try to classify them from time to time, so you can throw some directories into the pack. Sometimes, I even create two or three directories for the same group-series-category-whatever before I realize there is an existing one with a slightly different name. And frequently a lot of files remain unclassified, many of which could fit into one of the directories I mentioned.

Of course … whenever a new file arrives to my home server, it gets thrown into that very same directory, so Chaos keeps spreading, as it always does.

To clarify things, lets show an example:

drwxrwxrwx 1 user user      4096 2010-07-03 11:50 01_Battlestar_Enterprise
drwxrwxrwx 1 user user      4096 2010-07-03 11:42 02_Startrek_Galactica
drwxrwxrwx 1 user user      4096 2010-07-03 11:50 03_battlestar.enterprise-season.1
-rwxrwxrwx 1 user user 220393472 2010-07-03 02:49 battlestar.enterprise.s1e01.avi
-rwxrwxrwx 1 user user 221227008 2010-07-03 02:50 Battlestar_Enterprise_1_22.mp4
-rwxrwxrwx 1 user user 195393472 2010-07-03 02:49 startrek.galactica.4x15.[ripper_22].mkv

As you can imagine, sorting things up can get really tedious, and there is no automatic way of doing it that I know of.

I had some time this morning and got fed up with it. Every little piece of help is more than welcome, and here is where Python comes to the rescue.

The solution:

There are dozens of ways to do this, but I ended up coding a quick hack to help me sort things out.
It just compares the names of files and directories, and estimates the similarities. Anything above a 50% match is usually correctly estimated.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

# (C) 2010, Taher Shihadeh
# Licensed: GPL v2

"""
The script works based only on names of files and directories in a
non-recursive manner.

It takes a path as parameter and tries to determine if the names of
the contents look alike.

It removes separator characters, numbers and file extensions prior to
the comparison.
"""

import os
import sys
import string
from operator import itemgetter

FAST = False # Change this to skip file-to-file comparisons
SEP  = '_-+~.·:;·()[]¡!¿?<>'

def main (path):
    lst1    = os.listdir (path)
    lst2    = lst1
    len_lst = len(lst1)
    count   = 0.0
    results = []

    for x in lst1:
        for y in lst2:
            if x==y:
                continue
            x_dir = os.path.isdir(x)
            y_dir = os.path.isdir(y)

            if FAST and not (x_dir or y_dir):
                continue

            result = {'A': (x, x_dir), 'B': (y, y_dir)}

            str1, str2 = x, y
            if not x_dir:
                str1,_ = os.path.splitext (x)
            if not y_dir:
                str2,_ = os.path.splitext (y)

            result['factor'] = compare (str1,str2)
            results.append(result)

        lst2.remove(x)
        count += 1
        print >> sys.stderr, '%.2f%% done' %((count / len_lst)*200)

    show(results)

def split (str1):
    trans = string.maketrans(SEP, ' '*len(SEP))
    return str1.translate(trans).split()

def clean (lst):
    assert type(lst) == list
    return filter(lambda x: not x.isdigit(), lst)

def compare (str1, str2):
    """Return similarity factor as percentage"""
    aux1 = clean (split (str1.lower()))
    aux2 = clean (split (str2.lower()))

    set_or  = set(aux1) | set(aux2)
    set_and = set(aux1) & set(aux2)

    return (float(len(set_and)) / float(len(set_or)))*100

def show (results):
    """Show most similar last"""
    for x in sorted(results, key=itemgetter('factor')):
        a,b = x['A'],x['B']
        if not b[1] and a[1]:
            a,b = b,a
        print '%.2f \t %s \t --> %s' %(x['factor'], a[0], b[0])

if __name__=='__main__':
    try:
        path = sys.argv[1]
    except IndexError:
        path = os.getcwd()

    main (path)

I don’t think any one is going to use it, but what the hell. It’s a big Internet ;-)

Everything was recorded, so we will upload the slides and videos of all our sessions really soon. For now, only the photo gallery is available. Take a look at the mugshots.

I’m really glad we could make this Summit. It surpased all my expectations. By far. It was an unbelievable experience, and we had lots of fun. Take a look at our family photo. If you want to know which of the guys above is me, here’s a clue: “In brightest day…”.

I’m really looking forward to the next summit. Cherokee Summit 2010 was awesome. I’m sure the next one will be even better.

I’m sure that meeting many of the developers of Cherokee in person will be the highlight for me.

If by any chance you’ll be in Madrid that weekend, don’t forget to register in time and join us.

It’s gonna be legen… wait for it… dary!

You can read Alvaro’s announcement, or you can check out the Summit web-site.

Cherokee will be an important topic, but it won’t be the only one. Those will be a couple of days fully dedicated to High Performance and Scalable Web topics, so there’s room for everyone to join in.  We are commited to reaching the 1.0 milestone of Cherokee by then, so we will also have a party to celebrate it.

It’s going to be fun. I’ll be a speaker at the summit and I’m really looking forward to personally meeting many of the members of the project. Thanks to our sponsors we’ve managed to make the event completely free, so don’t forget to register while we still have free spots!

UPDATE: We’ve written a little brochure (~100KB) that can be used to  let your colleagues know about the summit. Do not hesitate to send it to any coworker or friend who would be interested in attending a High Performance and Scalable Web event.

From here I’d like to thank P.V. Anthony for his invaluable advice on audio production and my old friend Sara Genge for lending her voice to the project (and for her awesome fiction writing, but that is another story).

We wanted to brag about our little baby. After all, not every serious web server out there has a killer interface to configure it. Take a look at our Cherokee Web Server introductory screen-cast.

You might want to see it at full screen for readability.

It’s just one of many to come. We’ve got some more planned, so I’ll let you know when they’re ready.

If you expect a party kit from us you’ve come to the wrong place. We actually believe our software is so good that it is a prize for its own merits. It has been a while since I last announced one of our releases, mainly because I didn’t have much to add besides what was told on the official announcements. As always, a lot of development effort is being invested in our flag product, and this is something that doesn’t go by unnoticed. This weekend we decided to release Cherokee 0.99.25. As you can tell by the .25 part, lots of fixes and enhancements have been added steadily release after release.

I hope you enjoy it. We’ve tried to update all the documentation for this release, and we’ve automated most of the recipes in our cookbook by adding lots and lots of configuration Wizards, so hopefully you’ll be able to set up anything in a matter of seconds. As always feedback and feature requests are more than welcome at the mailing lists. Here are links to download the tarball and the online documentation.