Friday 1 May 2009

Automatic mp3 genre tagging: is it possible?

Genres are a tricky business and crawling the web has shown that listeners either love or hate them. Right now, I haven't formed a final opinion on whether I like them or not and if they're worthwhile, but I haven't fully committed to using them on my collection yet.

However, I have done a fair bit of research on the matter, and that's what I want to lay out here as it may save some of you a lot of time to get to where I am (still without a solution, but a lot wiser nonetheless). Apologies for the length of this particular diatribe, but it can't be made any more succinct.

My previous posts on mp3 tagging have focused on processes, software and automated ways of beating mp3s and tags into shape. After processing my entire collection, I sat back and saw that two things that were still missing:

  • Consistent genres
  • Ratings
I am not dealing with ratings here - there's a whole post in itself.

I wanted to get my genres into shape, not because of my crusade against badly tagged mp3s, but because they have the potential to help hugely in coordinating my listening and building playlists of similar types of music. This would be a necessary step in reaching my mp3 nirvana.

As a first foray into genre tags, I thought I would start by looking for an automatic method for populating the field. It made sense to me that with all the tagging software and mp3 databases out there, there should be a successful solution. Not so. However, here's what I did find...

POTENTIAL GENRE SOURCES

There are a number of good databases out there that genre population could come from. I thought these ones could do the trick if there were a way of getting the data into an ID3 tag:

I did a test to see which was best, looking at the legendary Led Zeppelin as a case study. I thought there was sufficient ambiguity in how to classify their music to generate quite a varied set of results. And I like them quite a lot. And I'm English.

So following the order of my bullet points above, I set about going through each of these sources starting with the old CDDB, Gracenote. This one did not help me much. I couldn't find a way of seeing the genre, even when I chose an album (Led Zeppelin I):

Hence, I decided this was not a great database for genres. I guess this is fine since it is mainly designed as a database of track info, not a method for organising individual music libraries, so I'll let it off.

Freedb wasn't a hell of a lot better than Gracenote. I found that I could only get a genre returned by disc, not by artist or track:

Also, it was not consistent between different instances of the same disc. Under the same album (Led Zep I), I got rock, blues, misc and even reggae! The only reggae-ish track I can think that the Zep did was D'yer Maker - not on this album. So, not too impressed here, but again, this is more a database of track info rather than for structuring your music in any way.

Discogs is a nice site, but once again, I could only get results returned by disc rather than artist or track. However, I noted that it was split into first Genre and then Style:

I was pretty happy with the simple set of results it returned as well - Rock as a genre, then split into Styles of Classic Rock and Blues Rock. Not bad.

I was sure the iTunes store wouldn't disappoint too much, and sure enough it did an alright job, returning quite a few applicable genres:

However, the more genres that get returned, you start thinking "well, which one do I choose for my tracks". This was starting to look like a very complicated idea, hence why no one has put together a bulletproof method or database yet.

Next was the mighty Last.fm. On first impressions, it looked promising, with 5 applicable results returned as 'tags':

However, on clicking the 'See more…' link, I was shocked to find a massive cloud of tags. It was apparent that this was heavily influenced by all the users tagging tracks and creating a muddled mass of genre information:

'awesome', 'dance', 'favs', 'indie pop', 'punk', 'yeah'? Give me a break…
So, suffice to say, I was not too impressed with Last.fm as a tagging source for genres.

Finally I came to Allmusic, sometimes known as AMG (All Music Guide). This was quite impressive. Firstly, artists are broken down into a Genre and then subdivided into Styles, just like Discogs had gone for. Good and applicable ones too:

Allmusic went one step further, though and put together 'Moods' and 'Themes' as well. Although there were a lot of moods, I began to think that these could well be useful in generating playlists. These are the moods and themes for the Led Zep I album:

Furthermore, just take a look at this link to see the pretty comprehensive breakdown of their Pop/Rock styles.

I thought that I could use the Genre, Styles and Moods in my ID3 tags as follows:

  • AMG Genre maps to ID3 Genre (there is only one)
  • AMG Styles can go into the the ID3 Comments, with underscores instead of spaces and separated by a space or a period (I am currently using Comments for artist bios, but I have realised I never look at these)
  • AMG Moods can go into the ID3 Custom 1 field in MediaMonkey for the moment
Therefore, I think I might have found a winner in Allmusic. Now If only I could find a way to get this data into my mp3 tags without any huge manual effort…

AUTOMATIC GENRE POPULATION

I searched for quite some time on this, so I think I can safely say without any doubt that there is no way to automatically populate your mp3 genre tags directly from Allmusic.

It seems as there used to be a solution called the Allmusic ID3 Tag Fixer by KRKeegan, but it seems as though Allmusic changed their terms of service back in 2007 which has prevented anyone from using the data. Humbug!

I also found that the well respected Mp3tag used to have Allmusic as one of its web sources framework (meaning you could select it as a tagging source), but with the terms of service this stopped working in 2007 as well.

Lastly, I found a post about a script that a chap called Steegy had written for MediaMonkey called Web Sources Tagger that could access Allmusic which has since disappeared.

How I wish I had a time machine…

So my plans were scuppered at the first hurdle. So I looked at what else was available. I found that Mp3tag could use amazon, discogs and freedb as web sources, but having seen what Allmusic had to offer, I decided not to proceed as I would always be left wanting.

I went a bit further and found another Media Monkey Script by Trixmoto called 'Last.fm genre script' and gave it a shot for a laugh. It did exactly what it was meant to, but if you cast your eyes up to the Last.fm results above, you can imagine the mess my genres were left in. The idea of this was to get something consistent going.

I think I ended up with many, many more genres than I had originally - 1047 in all, including such gems as...

  • Brutal Death Metal With Nordic Influence And Guest Vocalist Satan
  • Candy Ravers Rules The Universe
  • Existential Saturdays
  • Have Other Music by This Artist
  • I-feel-oh-so-classy-music
  • Jangly American Guitar Pop
  • Plunderphonics
  • Sing Yer Wee Heart Out
…and those are just some of the clean ones!

A great script, though - if only Trixmoto could apply it to Allmusi. Unfortunately it seems as though our hands are tied in terms of access to the data. My collection remains tagged like this, but I'm not bothered as I don't use the genres yet anyway - I need a solution.

So, still no success, and my dream of automatically tagging my genres was quickly fading [if I have missed anything, I would love to know about it, so please comment with your feedback below].

Hence, I set about seeing if anyone had any guidelines on how to do it manually without too much bother...

MANUAL GENRE POPULATION

I went trawling the Internet again to see what I could drag up. The most useful thing I found was a great post by Daniel R Stout called 'Organizing iTunes: Simplify your genre list'.

I won't repeat everything that Daniel says here - you can read his blog - but as a synopsis here are the main points:

  • Less genres are better
  • Have a basic list to choose from
  • Use smart playslists to filer music on genre to sturcture your listening
He also suggests a quick way of manually getting your genres into shape:

"It’s an easy thing to change the genre on many songs. Create a playlist that sorts by existing genre data. You can find out what genres are in your music library by doing a Get Info on any song and then clicking the dropdown menu for the genre of that song. So create a playlist that grabs some genres that fit under, say, Alternative & Punk. Then highlight a bunch of songs together and do a Get Info and change the genre for all of those songs. In my case, it took less than an hour to get my 20,000 song library into shape."

He then provides a list of 25 basic genres to start with. Which he further breaks down into sub-genres, if needed. This does end up as one long list which can be a little difficult to digest, so I have created a handy table in Excel that you can find here. Apologies for the Excel pastel colours.

This should be a useful lookup tool for anyone wanting to go ahead and manually tag their genres. I may go ahead and adopt this method of genre tagging, but I think it would take me longer than the hour suggested for 20,000 tracks. I just need to sum up the courage.

I would also like to credit the commenters (Lucas, amazon_blonde and Nick) for this post of mine on organising your iTunes library. There are some useful insights and another list of genres from Lucas that may suit you better if you are going the manual route.

THE FINAL ANALYSIS

As I think I have shown, there are no bulletproof ways to automatically tag your genres consistently. At least not yet.

Also, genres are a very personal thing - one person's 'Prog Rock' is another person's 'Cheesy Listening'. Hence it may be best to just take the plunge and manually tag them with something bespoke that simply works for you. The problem is that this takes time; and time is a resource many of us are not blessed with.

Another point to consider is how much detail do you put into it? Or, to coin a phrase, what is your 'depth of genre'? Is it best to have 25 basic all-encompassing genres, or to go to town like the Allmusic system and have a genre broken down into multiple styles and moods?

You can also argue over whether individual tracks should have different genres, or if genres should be grouped by album or artist. I think that single track genres would be best if you are going the detailed route, but artist grouped genres would be superior for a 25 genre system.

Lastly, one might put it out there that genre tagging is a complete waste of time and will only bring me closer to my own 'Stairway to Heaven'.

Personally, I think this post is using too may of my little grey cells and I'm going to sit on it for a while and see what debate it generates. Over to you...