Automatic mp3 genre tagging: is it possible?

Genres are a tricky business and crawling the web has shown that listeners either love or hate them. Right now, I haven't formed a final opinion on whether I like them or not and if they're worthwhile, but I haven't fully committed to using them on my collection yet.

However, I have done a fair bit of research on the matter, and that's what I want to lay out here as it may save some of you a lot of time to get to where I am (still without a solution, but a lot wiser nonetheless). Apologies for the length of this particular diatribe, but it can't be made any more succinct.

My previous posts on mp3 tagging have focused on processes, software and automated ways of beating mp3s and tags into shape. After processing my entire collection, I sat back and saw that two things that were still missing:

• Consistent genres
• Ratings

I am not dealing with ratings here - there's a whole post in itself.

I wanted to get my genres into shape, not because of my crusade against badly tagged mp3s, but because they have the potential to help hugely in coordinating my listening and building playlists of similar types of music. This would be a necessary step in reaching my mp3 nirvana.

As a first foray into genre tags, I thought I would start by looking for an automatic method for populating the field. It made sense to me that with all the tagging software and mp3 databases out there, there should be a successful solution. Not so. However, here's what I did find...

POTENTIAL GENRE SOURCES

There are a number of good databases out there that genre population could come from. I thought these ones could do the trick if there were a way of getting the data into an ID3 tag:

• Gracenote (formerly CDDB)
• Freedb
• Discogs
• iTunes store
• Last.fm
• Allmusic

I did a test to see which was best, looking at the legendary Led Zeppelin as a case study. I thought there was sufficient ambiguity in how to classify their music to generate quite a varied set of results. And I like them quite a lot. And I'm English.

So following the order of my bullet points above, I set about going through each of these sources starting with the old CDDB, Gracenote. This one did not help me much. I couldn't find a way of seeing the genre, even when I chose an album (Led Zeppelin I):

Hence, I decided this was not a great database for genres. I guess this is fine since it is mainly designed as a database of track info, not a method for organising individual music libraries, so I'll let it off.

Freedb wasn't a hell of a lot better than Gracenote. I found that I could only get a genre returned by disc, not by artist or track:

Also, it was not consistent between different instances of the same disc. Under the same album (Led Zep I), I got rock, blues, misc and even reggae! The only reggae-ish track I can think that the Zep did was D'yer Maker - not on this album. So, not too impressed here, but again, this is more a database of track info rather than for structuring your music in any way.

Discogs is a nice site, but once again, I could only get results returned by disc rather than artist or track. However, I noted that it was split into first Genre and then Style:

I was pretty happy with the simple set of results it returned as well - Rock as a genre, then split into Styles of Classic Rock and Blues Rock. Not bad.

I was sure the iTunes store wouldn't disappoint too much, and sure enough it did an alright job, returning quite a few applicable genres:

However, the more genres that get returned, you start thinking "well, which one do I choose for my tracks". This was starting to look like a very complicated idea, hence why no one has put together a bulletproof method or database yet.

Next was the mighty Last.fm. On first impressions, it looked promising, with 5 applicable results returned as 'tags':

However, on clicking the 'See more…' link, I was shocked to find a massive cloud of tags. It was apparent that this was heavily influenced by all the users tagging tracks and creating a muddled mass of genre information:

'awesome', 'dance', 'favs', 'indie pop', 'punk', 'yeah'? Give me a break…
So, suffice to say, I was not too impressed with Last.fm as a tagging source for genres.

Finally I came to Allmusic, sometimes known as AMG (All Music Guide). This was quite impressive. Firstly, artists are broken down into a Genre and then subdivided into Styles, just like Discogs had gone for. Good and applicable ones too:

Allmusic went one step further, though and put together 'Moods' and 'Themes' as well. Although there were a lot of moods, I began to think that these could well be useful in generating playlists. These are the moods and themes for the Led Zep I album:

Furthermore, just take a look at this link to see the pretty comprehensive breakdown of their Pop/Rock styles.

I thought that I could use the Genre, Styles and Moods in my ID3 tags as follows:

• AMG Genre maps to ID3 Genre (there is only one)
• AMG Styles can go into the the ID3 Comments, with underscores instead of spaces and separated by a space or a period (I am currently using Comments for artist bios, but I have realised I never look at these)
• AMG Moods can go into the ID3 Custom 1 field in MediaMonkey for the moment

Therefore, I think I might have found a winner in Allmusic. Now If only I could find a way to get this data into my mp3 tags without any huge manual effort…

AUTOMATIC GENRE POPULATION

I searched for quite some time on this, so I think I can safely say without any doubt that there is no way to automatically populate your mp3 genre tags directly from Allmusic.

It seems as there used to be a solution called the Allmusic ID3 Tag Fixer by KRKeegan, but it seems as though Allmusic changed their terms of service back in 2007 which has prevented anyone from using the data. Humbug!

I also found that the well respected Mp3tag used to have Allmusic as one of its web sources framework (meaning you could select it as a tagging source), but with the terms of service this stopped working in 2007 as well.

Lastly, I found a post about a script that a chap called Steegy had written for MediaMonkey called Web Sources Tagger that could access Allmusic which has since disappeared.

How I wish I had a time machine…

So my plans were scuppered at the first hurdle. So I looked at what else was available. I found that Mp3tag could use amazon, discogs and freedb as web sources, but having seen what Allmusic had to offer, I decided not to proceed as I would always be left wanting.

I went a bit further and found another Media Monkey Script by Trixmoto called 'Last.fm genre script' and gave it a shot for a laugh. It did exactly what it was meant to, but if you cast your eyes up to the Last.fm results above, you can imagine the mess my genres were left in. The idea of this was to get something consistent going.

I think I ended up with many, many more genres than I had originally - 1047 in all, including such gems as...

• Brutal Death Metal With Nordic Influence And Guest Vocalist Satan
• Candy Ravers Rules The Universe
• Existential Saturdays
• Have Other Music by This Artist
• I-feel-oh-so-classy-music
• Jangly American Guitar Pop
• Plunderphonics
• Sing Yer Wee Heart Out

…and those are just some of the clean ones!

A great script, though - if only Trixmoto could apply it to Allmusi. Unfortunately it seems as though our hands are tied in terms of access to the data. My collection remains tagged like this, but I'm not bothered as I don't use the genres yet anyway - I need a solution.

So, still no success, and my dream of automatically tagging my genres was quickly fading [if I have missed anything, I would love to know about it, so please comment with your feedback below].

Hence, I set about seeing if anyone had any guidelines on how to do it manually without too much bother...

MANUAL GENRE POPULATION

I went trawling the Internet again to see what I could drag up. The most useful thing I found was a great post by Daniel R Stout called 'Organizing iTunes: Simplify your genre list'.

I won't repeat everything that Daniel says here - you can read his blog - but as a synopsis here are the main points:

• Less genres are better
• Have a basic list to choose from
• Use smart playslists to filer music on genre to sturcture your listening
  

He also suggests a quick way of manually getting your genres into shape:

"It’s an easy thing to change the genre on many songs. Create a playlist that sorts by existing genre data. You can find out what genres are in your music library by doing a Get Info on any song and then clicking the dropdown menu for the genre of that song. So create a playlist that grabs some genres that fit under, say, Alternative & Punk. Then highlight a bunch of songs together and do a Get Info and change the genre for all of those songs. In my case, it took less than an hour to get my 20,000 song library into shape."

He then provides a list of 25 basic genres to start with. Which he further breaks down into sub-genres, if needed. This does end up as one long list which can be a little difficult to digest, so I have created a handy table in Excel that you can find here. Apologies for the Excel pastel colours.

This should be a useful lookup tool for anyone wanting to go ahead and manually tag their genres. I may go ahead and adopt this method of genre tagging, but I think it would take me longer than the hour suggested for 20,000 tracks. I just need to sum up the courage.

I would also like to credit the commenters (Lucas, amazon_blonde and Nick) for this post of mine on organising your iTunes library. There are some useful insights and another list of genres from Lucas that may suit you better if you are going the manual route.

THE FINAL ANALYSIS

As I think I have shown, there are no bulletproof ways to automatically tag your genres consistently. At least not yet.

Also, genres are a very personal thing - one person's 'Prog Rock' is another person's 'Cheesy Listening'. Hence it may be best to just take the plunge and manually tag them with something bespoke that simply works for you. The problem is that this takes time; and time is a resource many of us are not blessed with.

Another point to consider is how much detail do you put into it? Or, to coin a phrase, what is your 'depth of genre'? Is it best to have 25 basic all-encompassing genres, or to go to town like the Allmusic system and have a genre broken down into multiple styles and moods?

You can also argue over whether individual tracks should have different genres, or if genres should be grouped by album or artist. I think that single track genres would be best if you are going the detailed route, but artist grouped genres would be superior for a 25 genre system.

Lastly, one might put it out there that genre tagging is a complete waste of time and will only bring me closer to my own 'Stairway to Heaven'.

Personally, I think this post is using too may of my little grey cells and I'm going to sit on it for a while and see what debate it generates. Over to you...

MediaMonkey scripts to make your mp3 tags consistent

Hi there all you Lifehacker visitors! Thanks for popping into the Tippopotamus.
If you enjoy this post, you might also like this one on getting lyrics into iTunes the easy way and this one on how to process new music.
I [will be posting later this week] have now posted on music genre tagging and how it can be used to structure your listening.

I hate it when tags don't match up. For example, depending on where your tag data has come from you could end up with a host of different artist names:

• Guns and Roses
• Guns n Roses
• Guns N Roses
• Guns 'n' Roses
• Guns 'n Roses
• gunsnroses
• GnR
• GnFR

There are a plethora of different things that can be wrong with tags - missing track numbers, no album art, volumes and disc numbers - the list goes on. These bother me, but may not irk you. If you are like me, correcting this sort of mess is something that would require a lot of manual tagging work, since finding the files is difficult and correcting them is a labourious process.

Or so I thought.

MediaMonkey comes with a few built-in functions and scripts that can help you to tame your mp3 collection. However, the real power comes in the scripts nice people have written to solve tagging problems. Here follows a synopsis of what I have found useful…

1. Built-in MediaMonkey Features

Firstly, you can search for missing data very easily. Under Library -> 'Files to Edit'...

...you can find folders that will display all tracks with:

• Unknown Title
• Unknown Artist
• Unknown Album
• Unknown Genre
• Unknown Year
• Unknown Rating
• Unknown Album Art

These are great for finding tracks and albums that need to be tagged. Follow my previous post that shows you how to tag tracks using MediaMonkey to get these all sorted.

2. Built-in MediaMonkey Scripts

Next, MediaMonkey has a couple of built-in scripts that can help with some menial tasks. These are found under Tools -> Scripts.

Auto-increment Track #s

This one takes the tracks you have highlighted in the main window and gives you the opportunity to number the tracks in chronological order, starting at a track number you specify.

This is useful when the tag editor your have used to automatically tag your mp3 album has used a different version (e.g. US to UK) which has bonus tracks missing or tracks in a different order. This script helps sort this out without having to try and find the UK version of the album or re-numbering the files manually.

Swap Artist and Title

I find that automatic tagging sometimes uses data that has the artist and track title inverted. This script simply swaps them around. Believe me, I wish I had found this one earlier as I have wasted enough of my life swapping these manually.

3. User Created MediaMonkey Scripts and Plug-ins

The real powerhouses for getting your collection tagged nicely are these three scripts that MediaMonkey fans with the skills-to-pay-the-bills have written specifically to help out. There are many more out there, but after extensive searching, these are the ones I found to be the best:

'Find Missing The' script by Trixmoto

This will search through all the tracks you have in the main window (I make sure I have my whole collection displayed) and finds where you have artists with 'The' in front in some cases and without in others. It also looks for those that have a comma and The (i.e. ', The') as a suffix. It brings up a confirmation box before you have to accept the changes. This was useful for bands like The Beatles which I had in three different ways ('The Beatles', 'Beatles' and 'Beatles, The').

A word of caution, though - it also looked through the track title fields as well and where a song name matched an artist that needed changing, it also wanted to change the track title (e.g. 'The Jam' as a band and the track 'Jam' by Michael Jackson resulted in the track being renamed 'The Jam'. this required a little bit of remedial work to correct these track names, but a lot less than correcting all the missing 'The's in my collection.

The detailed description can be found here and the download can be found here.

'Case and Leading Zero Fixer plug-in by Bex

This plug-in is hugely useful in getting your upper-case and lower-case use in sync with each other. I used it mainly for its secondary function, namely adding or removing leading zeroes to the track number. It was by far the simplest way to do this and made everything look nicer in iTunes. Once installed, this one is found under Tools -> Options -> Library -> Case & Leading Zero Fixer.

A full guide on its use can be found in the MediaMonkey forums here. The download is here.

'Tagging Inconsistencies' plug-in by Bex

This is the grandaddy of them all and is the one you can't do without. It's a bit too complicated to explain all here, but suffice to say, it does what it says - it fixes tagging inconsistencies. It searches for inconsistencies by Album, Folder, Person, Track and Genre and retuns them for you to correct. The correction must be done manually, but believe me, 99% of this particular job is finding what needs correcting. If you think you have a beautifully tagged collection, run it through this and you will be surprised!

When it has been installed, the 'nodes' it uses to retun the inconsistencies can be found under your tree on the left hand pane of MediaMonkey: Library -> Files to Edit -> Tagging Inconsistencies.

Again, there is no point in me writing a huge guide for this as there is plenty of information on the MediaMonkey forums here. However, I found that the most useful way of learning how to use this was to actually just get into it and start experimenting. The download can be found here.

As a post-note, I would always suggest backing up your collection before you start on a festival of tag editing. You never know if something might go wrong or if you might prefer to have it the way you previously wanted it. I have never had a disaster, but there again, I tried and tested these on a duplicate before I got into using them properly. Good luck!

Everyday tagging with MediaMonkey

MediaMonkey is IMHO the best tag editor out there. It's not necessarily as good looking as iTunes, but for bulk tagging of new tracks, it can't be beaten.

In my introductory post on dealing with new music, I showed the process involved in getting new tracks from the moment you get them to finally having them in iTunes with the rest of your library. This one will deal with the individual steps within MediaMonkey, but assumes you have read that post in order to get to this point.

The tag editing works on an album-by-album basis (or track-by-track for single songs)...

1. Click on the your album folder in the tree on the left hand side

2. In the main pane on the right, highlight all the tracks in the album by using Edit --> Select All or the keyboard shortcut Ctrl-A.

3. Right click on any track and select 'Auto-Tag from Web' on the context menu. You can use the keyboard shortcut Ctrl-L if you prefer.

This brings up a new window.

4. Depending on the popularity of the album you are looking up you may get immediate results. I find I often need to hone the search a little. I do this by typing something more specific about the artist in the search box at the top left of the screen.

Here is an example after trying to tag 'Hotel Costes, Vol.8'. MediaMonkey went straight for the album artist, which returned no results:

By typing in 'Hotel Costes 8' I immediately got some results:

You can choose the best fitting results by looking through the search results with the dropdown function on the search box. In this case, only one result was returned. You can often find better album art and artist commnetary if you browse through a little bit.

5. Make sure you have all the checkboxes ticked to start with. Anything in your track tags that will change is highlighted in bright yellow so that you don't miss it.

6. Do a good check of the data that is going to be changed to make sure you are happy with it. Worth checking here are:

• The right number of tracks
• Are tracks names and artist names interchanged (this sometimes happens with compilations)?
• Are track names and artist names combined in the same field (again a problem with compilations sometimes)?
• Are the track numbers changing significantly?

Most of these can be changed manually after you have been through this step, but if there is another search result that gives a better fit, it will save you time.

7. Press the 'Auto-Tag' button at the bottom right (or press 'Close' if you don't want to accept the tag results).

8. You should now be back in the main MediaMonkey window. Check the results again to see if you need to do any manual correcting. I often do a little bit of tweaking so that the results fit with my naming convention in iTunes (e.g. for Volumes and First Name-Last Name etc.). You can do this by right clicking on individual or multiple tracks if you want to edit information for all the tracks together) and choosing 'Properties' at the bottom.

You are presented with a new window showing all the available fields to populate. Notice the tabs at the top which give more options. Specifically, if you could not get results with album art and you have managed to find a jpeg elsewhere, you can embed it in the tag using the tab called 'Album Art'.

9. At this point, I used to go ahead and rename the files with the 'Auto-Organize Files' context click or the keyboard shortcut Ctrl-R, but I no longer do this as I have iTunes manage my files. It is actually a great relief as I eventually realised I never used the filenames for searching. I only ever used my folders and now I just use iTunes. If you are an mp3 file naming convention junkie, like I was, you can go to town with this function, but I would strongly suggest asking yourself if it is actually worth the time and effort…

That's it - that's all I do to commission files within MediaMonkey. Unfortunately this is where I unceremoniously dump the singing simian. There are some other whole-library tagging adventures I have had with MediaMonkey that I was very impressed with, but I will post about those in the future as they are gargantuan one-off efforts. This should allow you to get quality and consistently tagged tracks feeding into your collection, regardless of the software you use to play your music with.

Dealing with new music

I seem to always have lots of music waiting to be listened to. Whether it's bought or recommended and shared by friends, it always needs to go through my commissioning process before it gets to be in my iTunes library.

This is my tried and tested method for getting new tracks tagged and sorted just the way I want them. What this post does not cover is what I did with the 25,000 tracks I had before I worked this out (which is the real 'odyssey' to be honest). I thought I had best detail the everyday process first.

There are 10 easy steps, as follows:

1. Put all new tracks into one root folder (let's call this 'new' for the purposes of this post), whether they are bought, shared or ripped (sub-folders do not need to be organised in any particular fashion.

2. Examine all the files to see whether they are all in mp3 format or not. My entire music collection is in mp3 (bar audiobooks). This way I should never have any compatibility issues in the future.

3. Convert any files in the 'new' folder without an mp3 extension using dBpoweramp. It is simple and effective. You just need to make sure you match the bitrate. This process is addressed in this post.

4. Delete all the non-mp3 files from the 'new' folder.

5. Open up MediaMonkey and point the directory at the 'new' folder. Choose 'All' so that all the tracks in the folder are shown.

6. Go through each album (or track if you have singles - I rarely do) and tag them automatically with the handy context menu or Ctrl-L keyboard shortcut. A more detailed description can be found here.

I no longer worry about changing the filename to my strict naming convention, since this is now irrelevant as I have allowed iTunes to manage my collection.

7. Close MediaMonkey and open iTunes.

8. Import your nice clean files into iTunes by choosing File --> Add Folder to Library and pointing it to your 'new' folder. This will copy the files from the 'new' folder into the nominated iTunes library location. iTunes will rename the files according to its simple convention.

9. Verify your new tracks are in iTunes with a quick search or by building a smart playlist with today's date as the 'Date Added' field.

10. Delete the corresponding files in your 'new' folder since these have already been duplicated by iTunes.

The useful thing here is that your tagged files will have been imported into iTunes all on the same date. Hence, you can create a simple smart playlist to group all these together into one place which then slowly empties as you listen to them. This way you can effectively commission the music and make sure you've at least listened to it all before it joins the rest of the 25,000 tracks.

An MP3 Odyssey: from MusicMatch to WinAmp to WMP to MediaMonkey to iTunes

I 'acquired' my first mp3 file in early 2000 with the original Napster. Since then my collection has blossomed from a handful of files to an amazing 25,000. Part of this growth involved ripping my entire CD collection in 2003 when I got my first iPod (2nd gen).

Ever since I converted to an 'intangible' music storage system, I have struggled to manage it effectively. This has meant that the last time I listened to a lot of my favourite music was back in 2003 when I first went all mp3.

Being slightly anally retentive, I've never been happy with simply ripping a CD or procuring some music without having it perfectly tagged. I won't get into this yet, but suffice to say, my mp3 collection is (IMHO) very well ordered.

As can be seen by the title of this post, I have tried a number of different pieces of software, tag editors, processes and procedures to tame my huge music collection over the years. I think I'm finally almost there with a user-friendly solution that might just last. On looking back, the goal should never have been about constructing a beautifully ordered folder taxonomy and tagging like I had an OCD. It should have been about the user experience - and not just mine at that. A system that works even for the lay-person is what's needed and that's where I'm at. Almost.

So, first a bit of history; I was initially introduced to ripping music to mp3 using MusicMatch Jukebox. I used this religiously until it was taken over by Yahoo! when it seemed to loose a few of the functions I counted on the most; namely automatic tagging. I played with WinAmp for a short while, and also Windows Media Player. WMP was pretty good as a player, but I was disappointed in its tagging facility, so I went looking elsewhere as my directory of untagged music grew larger and larger over a couple of years.

After reading the excellent Lifehacker, I eventually found that I could do all the tagging I used to do (and more) with MediaMonkey. I was very impressed with it and after a short while was tagging quicker than ever and had caught up with my major backlog. I have made a few discoveries recently that make it even better (all to be revealed in good time).

Unfortunately, I wasn't happy with MediaMonkey as a player. I'm not sure why - it has tonnes of features and looks good - I just never really clicked with it from that point of view. I then decided to conform and try the Big Kahuna of them all, iTunes. I'd read so much about why iTunes was great, but had never persevered further than just syncing my iPod then iPhone with it. I had never used it as a player. that has all now changed and I must say, I am converted and loving it.

I like its simple, non-intrusive interface. I like the fact that I can make smart playlists using logic (I had always avoided playlists beforehand). I love the fact that the Genius function, working together with a well tagged music library, can get my 'better half' subconsciously bopping after choosing just one funky song (this is the accessibility and usability part of my odyssey). Altogether, I think it's great, but I had to grow up and get rid of two precious things: my folder taxonomy and my file naming convention. Once I'd got over this and learnt to love iTunes looking after my library, all I needed to focus on was the tags. In retrospect, this is somewhat of a relief since it's one less thing to worry about.

I now have an even better tagged and ordered system. We listen to music more and find what we want to listen to more easily. There are still some little things I need to crack before I am completely content, but that will come in time. What I plan to do here is to blog about each thing I did and changed, why I did it and how I did it. This way I can share what has given me a bit of a music renaissance.