Implemented database cleaning

[gs11]

Files no longer found on the file system are now removed as part of the database update process.
Thus, import has been renamed to update in some parts of the view.

This fixes issue #129

[MightyCreak]

LGTM

[ltguillaume]

This doesn't remove unused cover art.

[gs11]

Good point, I'll fix that. I guess playlist membership should be removed as well if the database design doesn't use cascade for deletes.

[MightyCreak]

I guess playlist membership should be removed as well if the database design doesn't use cascade for deletes.

If we want to do something very nice (maybe a v2), the deleted songs should go into a "deleted_songs" table. Then the playlist will show the song grayed with a text message "Song has been removed". If there are no more dependencies like that, then the song can be removed from the "deleted_songs" table.

[gs11]

@ltguillaume Cover art & playlist membership removal now implemented

[gs11]

@MightyCreak Yeah, that might be feasible in the future. I took the simpler road for this pull req.

[ltguillaume]

@gs11 I don't think this will work: as far as I remember, the names of the cover art files are MD5 checksums, so if a song's cover art results in the same file, with the same checksum, it will re-use the existing file. So the relation cover art - song is one to many. I had a glance at your code, and it seems you'll remove the cover art without checking if other songs are using these files as cover art.

[MightyCreak]

@gs11 That's really all right, I am already glad you did this feature since I was waiting for someone to do it for a very long time! ;)

@ltguillaume About the MD5 checksums, I guess the MD5 is based on the song file? If it is, then I guess the idea would be to do SELECT COUNT(*) FROM songs WHERE cover = '<the md5>.jpeg' for each deleted songs (or for each unique md5 delete ;) ), if there is no result, we can remove the cover!

That might increase the update time, but I, for one, don't really care if it takes 1 min or 5, as long as the database is clean afterwards.

[gs11]

@MightyCreak @ltguillaume I'll add a lookup to make sure that the covers are only removed once all references are gone. The MD5 is btw based on the artist & album.

Btw, what has happened to the project/developers? The pull request queue is growing.

[ltguillaume]

I think I did this exactly the same way in my piece of code, yeah, just SELECT COUNT.

I think I remember that the import process via web UI vs the import process via the command line interface is completely separate, so you'll have to implement these changes twice, correct?

This was my code ltguillaume@6d0ffa7 but it's based on an older development commit, so things might have changed,

[gs11]

Finally got around to look at this again.
However, I'm struggling with getting CakePHP to work with my query.

Working SQL statement:

SELECT s1.id, s1.cover, COUNT(s2.id) files_with_cover FROM songs s1 INNER JOIN songs s2 ON s2.cover=s1.cover WHERE s1.source_path=$path`

CakePHP version:

$result = $this->Song->find('first', array(
    'joins' => array(
        array(
            'table' => 'songs',
            'alias' => 'Song2',
            'type' => 'INNER',
            'conditions' => array('Song2.cover = Song.cover')
        )
    ),
    'fields' => array('Song.id', 'Song.cover', 'count(Song2.id)'),
    'conditions' => array('source_path' => $file)
));

However, the query only hangs here. The idea is to only delete the cover if count is 1.
Other alternatives are:

• Do a separate query to check files with cover (I'd like to avoid this)
• Use raw sql ($this->Song->query('SELECT [...]')

Any ideas?

[ltguillaume]

I think it's a little bit complicated, that's for sure :P

I just did this after I removed the song.

if (!empty($cover) && $this->Song->find('count', array('conditions' => array('cover' => $cover))) == 1) {
delete cover file...
}

I think my code here ltguillaume@6d0ffa7 might be hacky, I don't know (and it's only for the command line interface), but it has been working fine for months.

[gs11]

Fixed the above query issue, my condition was ambiguous since source_path exist in both joined table.
Found out that there are logs written at /app/tmp/logs which were very helpful.

However, somehow the query returns two rows where the first row contains id & cover and the second contains the count. Troubleshooting..

[gs11]

Ok, issues resolved and this now seems to work just fine! Merging to my development-new branch.
The cover files are now only removed if the last song referencing it has been removed.
Implementation in command line utility deliberately left out for now