[Logo] Jaikoz and SongKong Forums
  [Search] Search   [Recent Topics] Recent Topics   [Members]  Member Listing   [Groups] Back to home page 
[Register] Register / 
[Login] Login 
moving unmatched files  XML
Forum Index -> SongKong Issues Go to Page: 1, 2 Next 
Author Message
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

I noticed when I have set sk to move only matched files and i dont have match from discogs turned on, that i am still getting some albums that are being moved over that do not have mb release ids. For example, this last run, moved over 217 albums, but only 201 had mb release ids and could be matched in jaikoz.

Does sk use only the current info that it has found in that run, or does it consider old mb and discogs info that was already in the tags when deciding what to move over as matched albums?

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

I tried it again on the same group, this time making sure to delete everything but album, artist, album artist, song, track number, and date in jaikoz. I then loaded them all into sk and ran a fix and it still ended up moving 15 albums over that did not have mb release ids. Not sure why sk is seeing these extra ones as completed albums even though they dont have mb release ids and the discogs match is turned off.

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

Do they have Discogs Release Urls because if they do then they will get moved because the save/move tasks just looks at ids in the files rather than ids added in the last run.

The logic behind this was that if you have a music folder that contained a 1000 songs, that you fixed with SongKong and then added another 100 songs to that folder, then changed the filename mask and just elect to fix the new 100 songs it allows the the new rename mask to be applied to all 1100 songs without having to rematch all 1100.

But perhaps that is too confusing, I'm running into the same issue as Jaikoz of trying to provided a solution flexible enough for everyone.

thanks Paul (Administrator)
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

I cleared out all the discogs and mb info before i ran it. I unchecked the match to discogs option, but i did have the option to add discogs supplementary info checked. Not sure if it added any info on those songs.

Would be nice if it is set to only match complete mb releases to only move the completed mb releases when it saves them.

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

I tried it again, this time unchecking even the add discogs additional info. It works as expected that way. It would be nice though if I could still have the benefit of using discogs to fill in any blank info that musicbrainz cant find and not break the match and move only complete albums.


Would be great if there was an option or way for it to ignore existing data as well. Seems a little redundant to have to load everything first into jaikoz and delete all the mb and discogs fields before i can fix it in sk.

So close to being a 1 click solution

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

greengeek wrote:

Would be great if there was an option or way for it to ignore existing data as well. Seems a little redundant to have to load everything first into jaikoz and delete all the mb and discogs fields before i can fix it in sk.
 

Good point, so yes it does ignore all that for song matching the problem lies with song naming/moving so I think it would probably be better if song renaming/moving only worked on songs that had been matched in that run.

I don't understand the Discogs issue though, if you cleared out the data and if you have Only allow match if all songs in one folder match to one album enabled and then Search for a Discogs Match disabled and only have Update from Discogs enabled then the only things that should have Discogs Url Ids would be songs that were matched to MusicBrainz, hence only the releases matched to MusicBrainz should be moved.

Do you still have your reports, or could you retest ?

thanks Paul (Administrator)
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

Ive made the following chnages for the new release, in line with your thoughts I think

1. Songs are only renamed/move to matched if matched in this invocation
2. Unmatched sections now count unmatched to release
3. Songs matched to recording are moved to unmatched folder if option enabled to prevent folders being split up.
4. If Ignore already matched option is enabled then nothing at all is done with those files including renaming/moving. (The old behaviour will be available in amended form as part of SONGKONG-94 (Update from Metadata option)

thanks Paul (Administrator)
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

As far as the discogs issue goes.

1. If I have the following Checked:

Only allow match if all songs in folder match to one album
Only allow match if all songs in album were matched

and then I have Unchecked the follow:

Update from Discogs
Search for a Discogs match

If I set it up this way, then songkong performs as I would expect it to and only moves complete matches that can check out in jaikoz as completed with no missing songs. All songs have a MB release tied to it.

2. If I do the same thing above, but this time I Check

Update from Discogs

The same files get moved from above, plus some additional albums. Even though Search for a Discogs match is unchecked, SK seems to still be trying to use the discogs info it got from Update from Discogs to make matches and move them.

Running a report in jaikoz will show albums that could not be checked as they do not have a matching MB release id.

It would be nice if this only did what the help description shows where it adds any additional info that is not already in the file that discogs may provide without overwriting the musicbrainz info. And SK not try to use this additional info to create a Discogs Match. It seems that should only happen if the option to Search for a Discogs match is checked.


I dont have the log files currently, but I did keep a copy of all my source files that were unaltered by SK, so i can reproduce any of this or test if you like or need those logs recreated.

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

Okay lets just see if the problem goes away with the new version due to the changes Ive made.

thanks Paul (Administrator)
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

Installed the latest version of SK. Have only tested 1 scenario so far and it appears to still be doing the same thing.

I ran sk on about 4,000 files without cleaning them first. So a lot of them have old MB and Discogs data in them.

I have Checked:

Only allow match if all songs in folder match to one album
Only allow match if all songs in album were matched
Update from Discogs

I have Unchecked:

Search for a Discogs match

It moved over a bunch of files that were not updated as well as ones that were unmatched. Jaikoz should unmatched files as well as files that needed to be saved as they were not updated to 2.4 in sk.

Should this still be happening? Do I still need to have both Update from Discogs and Search for a Discogs match unchecked as well as cleaning out all the old mb and discogs data first?

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

Hmm, can you please email me your support files

thanks Paul (Administrator)
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

Just tried it again, this time doing it the way that worked in the last version. I cleared out all the mb and discogs info in jaikoz and unchecked the discog options in sk and even the long way that worked before is no longer working

It moved over 5 albums that are not complete.

Emailed you the support files at support@jthink.net

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

Ideally what I would like to be able to do is just point sk to all my music files and match and move all the complete albums.

It would be ideal if it had a setting where it would:

1. do its normal magic, ie fingerprint, look at existing album, artist, song, track# etc tags to identify the tracks/album

2. match to a musicbrainz release that has the exact same amount of tracks as the ones in the folder.

3. fill in any empty fields with discogs data.

4. move the matched albums to the move folder, but only mb match releases, do not consider discog match releases as a match if the search for discogs match is unchecked. Do not consider any old mb or discogs data that was already in the id3 tags if not obtained from the current session for match/moving purposes.

That way one can move all the albums that have been verified as complete and correct on musicbrainz and have all the latest tag info and artwork from musicbrainz with discogs filling in the blanks.

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

That is exactly what the latest version is meant to do, just looking at your support files now

thanks Paul (Administrator)
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

Thanks, sending the Jaikoz Missing Songs Report was a nice touch makes it much easier to spot the problems

So I went through the missing songs list, the first incomplete album was

1. UFO - Covenant (dd34ce36-a59c-45ba-b549-0251a936481c) only 1 of 18 tracks

You started of with 18 songs in
C:\temp\u\UFO\(2000) Covenant (DE)

One song (02-03 - Let It Roll.mp3) has been matched and moved the other have not.

What has happened is after trying to match a folder SongKong does duplicate detection within the folder. The idea is if you have backups or multiple copies of some files then SongKong is never going to be able to match the whole folder to one album so we look for duplicates based on acoustid, and if that fails songname and if any duplicates are found we then split them into separate groups and try and match those individual groups.

Looking at the log I can see that Let It Roll has been identified as a duplicate and put it in its own group, and then because just one song it was able to match it to an album.

My first thought was that two versions of this song must be on the same release but that was not the case, the problem was that the acoustid for this track has been linked two different recordings Let it Roll and the Kids

file:///C:/SongKongSupport/GreenGeek200/SongKongSupport/FixSongsReport00001.html

So I need to improve duplicate identification to deal with this scenario.

Then I was wondering why it was the match was allowed anyway because you had Only allow match if all songs in album were matched enabled, I think there is some special case handling for single tracks that isn't working here.

BTW you can remove these incorrect links on Acoustid by clicking the unlink button and I have asked the developer about a housekeeping task to detect acoustids that are linked to multiple completely different songs (not just different versions of same song) by the same artist because these indicate an incorrect match.

2. Then I looked at U2 Chokecherry issue
http://musicbrainz.org/release/970bfcbd-cb06-466e-b670-0dadc24270da
this one is going wrong because the release contains multiple version of the same track. I'm wasn't sure how to deal with this as they are a good canditate for potential duplicate and because the original release match failed its not clear both belong on an album, but then once again shoudn't match because whole album is not matched anyway.

3. The Ultimate 7" & 12" Collection (disc 10) (b70aa0ca-87ee-4e77-a354-d3efe6cd8627)

This one is interesting because it has matched more than one song , Reminder:I'll look at this one later

thanks Paul (Administrator)
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

1. I wasn't right about (1) because the duplicate grouping is first done only on the acoustids and both songs have different acoustids, but if no dups are found based on acoustids it does it by title and this is where the problems originated.

Earlier on after failing to match the whole folder to a release it does recording only matches based on acoustids to ensure we have at least some basic information, if your song already has data it only uses acoutids if the acoustid matches to just one song, however in the case of This Kids the acoustid http://acoustid.org/track/e9c481e7-d9d2-4d5b-a8ea-d54210feffa3 does only match one song, but it matches the wrong song, 'Lets Roll' and does get modified, and hence at dups stage it finds two songs with the title Lets Roll

So maybe recording only matches should only set title if blank, but the problem is often titles are not blank but full of garbage such as 'Track 1' and it is a shame not to fix them. I could finesse to check if title is blank or Track* but I'm going to miss cases where it is just marked to a wrong title, maybe better would be to ignore unless the acoustid sources count is a certain threshold, this one only has two sources.

thanks Paul (Administrator)
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

4. The Undertones - Anthology

The missing songs report says 3 songs were matched but looking at the SongKong report only one song was actually matched You?ve Got My Number and this occurs twice on the album albeit different versions.

So with the changes I have made this would no longer be matched if you have Only allow match if all songs in folder match to one album enabled.

thanks Paul (Administrator)
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

3. The Ultimate 7" & 12" Collection (disc 10) (b70aa0ca-87ee-4e77-a354-d3efe6cd8627)

Once again SongKong only actually matched one song North and SOuth of the River so this should be fixed by the changes I have made.

thanks Paul (Administrator)
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

As I expect to have a simpler solution to the Discogs Image Authentication problem available soon I expect to have a new release out end of next week that solves these issues (already fixed in codebase)

But in the meantime it would be interesting if you could repeat the test with the Discogs options enabled to see if there is an additional Discogs related issue that I need to fix.

If you can do this please send support files including report and Jaikoz missing songs report as well.

thanks Paul

thanks Paul (Administrator)
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

Sure thing. I will copy the same source files I used in this last test and send it over to you so you can do a direct compare between the two runs.

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

Sent

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

Hi, thanks so from what I can see it is no worse then the previous report, i.e there are no additional problems caused by enabling the Discogs options.

thanks Paul (Administrator)
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

Okay, this should now be fixed in SongKong 1.24 please give it another go.

thanks Paul (Administrator)
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

Thanks, Ill give it a try with that same folder set and let you know how it worked.

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

I tried the latest version of SK and it seems to work better. It still failed on moving a non matched album. Also I noticed that it does not update the tags that are 2.3 to 2.4 when the save 2.4 option is checked. Would be nice if SK did a save before moving on ALL files, even the ones that it did not change so that they are all the same id3 version if they are being moved. Jaikoz showed a bunch of files that needed to be saved and the only reason was they were 2.3 and not 2.4.

I have emailed my sk support files along with jaikoz reports.

Thanks!

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

Thankyou, found the bug it is an issue that only affects folder with a single song in them http://www.jthink.net:8081/browse/SONGKONG-595

I don't quite follow what you mean about the tag versions, could you go into a bit more detail please.

thanks Paul (Administrator)
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

Sure, the source music is a mixture of both id3v2.3 and id3v2.4. I did some media monkey updates previously that saved over some of the files converting them to id3v2.3. I have songkong and jaikoz both set to save files as id3v2.4 to keep everything consistent. I run songkong and it does it's matching and moving. I am assuming when it is doing it's updating that it is not saving every song that it checks. Maybe songs that did not have any new info found or did not get updated are not saved before being moved? So I then run jaikoz, on the sk completed songs, and I see that I have a bunch of songs that jaikoz wants to save again as they are still using id3v2.3 and have not been updated to the id3v2.4 version.

Could songkong work like jaikoz does in that it will verify that all songs are using and saved with the id3 version set in the user options?

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

It should do that if you have the option set in the Saved tab, and for me it is doing that I haven't been able to replicate the issue so far, in fact I am seeing the opposite problem of SongKong sometimes saving files when there is no need http://jthink.net:8081/browse/SONGKONG-600

thanks Paul (Administrator)
greengeek

Pro
[Avatar]

Joined: 18/09/2007 02:50:48
Messages: 435
Offline

Hmm, guess I will have to try it again. I know it was set to v24, but I am not positive that I may have overlooked it and had it set to Same as or v24.

Unfortunately I have uninstalled both programs from that PC as I am in the process of upgrading my home PC. Once I get windows setup on the new box, Jaikoz and SongKong will be some of the first programs to be loaded on. Then their user prefs transferred over.

I am excited for the next update on SK. Can't wait to test it on a catalog set, and then run it against my full collection!

KevinBluemel.com
Contemporary Instrumental Musician and Composer. Stop by to listen, watch videos, and get free downloads of songs and sheet music.
[WWW]
paultaylor

Pro
[Avatar]

Joined: 21/08/2006 09:21:27
Messages: 7340
Offline

Moving a non matched album issue should now be fixed in SongKong 1.25

thanks Paul (Administrator)
 
Forum Index -> SongKong Issues Go to Page: 1, 2 Next 
Go to:   
Powered by JForum 2.1.6 © JForum Team