[jbinkley60]

Well, I'm pretty happy with it so far fr the movies but if you have any other minor tweaks to improve it then I am all for it.

BTW, I tried it also with my TV episodes and it found some dupes which I was never aware of. However, I noticed that it listed some episodes  that did not match any dupes. I wonder why?

Kudos to you for this add-on. It is a BIG help indeed.

If you want to put it in debug mode, run an episodes dupe check, upload the logfile and tell me which ones you think are shouldn't match I can look at it.  I suspect it is a file match where the file name is the same but the show, season, episode is different.   If so, the next release will help identify that by providing all of the metadata information but will still flag as a dupe since the file names are identical.  This is because Kodi prefers an episode naming standard something like:  s0xe0x ...  which would tend to be unique. 


Thanks,

Jeff

[ManilaByNight]

Well, I'm pretty happy with it so far fr the movies but if you have any other minor tweaks to improve it then I am all for it.

BTW, I tried it also with my TV episodes and it found some dupes which I was never aware of. However, I noticed that it listed some episodes  that did not match any dupes. I wonder why?

Kudos to you for this add-on. It is a BIG help indeed.

If you want to put it in debug mode, run an episodes dupe check, upload the logfile and tell me which ones you think are shouldn't match I can look at it.  I suspect it is a file match where the file name is the same but the show, season, episode is different.   If so, the next release will help identify that by providing all of the metadata information but will still flag as a dupe since the file names are identical.  This is because Kodi prefers an episode naming standard something like:  s0xe0x ...  which would tend to be unique. 


Thanks,

Jeff

Thanks for the info. Anyway, here is a partial screenshot of what I see:



When I take a  look at it, the paths are different and the years or am I misinterpreting the results? Anyway, whichever, I am able to make out the dupes just by looking at the report. Great job!

[jbinkley60]

Well, I'm pretty happy with it so far fr the movies but if you have any other minor tweaks to improve it then I am all for it.

BTW, I tried it also with my TV episodes and it found some dupes which I was never aware of. However, I noticed that it listed some episodes  that did not match any dupes. I wonder why?

Kudos to you for this add-on. It is a BIG help indeed.

If you want to put it in debug mode, run an episodes dupe check, upload the logfile and tell me which ones you think are shouldn't match I can look at it.  I suspect it is a file match where the file name is the same but the show, season, episode is different.   If so, the next release will help identify that by providing all of the metadata information but will still flag as a dupe since the file names are identical.  This is because Kodi prefers an episode naming standard something like:  s0xe0x ...  which would tend to be unique. 


Thanks,

Jeff

Thanks for the info. Anyway, here is a partial screenshot of what I see:



When I take a  look at it, the paths are different and the years or am I misinterpreting the results? Anyway, whichever, I am able to make out the dupes just by looking at the report. Great job!

It's showing them as dupes because the TV Show name, season and episode all match.  How do you tell these apart when you are browsing by TV Show ?  I can see if you are browsing by files / path but the TV shows have the same name so as far as the Kodi database is concerned both TV series are the same TV Show.  I don't believe that is best practice but I'll defer to the Kodi experts here..


Thanks,

Jeff

[Karellen]

How do you tell these apart when you are browsing by TV Show ? 

If browsing the library, the shows will have different artwork and most likely show the Year in the sub-title.
A bit like Doctor Who which now has 3 tv shows all named the same.

[HeresJohnny]

Out of curiosity which output format do you prefer, GUI or text file ?

I started with the GUI but soon switched to the text file. As I was using a file manger to track down and delete duplicates that mode made window switchiing easer. I only used 2 categories as I don't have any music videos in the traditional sense of the word but that was very fast.

[jbinkley60]

If browsing the library, the shows will have different artwork and most likely show the Year in the sub-title.
A bit like Doctor Who which now has 3 tv shows all named the same.

Ok, I can see how that could work but when doing this won't all the episodes from multiple series (all with the same TV Show name) end up under the same TV Show and seasons ?  For instance take Night Court when had 2 series, one in 1984 and another in 2023.  If I add them all to Kodi as "Night Court" won't all of the episodes from both season 1s end up under season 1 of the combined Night Court TV Show name in Kodi ?  I believe the scraper sites typically make the TV Show names different to avoid this.

I'll defer to whatever folks want to do here.  I could add another check like the First Aired or Premiered Date to delineate a situation like this when dupe checking.  I guess I am a bit of a purist here and think TV Show names are unique but I'll defer to the majority. 


Thanks,

Jeff

[jbinkley60]

I started with the GUI but soon switched to the text file. As I was using a file manger to track down and delete duplicates that mode made window switchiing easer. I only used 2 categories as I don't have any music videos in the traditional sense of the word but that was very fast.

I kind of figured this scenario, which is why I created options for the output.  I would suspect that once folks do a cleanup they might use the GUI more for small numbers of dupes.

Jeff

[Karellen]

but when doing this won't all the episodes from multiple series (all with the same TV Show name) end up under the same TV Show and seasons ? 

If you do it correctly it won't.
If you just throw all the episodes into one folder, then yes they will end up in the same show. But if you do it correctly and separate the two shows into two folders as they are meant to be, then the scraper will figure it out by the folder name which contains the (year).
No different to movies and their remakes that are named the same - Dune (1984) and Dune (2022)

[jbinkley60]

If you do it correctly it won't.
If you just throw all the episodes into one folder, then yes they will end up in the same show. But if you do it correctly and separate the two shows into two folders as they are meant to be, then the scraper will figure it out by the folder name which contains the (year).
No different to movies and their remakes that are named the same - Dune (1984) and Dune (2022)

Understood but in the screenshot example about the Show name is the same  "Goosebumps" for both vs. Goosebumps (1995) and Goosebumps (2023) so the scraper didn't make the TV Show names unique.  From a database duplicate comparison the TV Show name, the season and episode all say they are the same episodes. 

So to support this scenario and not have them show up as duplicates, I need to add a fourth field check.  I don't suggest using path of file name because there is no guarantee where someone might out something nor how they name their underlying media files.  The only fields which I can see which may be unique are First Aired or Premiered Date.


Jeff

[Karellen]

so the scraper didn't make the TV Show names unique

That is up to the scraper site, not the scraper. The scraper just downloads the data, it doesn't modify it.
You could also check UniqueID, but that might be problematic also, because even though the ID's are unique at each scraper site, they are not unique across the three scraper sites. So TVDB may use an ID that TMDB uses for another show/episode/movie.

[jbinkley60]

That is up to the scraper site, not the scraper. The scraper just downloads the data, it doesn't modify it.
You could also check UniqueID, but that might be problematic also, because even though the ID's are unique at each scraper site, they are not unique across the three scraper sites. So TVDB may use an ID that TMDB uses for another show/episode/movie.

Thanks.  But in general do you believe the scraper sites will typically make the TV Show names unique or is it likely that there will be a lot of this ?  I typically use TVDB and my experience is that it is pretty good about making the TV Shows unique.  An option is to leave as is and deal with the false positives vs. tightening the matching and having false negatives where dupes are missed.  That's the approach I am using with file name dupes. 


Jeff

[Karellen]

But in general do you believe the scraper sites will typically make the TV Show names unique or is it likely that there will be a lot of this ? 

Most are like this and seems that TVDB are the exception, though even TVDB's method of adding the (year) is questionable and causes problems as was seen recently with the new Doctor Who

.  An option is to leave as is and deal with the false positives vs. tightening the matching and having false negatives where dupes are missed

Yea, it is a difficult decision. Those users with very large libraries will find the list of false positives too large to be useful, while others with smaller libraries won't be too worried about the false positives.
Maybe a setting - "Strict Matching" or something like that?

Maybe you can check the logic that @CrystalP implemented for Video Versions matching?

[jbinkley60]

Yea, it is a difficult decision. Those users with very large libraries will find the list of false positives too large to be useful, while others with smaller libraries won't be too worried about the false positives.
Maybe a setting - "Strict Matching" or something like that?

That is an option and something I used for the file table cleaning due to similar reasons.  That would be easy to implement and allow the user to select the current 3 parameters or a 4th for stricter comparison.  For the 4th I am still looking at First Aired or Premiered Date, with a strong leaning towards First Aired since that aligns to the episode itself vs. Premiered which aligns more with the TV series. 

In your experience are the scraper sites and scrapers pretty good about populating the First Aired values in the Kodi database ?  My experience with TVDB and the Mezzmo Kodi addon has been pretty good with this field.  I just checked my episode table, which has 18,000 episodes.  All have a first aired date but that is using Mezzmo's scraper against TVDB.   I suppose 100% accuracy isn't necessary as long as at least one of the series in question has First Aired dates in the episode table.


Thanks,

Jeff

[jbinkley60]

The problem/assumption with those formats is you're assuming we are on a PC/MAC where we can "Save/SaveAs", navigate, open, review, and then decide etc.

If you're on a Fire device, like: My Airstream Trailer, My living Room TV, My bedroom TV, My Kitchen TV, My Billiards Room TV... there is only the hand held remote to contend with and it makes it difficult to work with trying to save it as a physical format and open. If it did do that way, you'd have to initiate it as an auto launch/open after generation, allow user to use/navigate, then auto delete on close to ensure the files don't fill up the limited flash drive space those devices have for each time the application is initiated for that function.

Isn't there a way to create that file as you need it, read/parse it to the screen, and delete it on action/cancel?

IMHO, it really needs to be onscreen like each individual one you do now, there isn't a way to build each, and then concatenate them all together for a use case? It isn't like you are going to selectively "clear" an individual once you do a "view all" option, most will just run the clear all and be on their way. We just want to SEE what is invalid and it intends to remove... and then be able to cancel out if we know something is amiss and then use the individual ones you have in place otherwise. That is why I mention above to build it as behind the scenes, build, parse, combine, view, delete on action/cancel.

It seems Kodi allows array data (or static memory), each log would/should be a giant array of each log indexed from 0 to the end and then just pump it to the screen like you do for each individual... 

I realize I'm short handing it, but that is the 30,000 foot view of how I see it based on all the work you've already done and "should" be an incremental step... but then again, I could be wrong.

I've started looking at the AnalyzeAll feature request.  As you've noted above, it is somewhat incremental.  I've looked through the code and implementing this functionality shouldn't be too much work and I can leverage the approach I just am wrapping up with the  Dupe detection to give the user the option of a GUI, text file or both output. 

Right now I don't delete older copies of the dupe detection text file.  I have some addon code which allows selecting how many copies to keep or I could just provide a simple option of only keeping the last video database analyzer / duplicate media output file.   I am leaning towards simple but would like feedback.

I should be able to have something to test by the end of the week which will have the AnalyzeAll feature.  I want to wrap up the dupe detection feature first before starting on AnalyzeAll.


Thanks,

Jeff

[Karellen]

In your experience are the scraper sites and scrapers pretty good about populating the First Aired values in the Kodi database

Yes, it is a forced date. You can't add the first episode without it.