Win Duplicate movies and tv episodes after upgrade to Kodi 17
#1
Hi!

I just upgraded my Kodi 16.1 to 17.0 RC2.
First after upgrade I couldnt play any movies, it said the files where missing.
Restarted Kodi and it started to sync for new media.
After that almost every movie and tv episode seems duplicated.

I have made sure that the same media source isnt added twice.
The duplicates point to the same files,
Only movies and tv-series that are rar-archived are duplicated.
Only difference seems to be the case of some encoded characters in filenameandpath/basepath
rar://Y%3a%5cvideo
vs
rar://Y%3A%5Cvideo

Its the one with capitol letters that is new
Strange is that dateadded is the same on both copies although votes and plot and some other metadata is different.

Se below for detailed example

Does anyone know why this happend?
And how I can get rid of the duplicates?
I,got a backup of the userdata folder
I'm thinking of removing all sources and start from scratch but then I guess all watchedstates are gone.

Any ideas?

Example from lib export xml file
Version 1
<path>Y:\video\film\hd\Nerve.2016.720p\</path>
<filenameandpath>rar://Y%3a%5cvideo%5cfilm%5chd%5cNerve.2016.720p.rar/Nerve.2016.720p.mkv</filenameandpath>
<basepath>rar://Y%3a%5cvideo%5cfilm%5chd%5cNerve.2016.720%5cnerve.2016.720p.rar/Nerve.2016.720p.mkv</basepath>

Version 2
<path>Y:\video\film\hd\Nerve.2016.720p\</path>
<filenameandpath>rar://Y%3A%5Cvideo%5Cfilm%5Chd%5CNerve.2016.720p%5Cnerve.2016.720p.rar/Nerve.2016.720p.mkv</filenameandpath>
<basepath>rar://Y%3A%5Cvideo%5Cfilm%5Chd%5CNerve.2016.720p%5Cnerve.2016.720p.rar/Nerve.2016.720p.mkv</basepath>


Running on Windows 10, all media from Synology NAS via SMB-share and mapped network drive.

One more thing, I think i before used a imdb plugin, so i did get ratings from Imdb, maybe it has to do with that.
The old copy says
<uniqueid type="imdb" default="true">tt3531824</uniqueid>
The new one
<uniqueid type="unknown" default="true">tt3531824</uniqueid>

Thanks..
Reply
#2
Looks like the rar files from one set is seen as a new file in the other version, case sensitive.

rar://Y%3a%5cvideo
vs
rar://Y%3A%5Cvideo

I would handle this by clearing out your sources, re-importing your exported back-up then export that library to separate files, so the new system has a chance to write back out the meta-data using overwrite. Then bring a scraper back online for the new/missed material I'm not sure this is the solve though. You might end up starting from scratch. The screw-up seems to be how the files are unrared and not sure if this is in the archiver or how Kodi is writing this out.
Reply
#3
Thanks for the answer!

Sounds a little complicated.
I'm lost after export library to separate files Smile

Maybe convert to mysql database and scripting Smile
Reply
#4
(2017-01-04, 14:54)gr00ve Wrote: I'm lost after export library to separate files Smile
Maybe convert to mysql database and scripting Smile

When you go to back up your library 'export', then you are presented with two options, single file or separate, choose the later and Kodi will try and place all the meta-data for the video in the video's folder; a further option will allow for overwrite of existing .nfo and thumbs. It's at this point the 'new' overwrite (from your old export) will use what is in the library, for future scrapes and case duplicates might be avoided.

Editing the mysql database and scripting by hand is always and option, perhaps notepad++ would make fast work with the right macro.
Reply
#5
I solved it by scripting the video database file via SQLLite Smile

Downloaded SQLLite and opened MyVideos107.db database file.

Below is my solution that fixed my problem

Code:
-- update playcount on new movies from old movies
update files set playCount = 1
where idFile in (
select f2.idFile from movie as m1, files as f1, movie as m2, files as f2
where m1.idFile = f1.idFile
and f1.playCount > 0
and substr(m1.c22,0,14) = 'rar://Y%3a%5c'
and m1.c23 = m2.c23
and m1.idMovie <> m2.idMovie
and m2.idFile = f2.idFile);


-- delete old movies
delete from movie
where idFile in (
select idFile from files
where substr(strFilename,0,14) = 'rar://Y%3a%5c'
)

-- delete old episodes
delete from episode
where idFile in (
select idFile from files
where substr(strFilename,0,14) = 'rar://Y%3a%5c'
)

-- delete old files
delete from files
where substr(strFilename,0,14) = 'rar://Y%3a%5c'
Reply
#6
Missed to copy playCount for episodes, but somehow it knows what I have watched Smile
Reply
#7
thank you, I had the same issue and it helped me,
in my case it was %2F character.
Reply
#8
I reported this here: http://trac.kodi.tv/ticket/17266
Reply
#9
Strange thing it happend to me again.. :S
Same problem, this time I used Xaspers solution from the ticket he reported.
Dont know when it happend but it seems to be ok now, we'll se for how long
Reply
#10
I am having a similar issue. I think from accidentally installing both the Rar Archive and Archive add-ons. Now I have two entries (sometimes three, where one just plays audio with the movie thumnail on the screen) for every Rar'ed movie and show.

Unfortunately, it turns out I have a lot of these in my library.

A lot of this was over my head, as I've never been able to figure out any SQL solutions to Kodi, and even if I were able to figure that out, I can't see where the files are pointing to two differently cased directories. They only show up once under the Video Files section, only in the scraped library are they duplicated.

I tried Cleaning the library, they don't get removed. I tried again after disabling all archive add-ons. I tried upgrading to Matrix and downgrading back to Leia. I tried Reimporting my library under Media Settings (although that didn't do anything).

I'm out of ideas. Any help is much appreciated.8.5.2
Reply
#11
The easiest way to clear your library is to go to your Source and set it to None and save that. Answer Yes to removing the entries and cleaning the library. Once that is done, set it back to Movies and rescrape the source, making sure you have only 1 archive addon enabled. But having both should not be causing duplicates. It is some setting that is wrong. Scrape one movie that causes duplicates and provide the Debug Log.

You can check where your library entries are pointing to...

Image
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply
#12
(2020-11-19, 07:05)Karellen Wrote: The easiest way to clear your library is to go to your Source and set it to None and save that. Answer Yes to removing the entries and cleaning the library. Once that is done, set it back to Movies and rescrape the source, making sure you have only 1 archive addon enabled. But having both should not be causing duplicates. It is some setting that is wrong. Scrape one movie that causes duplicates and provide the Debug Log.

You can check where your library entries are pointing to...

Image
Thanks for the info. I didn't notice that on the Refresh tab.

Before I jump the gun and wipe out my entire library source, I want to make sure it will actually fix anything first.

So, different movies have different issues. I'll start with one of the Harry Potter films.

For the first entry, the directory listing is "rar://...hpatootp-bdrip-chivalric.mp4"

For the 2nd, it is  exactly the same.

For the 3rd, which does nothing when you try to play it, it is the full directory listing of the rar file: "Z:\Movies\Harry.Potter.and.the.Order.of.the.Phoenix.2007.BDRip.X264-CHiVALRiC\hpatootp-bdrip-chivalric.rar"

Some other movies will have just the 1st and 2nd example, which are the same, but not the broken 3rd example.

For some other movies, the one formatted like the 3rd example, do play, but they play only audio with the thumbnail. Not sure what that is about.

As far as the issue being possibly a setting issue, the only setting on the RAR archive support addon is for whether or not you want it to ask to unpack.

You mentioned sending the log after unscraping one file, but I'm not sure how to do that, since 1) They're already scraped, and 2) I only know how to have the automatically done for the whole library.

Thanks again for any and all help.
Reply
#13
I noticed that Kodi is crashing after running scans.

My last log has several lines like this:

2020-11-19 11:39:47.916 T:1100   ERROR: AddOnLog: RAR archive support: CRarFile::Open: Open failed to open file in cache: C:\Users\chugn\AppData\Roaming\Kodi\cache\binary-addons\vfs.rar-temp/rarfolderv01100/

If I open that directory in file explorer, there is a RAR file there that is also in my library.

There are 165 folders like this with different movies in them. All of them are also in my library. Could this be what is causing the issue? Maybe if I delete them from there?

It also has several blocks like this:
2020-11-19 12:01:55.696 T:11316   ERROR: AddOnLog: RAR archive support: CRARControl::ArchiveExtract: unknown error 16 on 'Z:/Movies/Watchmen.2009.ULTIMATE.CUT.DVDRip.XviD-NODLABS/Subs/watchmen.2009.ultimate.cut.dvdrip.xvid-nodlabs.subs.rar'
2020-11-19 12:01:55.696 T:11316   ERROR: AddOnLog: RAR archive support: CRarManager::CacheRarredFile: failed to extract file:
2020-11-19 12:01:55.696 T:11316   ERROR: AddOnLog: RAR archive support: CRarFile::Open: Open failed to cache file
2020-11-19 12:02:23.922 T:11316   ERROR: XFILE::CDirectory::Create - Error creating
2020-11-19 12:02:23.923 T:11316   ERROR: Previous line repeats 2 times.8.5.2
Reply
#14
Here is my full log.
https://pastebin.com/00F5540G8.5.2
Reply
#15
(2020-11-19, 20:53)Chizad Wrote: Here is my full log.
https://pastebin.com/00F5540G8.5.2
It is a dead link. Try Kodi Paste Site

Also provide a list of the files and folders contained in the rar file.
My Signature
Links to : Official:Forum rules (wiki) | Official:Forum rules/Banned add-ons (wiki) | Debug Log (wiki)
Links to : HOW-TO:Create Music Library (wiki) | HOW-TO:Create_Video_Library (wiki)  ||  Artwork (wiki) | Basic controls (wiki) | Import-export library (wiki) | Movie sets (wiki) | Movie universe (wiki) | NFO files (wiki) | Quick start guide (wiki)
Reply

Logout Mark Read Team Forum Stats Members Help
Duplicate movies and tv episodes after upgrade to Kodi 170