Sorting through years is broken?
#1
Hello!
I recently discovered Kodi and I'm trying to organize around it to create my media center.
I've got around 3000 movies, so efficient sorting is crucial to me.
I've noted weirdnesses in the "Years" and "Genres" categories. Obviously, as you must know, sevreal movies are often sorted in very questionnable genres and some of them are never sorted anywhere. But I'm mostly ok with that since I don't see myself browsing through genres all that often.

"Years" on the other hand is a feature I was excited about but it seems to be broken as well.
Obviously it's hard to double check everything when you have so many movies but one year I use as reference is 1999. The Matrix is missing and so is The Phantom Menace to name a few. I know it is not just a matter of putting the year in the file name since The Mummy hasn't got the date and it's popping in the right year.

Is the category broken? How does it work? How could I fix it?
Thank you
Reply
#2
(2020-03-22, 12:40)macromaster Wrote: Obviously it's hard to double check everything when you have so many movies but one year I use as reference is 1999. The Matrix is missing and so is The Phantom Menace to name a few.

When I select Movies / Years / 1999, I get to see both those movies in the listing. So I'm not quite sure what's up with your setup.
Any chance you are using a different skin than the system default Estuary?

Perhaps enabling debugging, restarting Kodi, display the 1999 year listing again, could maybe show something in Kodi's debug log (wiki).
Reply
#3
Perhaps it's a scraper issue.  Might be worth getting up the info dialog for The Matrix and clicking the 'refresh' button to re-scrape the info and see if it picks up the year.
Learning Linux the hard way !!
Reply
#4
(2020-03-22, 13:54)Klojum Wrote:
(2020-03-22, 12:40)macromaster Wrote: Obviously it's hard to double check everything when you have so many movies but one year I use as reference is 1999. The Matrix is missing and so is The Phantom Menace to name a few.

When I select Movies / Years / 1999, I get to see both those movies in the listing. So I'm not quite sure what's up with your setup.
Any chance you are using a different skin than the system default Estuary?

Perhaps enabling debugging, restarting Kodi, display the 1999 year listing again, could maybe show something in Kodi's debug log (wiki)
All default, skins and such. I will try the debug thing. I often rescrap everything to get everything right in one go, correcting along the way my filenames (sequels not displaying properly if the year isn't included and such).

I did a refresh on Episode I and it finally entered 1999. Which is strange because it was in my titles list and had the correct info already. Could it be Kodi messes up when it's batch scrapping too many movies?
As for Matrix, my filename was "The Matrix 1999" and it was flat out unlisted. Double checked if the movie was indeed called "The Matrix" and not "Matrix" but it is indeed "The Matrix" from 1999. Refreshing it also got me the proper info and put it in the correct year.
Although it is possible I added The Matrix recently and I didn't exable refresh library on start. Honestly can't tell.

Thank you for your advice. I'll do a wide refresh and come back with some debug log. But I definitely think there's something fishy here and there.
Reply
#5
(2020-03-22, 14:29)macromaster Wrote: Could it be Kodi messes up when it's batch scrapping too many movies?

Although that's possible I think it's more likely that whichever site you are scraping from simply becomes 'overwhelmed' by the amount of data being requested.  I know for a fact that when scraping music for instance, that the musicbrainz website is limited to one request per second from an individual IP and if you try and go quicker, it just doesn't return any info for those queries.  I'd guess that a similar thing is in play here and that if you scrape your stuff in smaller batches you'll get a better success rate.  I could of course be totally wrong, but a debug log (wiki) should show up that issue quite clearly.
Learning Linux the hard way !!
Reply
#6
Scraping is done per video, movie or tv episodes. I don't think the size of a media collection should matter.
Reply
#7
(2020-03-22, 15:09)black_eagle Wrote:
(2020-03-22, 14:29)macromaster Wrote: Could it be Kodi messes up when it's batch scrapping too many movies?

Although that's possible I think it's more likely that whichever site you are scraping from simply becomes 'overwhelmed' by the amount of data being requested.  I know for a fact that when scraping music for instance, that the musicbrainz website is limited to one request per second from an individual IP and if you try and go quicker, it just doesn't return any info for those queries.  I'd guess that a similar thing is in play here and that if you scrape your stuff in smaller batches you'll get a better success rate.  I could of course be totally wrong, but a debug log (wiki) should show up that issue quite clearly. 

If that is the case. Could we slow down the rate on which Kodi scrapps? Or is there an Add-on to ask confirmation for each scrapped movie?

I just scrapped the whole library again. This time Matrix still didn't show up in 1999. Simply scrolling in my files and hovering on Matrix, the cover appeared blank but popped up a couple of seconds later. Came back in 1999 and it appeared. I'm puzzled.
Again, Matrix is just one example, but my library is full of diverse movies I don't know so well. The point is to be able to browse through the ones I don't know so much, by actors, or years or directors so I really want this to work well.

Here's the log for when it scrapped Matrix I think
Quote:2020-03-22 14:38:05.031 T:2784   DEBUG: VideoInfoScanner: Adding new item to movies:E:\The Matrix 1999.mp4
2020-03-22 14:38:05.161 T:5152   DEBUG: CCurlFile::GetMimeType - http://image.tmdb.org/t/p/original/sFsEe...c1DA49.jpg -> image/jpeg
2020-03-22 14:38:05.161 T:5152   DEBUG: CurlFile::Open(0x1f7051687e0) http://image.tmdb.org/t/p/original/sFsEe...c1DA49.jpg
2020-03-22 14:38:05.263 T:2784   DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting definitions
2020-03-22 14:38:05.263 T:2784   DEBUG: CAddonSettings[metadata.themoviedb.org]: trying to load setting definitions from old format...
2020-03-22 14:38:05.263 T:2784   DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting values
Then it skipped to The Matrix Revolutions (don't have Reloaded)
This is when I hovered the film I think
Quote:2020-03-22 14:46:43.967 T:12600    INFO: ffmpeg[3138]: Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'E:\The Matrix 1999.mp4':
2020-03-22 14:46:43.967 T:12600    INFO: ffmpeg[3138]:   Metadata:
2020-03-22 14:46:43.967 T:12600    INFO: ffmpeg[3138]:     major_brand     : isom
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:     minor_version   : 1
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:     compatible_brands: isom
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:     creation_time   : 2013-01-30T19:36:47.000000Z
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:   Duration: 02:16:17.96, start: 0.000000, bitrate: 2334 kb/s
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:     Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x796 [SAR 1:1 DAR 480:199], 1827 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc (default)
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:     Metadata:
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:       creation_time   : 2013-01-30T18:17:03.000000Z
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:       handler_name    : GPAC ISO Video Handler
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:     Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 377 kb/s (default)
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:     Metadata:
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:       creation_time   : 2013-01-30T13:58:43.000000Z
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:       handler_name    : Sound Media Handler
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:     Stream #0:2(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 124 kb/s (default)
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:     Metadata:
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:       creation_time   : 2013-01-30T14:58:29.000000Z
2020-03-22 14:46:43.968 T:12600    INFO: ffmpeg[3138]:       handler_name    : Sound Media Handler
2020-03-22 14:46:43.968 T:12600   DEBUG: CDVDDemuxFFmpeg::AddStream ID: 0
2020-03-22 14:46:43.968 T:12600   DEBUG: CDVDDemuxFFmpeg::AddStream ID: 1
2020-03-22 14:46:43.968 T:12600   DEBUG: CDVDDemuxFFmpeg::AddStream ID: 2
2020-03-22 14:46:43.968 T:12600  NOTICE: Opening stream: 0 source: 256
2020-03-22 14:46:43.968 T:12600  NOTICE: Creating video codec with codec id: 27
2020-03-22 14:46:43.969 T:12600  NOTICE: CDVDVideoCodecFFmpeg::Open() Using codec: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
2020-03-22 14:46:43.969 T:12600   DEBUG: CDVDVideoCodecFFmpeg - Updated codec: ff-h264
2020-03-22 14:46:43.969 T:12600   DEBUG: CVideoPlayerVideo::OpenStream - open stream with codec id: 27
2020-03-22 14:46:43.969 T:12600  NOTICE: Creating video thread
2020-03-22 14:46:43.969 T:12600   DEBUG: CEdl::ReadEditDecisionLists - Checking for edit decision lists (EDL) on local drive or remote share for: E:\The Matrix 1999.mp4
2020-03-22 14:46:43.969 T:16884   DEBUG: Thread VideoPlayerVideo start, auto delete: false
2020-03-22 14:46:43.969 T:16884  NOTICE: running thread: video_thread
2020-03-22 14:46:43.969 T:12600  NOTICE: Opening stream: 1 source: 256
2020-03-22 14:46:43.969 T:16884   DEBUG: CVideoPlayerVideo - CDVDMsg::GENERAL_PAUSE: 0
2020-03-22 14:46:43.969 T:12600  NOTICE: Finding audio codec for: 86018
2020-03-22 14:46:43.969 T:12600  NOTICE: CDVDAudioCodecFFmpeg::Open() Successful opened audio decoder aac
2020-03-22 14:46:43.970 T:12600  NOTICE: Creating audio thread
2020-03-22 14:46:43.970 T:12600  NOTICE: Opening stream: 0 source: 1024
2020-03-22 14:46:43.970 T:16876   DEBUG: Thread VideoPlayerAudio start, auto delete: false
2020-03-22 14:46:43.970 T:16876  NOTICE: running thread: CVideoPlayerAudio:Tonguerocess()
2020-03-22 14:46:43.970 T:16876   DEBUG: CVideoPlayerAudio - CDVDMsg::GENERAL_PAUSE: 0
 
Then found it mentionned here
Quote:2020-03-22 14:46:45.475 T:1596    INFO: Deleting settings information for files E:\The Matrix 1999.mp4
2020-03-22 14:46:45.476 T:10844   DEBUG: CSaveFileState:Big GrinoWork - Saving file state for video item E:\The Matrix 1999.mp4
2020-03-22 14:46:45.476 T:3236   DEBUG: Radio UECP (RDS) Processor - delete CDVDRadioRDSData::~CDVDRadioRDSData
2020-03-22 14:46:45.476 T:14228   DEBUG: CAESinkDirectSound::Initialize: secondary buffer created
2020-03-22 14:46:45.477 T:14228   DEBUG: CAESinkDirectSound::Initialize: Initializing DirectSound with the following parameters:
2020-03-22 14:46:45.477 T:14228   DEBUG:   Audio Device    : EP-HDMI-RX (NVIDIA High Definition Audio)
2020-03-22 14:46:45.477 T:14228   DEBUG:   Sample Rate     : 44100
2020-03-22 14:46:45.477 T:14228   DEBUG:   Sample Format   : AE_FMT_FLOAT
2020-03-22 14:46:45.477 T:14228   DEBUG:   Bits Per Sample : 32
2020-03-22 14:46:45.477 T:14228   DEBUG:   Valid Bits/Samp : 32
2020-03-22 14:46:45.477 T:14228   DEBUG:   Channel Count   : 2
2020-03-22 14:46:45.477 T:14228   DEBUG:   Block Align     : 8
2020-03-22 14:46:45.477 T:14228   DEBUG:   Avg. Bytes Sec  : 352800
2020-03-22 14:46:45.478 T:14228   DEBUG:   Samples/Block   : 32
2020-03-22 14:46:45.478 T:14228   DEBUG:   Format cBSize   : 22
2020-03-22 14:46:45.478 T:14228   DEBUG:   Channel Layout  : FL, FR
2020-03-22 14:46:45.478 T:14228   DEBUG:   Channel Mask    : 3
2020-03-22 14:46:45.478 T:14228   DEBUG:   Frames          : 661
2020-03-22 14:46:45.478 T:14228   DEBUG:   Frame Size      : 8
2020-03-22 14:46:45.478 T:14228   DEBUG: CActiveAESink::OpenSink - DIRECTSOUND Initialized:
2020-03-22 14:46:45.478 T:14228   DEBUG:   Output Device : default
2020-03-22 14:46:45.478 T:14228   DEBUG:   Sample Rate   : 44100
2020-03-22 14:46:45.478 T:14228   DEBUG:   Sample Format : AE_FMT_FLOAT
2020-03-22 14:46:45.478 T:14228   DEBUG:   Channel Count : 2
2020-03-22 14:46:45.478 T:14228   DEBUG:   Channel Layout: FL, FR
2020-03-22 14:46:45.478 T:14228   DEBUG:   Frames        : 661
2020-03-22 14:46:45.478 T:14228   DEBUG:   Frame Size    : 8
2020-03-22 14:46:45.478 T:19276   DEBUG: CActiveAE::ClearDiscardedBuffers - buffer pool deleted
2020-03-22 14:46:45.489 T:1596   DEBUG: Previous line repeats 1 times.
2020-03-22 14:46:45.489 T:1596   DEBUG: CApplication::OnPlayBackStopped: CApplication::OnPlayBackStopped
2020-03-22 14:46:45.498 T:3236   DEBUG: CGUIWindowManager:TonguereviousWindow: Deactivate
2020-03-22 14:46:45.498 T:3236   DEBUG: ------ Window Deinit (VideoOSD.xml) ------
2020-03-22 14:46:45.498 T:3236   DEBUG: ------ Window Deinit (VideoFullScreen.xml) ------
2020-03-22 14:46:45.500 T:3236   DEBUG: CGUIWindowManager:TonguereviousWindow: Activate new
2020-03-22 14:46:45.500 T:14228   DEBUG: CAESinkDirectSound::CheckPlayStatus: Resuming Playback
2020-03-22 14:46:45.502 T:3236   DEBUG: ------ Window Init (MyVideoNav.xml) ------
2020-03-22 14:46:45.503 T:3236   DEBUG: CGUIMediaWindow::GetDirectory (E:\)
2020-03-22 14:46:45.503 T:3236   DEBUG:   ParentPath = [E:\]
2020-03-22 14:46:45.574 T:3236   DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting definitions
2020-03-22 14:46:45.574 T:3236   DEBUG: CAddonSettings[metadata.themoviedb.org]: trying to load setting definitions from old format...
2020-03-22 14:46:45.574 T:3236   DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting values
2020-03-22 14:46:45.575 T:3236   DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting definitions
2020-03-22 14:46:45.575 T:3236   DEBUG: CAddonSettings[metadata.themoviedb.org]: trying to load setting definitions from old format...
2020-03-22 14:46:45.576 T:3236   DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting values
2020-03-22 14:46:45.750 T:500   DEBUG: Thread BackgroundLoader start, auto delete: false
Reply
#8
(2020-03-22, 15:13)Klojum Wrote: Scraping is done per video, movie or tv episodes. I don't think the size of a media collection should matter.

It's not the size, but the rate you are hitting the info site which is also getting hit by multiple other people at the same time.  Lets say you scrape 1000 videos and get 900 returned successfully.  You've got 100 videos with no or partial info.  If you scrape 100 videos and get the same failure rate then you only have 10 to redo.  So it makes sense to do smaller blocks no?
Learning Linux the hard way !!
Reply
#9
or maybe it's been scrapped here and skipped to Police Academy for some reason... And came back... What the...

* log removed by moderator *

Link: https://paste.kodi.tv/akupuxemip.kodi
Reply
#10
(2020-03-22, 16:20)macromaster Wrote: or maybe it's been scrapped here

Ehh... Which part of "Do not post logs in the forum. Use https: //paste.kodi.tv/ instead." is unclear to you? We prefer to keep our forum server lean.
Reply
#11
(2020-03-22, 17:28)Klojum Wrote:
(2020-03-22, 16:20)macromaster Wrote: or maybe it's been scrapped here

Ehh... Which part of "Do not post logs in the forum. Use https: //paste.kodi.tv/ instead." is unclear to you? We prefer to keep our forum server lean. 
My bad, sorry.
Reply

Logout Mark Read Team Forum Stats Members Help
Sorting through years is broken?0