[itombs]

Hi, a few weeks ago I posted a xbmc problem running in my xbox:

I have a video folder with 3.000 movies in differents name folders, the films are scraped and they have thumbtails and info.

The problem is that when I return from a movie folder, for example "/video/Matrix/", to the parent folder where I have the 3.000 movies ( "/video/" ) XBMC takes about 25 seconds "working" and its very unconfortable.

A few weeks ago I didnt know the cause of this and some people at this forum say me that this action should be quickly and my problem was I have video folder in a network resource.

I investigated this and know I say that the problem is not that I have video folder in a network resource.

XBMC takes a long time working because is charging thumbtails.

I try delete MyVideos.db and the time using to return video folder was the same but, when I rename "/E/Apps/XBMC/UserData/Thumbnails/Video/" to "/E/Apps/XBMC/UserData/Thumbnails/Video NO BORRAR/" the problem desapeared (and the thumbtails too, of course)

So, the problem is:
When xbmc open a video resource with thumbtails takes time loading thumbtails, more thumbtails -> more time and, if you have less than 750 folders with thumbtails to load this time is very small but, if you have more video folders with thumbtails to load, the time "thinking" or loading increases a lot, in my case with 3.000 thumbs it takes about 25 seconds.
The first time we open video folder is "normal" that xbmc takes time charging thumbtails but when we open /Video/Matrix/ and then return /Video/, XBMC loads thumbtails again and takes others 25 seconds

I think, if when we are in Video folder and press info button and then press back and RETURN to Video folder XBMC doesnt take time charging thumbs, could be possible do that when I return from /Video/Matrix to /Video/ doesnt takes time charging thumbtails , isnt?

The same problem appears when we play a movie and come back to video folder, for exaplme, we are in /Video/ and we press play buton in Matrix this action open media player with /Video/Matrix/Matrix.avi and when we press STOP, xbmc return /Video/ folder and takes 25 seconds

And the same problem apperas in library mode when I open films folder, genres etc etc...

My solutions to the problem could be:

a) Load once thumbtails and keep then in memory or similar to avoid RE-charging it. This option could be not possible because xbox only have 64 mbs mem.

b) XBMC loads in the same way thumbtails BUT in a second plane like when xbmc scans new content with a scrapper for the library. This option return control to user quickly when return to video folder but the xbmc takes 25 seconds to show all thumbtails.

c) A mixing between a) and b), XBMC "remember" the video thumbtails near the video file that we play/open and when returns back video folder we have loaded the thumbtails we see and the rest appears gradually.


I give you "solutions" but i dont know xbmc code and maybe this options could be unimplementables or errors, but i really want a xbmc able to manage folders with hundreds and tousands of thumbtails. I think this is very very important for a good media center like XBMC.

There is any hope that XBMC takes less time loadings thumbtails ?

Thanks a lot and I can test others options to try minimize the loading time if you want.

[spiff]

it's not much hope on the xbox no. the combination of no ram to spare and an utterly crappy fs which turns to a grinding halt with many files is hard to overcome :/

[itombs]

it's not much hope on the xbox no. the combination of no ram to spare and an utterly crappy fs which turns to a grinding halt with many files is hard to overcome :/

So, is impossible to find a solution that allow xbmc be able to manage big number of folders with thumbtails

The re-loading thumbtails in a second plane option seems to me could be able, in some skins fanarts load gradually when cursor is in video file folder and the action is similar.

Id like try to test others 2 options
a) Put video files directly into /video/ folder, for example "/Videos/Matrix.avi" and generate thumbtails and info in this way, but i dont no if xbmc will re-loading all thumbtails again when we stop a video and return videos folder.

b) Make 3.000 .tbn with the name of the folders and copy them into each folder movie for example Matrix.tbn into /Videos/Matrix/Matrix.tbn
I have hope that in this wat load thumbtails coulb be quickly that load thumbtails from "/E/Apps/XBMC/UserData/Thumbnails/Video/" , is it possible??


Thanks and please give me a hope, xbmc with no support to manage big numbers of files with thumbtails is a xbmc very limited.

[jmarshall]

How do you know the problem is with the thumbnails?

Have you debugged it to prove this fact?

[cold_realms]

If you must stick with the good ol' xbox try refining your filesystem a bit, instead of 3000 movies in from one root try /movies/Action/Matrix/matrix.avi
So that only action movies are in the next level, same for any other types of movies.... I could not imagin having to scroll through every movie in my collection to find the one I want, let alone 3k worth!

[itombs]

How do you know the problem is with the thumbnails?

Have you debugged it to prove this fact?

I am 99% sure that the problem is loading the thumbtails. I tested it, I deleted all database and keep .tbns in "/E/Apps/XBMC/UserData/Thumbnails/Video/" and xbmc takes the same 25 seconds loading or "thinking" without give me the control.
Otherwise I renamed "/E/Apps/XBMC/UserData/Thumbnails/Video/" to "/E/Apps/XBMC/UserData/Thumbnails/Video2/" and xbmc doesnt take more than 1 second when i come back from video/matrix to /video.
So, I am 99,9% sure that the problem is the load of 3.000 thumbnails.

I hope that this problem could be corrected because is really unconfortable, do the test and try to open a folder with 1500 folders with thumbnails and i am sure that your xbmc takes a more than 10 seconds loading images.

Also, when you have a big database its imposible to do a "scanning for new content" but this is other problem, the more important now for my is find a solution that avoid xbmc manage thousands of video folders with thumbnails without the large number of seconds loading.

I think that the best solution coluld be a option for "Large thumbnails files folders" that avoid come back or open this folders without load all the thumbnails and that the thumbnails appears gradually.

If I can do something to help or test in my xbmc I will do it. Really thanks a lot for try help me. You are the only people can repair it.

PS: I debugged too and time wasted are "retrieve the directory" :
http://www.xbmc.org/forum/showthread.php...post240455

[itombs]

If you must stick with the good ol' xbox try refining your filesystem a bit, instead of 3000 movies in from one root try /movies/Action/Matrix/matrix.avi
So that only action movies are in the next level, same for any other types of movies.... I could not imagin having to scroll through every movie in my collection to find the one I want, let alone 3k worth!

I think about it but isnt a solution for me, my own library video mode manage the movies and arrange they according to genre, year...

Scrolling were a problem to me but I solved the problem simply, i keymap 2 keys in the controller, one to up letter and other to down letter. Belive me, I waste 7 seconds to travel arround all movies from "0" to "Z"

Thanks for the idea, but I search some different, i search solve the manage problem.

[jmarshall]

Note that XBMC does not load the thumbnails at all when you generate a list. Instead, it is just doing a stat() on the files to see if they exist. This *may* take that long if your xbox filesystem is very fragmented I guess.

We want to redo the thumbnail system to reduce the hitting to disk. I think it can be done with reasonably minimal memory overhead, but am not 100% sure that it's the best way to do things at this point.

It certainly isn't focusing on the xbox at this point, however, as the majority of the team have moved on to the other ports.

Cheers,
Jonathan

[itombs]

Note that XBMC does not load the thumbnails at all when you generate a list. Instead, it is just doing a stat() on the files to see if they exist.

Cheers,
Jonathan

Ok but I think that the stat() function only uses information from the files (folders or files .avi) and there are not information about thumbnails, the thumbnails are in other xbox folders and , in any case if your function stat() calculated some thing about thumbnail information like the thumbnail hash with the path folder or other things you like, the time without thumbnails (when i rename "/E/Apps/XBMC/UserData/Thumbnails/Video/" to "/E/Apps/XBMC/UserData/Thumbnails/Video2/" ) should be 25 seconds and in practice, in my xbox, this time is 0.5 seconds so, when we come back to the parent folder with thumbnails xbmc do a stat()*1 on the files to see if they exist AND or the stat() use thumbnails or xbmc loads thumbnails or other option i dont know, but some more action relative to thumbnails happens.

*1 = really its not a stat() on all the files, is only on the files that previously stat() return yes because if you have a multipath directory with 3 network HDDs and 1 network HDD is off when you open first time the folder returns an error but when you open a folder in this parent folder and come back to the parent folder, the stat() is only in the 2 networks hdds that were ON and if the third HDD is power on in this time, xbmc dont see it, this is a mistake is you use network hdds with auto turn off option.

This *may* take that long if your xbox filesystem is very fragmented I guess.

This option could be, i hope you have reason. Tonight I will prepared other xbox with a new HDD formated and this xbox will scraped all night, tomorrow i will tell you if this are the problem.
But i think, if the problem is that my xbox filesysetm is very fragmented and XBMC does not load the thumbnails at all when you generate a list, why takes 25 seconds when i have thumbnails in the folder i come back and instead takes 0.5 seconds when i dont have thumbnails?

We want to redo the thumbnail system to reduce the hitting to disk. I think it can be done with reasonably minimal memory overhead, but am not 100% sure that it's the best way to do things at this point.

This will be much apreciated, really most parts of xbmc works very very fine and its "increible" that xbmc uses 25 seconds to open a folder with 3.000 thumbnails.

It certainly isn't focusing on the xbox at this point, however, as the majority of the team have moved on to the other ports.

I understand it, but if the team get a xbmc version that allows manage folders with thousands of thumbtails views the xbox port will be PERFECT to uses it like media center, to manage, play and catalog movies.
Its a pity that xbmc have a huge power to play and catalog videos and this power is limited because manage folders with a long number of thumbtails is very unconfortable.


Thanks a lot. Tomorrow I post if the problem is that my xbox filesystem is very fragmented.

[jmarshall]

The stat() is on the thumbnail files. We stat to see if they exist and are cached or not.

[itombs]

The stat() is on the thumbnail files. We stat to see if they exist and are cached or not.

AAAh ok, sorry for my mistake.
Im preparing a new xbox with a new and formated hdd, a new xbmc instalation for scrapper all night.

Tomorrow we speak.

Byes.

[itombs]

Hi, i tested if the problem is my xbox filesystem was fragmented and isnt the problem.

I prepared a new xbox, formatted the hdd, installed the newest version of xbmc and scrapped the movie folder all day.
Now, with about 1500 movies scrapped with info and thumbnails the time used to come back from /movies/matrix/ to movies is 20 seconds so the problem is the xbmc.
Xbmc can not manage folers with longer numbers of thumbnails without an unconfortable waste of time.

I desire that devlopers fix this, a new thumbnail system or the posibility to return control to user without load all thumbnails and that thumbnails appears gradually.
Please give me a hope, I think this "bug" is really a big pity, if xbmc could manages folders with a lot of thumbnails will be PERFECT.

Tell me that you, developers, team... will check this "problem", if xbmc-team achieve to play movies in 720p in a xbox is really probably do a xbmc that manages folders with a lo of thumnails.

Thanks a lot and if i could doa nything to help you with this, tell it me.

Byes.

[jmarshall]

See if it happens on a win32, linux, or os x machine. If not, there's not a lot we're going to do about it short-term at least, as that is where our focus lies.

[itombs]

See if it happens on a win32, linux, or os x machine. If not, there's not a lot we're going to do about it short-term at least, as that is where our focus lies.

Ok, tomorrow I see if it happens first on a win32 and if I can on a linux system.
I remember that in win32 it happened but, obviusly, in minor measure because my PC is 3 GHz and Xbox is 700 MHz but, im think if the "problem" exists or the implementation of thumbtails system is slow on xbox the same "error" is on the other ports too but, when the port runs into a more powerfully machine, this obvius less.

Also, if developers are working on win32, linux, or os x ports of xbmc, is very very important to improve the system thumbtail because the use of thumbnails are increase day by day with new skins and new views and people day by day are using xbmc like database view of theirs movies and music discs and well, its rare to have 3.000 movies in a database but, is it rare to have 3.000 music disc? I think no.

Now, thinking about the "problem" of manage thumbtails Ild know if xbmc do the stat() in case the folders have a .tbn because in this case a possible solution is keep thumbtails onto the movie folder as Matrix.tbn etc etc
Can you tell me exactly how you make the stat()? and, XBMC is open source? Ild like see the code of thumbnail system because I think that the problem could be in the function that makes the hash and search thumbnails of folders when xbmc list a folder.

Thanks a lot and sorry if i am insistent but im sure that a lot of people will need this improve in future, win32, linux and os x ports are very differents in the "portability" and access, a xbox is now 50$ and its PERFECT like Media Center with xbmc, a lot of people use xbmc a xbox, I think xbox users are more than the other 3 ports users but, is true, the other ports have more power and possibilitys but i think are complementarys, i have win32 xbmc and xbox xbmc and ALWAYS i use xbox xbmc and when i travel i carry my xbox wih the xbmc and, like me, a lot of people so please works on win32, linux an os x ports but dont forget xbox port, at least in this necessary things.


Tomorrow i will tell you if win32 port have the same behaviour with folders with a lot of thumbnails.

Byes.

[itombs]

Hello, i tested if it happens on win32 with a Quad 2,4 Ghz with 4 Gbs Ram and time wasted to return /video/ from /video/matrix is very short but, its normal, the machine is very very powerfull.
If the code is the same on xbox and win32 port, in a PC slower time probably will be longer.
So, is not probably an update of thumbnail system in a no-far future?
Bye.