[pieh]

Hi,
this is continuation of pilluli's thread.

Short overview:
This patch allow XBMC to check if library items are available / accessible when we navigate through libraries. This is useful for people who store their media on "not always turned on" devices - such as: external portable hard drives or network shares on normal desktop computers/notebooks.

Details:
Information about availability of item is storem in CFileItem's bool property "unavailable". This way both skins and code itself has easy access to it.
For the need of this feature i created abstract class CAvailabilityChecker with abstract method bool virtual Check(CFileItem *pItem) = 0; (return true if pItem "unavailable" property has changed). For now only Video items checking is available (through CVideoAvailabilityChecker class inheriting from CAvailabilityChecker). These classes reside now in ThumbLoader.h.

Modifications for CVideoThumbloader class where main work of this feature is done:

• Added protected members:
  • CVideoAvailabilityChecker m_AvailChecker; - class checking and updating CFileItem "unavailable" property, providing info if GUI update is needed
• Reseting count of items with change "unavailable" property
  
  Code:

  void CVideoThumbLoader::OnLoaderStart()
{
  m_AvailChecker.ResetCounter();
}
• If any item has changed "unavailable" property - update GUI
  
  Code:

  void CVideoThumbLoader::OnLoaderFinish()
{
  if (m_AvailChecker.AnyChanges())
  {
    CGUIMessage msg(GUI_MSG_NOTIFY_ALL, 0, 0, GUI_MSG_FILTER_ITEMS, 1);
    g_windowManager.SendThreadMessage(msg);
    m_AvailChecker.ResetCounter();
  }
}
• If user enabled this feature - availability checking is done
  
  Code:

  bool CVideoThumbLoader::LoadItem(CFileItem* pItem)
{
  // real file availability checking is done here - first lets check if user enabled this feature
  if (g_guiSettings.GetBool("videolibrary.fileavailabilitychecking"))
  {
    m_AvailChecker.Check(pItem);
  }

  //continue standard code
  if (pItem->m_bIsShareOrDrive
  ||  pItem->IsParentFolder())
    return false;
  
  [...]
• Logic used to determine what actualy CFileItem is:
  
  Code:

  if(pItem->m_bIsFolder)
{
  CVideoInfoTag *tag = pItem->GetVideoInfoTag();
  if(tag->m_iEpisode > 0)
  {
    // tvshow or tvshow's season
    if(tag->m_iSeason > -1)
    {
      // it's tvshow's season
    }
    else
    {
      // it's tvshow
    }
  }
}
else
  // it's either movie or tvshow's episode - we can simply check CFileItem->Exists()

Notes:
I would like to start that by default this feature is disabled - so XBMC doesn't do any additional work until user will enable this feature in settings -> video -> library.
As all "unavailable" info is stored in property, adding this feature doesn't force skinners to update skins to keep XBMC working. More or less anyway because without skin modification end user won't see any effects of availability checking. So enabling this feature with skin that doesn't support displaying unavailable information won't give any effects - unavailable items will still look like normal ordinary items.

Any comments are welcome!

Quick look on effects:
New item in Settings -> Video -> Library (disabled by default)


Modified Confluence skin with support of unavailability checking feature - if "Enable availability checks" is enabled in settings new option is visible - "Hide unavailable"
with patch v2 "Hide Unavailable" is change to "Hide unavailable" - lowercased to fit GUI conventiion


Modified Confluence skin with support of unavailability checking feature - if "Hide unavailable" is checked it filters out unavailable items


Modified Confluence skin with support of unavailability checking feature - TVShows example


Modified Confluence skin with support of unavailability checking feature - TVShow's Seasons example


Changelog:

• 2010-09-14 23:55
  • Patch v2d
    
  • Fixed: wrong item was focused when user was re-entering navigation window with Hide Unavailable toggle turned on
• 2010-09-11 23:28
  • Patch v2c
    
  • Added pilluli's changes: Not using cache values for checking file / directory existance
    
  • If only 1 season of tvshow is available, flatting tvshows is set to 'only if 1 season' and 'Hide unavailable' is toggled navigation GUI automaticly go to only available season but thinking it's still in season navigation to preserve navigation history. If flatting tvshows is disabled it will remove "All Episodes" item from Navigation leaving only 1 available season to pick.
    
• 2010-09-03 19:16
  • Patch v2b
    
  • Added new setting in AdvancedSettings:
    
    Code:

    <videolibrary>
  <availabilitycheckingbyfile>true / false</availabilitycheckingbyfile>
</videolibrary>
    by default set to false, if set to true - force checking files instead of just paths. Read pilluli's post in patch ticket to know when to use this setting.
    Note: file checking is slower than path checking, so if You don't need it - don't turn it on
    
  • Added 2 new methods to CVideoDatabase (needed to perform file checking)
    • bool GetFileNamesForTvShow(int idShow, std::vector<CStdString>& paths);
    • bool GetFileNamesForSeason(int idShow, int iSeason, std::vector<CStdString>& paths);
  • Encapsulated unavailable property changed in item list flag to CAvailabalityChecker
    
• 2010-09-03 02:07
  • Patch v2
    
  • Code reformated to use xbmc coding conventions
    
  • Partially merged hiding unavailable items into GetFilteredItems method (didn't want to put it in Video dedicated class as I have plans to add support for Music items too). Now I reuse GUI_MSG_FILTER_ITEMS msg to notify GUI.
    
  • Different approach aplied: instead of checking files, now paths are checked (suggested by jmarshall on patch ticket)
    
  • Added 2 methods in CVideoDatabase to improve availability checking performance (both based on bool GetPathsForTvShow(int idShow, std::vector<int>& paths) ):
    • bool GetPathsForTvShow(int idShow, std::vector<CStdString>& paths);
    • bool GetPathsForSeason(int idShow, int iSeason, std::vector<CStdString>& paths);
• 2010-09-01 16:04
  • Patch v1
  • First dirty working solution

Links:

• trac ticket http://trac.xbmc.org/ticket/10047
  
• patch v2d for xbmc http://wojtek.piechowiak.eu/mich/unavail..._v2d.patch
  
• patch for confluence skin http://wojtek.piechowiak.eu/mich/confluence_patch.patch
  
• icon http://wojtek.piechowiak.eu/mich/unavailable.png - need do be copied to addons/skin.confluence/media and then rebuild confluence skin

[takoi]

Do want! Why dont you create a patch and put it up on trac. Much easier for everyone

[pieh]

Do want! Why dont you create a patch and put it up on trac. Much easier for everyone

i will do it, when ill check it in every possible way it's working correctly - for now i have tested it just on local sources - so i can't say it's bug-free

i posted it to get some feedback to know if i should continue to devolope it or just leave it - for me it does the job in this form cause i dont use any network shares or stuff like that and to know if devs would like to support me with their suggestions about implemantation of this feature

edit:
or actually - You are right - if there are people who would like help me in doing it - check first post for links

changes to confluence skin are quite "not very nice looking" - i hope ill get support from skinners if this patch will be continued

[takoi]

ok. care to put it up here then so i dont have to go around editing a bunch of files. Also, can you include the gui part?

[pieh]

check first post for links

[pieh]

first post updated - check changelog

[grizzi]

In conjunction with this patch, is it possible to get XBMC to ping a mapped network drive, so that it doesn't ask to remove the path when you didn't have the external device turned on when u started your HTPC?

As an example, i have a local server for all my media, and this aint turned on all the time, so if i turn on my HTPC first, windows wont reconnect with my mapped drives before i refresh them, and XBMC can't do it by itself which aint very handy on a HTPC if u dont want to tab out of XBMC.

[pieh]

Well, this would be platform dependend I think.

As You said - mapped network drive has this flaw that either remote device must be turned on when windows is starting or You must refresh it manually. Maybe instead of using mapped drive You could just use normal network share in XBMC?

[AnalogKid]

In conjunction with this patch, is it possible to get XBMC to ping a mapped network drive, so that it doesn't ask to remove the path when you didn't have the external device turned on when u started your HTPC?

As an example, i have a local server for all my media, and this aint turned on all the time, so if i turn on my HTPC first, windows wont reconnect with my mapped drives before i refresh them, and XBMC can't do it by itself which aint very handy on a HTPC if u dont want to tab out of XBMC.

Don't use drive mapping, there's a nicer / better way.

Use symbolic links:

so

X:\My Virtual Media Folders\
X:\My Virtual Media Folders\Movies <---- symbolic link to remote drive
X:\My Virtual Media Folders\TV <---- symbolic link to remote drive
X:\My Virtual Media Folders\Music <---- symbolic link to remote drive


Then, create the sources in XBMC the usual way pointing to X:\My Virtual Media Folders\Movies etc.

THIS way, you can remap the Movies folder to a new drive without screwing up your library, AND you only need one drive letter for any number of remote drives (or even local ones for that matter).

[pilluli]

Hi grajen3,

I have added a patch to make flatten of TVshows (if only one season) work for unavailable items as it now works for watched items. If there are only files available for one season the tvshow is flatten.

You can find the patch in the trac ticket (only touches one file). I guess it will be perfect if you test it also and add it to the general unavailability patch. It works for me here.

However, murphy law stroke again and fixing the flatten tvshow thing I found another bug in the unavailability patch. Well actually it does not seem to be in it but triggered by it. Some of my tvshows show as unavailable when they really are available. This happens in the tvshow view only and not in the episode view. So for instance a tvshow with 5 episodes (all available) shows as unavailable in the tvshow list but as I click on it it shows available all 5 episodes. I am using checkbyfile and not checkbydirectory. Also notice that some tvshows do work and some don't.

I still haven't figured out really why but I have check a little bit the code and it seems as if using the cache is causing the problem. If I change the line 445 of ThumbLoader.cpp from:

Code:

if (directory.Exists())

to

Code:

if (directory.Exists(false))

It works. Again, I don't know why some tvshows work and other don't or why some are in the cache and other aren't Reading the cache code there seems to be some comments about fixing this kind of behavior but haven't gone further than that...

Have you noticed something like this?

[pieh]

Hi, im now on kinda holidays - im visitting my friend so I don't rly work on this since sunday, but i remember that i tried doing what u did - adding additional loop in DirectoryNodeSeasons and it didn't rly work for me (I thought it was caused by the fact that actual file/path checking was done in background thread and therefore unavailable properties weren't set yet when GetContent from DirectoryNodeSeasons was called)

I'll take a look on it again when i'll be back at home.

Thanks for Your input pilluli!

[pilluli]

Hi, im now on kinda holidays - im visitting my friend so I don't rly work on this since sunday, but i remember that i tried doing what u did - adding additional loop in DirectoryNodeSeasons and it didn't rly work for me (I thought it was caused by the fact that actual file/path checking was done in background thread and therefore unavailable properties weren't set yet when GetContent from DirectoryNodeSeasons was called)

Note that I am using checking by file not directory. Maybe that also affects it. I thought the same as you but in all my tests the unavailable property is already set and ready to use...

Have a nice holidays!! :-)

[pieh]

Back from holidays

Hi grajen3,

I have added a patch to make flatten of TVshows (if only one season) work for unavailable items as it now works for watched items. If there are only files available for one season the tvshow is flatten.

You can find the patch in the trac ticket (only touches one file). I guess it will be perfect if you test it also and add it to the general unavailability patch. It works for me here.

I tested Your patch and as You said - it works. I thought I did exactly same code as You (with 1 change that i will comment later) and it didn't work ... But now magicly it does work That 1 change in DirectoryNodeSeasons.cpp is change in if sentence:

Code:

from:
  if (!bFlatten && g_guiSettings.GetBool("videolibrary.fileavailabilitychecking"))

to:
  if (!bFlatten && g_settings.m_bMyVideoShowUnavailableMode)

that's just to enable it when 'Hide unavailable' is toggled and not when just general feature is enabled, but that is just minor change.

However, murphy law stroke again and fixing the flatten tvshow thing I found another bug in the unavailability patch. Well actually it does not seem to be in it but triggered by it. Some of my tvshows show as unavailable when they really are available. This happens in the tvshow view only and not in the episode view. So for instance a tvshow with 5 episodes (all available) shows as unavailable in the tvshow list but as I click on it it shows available all 5 episodes. I am using checkbyfile and not checkbydirectory. Also notice that some tvshows do work and some don't.

I still haven't figured out really why but I have check a little bit the code and it seems as if using the cache is causing the problem. If I change the line 445 of ThumbLoader.cpp from:

Code:

if (directory.Exists())

to

Code:

if (directory.Exists(false))

It works. Again, I don't know why some tvshows work and other don't or why some are in the cache and other aren't Reading the cache code there seems to be some comments about fixing this kind of behavior but haven't gone further than that...

Have you noticed something like this?

I check my patch only my development machine so i don't have big library and didn't notice yet such behaviour, but anyway I think it's logical to not use cached values - as we want to check if the item is available in this moment and not if it was available when value was cached, right?

Just wonder if there is anything more we could add here or I can move on to work on Music Library part.

[pilluli]

Hi,

I check my patch only my development machine so i don't have big library and didn't notice yet such behaviour, but anyway I think it's logical to not use cached values - as we want to check if the item is available in this moment and not if it was available when value was cached, right?

Yes, i think it is better to not use cache, at least in the checkingbyfile mode.

I found another small bug, when browsing movies with hide unavailable set to true. If I start playing a movie (somehow at the end of the list, it should have some unavailable movies before it), when I stop the movie the focus does not return to the same movie.

About the flatten thing, you are right. it does not work. Flatten checks availability before it is actually set. To be honest, I am slowly concluding again that the only way to solve this is either make the checking synchronous or store the availability in the db for future references...
well, or just live without it, flatten and unavailable do not talk to each other and it is just one click more anyways

[pieh]

Hi,



Yes, i think it is better to not use cache, at least in the checkingbyfile mode.

I found another small bug, when browsing movies with hide unavailable set to true. If I start playing a movie (somehow at the end of the list, it should have some unavailable movies before it), when I stop the movie the focus does not return to the same movie.

About the flatten thing, you are right. it does not work. Flatten checks availability before it is actually set. To be honest, I am slowly concluding again that the only way to solve this is either make the checking synchronous or store the availability in the db for future references...
well, or just live without it, flatten and unavailable do not talk to each other and it is just one click more anyways

actually im doing some progress on different approach to flatting stuff - during checking I check if we browse TV seasons and if there is only 1 season. in thumbsloader's onLoaderFinish I check if flatting is enable and if just 1 season is available and I update GUI to flatten. The problem is that if I do this that way going to parent directory keeps falling into loop - im in tvshow's seasons navigation, if just 1 season is available automaticly turn to episodes navigation, but when i go back to parent directory i go to tvshow's seasons navigation will again automaticly turn to episodes - need to figure out history/gotoparentdirectory stuff to make it work. We can't leave it that way - if user has enabled flatten that it should work always - we can't say Flatten is working (but not with unavailability stuff)

I don't think that checking availability synchronous is the solution here - to much work to do in large libraries, so blocking context will be painful for end user.

Database entries would be nice, but imo just to store last availability status - but basing on database data to hide/show unavailability won't be always 100% accurate

Cheers