Pictures database and library mode; scan EXIF and IPTC metadata tags into a SQL db

  Thread Rating:
  • 2 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Thread Closed
fungify Offline
Senior Member
Posts: 146
Joined: Jul 2010
Reputation: 2
Post: #301
(2012-03-29 10:12)Xycl Wrote:  Yes, an excluded directory is not scanned.
My Laptop scans several pictures per second (i7 2620m) but the pictures have a resolution of only 5 MPixels.
I know that the larger the pictures are the longer it will take to scan them.

EDIT: And I know that even videos are scanned. This takes a very long time.
My photo collection is around 15gb in some just under 7000 files (just cleaned some dups). Scanning the collection for the first time with newest version of the addon took more than 12 hours on a dedicated Core I3. It seems to me that it didn't take this long on previous versions. The only difference I could think of was that this time I excluded some folders from scanning - which as you say should actually decrease the time.

But it might just have been a glitch at my end.

Will start translating.

FFY
find
fungify Offline
Senior Member
Posts: 146
Joined: Jul 2010
Reputation: 2
Post: #302
Just finished rescanning all pictures once again with (movie)folders excluded which took around 11 hours.

Found a bug - which might help explain why I can't see the pictures related to a person if the persons has a large number of images attached.

My image collection is just under 7000 images. I have persons tagged on around 3500, which leaves 3500 without any people tags. When selecting Browse by Persons, the count of images for each person seems correct, but the count for Pictures without persons is way off. In my case it claims that 6870 pictures have no persons tagged - even though just one of my persons is tagged in 1830 of the 7000 images (yes - I am a proud father with an over active camera triggerSmile)

Would someone please test and confirm?

FFY
find
Xycl Online
Fan
Posts: 362
Joined: Feb 2012
Reputation: 7
Location: Germany
Post: #303
(2012-03-30 01:02)fungify Wrote:  Just finished rescanning all pictures once again with (movie)folders excluded which took around 11 hours.

Found a bug - which might help explain why I can't see the pictures related to a person if the persons has a large number of images attached.

My image collection is just under 7000 images. I have persons tagged on around 3500, which leaves 3500 without any people tags. When selecting Browse by Persons, the count of images for each person seems correct, but the count for Pictures without persons is way off. In my case it claims that 6870 pictures have no persons tagged - even though just one of my persons is tagged in 1830 of the 7000 images (yes - I am a proud father with an over active camera triggerSmile)

Would someone please test and confirm?

FFY

There's a typo in the code.
The number behind pictures without pesons is the number for pictures without categories.
But the displayed pictures are correct.
I'll correct the number in the next release.

No log no help.
How to submit a log file: XBMC-Wiki

Main page: https://github.com/Xycl
Repository: Xycl Repository
find
fungify Offline
Senior Member
Posts: 146
Joined: Jul 2010
Reputation: 2
Post: #304
Good to hear. I've been looking at the logfiles for what happens when I select a person with a lot of images attached and this fails. This is what the log says:

Code:
01:12:52 T:5048   ERROR: C:\Program Files (x86)\XBMC\system\python\Lib\urllib.py:1224: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal res = map(safe_map.__getitem__, s)

01:12:52 T:5048    INFO: -->Python script returned the following error<--
01:12:52 T:5048   ERROR: Error Type: <type 'exceptions.KeyError'>
01:12:52 T:5048   ERROR: Error Contents: (u'\xe5',)

01:12:52 T:5048   ERROR: Traceback (most recent call last):  
File "C:\Users\Administrator\AppData\Roaming\XBMC\addons\plugin.image.mypicsdb\default​.py", line 1399, in <module> m.show_pics()
File "C:\Users\Administrator\AppData\Roaming\XBMC\addons\plugin.image.mypicsdb\default​.py", line 1332, in show_pics fanart = xbmcplugin.getSetting(int(sys.argv[1]),'usepicasfanart')=='true' and join(path,filename) or picfanart
File "C:\Users\Administrator\AppData\Roaming\XBMC\addons\plugin.image.mypicsdb\default​.py", line 206, in addPic quote_plus(picpath),
File "C:\Program Files (x86)\XBMC\system\python\Lib\urllib.py", line 1230, in quote_plus s = quote(s, safe + ' ')
File "C:\Program Files (x86)\XBMC\system\python\Lib\urllib.py", line 1224, in quote res = map(safe_map.__getitem__, s) KeyError: (u'\xe5',)

01:12:52 T:5048    INFO: -->End of Python script error report<--

Larger fragment of the log here: http://pastebin.com/KHDdwAEN

So the error seems to be somewhat related to the unicode conversion?

FFY
(This post was last modified: 2012-03-30 13:27 by fungify.)
find
fiveisalive Offline
Fan
Posts: 371
Joined: Jul 2009
Reputation: 0
Location: United States
Post: #305
(2012-03-27 20:55)Xycl Wrote:  
(2012-03-27 19:33)fiveisalive Wrote:  I wonder if you'd consider putting up your repo:

http://www.xycl.de/xbmc/repository.xycl....dharma.zip

on the list at:

http://wiki.xbmc.org/index.php?title=Uno...positories

that way the Repositories Installer Add-on can more easily install it. I had to manually search for the link to the zip file from your site, manually download it, then install it within XBMC. Thanks.

Done

Doesn't show up in the list of repos. I think the addon expects a URL in column 1.
find
Xycl Online
Fan
Posts: 362
Joined: Feb 2012
Reputation: 7
Location: Germany
Post: #306
(2012-03-30 12:48)fungify Wrote:  So the error seems to be somewhat related to the unicode conversion?

FFY

I found the unicode error and the reason for the progress dialog when including/excluding directories.
The new version will be available in 3-4 days .
(2012-04-01 05:55)fiveisalive Wrote:  Doesn't show up in the list of repos. I think the addon expects a URL in column 1.

I use a web server and not git or subversion to host the zip.
Due to security reasons directory listing is disallowed.
Therefore I cannot provide an URL for column 1.

No log no help.
How to submit a log file: XBMC-Wiki

Main page: https://github.com/Xycl
Repository: Xycl Repository
(This post was last modified: 2012-04-01 13:04 by Xycl.)
find
fiveisalive Offline
Fan
Posts: 371
Joined: Jul 2009
Reputation: 0
Location: United States
Post: #307
(2012-04-01 13:00)Xycl Wrote:  
(2012-04-01 05:55)fiveisalive Wrote:  Doesn't show up in the list of repos. I think the addon expects a URL in column 1.

I use a web server and not git or subversion to host the zip.
Due to security reasons directory listing is disallowed.
Therefore I cannot provide an URL for column 1.

I don't think the URL has to be related to the actual .zip file, it could just be to your homepage.
find
fungify Offline
Senior Member
Posts: 146
Joined: Jul 2010
Reputation: 2
Post: #308
(2012-04-01 13:00)Xycl Wrote:  I found the unicode error and the reason for the progress dialog when including/excluding directories.
The new version will be available in 3-4 days .
You're the man!

FFY
find
pedr0 Offline
Junior Member
Posts: 23
Joined: Feb 2011
Reputation: 0
Post: #309
Just finished reading this thread. Very excited by what this promises. Shame solexalex disappeared last March but thanks Xycl for breathing life into it.

'scuse the ignorance, but I'm not sure what repo to add. Should I add Xycl's abandoned repo or the passion repo? Does the passion repo exclude a required add-on? And how do I find Xycl's repo if the missing URL means that the repo doesn't show up in the list?

Cheers!
find
fiveisalive Offline
Fan
Posts: 371
Joined: Jul 2009
Reputation: 0
Location: United States
Post: #310
(2012-04-05 00:19)pedr0 Wrote:  Just finished reading this thread. Very excited by what this promises. Shame solexalex disappeared last March but thanks Xycl for breathing life into it.

'scuse the ignorance, but I'm not sure what repo to add. Should I add Xycl's abandoned repo or the passion repo? Does the passion repo exclude a required add-on? And how do I find Xycl's repo if the missing URL means that the repo doesn't show up in the list?

Cheers!

You can just visit http://wiki.xbmc.org/index.php?title=Uno...positories in your web browser and manually download the .zip file for the repo (it's towards the bottom of the list). I'm going to add a top-level URL to see if it makes it show up in the Repositories Installer Add-on.

Update: I took the liberty of adding the link to: http://www.xycl.de/de/xbmc/addons-und-programme.html to the first column, now the two Xycl repos should show up in the Repositories Installer Add-on.
(This post was last modified: 2012-04-05 19:02 by fiveisalive.)
find
fmronan Offline
Fan
Posts: 600
Joined: Dec 2010
Reputation: 4
Post: #311
You can install by repo passion in xbmc or with the zip file too
http://passion-xbmc.org/addons/?Page=Vie...e.mypicsdb
Is it possible to add property for recently or random pictures in the home, like recently movie or other media?
(This post was last modified: 2012-04-05 19:11 by fmronan.)
find
mgmartin Offline
Junior Member
Posts: 4
Joined: Mar 2012
Reputation: 0
Post: #312
Thanks for putting this addon together, it is filling a distinct hole in XBMC's capabilities.

I've noticed that the browse_folder function scanpath.py tends to bomb out at the stat functions if it encounters any files or folders without read permissions.
This is probably OS-specific and partially a user issue...
HOWEVER - In the name of bulletproofing, would it be possible to insert some exception trapping per-file or per-folder ?
I crudely hacked some try/except blocks around the code to enable it to scan my pics, but this might be handled more elegantly by the original authors.

Thanks again !
find
Xycl Online
Fan
Posts: 362
Joined: Feb 2012
Reputation: 7
Location: Germany
Post: #313
(2012-04-05 18:53)fiveisalive Wrote:  Update: I took the liberty of adding the link to: http://www.xycl.de/de/xbmc/addons-und-programme.html to the first column, now the two Xycl repos should show up in the Repositories Installer Add-on.

Thanks.
It works.
(2012-04-05 19:09)fmronan Wrote:  Is it possible to add property for recently or random pictures in the home, like recently movie or other media?

You want to go directly into "Uploads from last n days" or "The n latest pictures"?
PHP Code:
ActivateWindow(Pictures,"plugin://plugin.image.mypicsdb/?method='recentpicsdb'&viewmode='view'&action='showpics'",return)
or
ActivateWindow(Pictures,"plugin://plugin.image.mypicsdb/?method='lastpicsshooted'&viewmode='view'&action='showpics'",return) 

No log no help.
How to submit a log file: XBMC-Wiki

Main page: https://github.com/Xycl
Repository: Xycl Repository
(This post was last modified: 2012-04-06 20:06 by Xycl.)
find
fmronan Offline
Fan
Posts: 600
Joined: Dec 2010
Reputation: 4
Post: #314
good questionFrown
the last added , i suppose
find
Xycl Online
Fan
Posts: 362
Joined: Feb 2012
Reputation: 7
Location: Germany
Post: #315
See my updated reply.

No log no help.
How to submit a log file: XBMC-Wiki

Main page: https://github.com/Xycl
Repository: Xycl Repository
find
Thread Closed