•   
  • 1
  • 20
  • 21
  • 22
  • 23(current)
  • 24
  •   
Release WatchedList - service to automatically save/restore watched state
Finally I rescraped all movies and put manually watched state and date in each files
5 hours of works
Reply
(2019-06-22, 08:32)schapplm Wrote:
(2019-06-05, 17:39)Tobby Wrote: Hi. Every time I run WatchedList on one of my TVs I get this error "WatchedList Error: Doing a database backup (Invalid argument)" and it doesn't create a new backup zip. Is this a known issue? Couldn't find anything in the thread about it.
The first one is only the displayed message in Kodi. In the log, you have
 
Code:
2019-06-05 17:27:41.642 T:9706 ERROR: service.watchedlist: database_backup: Error creating database backup /storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/userdata/addon_data/service.watchedlist/dbcopy/20190605_172741-watchedlist.db.zip: (22) Invalid argument
In the code for creating the zip file, this error is raised either as ValueError (e.g. "timestamps before 1980") or IOError (e.g. "permission denied"). Can you please check, if you have write permission on the Android device and if you have files with time stamp before 1980?
Same problem here on my Android device. Write permissions is not the issue. All addons can write to the internal storage and external (usb) hard drive.

Your suggestion is to check whether there are "files" with a timestamp before 1980.

Please explain "files" to me. Which files/folders?
Reply
The file in question should only be "watchedlist.db" from ".kodi/userdata/addon_data/service.watchedlist/".
Other possible problems may be that the folder "dbcopy" can not be created.
I can not reproduce the situation on my hardware so I can only guess.
If you use dropbox synchronization, then you can disable the database zip backup in the settings.
Running without the backup should also be possible since the addon proved to be very stable the last years.
My addon: service.watchedlist
My HTPC: Kodi 18.3; Ubuntu 16.04 LTS x64; AMD A6-6400K; MSI FM2-A55M-E33; 4GB Kingston HyperX RAM; Streacom FC8B Evo with 120W picoPSU
Reply
@schapplm, I can see the watchedlist.db in the addon_data folder.

Let me explain a little further. I'm running Kodi 18.3 on an Android device. I've set the number of backups to 1.

I can see the backup file in the addon_data folder: yyyymmdd_hhmmss-watchedlist.db.zip. So it appears that it does the backup task, but the funny thing is, that the zipfile is empty.

I will set the number of backups to 0, if you're unable to reproduce the problem and can't fix it.

The backup addon from the Kodi repository does the backup job flawlessly. Maybe you could use a part of the code of this addon.
Reply
The backup addon seems to use the same module zipfile, but uses a different method writestr instead of write in my addon. Since my code is only a two-liner, I would prefer to adapt it.
There were already some python issues with another module on Android devices:
(2017-12-31, 17:30)schapplm Wrote: The Dropbox authentification mechanism seems to be broken for Android devices. LibreElec seems to be a full linux distribution, but perhaps with a reduced set of python libraries. (I don't use it so I am not sure there).
Can you please check, if you can run the zip command in a normal linux terminal on your device?
Open a terminal, enter python and in the python terminal (with ">>>" on the beginning of the line) enter

import zipfile
zipfilename='/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/userdata/addon_data/service.watchedlist/test.zip'
dbpath='/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/userdata/addon_data/service.watchedlist/watchedlist.db'
with zipfile.ZipFile(zipfilename, 'w') as zf:
zf.write(dbpath, arcname='watchedlist.db', compress_type=zipfile.ZIP_DEFLATED)
This is the code run by the watchedlist addon. Is the zip file created with this code?
My addon: service.watchedlist
My HTPC: Kodi 18.3; Ubuntu 16.04 LTS x64; AMD A6-6400K; MSI FM2-A55M-E33; 4GB Kingston HyperX RAM; Streacom FC8B Evo with 120W picoPSU
Reply
Still using this addon to this day - thanks. :-)

Besides that, i'm still searching for a reliable way to make a user friendly "To Watch"-Watchlist with movies i want to watch in KODI.

Thinking about something similar like in your addon:
One database file with imdb numbers of movies which are part of the so called "To watch"-list. Adding and removing could be completely manual by a context menu item "Add to Watchlist" "Remove from Watchlist". (No automatic removal from the list after watching the movie to keep it simple for now.)

Is that something that crossed your mind as additional functionality or separate addon? (Not an easy task and probably too time consuming, just wondering)

Greetings
A Confluence ZEITGEIST (Modernized Confluence Skin)
Reply
(2019-08-17, 11:27)schapplm Wrote: The backup addon seems to use the same module zipfile, but uses a different method writestr instead of write in my addon. Since my code is only a two-liner, I would prefer to adapt it.
There were already some python issues with another module on Android devices:
(2017-12-31, 17:30)schapplm Wrote: The Dropbox authentification mechanism seems to be broken for Android devices. LibreElec seems to be a full linux distribution, but perhaps with a reduced set of python libraries. (I don't use it so I am not sure there).
Can you please check, if you can run the zip command in a normal linux terminal on your device?
Open a terminal, enter python and in the python terminal (with ">>>" on the beginning of the line) enter

import zipfile
zipfilename='/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/userdata/addon_data/service.watchedlist/test.zip'
dbpath='/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/userdata/addon_data/service.watchedlist/watchedlist.db'
with zipfile.ZipFile(zipfilename, 'w') as zf:
zf.write(dbpath, arcname='watchedlist.db', compress_type=zipfile.ZIP_DEFLATED)
This is the code run by the watchedlist addon. Is the zip file created with this code? 
Hi @schapplm , I'm not sure if I have enough skills to achieve this, but I''ll give it a try and will let you know my findings.
Reply
(2019-08-18, 13:04)axbmcuser Wrote: way to make a user friendly "To Watch"-Watchlist[...]Is that something that crossed your mind as additional functionality or separate addon? (Not an easy task and probably too time consuming, just wondering)
I also have thought about this issue from time to time.
Creating a database with unique identifiers from within Kodi (e.g. imdb number) would require scraping the media somehow or obtaining the identifiers on other ways, so this would mostly be suitable for media that is already in the database.
There are already a few solutions for this. E.g., the trakt addon or lists in streaming addons like in the Netflix addon. There is also a "WatchList" addon, but I am not sure what it does.
I think you should start a discussion thread on this where you specify your requirements and compare them against existing solutions.
My addon: service.watchedlist
My HTPC: Kodi 18.3; Ubuntu 16.04 LTS x64; AMD A6-6400K; MSI FM2-A55M-E33; 4GB Kingston HyperX RAM; Streacom FC8B Evo with 120W picoPSU
Reply
(2019-08-18, 19:54)schapplm Wrote: Creating a database with unique identifiers from within Kodi (e.g. imdb number) would require scraping the media somehow or obtaining the identifiers on other ways, so this would mostly be suitable for media that is already in the database.
I always just thought about a "To Watch List" for items which already are in your KODI movie database. Maybe even just for movies.
I think including "all movies and espisodes of the world" just would bloat the idea beyond practicality. (Besides the nightmare of additional dev workload :-) )
 
(2019-08-18, 19:54)schapplm Wrote: There are already a few solutions for this. E.g., the trakt addon or lists in streaming addons like in the Netflix addon. There is also a "WatchList" addon, but I am not sure what it does.
I think you should start a discussion thread on this where you specify your requirements and compare them against existing solutions. 
Yes, thats a good idea. I already tried the existing possibilities. "Watchlist" is not a "To Watch List" addon. Had hopes for this one, but nope.
I'll ping you in the new thread when i have the time to create it. Thanks for your reply. :-)
A Confluence ZEITGEIST (Modernized Confluence Skin)
Reply
(2019-08-17, 11:27)schapplm Wrote: The backup addon seems to use the same module zipfile, but uses a different method writestr instead of write in my addon. Since my code is only a two-liner, I would prefer to adapt it.
There were already some python issues with another module on Android devices:
(2017-12-31, 17:30)schapplm Wrote: The Dropbox authentification mechanism seems to be broken for Android devices. LibreElec seems to be a full linux distribution, but perhaps with a reduced set of python libraries. (I don't use it so I am not sure there).
Can you please check, if you can run the zip command in a normal linux terminal on your device?
Open a terminal, enter python and in the python terminal (with ">>>" on the beginning of the line) enter

import zipfile
zipfilename='/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/userdata/addon_data/service.watchedlist/test.zip'
dbpath='/storage/emulated/0/Android/data/org.xbmc.kodi/files/.kodi/userdata/addon_data/service.watchedlist/watchedlist.db'
with zipfile.ZipFile(zipfilename, 'w') as zf:
zf.write(dbpath, arcname='watchedlist.db', compress_type=zipfile.ZIP_DEFLATED)
This is the code run by the watchedlist addon. Is the zip file created with this code?
Hi,

I've copied and pasted the code into the terminal to avoid typos. Hopefully I did not made an mistake. Here's a screenshot of the result (Indentation error: expected an indented block):

Image
Reply
The last line "zf.write..." has to be preceeded by two or four whitespaces ("indentation"). Unfortunately the code was not displayed correctly at first in my post.
My addon: service.watchedlist
My HTPC: Kodi 18.3; Ubuntu 16.04 LTS x64; AMD A6-6400K; MSI FM2-A55M-E33; 4GB Kingston HyperX RAM; Streacom FC8B Evo with 120W picoPSU
Reply
(2019-04-17, 21:40)HeresJohnny Wrote: I would like to add some observations related to the imdb ID.
[...] Unfortunately, the default scrapers in Kodi for movies and tv shows are now set to TMDB. As I already commented in the TMDB scraper thread, items which are scraped with that scraper have existing IMDB nos. forcibly removed and only the TMDB no. is added/kept in the db. The Universal Movie scraper on the other hand doesn't add the TMDB ID.
Karellen Wrote: "TheMovieDB-TVShows" replaced "TheTVDB" as the default scraper during Nov 2018. If you originally scraped with TVDB and are now rebuilding your library with NFO Files, ensure the correct scraper is active.

This messed up the situation a little for my video library (mySQL):
SELECT tvshow.idShow,tvshow.c12, uniqueid.type, uniqueid.value, tvshow.c00 FROM tvshow INNER JOIN uniqueid ON tvshow.c12 = uniqueid.uniqueid_id;
Fields: internal number of tv show, reference to uniqueid, type of id, id of the provider, title
Code:
169 22829 tmdb 60741 Ascension
170 22896 unknown 247897 Homeland
171 23551 unknown 121361 Game of Thrones
172 23588 tvdb 71663 Die Simpsons
I had a deeper look into the tvdb vs tmdb issue and it seems to have bigger implications on the WatchedList addon than I thought.
There is a table `uniqueid` that stores the id of the databases TheTVDB (tvdb) and TheMovieDB (tmdb) in the Kodi Video Library:
Code:
http://localhost:8080/jsonrpc?request={"jsonrpc": "2.0", "method": "VideoLibrary.GetTVShows", "params": {"properties": ["title", "imdbnumber", "uniqueid"], "sort": { "order": "ascending", "method": "title" } }, "id": 1}
{"imdbnumber":"10283","label":"Archer","title":"Archer","tvshowid":164,"uniqueid":{"tmdb":"10283","tvdb":"110381"}}
The key `idShow` in the tvshows table of the WatchedList database was only referring to the field `imdbnumber` of the JSON output, which from retroperspective is not a good choice for a unique identifier of watched status. It might be that I started with API version 4 instead of version 6, where the field uniqueid was introduced.

So the situation for me and the average user of this addon is that we have lots of entries in the database referring to the tvdb IDs which were scraped and watched over the last years. For new media (scraped differently) this data can not be used and duplicate entries are created. E.g. in my WatchedList database I now have "The Last Kingdom" as "63333" (tmdb) and "298566" (tvdb); at least with identical playcount data.
(2019-04-18, 06:36)schapplm Wrote: As for the tv shows I also support this to be done on scraper/database/json-API-level.
I have to think about a solution how to fix it at the end of this addon, since the data is now duplicate and old data can not be used. Creating an automatic upgrade mechanism of the table will take time, however.
My addon: service.watchedlist
My HTPC: Kodi 18.3; Ubuntu 16.04 LTS x64; AMD A6-6400K; MSI FM2-A55M-E33; 4GB Kingston HyperX RAM; Streacom FC8B Evo with 120W picoPSU
Reply
(2019-08-24, 16:01)schapplm Wrote: The last line "zf.write..." has to be preceeded by two or four whitespaces ("indentation"). Unfortunately the code was not displayed correctly at first in my post.
Can you edit your post and provide the correct code please? I prefer to copy and paste the code, instead of typing it.
Reply
@schapplm I'm really glad you had some time to look into the problem of tv show IDs. IMHO you can only tackle the symptoms but to find a cure for the root problem I think something must be done to harmonize scrapers.

Soomeone in the TMDB scraper thread pointed out that there was to readily available mechanism to identify an IMDB ID through TMDB. However, if such an ID already exists in the database - even in another place - maybe it could be read from there. In the end, the IMDB ID might even be used as a central value to avoid having to rescrape complete TV shows to change from TMDB to TheTVDB or vice versa. In the worst of cases, user could be asked for a IMDB ID while scraping when ambiguities are found.
Reply
(2019-08-24, 22:27)HeresJohnny Wrote: In the end, the IMDB ID might even be used as a central value to avoid having to rescrape complete TV shows to change from TMDB to TheTVDB or vice versa. In the worst of cases, user could be asked for a IMDB ID while scraping when ambiguities are found.
My approach would be to just use `uniqueid.tmdb` instead of `imdbnumber` as can be seen in the JSON output. (Keep in mind that `imdbnumber` is just the name of the field and does not refer to imdb ...). Instead of course uniqueid.imdb could be used. But this has to be created by the scraper, which is not the case at the moment. I asked in the tmdb thread for advice how to proceed in that matter.
My addon: service.watchedlist
My HTPC: Kodi 18.3; Ubuntu 16.04 LTS x64; AMD A6-6400K; MSI FM2-A55M-E33; 4GB Kingston HyperX RAM; Streacom FC8B Evo with 120W picoPSU
Reply
  •   
  • 1
  • 20
  • 21
  • 22
  • 23(current)
  • 24
  •   
 
Thread Rating:
  • 3 Vote(s) - 5 Average



Logout Mark Read Team Forum Stats Members Help
WatchedList - service to automatically save/restore watched state53