• 1
  • 18
  • 19
  • 20
  • 21(current)
  • 22
Release [Module] youtube-dl - multi-site playable URL resolver
I'm having the same issue since upgrading to Kodi 19. Whenever I click the 'download currently playing video' button nothing happens.

This is what the log says.
Quote:2021-06-14 17:13:54.507 T:9936 ERROR : EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <class 'ModuleNotFoundError'>
Error Contents: No module named 'YoutubeDLWrapper'
Traceback (most recent call last):
File "C:\Users\stefm\AppData\Roaming\Kodi\addons\script.module.youtube.dl\lib\main.py", line 56, in onClick
self.main.downloadPlaying()
File "C:\Users\stefm\AppData\Roaming\Kodi\addons\script.module.youtube.dl\lib\main.py", line 99, in downloadPlaying
from lib import YDStreamExtractor
File "C:\Users\stefm\AppData\Roaming\Kodi\addons\script.module.youtube.dl\lib\YDStreamExtractor.py", line 6, in
import YoutubeDLWrapper
ModuleNotFoundError: No module named 'YoutubeDLWrapper'
-->End of Python script error report<--
Reply
(2021-06-14, 20:32)Steph09 Wrote: I'm having the same issue since upgrading to Kodi 19. Whenever I click the 'download currently playing video' button nothing happens.

This is what the log says.
Quote:2021-06-14 17:13:54.507 T:9936 ERROR : EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <class 'ModuleNotFoundError'>
Error Contents: No module named 'YoutubeDLWrapper'
Traceback (most recent call last):
File "C:\Users\stefm\AppData\Roaming\Kodi\addons\script.module.youtube.dl\lib\main.py", line 56, in onClick
self.main.downloadPlaying()
File "C:\Users\stefm\AppData\Roaming\Kodi\addons\script.module.youtube.dl\lib\main.py", line 99, in downloadPlaying
from lib import YDStreamExtractor
File "C:\Users\stefm\AppData\Roaming\Kodi\addons\script.module.youtube.dl\lib\YDStreamExtractor.py", line 6, in
import YoutubeDLWrapper
ModuleNotFoundError: No module named 'YoutubeDLWrapper'
-->End of Python script error report<--

Hi. Anyone looking into this? I have the same issue.

Roger J.
Reply
Now that youtube-dl's development has stopped since june (6+ months ago), can we get a new version of the addon that uses yt-dlp?
Yt-dlp is the successor to youtube-dl, it has frequent updates and it is even suggested by the devs of youtube-dl on its reddit and github pages.
Reply
Speak of the devil, youtube-dl was updated today! I hope we will see a new version of the addon soon.
Reply
All indications are that youtube-dl is dead. yt-dlp appears to be the successor.

I am working on an update to script.module.youtube.dl that will add yt-dlp and allow an addon to specify which lib to use. The default will be yt-dlp. The intention is to remove youtube-dl before very long since it's value is minimal. I leave it mostly to help test and migrate. Unless you depend upon some code specific to one of the implementations, the change should be transparent.

I hope to have this out in several weeks. My testing is limited, since I use only one call to simply download trailers and metadata. If anyone is interested in testing, please let me know.

Frank
Reply
In a similar discussion on github, I found out that yt-dlp is python3 only. This means that its to no use for distros that come with python2 only, e.g. libreelec 9.x with kodi 18. It simply can not run there. So, the change will not be that transparent for those users, as any addon that depends on it will stop working.
Reply
Seems to me yt-dlp is the only viable option going forward.

scott s.
.
Reply
Perhaps I should just ditch the youtube-dl and completely switch to the yt-dlp. I won't be able to thoroughly test the youtube-dl wrapper code. I tried testing with some of the existing addons, but the problems I have found so far seem to be in the addon and not youtube-dl.
Reply
Jim_p, although I'm not volunteering to do the work, depending upon the use case, it is probably technically possible to add bridge code to exec yt-dlp so that it can run under python 3 and communicate via file or pipe. No, I don't want that job. Depends upon how desperate one is.
Reply
Since it seems youtbe-dl command a library is dead, I am in the works of updating script.module.youtube.dl to use yt-dlp instead. It appears to work well. Unfortunately, yt-dlp does NOT support python 2. It seems that youtube-dl is nearly useless at this point, so I am planning to remove it. I could leave youtube-dl there and allow it to be configured by setting, or a passed argument, but there doesn't seem to be much point in it at this point.

I'm hoping to have this done soon. I have only tested basic operations, since my addon only needs to use a small portion of it's capabilities.
Reply
I would say that this transition does not affect me because, right now, I have no addon that uses youtube-dl. Well, playthis by anxdpanic does use it, but I use it like twice a year Sad
Reply
I tried in the past to get it working in Matrix.  It seemed to add to the d/l queue, but I couldn't get the queue to run.  (using Runscript() from the UI while playing video).

scott s.
.
Reply
Where I am at:

After much internal debate, I decided to leave script.module.youtube.dl alone and create script.module.yt-dlp in its place. I did this for several reasons:
  1. I do not like addons putting exported code in lib, but rather a child directory with something identifying it what it belongs to. I'm concerned that the namespace (top level python path) could get a bit cluttered.
  2. Along with the previous change, the imports had to change as well. Sigh. Sorry.
  3. There were some who were using an older version on python 2 kodi, and I suppose wanted the door open to continue updating it.
So all in all, I decided to reduce the amount of disruption.

Progress so far:
  1. The port to the new yt-dlp lib is not so bad.
  2. The problem is that there seem to be a backlog of problems in the YouTubeDL wrapper code. Many of these appear to be due to the Python 3 migration. I have found places where strings are being converted to hex and back when they don't need to. It is also common to not specify encoding options on file operations.
  3. My big accomplishment today was to be able to download something playing on Youtube using the YouTubeDL-Control app. Never mind that the file name is very long garbage, etc. The main thing is that it sort of worked.
The plan is to continue pecking at the problems. I'll undoubtedly release prior to everything being fixed because I'm not that familiar with the code. I actually use it by calling the yt-dlp code directly and ignoring the Youtube.dl wrapper code.  Still, I can see value in the wrapper, once it works right.

Working on this did bring up a recent concern that I have with Kodi addons. There is a bug which a draconian fix was put in. It impacts locale, etc. In particular, it broke unicode characters in filenames. The work around is easy enough (encode the file path on file open to utf-8). But this may nab a number of people. Time will tell. It also causes locale.getdefaultlocale and getPreferredEncoding to behave less than perfectly. UTF-8 still works, but there are some subtle edge cases.

I'll try to have a workable version within a week. Some day I'll get to release RandomTrailers.
Reply
Good Progress on migrating to yt-dlp for the upcoming script.module.yt-dlp addon. This are a bit of a mess and I have done very limited testing, but I am hoping to have something deliverable in several weeks. Since the addon name will be different, other addons will not have a nasty surprise and will have opportunity to test and get fixes before switching. I had to make more changes to the code (YoutubeDLWrapper, etc.) than I like, but I found several things that had to be changed.

Unfortunately I found some problems in yt-dlp & youtube-dl that are greatly aggravated by Kodi's utf-8 hack for Turkish (but the 'fix' impacts many codesets). Most likely, this will prevent you from downloading any file with names that are non-ASCII. There are circumstances where non-ASCII content may cause trouble as well. I started hacking yt-dlp to see how difficult it would be to fix, but it is a very sizable chunk. There is code to attempt to address it, probably put in for Unicode Migration, but there are bugs and inconsistencies that likely only show up in stress situations such as these.

For the curious: the problem is that there is encoding for the contents of a file, that we all know and love. There is also encoding for the file name, which depends upon what the OS allows. Nearly all OS's these days support utf-8 for both file names and content. During startup Python does some magic and set up default encodings for file names as well as regular strings. The hack for Kodi disabled these, causing the default encoding for file names to be ASCII. The default (nearly) can not be changed after startup, but the consequences of changing it are bad, so don't go down that route. The default encoding for strings is utf-8, but there are some warts on the locale.
Reply
@fbacher thanks for the detailed post. Are you still working on this?

I don't know about Kodi's utf-8 hack for Turkish exactly, but I do notice Kodi changing reasonable locale and filesystem encoding settings. In my addon I was able to download files with non-ASCII names (from yle, not youtube) and display them correctly within Kodi. I will give a quick explanation of my approach here in becomes useful.

Overview:
- Extract filename (and download link) into python string (unicode code points).
- Set download directory, using `xbmcvfs.translatePath()` if required.
- Append filename (utf-8 encoded) to complete download path.
- Download file
- Open filename (utf-8 encoded) and write downloaded data as binary data.
- Pass filename (utf-8 encoded) and path to Kodi as list item.

python:

from urllib.parse import urlencode

filename = "non-ääsci"
url = "example.xyz/video/12345"

# Download path
target_dir = xbmcvfs.translatePath("special://temp").encode("utf-8", "surrogateescape")
filepath = os.path.join(target_dir, filename.encode("utf-8", "surrogateescape"))

# Download remote file.
res = requests.get(url, stream=True)

# Open filename (utf-8 encoded) and write downloaded data as binary data.
with open(filepath, "ab") as file:
    for chunk in res.iter_content(chunk_size=1024**2):
        file.write(chunk)

# Create list item entry for downloaded file.
list_item = xbmcgui.ListItem(label=filename.encode("utf-8", "surrogateescape"))
# From memory it is best to give unicode to urlencode
attrs = {"local_file_playback": filepath.decode("utf-8", "surrogateescape")}
param_string = urlencode(attrs, encoding="utf-8", errors="surrogateescape")
callback_url =  f"plugin://{addon_id}/?{param_string}"
listing = (callback_url, list_item, False)

Obviously this code is incomplete, and requires setting  relevant `addon_id` and `_handle` and having a function to handle "local_file_playback" parameter to play the video.

In summary:
- Changes Kodi makes to Python locale are ignored by explicitly using utf-8 for filenames.
- Changes Kodi makes to Python filesystem encoding are ignored by writing the file as binary data. i.e. video file is not a text format so encoding is not an issue.
- Filenames and paths are passed to Kodi functions are utf-8 encoded bytes.


Extra notes:
- Python os functions accept either bytes or unicode strings: return type matches input type.
- Everything in the OS is bytes (eg. filenames), so use bytes as input to os functions.
- Not all bytes have valid unicode representations when decoded from utf-8.
- Since the underlying bytes can possibly be invalid, error handling is required..
  The surrogateescape error mode represents the invalid utf-8 bytes as reserved unicode code points.
  It is the only error mode in python that provides lossless handling of the invalid utf-8 bytes.
  e.g. b"\xff" == b"\xff".decode("utf-8", "surrogateescape").encode("utf-8", "surrogateescape")
- Many Kodi functions can accept either python unicode strings or utf-8 bytestrings, but they don't accept python unicode strings containing reserved code points (i.e. unprintable utf-8 decoded using surrogateescape error handler.)
Reply
  • 1
  • 18
  • 19
  • 20
  • 21(current)
  • 22

Logout Mark Read Team Forum Stats Members Help
[Module] youtube-dl - multi-site playable URL resolver2
This forum uses Lukasz Tkacz MyBB addons.