Kodi Community Forum

Full Version: XBMC.Extract
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4
Edit. I figured out that Kodi is looking for a trailing slash in the directory stucture.

For anyone else interested, here is an 'extract all' python code for use with libarchive. This code expects you have installed 'Archive Support' (vfs.libarchive) addon from the Kodi repository.
python:

from urllib import quote_plus as url_quote

def extract_all_libarchive(archive_file,directory_to):
overall_success = True
files_out = list()
if 'archive://' in archive_file:
archive_path = archive_file
else:
archive_path = 'archive://%(archive_file)s' % {'archive_file': url_quote(xbmc.translatePath(archive_file))}
dirs_in_archive, files_in_archive = xbmcvfs.listdir(archive_path)
for ff in files_in_archive:
file_from = os.path.join(archive_path,ff).replace('\\','/') #Windows unexpectedly requires a forward slash in the path
success = xbmcvfs.copy(file_from,os.path.join(xbmc.translatePath(directory_to),ff)) #Attempt to move the file first
if not success:
xbmc.log(msg='Error extracting file %(ff)s from archive %(archive_file)s' % {'ff': ff,'archive_file':archive_file}, level=xbmc.LOGDEBUG)
overall_success = False
else:
xbmc.log(msg='Extracted file %(ff)s from archive %(archive_file)s' % {'ff': ff,'archive_file':archive_file}, level=xbmc.LOGDEBUG)
files_out.append(os.path.join(xbmc.translatePath(directory_to),ff))
for dd in dirs_in_archive:
if xbmcvfs.mkdir(os.path.join(xbmc.translatePath(directory_to),dd)):
xbmc.log(msg='Created folder %(dd)s for archive %(archive_file)s' % {'dd': os.path.join(xbmc.translatePath(directory_to),dd,''),'archive_file':archive_file}, level=xbmc.LOGDEBUG)
files_out2, success2 = extract_all_libarchive(os.path.join(archive_path,dd,'').replace('\\','/'),os.path.join(directory_to,dd))
if success2:
files_out = files_out + files_out2
else:
overall_success = False
else:
overall_success = False
xbmc.log(msg='Unable to create the folder %(dir_from)s for libarchive extraction' % {'dir_from': os.path.join(xbmc.translatePath(directory_to),dd)}, level=xbmc.LOGDEBUG)
return files_out, overall_success
Still trying to fix service.subtitles.legendastv ...

The goal is to extract zip or rar files with subtitles that, sometimes, could be inside directories.

I've managed to make it work with vfs.libarchive 1.0.5 on Leia (18.2) on Windows, using the above extract_all_libarchive (thank you very much @zachmorris !)
But the same code didn't work on Android (MiBox) and LibreElec (RPi)

The xbmcvfs.listdir throws:
Code:
GetDirectory - Error getting archive://storage/.kodi/userdata/addon_data/service.subtitles.legendastv/temp/5bcdcef91b7f4e3ab28efc2dddd4a5d9.rar

I’ve tried to read some other threads with similar problems but they point to different solutions: reboot, use vfs.libarchive combined with vfs.rar or never use both together, install additional O.S. packages, use external command line programs..,

And now I’m confused!

Please, can anyone answer these questions to help me understand:

1- Is vfs.libarchive 1.0.5 supposed to work on all systems to extract zip/rar with directories?
2- Is so, is there any additional installation/configuration besides the installation of the addon from Kodi repo?
3- Is reboot required? (didn’t fix on Android)
4- Do I need vfs.rar if I only use archive:// urls on extract_all_libarchive? Is there any change on xbmcvfs.listdir or copy behavior ? (I guess not but people said otherwise on other threads)
Got it!!

The problem was at the URL quote function
I was using urllib.quote and it only worked on Windows.
Changed to urllib.quote_plus and now it works on all systems!


Guess I'll answer my own questions...
 
Quote:1- Is vfs.libarchive 1.0.5 supposed to work on all systems to extract zip/rar with directories?

Yes!!!

 
Quote:2- Is so, is there any additional installation/configuration besides the installation of the addon from Kodi repo?
 
No. Everything works out of the box. Just install "Archive Support" from Kodi oficial repo
 
Quote:3- Is reboot required? (didn’t fix on Android)
 
No
 
Quote:4- Do I need vfs.rar if I only use archive:// urls on extract_all_libarchive? Is there any change on xbmcvfs.listdir or copy behavior ? (I guess not but people said otherwise on other threads)
 
No.
(2019-05-15, 03:04)henricos Wrote: [ -> ]Got it!!

The problem was at the URL quote function
I was using urllib.quote and it only worked on Windows.
Changed to urllib.quote_plus and now it works on all systems!

Good deal, I updated the code to make that clear
I have epg that is gzipped (epg.xml.gz).
However, it isn't a file in an archive - it's simply a gzipped file.

Anyway to extract this using archive:// ?
listdir shows no files or directories.
copy fails.
rename fails.
(2019-07-10, 03:38)matthuisman Wrote: [ -> ]I have epg that is gzipped (epg.xml.gz).
However, it isn't a file in an archive - it's simply a gzipped file.

Anyway to extract this using archive:// ?
listdir shows no files or directories.
copy fails.
rename fails.

Seems like it should work. The addon shows it should work with the following formats:
extensions=".7z|.tar.gz|.tar.bz2|.tar.xz|.zip|.rar|.tgz|.tbz2|.gz|.bz2|.xz|.cbr"

I tried the following:
zip - Works
7z - Works
tar - Works
bz2 - Fails
rar - Fails (works with rar:// url)
gz - Fails
xz - Fails
Others - Didn't try

I opened an issue in github here.
(2017-12-22, 11:45)henricos Wrote: [ -> ]
(2017-12-22, 01:51)spiff Wrote: [ -> ]you need vfs.libarchive as discussed further up in the thread for archive://, vfs.rar or vfs.libarchive for rar://, while zip is built-in.
 Great!

Last one... I guess  Blush

Can vfs.rar be automatically enabled by another addon during install ?

I've tried to use the import tag on addon.xml:

xml:

  <requires>
    <import addon="vfs.rar" version="2.0.2" />
  </requires>

But got this...

ERROR: CAddonInstallJob[myaddon]: The dependency on vfs.rar version 2.0.2 could not be satisfied    

Did you find any solution to requires/import any vfs (vfs,rar or vfs.libarchive)?
I tried but got error too :S

After kodi restart it works.
(2019-05-15, 07:29)zachmorris Wrote: [ -> ]
(2019-05-15, 03:04)henricos Wrote: [ -> ]Got it!!

The problem was at the URL quote function
I was using urllib.quote and it only worked on Windows.
Changed to urllib.quote_plus and now it works on all systems!

Good deal, I updated the code to make that clear  

Hey guys! I maintain 3 subtitles addon and the function does not extract everything inside a rar file. In a rar file with 2 directories and files inside it will only make one directory and copy one file.
I'm trying to make it work but there's not much time left in 24h...
Any ideias on what's missing?

Thanks.
(2019-11-07, 03:07)HiGhLaNdeR Wrote: [ -> ]
(2019-05-15, 07:29)zachmorris Wrote: [ -> ]
(2019-05-15, 03:04)henricos Wrote: [ -> ]Got it!!

The problem was at the URL quote function
I was using urllib.quote and it only worked on Windows.
Changed to urllib.quote_plus and now it works on all systems!

Good deal, I updated the code to make that clear  

Hey guys! I maintain 3 subtitles addon and the function does not extract everything inside a rar file. In a rar file with 2 directories and files inside it will only make one directory and copy one file.
I'm trying to make it work but there's not much time left in 24h...
Any ideias on what's missing?

Thanks.

Do you have an example file on the web somewhere?
(2019-11-07, 03:13)zachmorris Wrote: [ -> ]
(2019-11-07, 03:07)HiGhLaNdeR Wrote: [ -> ]
(2019-05-15, 07:29)zachmorris Wrote: [ -> ]Good deal, I updated the code to make that clear  

Hey guys! I maintain 3 subtitles addon and the function does not extract everything inside a rar file. In a rar file with 2 directories and files inside it will only make one directory and copy one file.
I'm trying to make it work but there's not much time left in 24h...
Any ideias on what's missing?

Thanks. 

Do you have an example file on the web somewhere? 
Yes of course.
Use this link https://gameover.com.pt/example.rar
Thanks.
(2019-11-07, 12:20)HiGhLaNdeR Wrote: [ -> ]
(2019-11-07, 03:13)zachmorris Wrote: [ -> ]
(2019-11-07, 03:07)HiGhLaNdeR Wrote: [ -> ]Hey guys! I maintain 3 subtitles addon and the function does not extract everything inside a rar file. In a rar file with 2 directories and files inside it will only make one directory and copy one file.
I'm trying to make it work but there's not much time left in 24h...
Any ideias on what's missing?

Thanks. 

Do you have an example file on the web somewhere? 
Yes of course.
Use this link https://gameover.com.pt/example.rar
Thanks.

Yes, that doesn't seem to work as you said. The xbmcvfs doesn't return all folders in the archive, so I think this is another bug.
Related, I made a script.module to more easily work extraction, and started a thread here.
(2019-11-10, 18:56)zachmorris Wrote: [ -> ]
(2019-11-07, 12:20)HiGhLaNdeR Wrote: [ -> ]
(2019-11-07, 03:13)zachmorris Wrote: [ -> ]Do you have an example file on the web somewhere? 
Yes of course.
Use this link https://gameover.com.pt/example.rar
Thanks. 

Yes, that doesn't seem to work as you said. The xbmcvfs doesn't return all folders in the archive, so I think this is another bug.
Related, I made a script.module to more easily work extraction, and started a thread here

Hello Zach,

been a fight between me and the "path's" to get the multi-folder and multi-file extraction...
after @alwinus fixed the vfs.rar I used the code you pasted earlier in this thread and finally I could make it work.
If you don't mind, and I will quote the code as yours, I will use your Function extract_all in my code.
When you get your addon running in the official repo I will use it instead.
You ok with this?
Thanks.

Regards.
(2020-04-10, 11:17)HiGhLaNdeR Wrote: [ -> ]
(2019-11-10, 18:56)zachmorris Wrote: [ -> ]
(2019-11-07, 12:20)HiGhLaNdeR Wrote: [ -> ]Yes of course.
Use this link https://gameover.com.pt/example.rar
Thanks. 

Yes, that doesn't seem to work as you said. The xbmcvfs doesn't return all folders in the archive, so I think this is another bug.
Related, I made a script.module to more easily work extraction, and started a thread here

Hello Zach,

been a fight between me and the "path's" to get the multi-folder and multi-file extraction...
after @alwinus fixed the vfs.rar I used the code you pasted earlier in this thread and finally I could make it work.
If you don't mind, and I will quote the code as yours, I will use your Function extract_all in my code.
When you get your addon running in the official repo I will use it instead.
You ok with this?
Thanks.

Regards.

Yes, the code is free to use
(2020-04-12, 20:41)zachmorris Wrote: [ -> ]
(2020-04-10, 11:17)HiGhLaNdeR Wrote: [ -> ]
(2019-11-10, 18:56)zachmorris Wrote: [ -> ]Yes, that doesn't seem to work as you said. The xbmcvfs doesn't return all folders in the archive, so I think this is another bug.
Related, I made a script.module to more easily work extraction, and started a thread here

Hello Zach,

been a fight between me and the "path's" to get the multi-folder and multi-file extraction...
after @alwinus fixed the vfs.rar I used the code you pasted earlier in this thread and finally I could make it work.
If you don't mind, and I will quote the code as yours, I will use your Function extract_all in my code.
When you get your addon running in the official repo I will use it instead.
You ok with this?
Thanks.

Regards. 

Yes, the code is free to use 
Thanks Zach.
I will pay you a few beer's when the COVID get's kicked out.
Cheers!
Has anybody else issues with 7z-files? I created a simple text file and compressed it with Windows version of 7-zip.

I used following code to test it (quite similar to zachs code above):
python:

import xbmcvfs
from urllib.parse import quote_plus

file_7z = "C:\\Users\\Malte\\Documents\\Archivetests\\Test.7z"
archive_file = 'archive://%(archive_file)s' % {'archive_file': quote_plus(xbmc.translatePath(file_7z))}
(dirs, files) = xbmcvfs.listdir(archive_file)
print(files)

When I run this code inside a Kodi addon, Kodi just crashes without any error message. When I change the 7z file to a zip file, the same code runs without any issues.

I tested several compression types (lzma, lzma2, bzip2, ...) and also tested with simple files or files in folders etc. All with the same result.

Also tested with Kodi Leia (version 1.0.7 of Archive support) and Matrix (version 1.2.0 of Archive support)

Does anybody has an idea what is wrong with my test?
Pages: 1 2 3 4