2007-08-20, 20:45
I need to learn how to script lol, i really want to port NZBPlayer to the XBOX.
2007-08-20, 21:22
optip, please don't let this project die!
Even if you are too busy to code at the moment, please update us or release the source.
Thanks.
Even if you are too busy to code at the moment, please update us or release the source.
Thanks.
2007-08-22, 02:22
I have written a frontend python script that can take an nzb file sort it into the correct order and also sort the segments into order. It then creates a object file of the PostIds and the group that each one is in.
Its very crude but does get the job done although theres no real error checking. I also added in the ability to download each of those chunks and when the file was complete to create the actual archive file. Like i say no real error checking and dont think python has the ability to do this ontop of xbmc trying to play the file..
Would this be possible?. I expect most people would want 8 threads atleast to get maximum data to keep the stream good.
Heres the Code if anyones interested in trying it. I have actually tried running it on the xbox. It wont display anything or start playing the file and it has been forced to abort after retrieving the first file..just put an nzb in the same folder as this file and run it..It has only been coded to work with nzbs with only one rar set anything else will most likely fail to sort..
Its very crude but does get the job done although theres no real error checking. I also added in the ability to download each of those chunks and when the file was complete to create the actual archive file. Like i say no real error checking and dont think python has the ability to do this ontop of xbmc trying to play the file..
Would this be possible?. I expect most people would want 8 threads atleast to get maximum data to keep the stream good.
Heres the Code if anyones interested in trying it. I have actually tried running it on the xbox. It wont display anything or start playing the file and it has been forced to abort after retrieving the first file..just put an nzb in the same folder as this file and run it..It has only been coded to work with nzbs with only one rar set anything else will most likely fail to sort..
Code:
import os,sys,traceback,string
from re import findall as reFindall, compile as reCompile,re.DOTALL as reDotAll, search as reSearch
import pickle,nntplib
############################################
## Change These to match your news Server ##
############################################
NEWZ_SERVER = <<CHANGE TO ADDRESS OF YOUR NEWS SERVER>>
NEWZ_USER =<<CHANGE TO YOUR USERNAME ON SERVER>>
NEWZ_PASS =<<CHANGE TO YOUR PASSWORD ON SERVER>>
NZB_FILE =<<NAME OF NZB FILE TO PROCESS THATS IN THIS DIRECTORY>>
############################################
HOME_DIR = os.getcwd().replace(";","")+"\\"
yenc42 = "".join([chr((i - 42) & 255) for i in range(256)])
yenc64 = "".join([chr((i - 64) & 255) for i in range(256)])
REALNAME = ''
def ParseNzb(data) :
try :
## Get start/end points ##
Exp = reCompile('(<file.*?</file>)',reDotAll)
Tar = reFindall(Exp,data)
FileData = {}
RARTYPE = None
for FTag in Tar :
Exp = reCompile('<file.*?subject=\"(.*?)\">')
Subject = reSearch('subject=\".*?\">',FTag).span()
Fields = str(FTag[Subject[0]+9:Subject[1]-2]).rsplit(' - ')
FileNm = reSub('"','',Fields[-1])
FileNm = reSub('"','',FileNm)
FileNm = reSub('"','',FileNm)
Chk1 = reSearch(' yEnc| \(',FileNm,reIgnoreCase)
if ( Chk1 ) :
FileNm = FileNm[:Chk1.start()]
Spaces = FileNm.rsplit(' ')
FileNm = Spaces[-1]
FileNm = FileNm.strip()
## Now see what type of file it is.
Record = -1
Type_1 = reSearch('part\d+\.rar$',FileNm)
Type_2 = reSearch('\.r\d+$',FileNm)
if ( Type_1 ) or ( Type_2 ) :
RecPos = reSearch('\d+$',FileNm).span()
Record = int(FileNm[RecPos[0]:RecPos[1]])
if (Type_2) : Record += 2
elif ( reSearch('.rar$',FileNm) ) :
Record = 1
if Record == -1 :
print 'Ignored -- '+ str(FileNm)
elif FileData.has_key(Record) :
print 'Duplicates -- Cannot Sort This File..'
raise Exception
else :
##print 'Processed --'+ str(FileNm) +' - In slot: '+ str(Record)
FileData[Record] = FTag
return FileData
except :
print '--ParseNzb Fail--'
traceback.print_exc()
return None
def WriteObj(Var,myFile):
try :
toFile = open(HOME_DIR + myFile,'wb')
pickle.dump( Var,toFile )
toFile.close()
return True
except :
print '--WriteObj Fail--'
traceback.print_exc()
return False
def ReadObj(myFile):
try :
toFile = open(HOME_DIR + myFile,'rb')
data = pickle.load(toFile)
toFile.close()
return data
except :
print '--ReadObj Fail--'
traceback.print_exc()
return None
def SortNzb(Fn) :
try :
if Fn.endswith('.nzb') :
f = open(HOME_DIR + Fn,'r')
data = f.read()
f.close
Result = ParseNzb(data)
if len(Result) == 0 or Result == None :
print 'Unable to Sort this file..'
else :
KeySort = Result.keys()
KeySort.sort()
## Now setup Regexp for GROUP and SEGMENT data.
GrpExp = reCompile('<group>(.*?)</group>')
SegExp = reCompile('<segment.*?number=[\"|\'](\d+)[\"|\']>(.*?)</segment>')
for Key,Segment in Result.items() :
Result[Key] = {}
Tar = reFindall(SegExp,Segment)
Grp = reFindall(GrpExp,Segment)
Result[Key]['Group'] = Grp
for Add in Tar :
Result[Key][int(Add[0])] = Add[1]
## Write the whole Object out to a file.close
if ( WriteObj(Result,'stream.obj') ) :
return True
return False
except :
traceback.print_exc()
return False
def YencDecode(FileIn):
try :
global REALNAME
Buffer = []
for line in FileIn :
if line == '' :
print '--Ignore : '+ str(line) +' --'
elif line.startswith('=y') :
HasName = reSearch('name=',line)
if ( HasName ) :
REALNAME = line[HasName.end():]
else :
print '--Ignore : '+ str(line) +' --'
else :
## This is real data
if line[-2:] == "\r\n":
line = line[:-2]
elif line[-1:] in "\r\n":
line = line[:-1]
splitData = line.split("=")
Buffer.append(splitData[0].translate(yenc42))
for data in splitData[1:]:
Buffer.append(data[0].translate(yenc64).translate(yenc42))
Buffer.append(data[1:].translate(yenc42))
return Buffer
except :
traceback.print_exc()
return None
def CheckPost(SortKeys,Type) :
try :
############################################################
## Types are 'CHECK' or 'DOWNLOAD' ##
## CHECK will stat all of the files to ensure they exist ##
## DOWNLOAD will download the file. ##
############################################################
DUMP = []
Newz=nntplib.NNTP(NEWZ_SERVER,119,NEWZ_USER,NEWZ_PASS)
for i in SortKeys :
print '---KEY--- :'+ str(i) +' --Group:'+ str(PostId[i]['Group'])
FILEARRAY = []
REALNAME = ''
## Goto Group
NullData = Newz.group(PostId[i]['Group'][0])
if len(NullData) == 0 :
print 'Group :'+ str(PostId[i]['Group'][0]) +' Doesnt exist'
## Now loop through the keys
SubKeys = PostId[i].keys()
SubKeys.sort()
TmpName = ''
FullChunk = []
for j in SubKeys :
if j != 'Group' :
## Perform an Action back on the Type passed.
if Type == 'CHECK' :
print '--Statting-- :'+ str(PostId[i][j])
Result = Newz.stat('<' + str(PostId[i][j]) +'>')
if len(Result) > 0 : return None,None
elif Type == 'DOWNLOAD' :
print '--Getting-- :'+ str(PostId[i][j])
Result = Newz.body('<' + str(PostId[i][j]) +'>')
TmpName = 'tmp-'+ str(i) +'-'+ str(j) +'.tmp'
FullChunk.append(Result)
if not ( DumpChunk(FullChunk) ) :
print '--Failed to Dump Chunk--'
raise Exception
raise Exception
except :
traceback.print_exc()
return None
def DumpChunk(Dump) :
try :
f = open(HOME_DIR + 'Dummy.rar','wb')
for Chunk in Dump :
Data = YencDecode(Chunk[3])
f.write(''.join(Data))
f.close()
os.rename(HOME_DIR + 'Dummy.rar',HOME_DIR + REALNAME)
return True
except :
traceback.print_exc()
return False
print '--Starting--'
if ( SortNzb(NZB_FILE) ) :
print '--NZB File sorted and saved as stream.obj--'
## File was sorted ok and written out to stream.obj ##
PostId = ReadObj('stream.obj')
if PostId != None :
print '--Read stream.obj from file into PostId dictionary has worked--'
SortKeys = PostId.keys()
SortKeys.sort()
if ( CheckPost(SortKeys,'CHECK') ) != None :
print '--All Files STAT OK **We can proceed** Yippee--'
Dump = CheckPost(SortKeys,'DOWNLOAD')
else :
print '--All files DID NOT STAT ok on the news server--'
else :
print '--Failed to read stream.obj into PostId--'
2007-09-24, 23:45
Bump in the hopes that someone keeps the dream alive - or that the developer subscribes to this thread 
2007-10-03, 22:20
I think everyone would. Hopefully a dev will take a look at this and pick this up.
2007-10-10, 05:23
Hey man. This would be a great addition to the Xbox. I hope you are able to get it going.
I'm sorry for not reading the whole thread, but read about 3 pages of it. But I had a few suggestions/questions, so I apologize if something is covered in the rest of the thread.
Have you tried this?
1. RAR and post to a news server in a RAR/PAR2 ratio that would work well
what you are trying to achieve. Smaller RAR sets and PAR sets. You make
them large enough to be useful and small enough that if a single or two
drop, then the stream is able to stay in tack.
2. You have the script pull ALL the headers from the server and update this
cache. It's a large list, but once cached, you only have to prune the oldest
and add the newest when you first start the script. After that you would
be able to:
a. Take and NZB file and compare to parts listing for those RAR files with
the headers available on the server.
b. Then you flag an NZB file [yellow, red or green] based on whether just
[a few, many or none] are missing, relatively speaking.
3. This would allow you to feed the script a list of NZB files.
a. This you could do with a newsreader such as Agent. You pull headers
and save any NZB files you are interested in. (I know there are those
wanting to use servers and such available on the internet to gather
feeds. I think that just brings unwanted attention to what we are
wanting to accomplish. But just my two cents.)
b. After you save the files, you FTP them over to a script directory on
your Xbox. Voila!, the script then will have a list that it can compare
to it's local header listing and tag.
4. This allows the script to find the uploaded media best suited for streaming
PART 2
1. If you know of any GUI programmers, have someone come up with an
encoder or script that will allow folks to leverage this.
2. Best settings for Winrar and QuickPar which a lot of uploaders use.
3. If using these settings doesn't "cost" too much to the end user that
encodes files for upload, it might catch on to encode them that way for
folks that stream.
OK, wanted to get all that out before I forgot it. If a lot of folks uploaded
with this working, alt.binaries.movies and alt.binaries.tv will become a
literal media shelf of video surfing. Wish I had the skills or the time to help
you. Later...
I'm sorry for not reading the whole thread, but read about 3 pages of it. But I had a few suggestions/questions, so I apologize if something is covered in the rest of the thread.
Have you tried this?
1. RAR and post to a news server in a RAR/PAR2 ratio that would work well
what you are trying to achieve. Smaller RAR sets and PAR sets. You make
them large enough to be useful and small enough that if a single or two
drop, then the stream is able to stay in tack.
2. You have the script pull ALL the headers from the server and update this
cache. It's a large list, but once cached, you only have to prune the oldest
and add the newest when you first start the script. After that you would
be able to:
a. Take and NZB file and compare to parts listing for those RAR files with
the headers available on the server.
b. Then you flag an NZB file [yellow, red or green] based on whether just
[a few, many or none] are missing, relatively speaking.
3. This would allow you to feed the script a list of NZB files.
a. This you could do with a newsreader such as Agent. You pull headers
and save any NZB files you are interested in. (I know there are those
wanting to use servers and such available on the internet to gather
feeds. I think that just brings unwanted attention to what we are
wanting to accomplish. But just my two cents.)
b. After you save the files, you FTP them over to a script directory on
your Xbox. Voila!, the script then will have a list that it can compare
to it's local header listing and tag.
4. This allows the script to find the uploaded media best suited for streaming
PART 2
1. If you know of any GUI programmers, have someone come up with an
encoder or script that will allow folks to leverage this.
2. Best settings for Winrar and QuickPar which a lot of uploaders use.
3. If using these settings doesn't "cost" too much to the end user that
encodes files for upload, it might catch on to encode them that way for
folks that stream.
OK, wanted to get all that out before I forgot it. If a lot of folks uploaded
with this working, alt.binaries.movies and alt.binaries.tv will become a
literal media shelf of video surfing. Wish I had the skills or the time to help
you. Later...
2007-10-10, 21:43
Newsgroup Archiver
http://sebsauvage.net/python/newsarchiver.py
Python script to fetch articles from a newsserver
http://www.enyo.de/fw/scripts/get-article.py
http://sebsauvage.net/python/newsarchiver.py
Python script to fetch articles from a newsserver
http://www.enyo.de/fw/scripts/get-article.py
2007-12-01, 19:01
Any progress on this. Where is optip?
This would be such a good addition to XBMC any developer willing to pick up the torch?
This would be such a good addition to XBMC any developer willing to pick up the torch?
Hi,
Sorry i've been away for so long... i hope to see progress on this in the next couple of weeks. I'm sure i've said that before but my circumstances have changed (i work a 9-5 now - rather than a 10-10! ) so I fully intend on getting something working in the next month or so. Will try and post regular updates!
Regards,
Patrick
Sorry i've been away for so long... i hope to see progress on this in the next couple of weeks. I'm sure i've said that before but my circumstances have changed (i work a 9-5 now - rather than a 10-10! ) so I fully intend on getting something working in the next month or so. Will try and post regular updates!
Regards,
Patrick
2008-01-20, 17:16
Great news!
Glad to see your back and enthusiastic about it. Any chance of a preview of what you have coded so far?
Need any input on anything?
Glad to see your back and enthusiastic about it. Any chance of a preview of what you have coded so far?
Need any input on anything?
2008-01-20, 20:20
optip - great to see you're back at it. We all have lives/jobs outside of XBMC, but should this project fall by the wayside again - could I request on behalf of the community that you release the source code you've come up with so far so one of us could pickup where you left off and complete it?
If you need testers or input during the development of this, count me in.
If you need testers or input during the development of this, count me in.
