Windows - Hydraulic

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
ckergald Offline
Member
Posts: 86
Joined: Jun 2011
Reputation: 2
Post: #1
Hydraulic is a simple windows tool that will create .strm files that can be used in conjunction with Pneumatic addon.

These .strm files can then be scanned into your xbmc library. For those not familiar Pneumatic allows streaming content from newsgroups/usenet.

There is a gui and command line versions, I've also included my shows.csv file to get you started. If you would like to start fresh with no shows pre-populated, just delete the shows.csv file and use the gui to build your own list.

I've created the command line version so it could be included as a startup script or scheduled task to update what shows are available.

Hydraulic will first search for a high definition source, if not found it will then match against any quality of source.

Shows that are off the air or unlikely to see a new release, or that you just don't want to scrub for each time can be removed from your show list after the initial scrub.

requires a newznab site such as nzb.su
will use atleast 2 api hits for each show added. Since newznab sites only respond with 100 results at a time, shows with a large number of releases avaiable will use additional api hits to gather those additional releases.

I've included the source code in the release (it's in c# 2008)

Hydraulic_1.3.zip
(This post was last modified: 2012-03-25 16:14 by ckergald.)
find quote
ldh830114 Offline
Junior Member
Posts: 21
Joined: Jul 2011
Reputation: 0
Post: #2
Hey... I like where this is headed. keep it up guys
find quote
dr99 Offline
Member
Posts: 84
Joined: May 2011
Reputation: 0
Post: #3
Nice app. Have you thought of having your program grab a list of releases for a show and then letting the user pick which episodes to make .strm file for, instead of all episodes?
(This post was last modified: 2012-03-16 05:06 by dr99.)
find quote
wireman99 Offline
Junior Member
Posts: 7
Joined: Mar 2012
Reputation: 0
Post: #4
Nice app with lots of potential but I am encountering a couple of problems. I am running Windows 7 64.

First problem when adding and scrubbing American Chopper Senior Vs Junior. Both Hydraulic and Hydro crash in version 1.1 and Hydraulic version 1.0. This happens on both the initial scrub and any subsequent scrubs. Also doesn't seem to matter if its the only show in the list or if there are others included.
This is the error I get from Hydraulic.

************** Exception Text **************
System.ArgumentException: Illegal characters in path.
at System.IO.Path.CheckInvalidPathChars(String path)
at System.IO.Path.NormalizePathFast(String path, Boolean fullCheck)
at System.IO.Directory.CreateDirectory(String path, DirectorySecurity directorySecurity)
at NewzNab_strm_creator_GUI.Form1.button3_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.5448 (Win7SP1GDR.050727-5400)
CodeBase: file:///C:/Windows/Microsoft.NET/Framework64/v2.0.50727/mscorlib.dll
----------------------------------------
NewzNab strm creator GUI
Assembly Version: 1.0.0.0
Win32 Version: 1.0.0.0
CodeBase: file:///C:/Media/Hydraulic_1.1/Hydraulic.exe
----------------------------------------
System.Windows.Forms
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.5446 (Win7SP1GDR.050727-5400)
CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.5453 (Win7SP1GDR.050727-5400)
CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.5420 (Win7SP1.050727-5400)
CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System.Xml
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.5420 (Win7SP1.050727-5400)
CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Xml/2.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
System.Web
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.5456 (Win7SP1GDR.050727-5400)
CodeBase: file:///C:/Windows/assembly/GAC_64/System.Web/2.0.0.0__b03f5f7f11d50a3a/System.Web.dll
----------------------------------------
System.Configuration
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.5420 (Win7SP1.050727-5400)
CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Configuration/2.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------

Second problem is when I remove a show using Hydraulic GUI it also removes all the configuration information from the csv file.

Also a question for you.

In another thread you stated.

"I updated Hydraulic, fixing a few bugs.
it also now uses a regex to match season and episode information. And attempts to only make 1 .strm file for each episdoe matched instead of 1 strm for each release matched."

In version 1.0 of Hydraulic it often returned results in several languages and version 1.1 gives no indication as to the language the strm file is for. How was this addressed?

Thanks for the hard work on this and look forward to seeing how this evolves.
find quote
ckergald Offline
Member
Posts: 86
Joined: Jun 2011
Reputation: 2
Post: #5
(2012-03-16 05:06)dr99 Wrote:  Nice app. Have you thought of having your program grab a list of releases for a show and then letting the user pick which episodes to make .strm file for, instead of all episodes?

Maybe eventually I may do this, kinda low on my priority list at the moment. I probably will add a search age option soon though so you can only grab recent releases once your initial set has been grabbed.

(2012-03-17 16:19)wireman99 Wrote:  Nice app with lots of potential but I am encountering a couple of problems. I am running Windows 7 64.

First problem when adding and scrubbing American Chopper Senior Vs Junior. Both Hydraulic and Hydro crash in version 1.1 and Hydraulic version 1.0. This happens on both the initial scrub and any subsequent scrubs. Also doesn't seem to matter if its the only show in the list or if there are others included.
This is the error I get from Hydraulic.
This was due to a : being in the title, it then tried to use this to create a directory and file name, I'm testing a new version that gets around this now and expect a release soon.

(2012-03-17 16:19)wireman99 Wrote:  Second problem is when I remove a show using Hydraulic GUI it also removes all the configuration information from the csv file.
Thanks I'll get that fixed.

(2012-03-17 16:19)wireman99 Wrote:  Also a question for you.

In another thread you stated.

"I updated Hydraulic, fixing a few bugs.
it also now uses a regex to match season and episode information. And attempts to only make 1 .strm file for each episdoe matched instead of 1 strm for each release matched."

In version 1.0 of Hydraulic it often returned results in several languages and version 1.1 gives no indication as to the language the strm file is for. How was this addressed?

Thanks for the hard work on this and look forward to seeing how this evolves.
Ver 1.0 just searched catagory 5000 in newznab looking for releases, this is everything mixed together.
Ver 1.1 searched catagory 5040 for HD and 5000 for SD, this was better but could still potentialy return foriegn languages on the SD scan

current version I'm working on searched 5040 for HD and 5030 for SD, for those not familiar neznab does a pretty good job at applying regex's and moving forigen releases to a seperate catagory 5020, if you search catagory 5000 you will get all tv including foriegn, but searching 5040 and 5030 should turn up just english releases. This isn't perfect and occasionaly a relase makes it through but there filtering and regex's are far better then what I could come up with.
find quote
ckergald Offline
Member
Posts: 86
Joined: Jun 2011
Reputation: 2
Post: #6
Not sure why I'm unable to attach a file to the forum posts today,
but the latest version can be downloaded here

I've applied fixes for shows containing problematic characters like : and *
applied the fix for bug reported by Wireman99 will now save your api key information when adding and deleting shows.

Major changes
1) at first run it will create a file called regexs.txt, it uses these regex's to match release information in the order they appear in the .txt file.
These match the majority of releases I've found but you can modify this file to match additional or specific releases. (if you do please share them)
2) files and folders are named by the showname from a tvrage lookup. I noticed in previous versions we would sometime have multiple folders for the same show with slight variations. This will correct that problem.
3) If a release is titled with a air date instead of season and episode information the 3rd regex grabs that air date, and does a lookup at tvdb to try and match the air date, if a match is found it grabs the coresponding season and episode information from tvdb and uses that in naming the file.
4) I've excluded the command line version, I've made significant changes to the code and it's rather complicated to move these changes back to the command line version.

Goals for next version
1) fix any bugs found from this version
2) add movies
(This post was last modified: 2012-03-17 22:17 by ckergald.)
find quote
dr99 Offline
Member
Posts: 84
Joined: May 2011
Reputation: 0
Post: #7
How does you program decide which release to get when there are multiple versions of the same episode? Does it look for bluray first then web-dl then hdtv like sickbeard does?
find quote
ckergald Offline
Member
Posts: 86
Joined: Jun 2011
Reputation: 2
Post: #8
(2012-03-18 01:49)dr99 Wrote:  How does you program decide which release to get when there are multiple versions of the same episode? Does it look for bluray first then web-dl then hdtv like sickbeard does?

Right now it's just doing most recent match, but that wouldn't be too hard to implement.
find quote
wireman99 Offline
Junior Member
Posts: 7
Joined: Mar 2012
Reputation: 0
Post: #9
ckergald, thanks for the update so quick. I'm really starting to like this program.

Ran into another issue though. Some of the matches coming up are mpeg2 which in my case XBMC doesn't seem to like. They either fail to play or only play through the first rar file. Seems XBMC is seeing the first rar file as the complete show. As an example the most recent match for Wipeout S01E02 is (Wipeout S01E02 All Work and No Play 1080i HDTV-UPSCALE DD5 1 MPEG2-TrollHD). When played XBMC sees this as 49 seconds in duration.

Now the ever present request. Would it be possible to implement the creation of season folders within the show folders? I find using season folders makes it easier to manage the strm files and meta data.

Again thanks for the hard work and look forward to where this goes.
(This post was last modified: 2012-03-19 01:59 by wireman99.)
find quote
ckergald Offline
Member
Posts: 86
Joined: Jun 2011
Reputation: 2
Post: #10
(2012-03-19 01:31)wireman99 Wrote:  ckergald, thanks for the update so quick. I'm really starting to like this program.

Ran into another issue though. Some of the matches coming up are mpeg2 which in my case XBMC doesn't seem to like. They either fail to play or only play through the first rar file. Seems XBMC is seeing the first rar file as the complete show. As an example the most recent match for Wipeout S01E02 is (Wipeout S01E02 All Work and No Play 1080i HDTV-UPSCALE DD5 1 MPEG2-TrollHD). When played XBMC sees this as 49 seconds in duration.

Now the ever present request. Would it be possible to implement the creation of season folders within the show folders? I find using season folders makes it easier to manage the strm files and meta data.

Again thanks for the hard work and look forward to where this goes.

I'll look into season folders, not sure how easy/difficult this will be to implement yet but I'll give it a try.
as for the mpeg2 releases I'm having similar issues with some titles, have not narrowed it down to mpeg2 or not but I suspect it may be an issue with pneumatic. Haven't done enough testing to know for sure yet.

Current development version is doing some basic quality checks now and trying to select the highest quality release instead of just writing the most recent version. It's doing this by 1st comparing resolution prefers in this order (1080p>720p>1080i>none) so a 1080p version will win of all others, if no resolution is listed any other resolution would win. If resolution matches then it looks at source and codec to decide.
find quote
wireman99 Offline
Junior Member
Posts: 7
Joined: Mar 2012
Reputation: 0
Post: #11
Agree that the problem isn't solely with mpeg2 files but also with xbmc and pneumatic.

After posting above I went on to experiment with several other files, most of them mkv's. Getting failure to play and strangely some even return the error of no rar's in nzb file, yet sabnzbd is downloading them. Haven't looked at the log files yet but as a quick test I used an external newsreader to stream the same file using MPC-HC. No problems streaming using this method.

I like the direction your going by filtering by quality. Another way to possibly help filter the results returned would be to use file size. I took a quick look at the newznab docs and there appears to be an api call for file size. My thinking is that files like mpeg2 are considerably bigger than x264 for the same quality, or in the case of those that don't have a high enough bandwidth connections they could restrict it to those they could stream. This possibly might allow excluding files that pneumatic and xbmc have problems with also (although I think this is less likely to work consistently). This is all conjecture on my part as I don't have much coding experience, but I'm trying to learn, so not sure how hard things like this would be to implement.

Looking forward to the next release and thanks for the app. It has inspired me to redouble my efforts to learn to code.
(This post was last modified: 2012-03-19 08:02 by wireman99.)
find quote
ckergald Offline
Member
Posts: 86
Joined: Jun 2011
Reputation: 2
Post: #12
(2012-03-19 08:00)wireman99 Wrote:  Agree that the problem isn't solely with mpeg2 files but also with xbmc and pneumatic.

After posting above I went on to experiment with several other files, most of them mkv's. Getting failure to play and strangely some even return the error of no rar's in nzb file, yet sabnzbd is downloading them. Haven't looked at the log files yet but as a quick test I used an external newsreader to stream the same file using MPC-HC. No problems streaming using this method.

I like the direction your going by filtering by quality. Another way to possibly help filter the results returned would be to use file size. I took a quick look at the newznab docs and there appears to be an api call for file size. My thinking is that files like mpeg2 are considerably bigger than x264 for the same quality, or in the case of those that don't have a high enough bandwidth connections they could restrict it to those they could stream. This possibly might allow excluding files that pneumatic and xbmc have problems with also (although I think this is less likely to work consistently). This is all conjecture on my part as I don't have much coding experience, but I'm trying to learn, so not sure how hard things like this would be to implement.

Looking forward to the next release and thanks for the app. It has inspired me to redouble my efforts to learn to code.

I would recomend also testing the same release using popeye's newznab plugin, you should get the same results either it ends after a few minutes, or the message about no rar's in the file. I'm seeing these same items ocassionaly as well. Also Popey's newznab plugin has the feature to create a .strm file for a individual release just not in bulk as I'm doing with hyrdaulic. Try and create a strm file from his plugin and see if that works. If both these fail then it's either a bad usenet release or a problem with pneumatic. And if it work in a external player like mpc-hc that would indicate the release is fine.

Size may be a bit tricky as hour long shows would naturaly have larger file sizes then half hour shows, so there would need to be someway to look up the show first to find it's length then divide that by the file size. I think tvrage returns the length so I'll look into it.

With the quality checks I have in place on the latest version we shouldn't turn up to many mpeg2 releases anyways
i'm first looking at resolution, 1080p wins all, 720p wins over 1080i and any releases with no resolution in the title, and 1080i wins only over releases without a resolution in the title.
If resolutions match I then check codec, x264 and h264 have the same priority and will win over all others, next is mpeg2 and finaly is xvid.
I also look at source and prefer in this order bluray, web-dl,hdtv,bdrip,dvdrip,pdtv,anything else.

As far as coding goes, I understand how to code, but could never cut it as a full time programer. I really just threw this together to give me something similar to icelibrary now that pneumatic supports .strm files. If couchpotato and sickbeard start creating these strm files there really would be no need for this app, and I expect they probably will as popeye's plugin improves.
(This post was last modified: 2012-03-19 12:46 by ckergald.)
find quote
wireman99 Offline
Junior Member
Posts: 7
Joined: Mar 2012
Reputation: 0
Post: #13
(2012-03-19 12:43)ckergald Wrote:  I would recomend also testing the same release using popeye's newznab plugin, you should get the same results either it ends after a few minutes, or the message about no rar's in the file. I'm seeing these same items ocassionaly as well. Also Popey's newznab plugin has the feature to create a .strm file for a individual release just not in bulk as I'm doing with hyrdaulic. Try and create a strm file from his plugin and see if that works. If both these fail then it's either a bad usenet release or a problem with pneumatic. And if it work in a external player like mpc-hc that would indicate the release is fine.

ckergald didn't mean to imply that the problem was with Hydraulic. That is exactly what I did and it failed to launch either directly from the newznab or nzbmatrix plugin or after creating the strm files. Whats strange is that whether I use Hydraulic, newznab to create the strm file or even try to directly launch from the newznab plugin it fails. Yet it successfuly sends the request to sabnzbd and gets downloaded. Even trying to use the incomplete route from the plugin returns the same error messages.

Posted the problem and log to Popeye's thread to see what he makes of it.

(2012-03-19 12:43)ckergald Wrote:  As far as coding goes, I understand how to code, but could never cut it as a full time programer. I really just threw this together to give me something similar to icelibrary now that pneumatic supports .strm files. If couchpotato and sickbeard start creating these strm files there really would be no need for this app, and I expect they probably will as popeye's plugin improves.

That is exactly what drew me to your program. The ability to have a extensive library yet not require massive amounts of hard drive space.

P.S. Looking forward to the addition of movies to the program.
(This post was last modified: 2012-03-19 18:15 by wireman99.)
find quote
ckergald Offline
Member
Posts: 86
Joined: Jun 2011
Reputation: 2
Post: #14
I've updated Hydraulic to ver 1.3
Hydraulic_1.3.zip

It now will try to pick the best quality for a tv release, supports season folders, and I've started adding movie support as well.
find quote
ursli Offline
Fan
Posts: 306
Joined: Nov 2011
Reputation: 7
Post: #15
Thanks for this, working as expected here.

To my question: Anyone already managed to create a regex to only create 720p.WEB-DL .strm files if it doesn't find 1080p ones? I tried fiddling around with the regex.txt file already but I lack to knowledge to make it work.
find quote
Post Reply