[barrygordon]

I am having real issues building my libraries by scanning my content.

My video content is in two main shares "My Movies" and "TV Programs". My movies contains BR and DVD Rips in Bluray or DVD Folder format (looks like an iso and has all the same files and directory structure). The TV programs are acually box set DVD's ripped in DVD format with each disk of the set having its own entry.

When I scan my movies using IMDB as the repository I get most of the movies found correctly but for some I get the stangest results. Other repositories are worse. If I use getmovie information from the context menu and try refresh and look at what is shown as possible matches I get some strange results. As an example It can not find an entry for "The Good, The Bad, and the Ugly" that is a proper match to the DVD, It finds entries but not the proper one. There were two movies called Crash. The more recent one is found. The older one is not in the choice list. The same is true of many others in my collection, none of which are very esoteric. DVDProfiler has found the proper entries for my entire collection. I think it uses CDDB but am not sure.

Is there a scanner that will look at the DVDprofiler? Either its data base or its exported xml file?

When I try TV Programs, it finds just one episode of Battlestar Galactaca. All episodes are ther as are 24, Alis, West Wing. TV shows are stored as DVD Rips, each disk having a seperate folder names with the show name, season number, episode content e.g "Alias: Season 1 Episodes 1-4". Perhaps the ":" is confusing it?

Any help/recommendations on how I can improve this or use multiple scanners, rename things etc. would be very helpful!!

[jmarshall]

The TV programs are acually box set DVD's ripped in DVD format with each disk of the set having its own entry.

Re-rip to individual episodes - it will save you a lot of hassle, and is the reason that nothing is working as it should. While you could probably write regexps to identify your episodes, to be honest it's going to be way better long term to have separate episodes.

No, nothing will use whatever "DVDProfiler" is. If that does stuff based on DVDs then it may use things like the DVD identifier to lookup which is obviously quite accurate. Once you only have a title, accuracy is a lot worse. If you add year to the name then you'll get better results.

Cheers,
Jonathan

[barrygordon]

JMarshall, Thank you or the reply. I think I understand what you are saying and I could re-rip to individual episodes and there is a lot of logic in doing that.

DVD Profiler is a complete management system for maintaining all the info about a DVD, CD, BR Disc. When I get a new item all I do is enter the UPC code from the box and everything else is fully automatic. It can put out an XML file which contains all the info about the disk including its ID. It is basically the same info that the scraping process retrieves minus the fanart. I drive my "pre XBMC" movie librarian from that xml file.

One of the DVDprofiler features is that peole can contribute additional info about a disk and upload it (much like a wiki) so eventually all of the known info gets filled in. Updated profiles can be downloaded at any time. It is really a very complete and easy to use system.

I am wondering if I couldn't add an entry (a simple xml or text file) in the folder for the item (e.g. the folder named "Alias Season 1 Episodes 1-4") that would contain the disk ID or other absolute identifying info which the profiler has and some how use it to drive the scraping operation. If you could point me to the place in the source where this is done I would be most appreciative. Just a general idea where to start looking would be a big aid.

From what I have read this would be like the .nfo file concept.

I have gotten all my movies properly scraped and adjusted with just a few missing parts (cover art generally) for some DVD's. My Music seems to be fine as it is all in tagged MP3 files. It is just the DVD's that are really TV Shows in boxed sets that are giving me the issue.

I assume (from what I have read) that the proper format for the folders containing each ripped single episode of a TV series would be
Name SxxEyy where xx is a 2 digit season number and yy is a two digit episode number and the contents of the folder (.ifo and .vob files) is irrelevant. Is this correct?

[jmarshall]

Right - with the disk ID you can do lots more, assuming you have a database backend to query.

Does DVDProfiler offer some sort of an http interface that you can query or something? If so, a scraper could be written. If not, it can't for now. Another option might be to write an addon to DVDProfiler to output XBMC's nfo format. Not sure if it allows that.

The plan eventually is to have XLST support (an xml file that describes how to translate one xml format into another) in the scrapers so that this sort of thing is doable. That gets all the logic for scraping various XML file formats out of XBMC and into addons. eg currently we support mymovie.xml or whatever it is internally, where ideally this stuff would be external so that when layouts change you don't need a new version of XBMC, you can just update the addon.

As for naming episodes, the format is:

<folder containing tvshows>/name of show (year)/folders/here/ignored/filename.s01e01.iso

I suggest not doing .ifo/vob - a single file is cleaner.

Cheers,
Jonathan

[barrygordon]

DVDProfiler maintains on the machine it is running on a data base, I believe using SQL. It also puts out (exports on demand so it is available) an XML file which is what I use and could obviously be querried.

DVDprofiler does allow for plugins which could be written to put out the nfo format desired by the current scrapers.

To aid this community (and obviously myself) I would like to look at the writing of a scraper that deals with the exported DVDprofiler XML data base that has everything in it. If you can aid me in finding out how to write such a scraper (some one said the were just XML files themselves which has me confused, as ther must be some process going on) I would take a look at doing that.

Cover art for all disks in the collection are kept as jpg files in a folder whose location is well known. Each disks cover art file is named with the disks ID number with a suffix of f for front and b for back. Avatar's front cover is named 024543656135f.jpg.

For clarification, the following is what the DVDprofiler has in its XML output file for the movie Avatar (hoping the forum scanner leaves it intact):

<DVD>
<ProfileTimestamp>2010-05-17T12:41:58.000Z</ProfileTimestamp>
<ID>024543656135</ID>
<MediaTypes>
<DVD>true</DVD>
<HDDVD>false</HDDVD>
<BluRay>true</BluRay>
</MediaTypes>
<UPC>024543-656135</UPC>
<CollectionNumber>450</CollectionNumber>
<CollectionType>Owned</CollectionType>
<Title>Avatar</Title>
<DistTrait/>
<OriginalTitle/>
<CountryOfOrigin>United States</CountryOfOrigin>
<ProductionYear>2009</ProductionYear>
<Released>2010-04-22</Released>
<RunningTime>162</RunningTime>
<RatingSystem>Film</RatingSystem>
<Rating>PG-13</Rating>
<RatingAge>13</RatingAge>
<RatingVariant>0</RatingVariant>
<RatingDetails>Intense Epic Battle Sequences and Warfare, Sensuality, Language and Some Smoking</RatingDetails>
<CaseType>HD Slim</CaseType>
<CaseSlipCover>true</CaseSlipCover>
<Genres>
<Genre>Fantasy</Genre>
<Genre>Action</Genre>
<Genre>Adventure</Genre>
</Genres>
<Regions/>
<Format>
<FormatAspectRatio>1.78</FormatAspectRatio>
<FormatVideoStandard>NTSC</FormatVideoStandard>
<FormatLetterBox>true</FormatLetterBox>
<FormatPanAndScan>false</FormatPanAndScan>
<FormatFullFrame>false</FormatFullFrame>
<Format16X9>false</Format16X9>
<FormatDualSided>false</FormatDualSided>
<FormatDualLayered>false</FormatDualLayered>
</Format>
<Features>
<FeatureSceneAccess>true</FeatureSceneAccess>
<FeatureCommentary>false</FeatureCommentary>
<FeatureTrailer>false</FeatureTrailer>
<FeaturePhotoGallery>false</FeaturePhotoGallery>
<FeatureDeletedScenes>false</FeatureDeletedScenes>
<FeatureMakingOf>false</FeatureMakingOf>
<FeatureProductionNotes>false</FeatureProductionNotes>
<FeatureGame>false</FeatureGame>
<FeatureDVDROMContent>false</FeatureDVDROMContent>
<FeatureMultiAngle>false</FeatureMultiAngle>
<FeatureMusicVideos>false</FeatureMusicVideos>
<FeatureInterviews>false</FeatureInterviews>
<FeatureStoryboardComparisons>false</FeatureStoryboardComparisons>
<FeatureOuttakes>false</FeatureOuttakes>
<FeatureClosedCaptioned>false</FeatureClosedCaptioned>
<FeatureTHXCertified>true</FeatureTHXCertified>
<FeaturePIP>false</FeaturePIP>
<FeatureBDLive>false</FeatureBDLive>
<FeatureBonusTrailers>false</FeatureBonusTrailers>
<FeatureDigitalCopy>false</FeatureDigitalCopy>
<OtherFeatures/>
</Features>
<Studios>
<Studio>Twentieth Century Fox</Studio>
<Studio>Dune Entertainment</Studio>
<Studio>Ingenious Film Partners</Studio>
</Studios>
<MediaCompanies>
<MediaCompany>Twentieth Century Fox Home Entertainment</MediaCompany>
</MediaCompanies>
<Audio>
<AudioTrack>
<AudioContent>English</AudioContent>
<AudioFormat>DTS-HD Master Audio</AudioFormat>
<AudioChannels>5.1</AudioChannels>
</AudioTrack>
<AudioTrack>
<AudioContent>English</AudioContent>
<AudioFormat>Dolby Digital</AudioFormat>
<AudioChannels>Dolby Surround</AudioChannels>
</AudioTrack>
<AudioTrack>
<AudioContent>Audio Descriptive</AudioContent>
<AudioFormat>Dolby Digital</AudioFormat>
<AudioChannels>5.1</AudioChannels>
</AudioTrack>
<AudioTrack>
<AudioContent>Spanish</AudioContent>
<AudioFormat>Dolby Digital</AudioFormat>
<AudioChannels>5.1</AudioChannels>
</AudioTrack>
<AudioTrack>
<AudioContent>French</AudioContent>
<AudioFormat>Dolby Digital</AudioFormat>
<AudioChannels>5.1</AudioChannels>
</AudioTrack>
<AudioTrack>
<AudioContent>Portuguese</AudioContent>
<AudioFormat>Dolby Digital</AudioFormat>
<AudioChannels>5.1</AudioChannels>
</AudioTrack>
</Audio>
<Subtitles>
<Subtitle>English</Subtitle>
<Subtitle>Portuguese</Subtitle>
<Subtitle>Spanish</Subtitle>
</Subtitles>
<SRP DenominationType="USD" DenominationDesc="United States (Dollar)" FormattedValue="$39.99">39.99</SRP>
<Actors>
<Actor FirstName="Sam" MiddleName="" LastName="Worthington" BirthYear="0" Role="Jake Sully" CreditedAs="" Voice="false" Uncredited="false"/>
<Actor FirstName="Zoë" MiddleName="" LastName="Saldana" BirthYear="0" Role="Neytiri" CreditedAs="" Voice="false" Uncredited="false"/>
<Actor FirstName="Sigourney" MiddleName="" LastName="Weaver" BirthYear="0" Role="Grace" CreditedAs="" Voice="false" Uncredited="false"/>
<Actor FirstName="Stephen" MiddleName="" LastName="Lang" BirthYear="0" Role="Colonel Miles Quaritch" CreditedAs="" Voice="false" Uncredited="false"/>
<Actor FirstName="Michelle" MiddleName="" LastName="Rodriguez" BirthYear="0" Role="Trudy Chacon" CreditedAs="" Voice="false" Uncredited="false"/>
<Actor FirstName="Giovanni" MiddleName="" LastName="Ribisi" BirthYear="0" Role="Parker Selfridge" CreditedAs="" Voice="false" Uncredited="false"/>
<Actor FirstName="Joel" MiddleName="" LastName="David Moore" BirthYear="0" Role="Norm Spellman" CreditedAs="" Voice="false" Uncredited="false"/>
<Actor FirstName="CCH" MiddleName="" LastName="Pounder" BirthYear="0" Role="Moat" CreditedAs="" Voice="false" Uncredited="false"/>
<Actor FirstName="Wes" MiddleName="" LastName="Studi" BirthYear="0" Role="Eytukan" CreditedAs="" Voice="false" Uncredited="false"/>
<Actor FirstName="Laz" MiddleName="" LastName="Alonso" BirthYear="0" Role="Tsu&apos;tey" CreditedAs="" Voice="false" Uncredited="false"/>
</Actors>
<Overview>A reluctant hero. An epic journey. A choice between the life he left behind and the incredible new world he&apos;s learned to call home. Return to James Cameron&apos;s &lt;i&gt;AVATAR&lt;/i&gt; — the greatest adventure of all time.</Overview>
<Discs>
<Disc>
<DescriptionSideA>Main Feature</DescriptionSideA>
<DescriptionSideB/>
<DiscIDSideA/>
<DiscIDSideB/>
<LabelSideA/>
<LabelSideB/>
<DualLayeredSideA>false</DualLayeredSideA>
<DualLayeredSideB>false</DualLayeredSideB>
<DualSided>false</DualSided>
<Location>BRC</Location>
<Slot>55</Slot>
</Disc>
</Discs>
<SortTitle>Avatar</SortTitle>
<LastEdited>2010-06-10T15:22:36.000Z</LastEdited>
<WishPriority>0</WishPriority>
<CountAs>1</CountAs>
<PurchaseInfo>
<PurchasePrice DenominationType="USD" DenominationDesc="United States (Dollar)" FormattedValue="$0.00">0</PurchasePrice>
<PurchasePlace/>
<PurchasePlaceType/>
<PurchasePlaceWebsite/>
<PurchaseDate>2010-04-23</PurchaseDate>
</PurchaseInfo>
<Review Film="0" Video="0" Audio="0" Extras="0"/>
<MediaBanners Front="Automatic" Back="Automatic"/>
<Events/>
<BoxSet>
<Parent/>
<Contents/>
</BoxSet>
<LoanInfo>
<Loaned>false</Loaned>
</LoanInfo>
<Tags/>
</DVD>

[jmarshall]

Unfortunately you can't write a scraper file for that (at least I don't think you can) as scrapers require an http URL.

Cheers,
Jonathan

[barrygordon]

Where there is a will there is a way. Are you saying that the scraper process uses some sort of an http request to get what it wants? If that is the case I guess I could write a small Http server and have it run on one of my systems to answer the request with info from the DVD profilers xml file as I believe all the info needed is there. The question is what should the request look like and what is expected as a response?

Could you point me to any info on this subject. I have looked in the wiki but not found real technical detail yet.

You have been a big help so far (seriously).

P.S. I rarely give up!

[barrygordon]

Well I did some more research and I think I have found all the info I need.

My first cut approach will be to use the .nfo file concept. I plan to write a program that will take the output of the DVDprofiler as represented by its XML file and produce for each dvd in the collection a corresponding .nfo entry and place that entry into the proper location in the tv episode or movie folder with the proper name.

The IMDB scraper has handled my movie collection with only one or two issues.