[garbear]

My guess is something like a recursive partition tree would work well for these data (rpart in R), though even something like naive Bayes would probably work out OK. You have some very good discriminators for episodes vs other (based on regexp matches) and discriminating musicvideos vs movies is pretty clear cut based on time I should think. Long-format music videos (concert footage etc.) will be trickier - things like audio track format might be useful here (higher quality track more indicative of a concert?) Further, things like foldernames can be grepped for typical things like "Movie", "Music", "Concert", "TV", "Show" (perhaps with other common languages included) which would help. You'd maybe use something like fstrcmp() and take the maximum value from each portion of the path?

topfs2: perhaps you could summarise what information (predictors) you have for each item that might be useful? Is it essentially all the DB variables? What else on top of the DB stuff?

Some predictors I can think of:

+ File runtime
+ Container and track formats
+-- Resolution
+ File name
+-- Presence of year in filename could indicate movie
+-- Split file (.cd1/2)
+-- Episode-based regexes, adjacent files with similar file name structure
+-- Presence of music video artist in music database
+ Tokenized path as a feature vector
+ Scraper results - catch-22 dilemma, but scraper results do verify our decision

For folder string matching, I think rooting ourselves to a particular lexicon will cause too many headaches. Alternatively, we could let a text classifier find a language-neutral pattern kinda like a modern day spam filter. I'm unfamiliar with decision trees (and always looking to learn), so I don't know if they would perform better than naive bayes classifiers here, but I imagine it's the same idea. For instance, I almost added "organizing movies by resolution" as a predictor (I used to do this). But if we do a text classifier, in addition to "Movies" and "TV Shows" in folder names, it'll automatically pick up on this as part of its normal operation.

A hypothetical question: to broaden our information base, what about proactively running 3 queries, each assuming a movie, a music video and a tv show? An empty/undesirable response would indicate a negative/poor test result. ofc this is hypothetical, a better idea could possibly be to submit a query to a different type of scraper retroactively if one type returns poorly.