Have begun some example code for an aggregate repo

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
anarchintosh Offline
Senior Member
Posts: 279
Joined: Jul 2010
Reputation: 4
Post: #1
Hi, i've begun some code so that there can be one repository (named AppStore) that can aggregate addons from every repository.
I probably won't finish it, but its a nice proof-of-concept.

PS. I will be updating this post

The idea is that we give it to a trusted xbmc fanboy to run all the time non-stop on an old computer.

Its a script that auto-runs a check of other repositories addons.xml.md5 for changes every five or ten minutes (this control can be modified).
If it sees a change, it will check the addons.xml for what exactly changed (by comparing it with a local copy) and then dl the new addons via http. (new plan is to not use git or svn checking-out). it will also re-download the md5 and addons.xml so it can now use them to compare to the online copies.

It will then use the datadirzipped=... parameter to check if all the addons conform to the xbmc zip specification (if not it will run a script to do this), and put them into to a special AppStore local repo, which it will then auto-magically commit via SVN to an AppStore googlecode page.

It also gets the repos from the Unofficial Repo wiki page, and watches it for changes, so as to add new repos. It also every 12 hours completely redownloads all the repo files, in case a coder makes a change to their repo file....

Voila! A single Unofficial repo that contains all addons not provided in the official repo, whilst still allowing coders to maintain their independent repo.

This is mainly drawn from a frustration at there already being tens of repos floating around, some of which only contain a few addons (mine is no exception), and the fact that its often hard to get committer access to someone elses. This way, you make your own repo, add it to unofficial repo wiki and it will then show up in the aggregate.


I'm now hosting the POC at collabedit, because i will update it from time to time. Feel free to make any changes as long as they conform to what the purpose of the script is meant to do. (see above^^) and also please bump the version number if you do, so i can see that changes have been made.Wink
(This post was last modified: 2011-04-27 10:54 by anarchintosh.)
find quote
pieh Offline
Team-Kodi Member
Posts: 672
Joined: Aug 2010
Reputation: 13
Location: Poland
Post: #2
Why do You need to checkout SVN/clone GIT? and then commit it to SVN again? (or I missundersood the idea). Simple file downloading wouldn't be enough? Every repository provides path to addons.xml and directory containing addon files:

Code:
<extension point="xbmc.addon.repository"
              name="Official XBMC.org Add-on Repository">
    <info compressed="true">http://mirrors.xbmc.org/addons/dharma-pre/addons.xml</info>
    <checksum>http://mirrors.xbmc.org/addons/dharma-pre/addons.xml.md5</checksum>
    <datadir zip="true">http://mirrors.xbmc.org/addons/dharma-pre</datadir>
  </extension>
Just do as XBMC - periodicly check if content of .md5 file changes - if changes fetch addons.xml and look for changes - if new version of addon - download it from "datadir_path/addon_id/addon_id-version.zip"
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #3
and the cry of thousands of sysadmins was heard as their servers were unnecessarily raped by thousands of users.
find quote
anarchintosh Offline
Senior Member
Posts: 279
Joined: Jul 2010
Reputation: 4
Post: #4
@grajen3
hmm actually that sounds like a better idea than doing checkouts, and should make the code less hacky.

@spiff
aha, well the idea is that it would only be run on 1 computer, and that it would check for new versions before doing downloads. and if it maintains an aggregate repo, it should ease the load on individual repos (presuming people begin use the aggregate)
if using grajen's idea, it should only be as heavy as xbmc.
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #5
sure, i wasn't all that serious.
find quote
anarchintosh Offline
Senior Member
Posts: 279
Joined: Jul 2010
Reputation: 4
Post: #6
spiff Wrote:sure, i wasn't all that serious.
no worries, server raping is one of my greatest skills
i have already be accused by icefilms as their main bandwidth problem Laugh
(although i think it was some1 elses addon actually)
find quote
vikjon0 Offline
---
Posts: 2,500
Joined: Apr 2009
Reputation: 8
Location: Sweden
Post: #7
But did not someone make a repo installer that cover most of the need and have none of the potential issues?
find quote
anarchintosh Offline
Senior Member
Posts: 279
Joined: Jul 2010
Reputation: 4
Post: #8
vikjon0 Wrote:But did not someone make a repo installer that cover most of the need and have none of the potential issues?
yes they did, but as the days go by, more people are creating new repos. i don't want to have to keep checking the repo installer for new repos

anyway, there won't really be issues with this, as i've got it pretty well worked out. once its done, some1 can just leave it sitting on an old computer running 24/7 and it will do it all automagically.

i was also bored Wink
find quote