Jump to content


Photo

Status of this?


  • Please log in to reply
37 replies to this topic

#1 Terrasque

Terrasque

    Prisoner

  • Members
  • 20 posts

Posted 15 May 2015 - 06:01 AM

Hey, this seem to have been dead for some years.. I wonder, what was the status, and what made it stop?

 

The reason I'm asking is because I'm working on something similar called YAMM - and someone pointed out this project.

 

I'm wondering if my project will hit the same problems this had, and if so I'd like to know them so I can figure out if it's pointless or the issues can be avoided or dealt with.


  • 0

#2 gpthree

gpthree

    Guard

  • Members
  • PipPip
  • 114 posts

Posted 15 May 2015 - 06:25 AM

What is this 'this' that you are referring to be dead?

 

EDIT: I went straight to the post from the main page, didnt notice this was under the SEMI-Automatic STEP forum.


Edited by gpthree, 15 May 2015 - 06:29 AM.

  • 0

#3 Nozzer66

Nozzer66

    Dragon Prince

  • Pack Authors
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2,498 posts

Posted 15 May 2015 - 06:30 AM

I believe he's referring to the Semi-Automated STEP Install.

 

The project The Tarrasque's working on looks like something that checks if you have all a mod's dependent mods loaded and if not automatically grabs and downloads those as well. I think that's it. Correct me if I mis-speak.


  • 0

#4 Terrasque

Terrasque

    Prisoner

  • Members
  • 20 posts

Posted 15 May 2015 - 07:32 AM

I believe he's referring to the Semi-Automated STEP Install.

 

The project The Tarrasque's working on looks like something that checks if you have all a mod's dependent mods loaded and if not automatically grabs and downloads those as well. I think that's it. Correct me if I mis-speak.

More or less, yes. So if you want to install GreatMod, and that depends on CoolMod and SomeLibrary, and CoolMod depends on SomeOtherLibrary and YetAnotherMod, and so on.. It'll create a list of all mods needed, and download them if possible. Right now I just have a simple test list up with a few mods, but it works well with them (and some of them require a total of 15+ mods installed to work)


  • 0

#5 DoubleYou

DoubleYou

    Wiki Stepper

  • Super Moderators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 4,480 posts

Posted 15 May 2015 - 03:31 PM

Unfortunately, the developer who was working on this for us disappeared, so it is pretty much dead.



#6 GrantSP

GrantSP

    The antipodean

  • Super Moderators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 4,012 posts

Posted 15 May 2015 - 07:13 PM

I notice you mentioned an 'experimental' MO plugin you have. Is it at a testing stage?



#7 hishutup

hishutup

    Daedric Prince

  • Super Moderators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2,592 posts

Posted 15 May 2015 - 08:46 PM

I thought the nexus didnt like hotlinking mod downloads and that was one of the biggest reasons



#8 Terrasque

Terrasque

    Prisoner

  • Members
  • 20 posts

Posted 16 May 2015 - 04:12 AM

I notice you mentioned an 'experimental' MO plugin you have. Is it at a testing stage?

Yes. Althrough the only thing it does so far is tell MO "Can you install a mod from this archive?" - same as opening an archive from MO does.

 

I have a post at the MO support forum asking about advice for expanding that.

 

 

I thought the nexus didnt like hotlinking mod downloads and that was one of the biggest reasons

 

Well, this isn't hotlinking. And it's not tied to any one site, and can be supported by any number of sites. Right now I have a small test service up so I can test and develop the system, and showcase how it works.

 

I'm hoping this becomes something that sites like Nexus and LoversLab is willing to support themselves some day.


  • 0

#9 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 20 May 2015 - 12:37 PM

Nexus wants users to visit their site and mod pages. An automated downloader subverts this, but I support such a utility if it allows the user to manually visit the Nexus mod pages manually ...

 

... for example, MO is only aware of mods (packages) previously obtained and linked to its Nexus counterpart (meta-information). This almost always requires that the user visits the Nexus to obtain the meta-info (the mod ID from the URL at least). Furthermore, once linked, MO mods include links to the corresponding Nexus mod page and integrates with the endorsement system, too. In this way, MO is, in part, a Nexus 'helper' mod management system that sits on top of the Nexus backbone (in addition to its base functionality). It does not subvert use of the Nexus, but rather enriches mod management with Nexus links and meta-info.

 

If you were to create a MO plugin that extends this functionality (which is what YAMM seems to be doing), then I think it could be very useful. The problem is standardization of meta-information on the host side (or by leveraging info from some ancillary service like LOOT/BOSS masterlists or Mlox, which manually/semi-automatically compile their own meta-info ... Wrinklyninja may have some tips). Mod authors (or the third party) also have to define all dependencies (implicit/explicit, upstream/downstream).

 

This is effectively like a Linux package management system that only works if there is a common standard that all parties agree to follow. It makes things simpler for the end user (and in terms of support by mod providers), but it has high ongoing maintenance cost for the providers. STEP has use for such a tool, since it would be relatively easy and ultimately time saving for us to maintain the meta-information as it relates to STEP; however, more broad application requires a concerted effort across dispirate asset/info providers.

 

STEP has wanted such a solution (as an MO extension) for a long time.



#10 Terrasque

Terrasque

    Prisoner

  • Members
  • 20 posts

Posted 22 May 2015 - 02:07 PM

That's nice to hear!

 

Yeah, Nexus of course want people to be part of their community, and also view ads / buy premium. And seeing the hosting cost they have, I'm not surprised. All mods added to a service can have a homepage associated with it, and if nexus provided a service it would be natural to set that link to the mod's page on nexus.

 

Let me see.. The meta data looks like this at the moment. Some things to note there: SKSE doesn't have a download link, only homepage entry. And Dawnguard is referenced in one recommended, but has no entry itself. This would let you easily define meta-packages for collections of other mods without actually having a file itself.

 

Right now I'm hosting a copy of the mods listed there, for testing and development purposes and demonstration. There's nothing technical that stops Nexus (or LoversLab, or step) to have their own json (aka a yamm service) listing mods and download locations. Currently the client compiles everything into a local database that it uses for searches and finding dependencies, and works cross services. So if STEP and Nexus both provided a yamm service, STEP could have meta-packages and a few mods unique to STEP, and recommend / depend on mods on Nexus.

 

It's also designed to be low resource cost, with the json file being just a static file served and client understanding etag when checking for updates. It also have a built in way to define file mirrors, and can optionally verify downloaded files via SHA256. File mirrors could be handled on an automated basis by volunteers too, and an idea I had was to add a field for pingback url, and a program to mirror a service and ping back the mirror url automatically.

 

I'm still working on it, but it's been slow going lately. Been busy with work and gaming :)

 

Currently working on:

 * Better MO integration

 * Better download UI

 

Looking into:

 * Torrent / Magnet link support via Transmission client and RPC. This should further help resource usage.

 * URL schema handling, for being able to click a link on a webpage for adding a service or fetch a mod

 

Also working on server software to easily host and manage a YAMM service. Looking into torrent integration and more user facing stuff, right now it's more or less just for uploading a file in an admin interface and enter metadata and auto generate the json.


  • 0

#11 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 22 May 2015 - 02:36 PM

OK, if we only need to host a json file, then it is totally doable I think.

  • Does your Py program provide an easy way to construct the json or is that a manual process? There should be a text-json conversion utility I could use though.
  • Does your program only depend on the json? In other words, will files be fetched from mod links defined in the json?
  • Will the MO plugin version fetch and deposit the mods into the defined MO "downloaded mods" path and then invoke MO's installation utility for each of those mods in turn? As in: download mod1 > install mod1 > ... download modn > install modn OR download mod1 ... download modn > install mod1 ... install modn
  • Do you have any plans to implement instructions for the automated MO-based installation/configuration of each mod? In other words, if the json had atributes defining installation/MO configuration, would it be feasible to invoke automated installation?

Ideally, all of this configuration would happen from within MO. One would launch MO, launch your app/plugin from MO, invoke the json, perform installation, perform MO configuration as we define in the json for each mod. Then the STEP json (or any other) could define the mod list, installation order, installation properties, MO mod config ... vuallah! ... semi-automated STEP via MO using your plugin. (as long as downloads occur from Nexus and pass appropriate tokens identifying the Nexus member downloading the mods and counters, etc. ... as a proxy for manually downloading each individual mod ... meaning that Nexus has no idea that these downloads are being performed by a service as a proxy of the Nexus member rather than the member).



#12 Terrasque

Terrasque

    Prisoner

  • Members
  • 20 posts

Posted 22 May 2015 - 03:43 PM

 

Does your Py program provide an easy way to construct the json or is that a manual process? There should be a text-json conversion utility I could use though.

The client program doesn't have a way to generate a json, but I made a simple django-based webapp for it. Code is at https://github.com/T...rasque/yammyamm

 

 

Does your program only depend on the json? In other words, will files be fetched from mod links defined in the json?

It only depens on the json info. If the json doesn't have a link to a file, it can't download it.

 

 

Will the MO plugin version fetch and deposit the mods into the defined MO "downloaded mods" path and then invoke MO's installation utility for each of those mods in turn? As in: download mod1 > install mod1 > ... download modn > install modn OR download mod1 ... download modn > install mod1 ... install modn

Right now it has a mod info and download window that looks like this. Clicking the "To MO" button will try to open the "install this mod" in MO (if yamm's plugin is installed). But I'm thinking of reworking that a bit

 

Do you have any plans to implement instructions for the automated MO-based installation/configuration of each mod? In other words, if the json had atributes defining installation/MO configuration, would it be feasible to invoke automated installation?

Not at the moment, but a meta-mod containing a fomod installer or an exe  or override files could perhaps do that? I'm not that well versed in actually installing mods, so don't know what would be needed there. Load order could be embedded in the json, with a priority field I guess. But I think that could get messy fast when you have a lot of mods. Again, I don't know the advanced things around mod installation.

 

 

(as long as downloads occur from Nexus and pass appropriate tokens identifying the Nexus member downloading the mods and counters, etc. ... as a proxy for manually downloading each individual mod ... meaning that Nexus has no idea that these downloads are being performed by a service as a proxy of the Nexus member rather than the member).

This could technically be done with a field for the nexus ID, but it would likely also make an enemy out of the Nexus admins, especially if it became popular. That's not my intention - I'd rather have their support :)

 

Edit: When that's said, I think the SA-STEP goals could be achieved, but that would need cooperation with the MO devs. I've been thinking about having option extra / extended json file for mods that can be downloaded for more metadata. So far I've only thought about screenshots and extended description, but having installation metadata could fit in there well. Acting on that metadata should be in MO's domain (or an MO plugin), but yamm can get the data to MO.


Edited by Terrasque, 22 May 2015 - 03:48 PM.

  • 0

#13 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 22 May 2015 - 04:20 PM

OK, I will play with your resources on Git over the next few days as I have time and get back to this thread.

 

Thanks for working on this!



#14 Terrasque

Terrasque

    Prisoner

  • Members
  • 20 posts

Posted 30 May 2015 - 07:51 PM

I don't know if you've had a chance to look at it yet, but I've worked on it when time allows, and just pushed a new release to master. I've also got running http://yamm.thelazy.net/ to both help development and to showcase the system more.

 

It contains 3 interlinked services, and I also got to test out and improve the new URL protocol. Getting closer to testing bittorrent, but there's a lot backend needed before I can do any meaningful tests there.

 

The MO plugin and the download window is still weak. For example, closing the window while it downloads will lead to aborted downloads, and no way to continue or re-download except deleting the file in explorer. But it should be good enough for testing it out, and it works :)

 

Right now, if you have the YAMM client installed with URL handling and visits yamm.thelazy.net, find some mod, and click "Open in YAMM" it will also ask if you want to add the service for that mod if you don't have it. And if the service points to other services, ask if you want to add them too. This is especially noticeable on the loverslab mods, where if you have no services and click on a mod it will first ask if you want to add the loverslab service, then ask about adding nexus, and then ask (from nexus) if you want to add the third, "default" service on the site. Then it will pop up a window for the mod with all requirements needed.

 

Which is pretty cool :) And demonstrates how it can work when multiple sites use this system.


  • 0

#15 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,258 posts

Posted 01 June 2015 - 09:53 AM

Great ... glad I have not tested yet, so I will get a first impression from your updated version.

 

I do intend to take a look, but I'm caught in lots of other fires that need tending. Wil post back though when I do. I am watching this thread.

 

Thanks again!




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users