Jump to content


Photo

Automated STEP Installer - Now in Beta


  • Please log in to reply
191 replies to this topic

#181 torminater

torminater

    Dragon Prince

  • Contributors
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2,222 posts

Posted 26 November 2013 - 04:21 PM

What we would need, would be a CRC check, but not just for installing the files from the downloads, but to create a meta-file (aka library) of all the files in a specific Skyrim install, which is done through MO. To be clear: 1) A pack author/STEP affiliate creates a custom Skyrim install/STEP CORE/STEP Extended on his own system through MO 2) After checking everything is installed correctly verification that all esps remain unaltered and no bsa are extracted during install process!!! 3) Now he creates a CRC of all the needed download packages for this install. All of them will be written in a meta-library and uploaded on the pack page. 4) He then runs a sort of CRC program or MH5 (or whatever it's called) through MO on his virtual Skyrim game folder. The program creates a specific code for each file in that virtual data folder and puts the file and folder structure plus all the generated codes into a meta file. 5) The user downloads all the specified mods' versions the pack author/STEP maintainer used. Then he downloads the generated meta-library file for the downloads and for the install from STEP wiki. 6) The user runs ASI through MO. Now there are 2 options: A) ASI creates a Skyrim mod folder where all files from all mods are put in one folder and then installed through MO. Good for easy install, bad for user control. ASI reads the meta-library with the download packages and extracts all packages which have a CRC checksum that is inside that meta file to a working directory. Now ASI checks all the extracted packages for files which have the same checksums as on the custom Skyrim install meta-library and copies/moves them to the respective location in the folder tree of the skyrim mod folder. B) ASI reads the meta-library with the download packages and extracts all packages which have a CRC checksum that is inside that meta file to each their own mod folders in MO\mods. The user still needs to set the correct priority order, plus convert archives that have structures which are not known to ASI to MO compatible archives, plus delete files that are not wanted by the pack author/STEP affiliate.
  • 0

#182 mothergoose729

mothergoose729

    Thane

  • Members
  • PipPipPipPipPipPip
  • 462 posts

Posted 26 November 2013 - 05:35 PM

Yeah that is pretty much the idea, and more or less what monty has been working on with skymerge. My program doesn't do that though, and I can't think of a way to integrate CRC checksums into its current structure. If monty or someone else ever got a program like that to work we would probably abandon what I wrote in favor of it. Problem is that program is exceedingly difficult to write.
  • 0

#183 MilesTeg

MilesTeg

    Citizen

  • Members
  • Pip
  • 91 posts

Posted 26 November 2013 - 06:24 PM

yes, I've drafted s.th. similiar: #my kind of directory walkthrough (for package maintenance): go through (extracted) dir a: -save: source crc , path/filename go through dir b (installed): -save: dest crc , path/filename - find source crc in dest crc - merge them - upload info to wiki from there it's fairly easy to generate a config file. Of course you'r right, the crc for the archives should be added as well. " it could be made to make any number of data folders" yes, that's what I thought the first time I tried it. or you just change the directory names and invoke it with a batch loop. that part seems mostly solved. nice work indeed :) edit: I'm currently concentrating on getting checksums/filenames and saving those to a file. I tried using the ModMerge bash scripts but they failed badly. I think I'll give skyrep itself another try...
  • 0

#184 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,245 posts

Posted 27 November 2013 - 12:56 AM

yes, I've drafted s.th. similiar:

#my kind of directory walkthrough (for package maintenance):

go through (extracted) dir a:
-save: source crc , path/filename

go through dir b (installed):
-save: dest crc , path/filename

- find source crc in dest crc
- merge them
- upload info to wiki


from there it's fairly easy to generate a config file.

Of course you'r right, the crc for the archives should be added as well.


" it could be made to make any number of data folders"
yes, that's what I thought the first time I tried it. or you just change the directory names and invoke it with a batch loop. that part seems mostly solved. nice work indeed :)

edit:
I'm currently concentrating on getting checksums/filenames and saving those to a file. I tried using the ModMerge bash scripts but they failed badly. I think I'll give skyrep itself another try...

Yes, that is the most concise way I have seen it stated. It is simpler to design something with the objectives clearly and simply stated. (it also make bit simpler for others to follow along and pitch in where they see an opportunity).

I am in favor of a one-click solution to start with. This satisfies the main objective: make it simple to install STEP:Core (or any mod recipe), period. Anyone that wants more fine-grained control is probably familiar with using the guide and installing that way. The more piecemeal and configurable version of the tool requires more logic to be built in, and would be inherently more complex ... phase II work I think.

Also remember that the program should not fail if all CRCs are not present or do not match, and it should simply ignore missing data (perhaps with a log showing the details).

Finally, It would probably be good to define milestones and begin working on the code within GitHub or something like that (maybe add to what MG has already started, as the new methods can be added to that repo and all doc kept in one place). That is a workflow call though, so it is up to the people working on this ... but if they want others to contribute freely and those contributions to be tracked ...

#185 MilesTeg

MilesTeg

    Citizen

  • Members
  • Pip
  • 91 posts

Posted 27 November 2013 - 11:37 AM

I am in favor of a one-click solution to start with. This satisfies the main objective: make it simple to install STEP:Core (or any mod recipe), period. Anyone that wants more fine-grained control is probably familiar with using the guide and installing that way. The more piecemeal and configurable version of the tool requires more logic to be built in, and would be inherently more complex ... phase II work I think.

exactly. And that's why we also need to be very precise in what should go into "our" finished STEP:Core. Common sense would probably be a "all DLC" strict Baseline configuration will all things "recommended" in the guide. Any objections against this?
As you said, there can be other options later (phase II) and I thing support for vanilla skyrim should be #1 priority, when we start working on that - we shouldn't leave players in the cold if they don't have all DLCs. But that probably doesn't concern us for some months. We should first try to get a stable installation process (with the mentioned setup) working.

Also remember that the program should not fail if all CRCs are not present or do not match, and it should simply ignore missing data (perhaps with a log showing the details).

Yes, that's probably the first thing the program will do: check files and their crc and give out errors if something doesn't seem right. The user should get a simple msg box: Continue anyway? Yes/No.

Finally, It would probably be good to define milestones

see http://wiki.step-project.com/SASTEP


and begin working on the code within GitHub or something like that (maybe add to what MG has already started, as the new methods can be added to that repo and all doc kept in one place). That is a workflow call though, so it is up to the people working on this ... but if they want others to contribute freely and those contributions to be tracked ...

have to set it up first, but it's on my todo list
  • 0

#186 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,245 posts

Posted 27 November 2013 - 12:04 PM

[quote='MilesTeg' pid='59303' dateline='1385570277']
[quote name=''z929669' pid='59266' dateline='1385531775']I am in favor of a one-click solution to start with. This satisfies the main objective: make it simple to install STEP:Core (or any mod recipe)' date=' period. Anyone that wants more fine-grained control is probably familiar with using the guide and installing that way. The more piecemeal and configurable version of the tool requires more logic to be built in, and would be inherently more complex ... phase II work I think.[/quote']
exactly. And that's why we also need to be very precise in what should go into "our" finished STEP:Core. Common sense would probably be a "all DLC" strict Baseline configuration will all things "recommended" in the guide. Any objections against this?
As you said, there can be other options later (phase II) and I thing support for vanilla skyrim should be #1 priority, when we start working on that - we shouldn't leave players in the cold if they don't have all DLCs. But that probably doesn't concern us for some months. We should first try to get a stable installation process (with the mentioned setup) working.

[quote]Also remember that the program should not fail if all CRCs are not present or do not match, and it should simply ignore missing data (perhaps with a log showing the details).[/quote]
Yes, that's probably the first thing the program will do: check files and their crc and give out errors if something doesn't seem right. The user should get a simple msg box: Continue anyway? Yes/No.

[quote]Finally, It would probably be good to define milestones[/quote]
see http://wiki.step-project.com/SASTEP


[quote]and begin working on the code within GitHub or something like that (maybe add to what MG has already started, as the new methods can be added to that repo and all doc kept in one place). That is a workflow call though, so it is up to the people working on this ... but if they want others to contribute freely and those contributions to be tracked ...[/quote]
have to set it up first, but it's on my todo list
[/quote]
Yes. Pure STEP:Core, as defined by the STEP Guide is a requirement. I and others think that all DLC should be required for STEP:Core henceforth in order to simplify things for modders and developers (and STEP teams).

Thanks for adding to Git!

#187 mothergoose729

mothergoose729

    Thane

  • Members
  • PipPipPipPipPipPip
  • 462 posts

Posted 27 November 2013 - 12:09 PM

With a CRC based approach you have to create multiple configurations and take a sort of "snapshot" of that. The simplest implementation would be to snapshot a vanilla only version, and then an all DLC version, and have the user pick between configurations. At least this has been the current design. It should be possible to layer configurations on top of each other. Remember, all files have unique keys, so if the program searches for keys that aren't there, and moves keys that are there, we can give the user some flexibility in terms of exactly what kind of installation they choose. It would increase the complexity of the algorithm some, and more importantly, the creation of the package would become more complex. Something to think about of we ever get this that far.
  • 0

#188 MilesTeg

MilesTeg

    Citizen

  • Members
  • Pip
  • 91 posts

Posted 27 November 2013 - 12:22 PM

Yes. Actually we need to take snapshots of every mod directory to keep it as clean (and layered) as possible. That will be the case from the beginning. And I'm pretty sure you leave out some mods the proof-of-concept version will still work with a vanilla skyrim. But for the start we should concentrate more on getting ONE working configuration (I'm neutral if that should be vanilla or all DLCs) and go on from there. When we get to a point where we can think about options/multiple configs/profiles THEN the first thing todo is assuring vanilla compatibility. But, yes, we should always keep the (later) goal of layered design in mind. If some of you don't have all DLCs I don't have a problem at all to go with vanilla first.
  • 0

#189 mothergoose729

mothergoose729

    Thane

  • Members
  • PipPipPipPipPipPip
  • 462 posts

Posted 27 November 2013 - 12:34 PM

I didn't think about that actually. With MO creating a flexible mods directory would not be that hard at all. Perhaps a scheme where the CRC program builds for a MO installation, and then an adaptation of my program could be made to build for a manual/NMM installation. The mod directories in an MO install are essentially all mini data folders, so packaging them up as archives and shuffling them together would not be that difficult at all with my program. EDIT: What I mean is, in MO you can add any number of mods to an installation and the "virtual data directory" has only the mods that are not overwitten, but the mods folder has all the files irrespective of what actually goes into the data directory. What this means is that when creating a mods folder to "snap shot" you can include any number of mods you like, both catering to and not catering to people with different DLCs, version, personal preferences, whatever, and it will all be there. The CRC program comes through and tags everything, and then the user just assembles the mods that suites them. A prompt for missing files would not be ideal with that, because then every user would be prompted for every possible mod that could be included, but it wouldn't much complicate the creation of packages or the algorithm that could be used to recreate them. As an after thought, my program could be used to package together an literal data folder structure, for people that want to manually install or use a different mod manager like NMM.
  • 0

#190 z929669

z929669

    Ixian Inventor

  • Administrators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 9,245 posts

Posted 27 November 2013 - 01:26 PM

Just as you guys want to create this ...
... but I think the default should be all DLC included, otherwise, there are 2#DLC (or 8) variations. The other 7 variations could be configured after, but we should be spearheading the push to incentivize users to obtain all DLC, as this is the best solution for the whole of the modding community authors and content supporters, and persons with all DLC should not suffer a cost imposed by a decision to primarily support those without them.

#191 MilesTeg

MilesTeg

    Citizen

  • Members
  • Pip
  • 91 posts

Posted 27 November 2013 - 01:47 PM

EDIT: What I mean is, in MO you can add any number of mods to an installation and the "virtual data directory" has only the mods that are not overwitten, but the mods folder has all the files irrespective of what actually goes into the data directory.

see http://wiki.step-project.com/SASTEP

I think MO support and use is a nonbrainer here :)

It's only a matter of time till we get a working MO-Batchinstaller (but that's just not priority right now). There are no hurdles in sight for this.
  • 0

#192 wolverine2710

wolverine2710

    Mod Author

  • Contributors
  • PipPipPipPipPipPip
  • 441 posts

Posted 27 November 2013 - 04:45 PM

Currently on holiday and have very limited time to check things. Since my last check a LOT of very postive stuff has happened. LOOKING GOOD guys. Perhaps next week around this time when I´m back there is an alpha which I can put gently on the rack and see what it is made of..... Such fond memories of AV and MO. I have this nasty gift/habit (but useful for my job as a Java progammer) to find these little critters, even if they try hide fromme. Again good job and nice to see that when one person has no time anymore (finals) that another person takes up the gauntlet... Gonna drink a few vinos and going to bed (he I´m holiday after all).
  • 0


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users