Automated STEP Installer - Now in Beta
Posted 20 November 2013 - 01:35 PM
Posted 20 November 2013 - 01:47 PM
Perhaps its time for a new start with ASI and leverage the tools which exist. What has been discussed in the last week is that maintenance can be minimized by letting do MO the hard stuff. As in a packager creates a certain setup in MO which consists of mods with and without installers, with and without difficult to parse badly packaged mods. After that he selects the MO plugin 'export ASI config' which creates an XML file. This XML contains info like nexusid, version, category, modname (in MO), filename of the zipfile AND for each mod all files associated with it. When a users want to install a complete S.T.E.P. 2.2.7 he starts MO and selects the plugin 'create mods from an ASI config'. That way there is no need for an ASI config file which included dos commands like move, robocopy, delete etc.
I have several ideas how you could optimize your standalone ASI further and create parts of the config automatically (with regard nexusid, category, modname) but I'm not sure that would be in your interest in the long run. As in save your sanity. ASI will keep getting bigger and the users want an UI to be able to install only a part of S.T.E.P. 2.27, want a list of mods they have to download (taken into account what they already have installed/downloaded) etc etc. All kind of stuff MO already does or has support for.
My advice: Use the knowledge gained, use the new insights which have surfaced (maintenance issue) and start fresh by creating a plugin for MO. I know first hand how frustrating it can be to have hit a wall and how irritating it can be to receive 'advice' which basically tells you 'close but no cigar'. The truth is, sometimes starting fresh is the best and only way to go. I wish I could do that in RL but managers don't like that and I often have to 'enhance' a program which you should not enhance. Costing more money (and blood sweat and tears and swe*aring) in the long run then doing it right and start fresh. Managers never learn..........
That said, should you want to continue with ASI as an standalone executable I will try my best to help you further.
EDIT: Noticed that Z had created a post when I was creating mine. He basically is saying the same as I am.
Posted 20 November 2013 - 04:09 PM
Posted 20 November 2013 - 06:07 PM
Posted 20 November 2013 - 06:31 PM
Posted 20 November 2013 - 06:32 PM
Posted 20 November 2013 - 08:47 PM
I would love to do it, but I am only an aspiring programmer with no real talent and so much other stuff to do around here :/
The python code for dealing with BCFs is in bosh.py in Wrye Bash (Mopy/bash/bosh.py) beginning around line 8000 (search for 'bcf'). This may be instructive at least for learning how to access the CRCs and use them and the path info to construct/deconstruct file hierarchy within archives.
Posted 21 November 2013 - 02:26 AM
Posted 21 November 2013 - 03:20 AM
That is what I had thought, but it seems that MG et al are not interested in using those BAT scripts, so I thought that they might be interested in using Python. It all seems like duplicated effort nevertheless.
Could ModMerge be wrapped up into a plugin for MO?
Posted 21 November 2013 - 03:55 AM
Posted 21 November 2013 - 10:51 AM
Posted 21 November 2013 - 11:51 AM
So a lot will obsolete but I dislike throwing away posts, so here it comes.
Lets try it from a different angle. Lets investigate the complexity of the code and what you need to do. Lets start with what you have tagged as the hardests part: Finding of files with a specific MD5/CRC inside a mod (zipfile). The trick is to use libraries which can do the hard work for you so you can concentrate on the fun stuff, ie ASI.
I've done a bit of digging. sfk gives you what you need: "create list of md5 checksums over all files in a folder.by default, this also includes all subfolders (recursive tree walk).". You could parse the output of that external program but perhaps we can perform that action ourself. What is needed is a C++ library which does the following: "Walk a directory/Recursively". This is part of a C++ library called Boost. Let call the function FileTree. You point filetree to the directory you want to traverse. Every call to Filetree gives you a different filename. You can calculate the MD5/CRC value for that file. Now you put both filename and MD5 value in a hash table. The 'key' is the MD5 number and the 'value' is the filename. Now finding the filename of a file (in a mod directory) with a certain MD5 value is reduced to doing a lookup of the hash table. Each MD5 value represents a filename.
When processing the ASI exporter XML file (AEX) you process each mod after each other and at the end create a mod in
Iirc Tannin mentioned that the used archiving library has a way of accessing CRC value inside a zipfile. Perhaps you can use that library to fill the hash table as well. Tannin is also accessing the filing system, so perhaps he knows a better library.
Creating an ASI exporter file
I'n not an XML expert but have used it in the past.
The ASI exporter creates an XML file which contains all the information about the current mod setup in MO. In the MO UI all this info is known. Not sure if there is atm an API which can be used from a plugin to access all that info. If not iirc Tannin said he would extend MO so that the required info is exposed. Lets assume that API is in place. What you most likely do is to built your own object/structure which hold all needed information. Now an XML file has to be created. There are two ways to do this. First method. Create an XML file with a certain structure as you deem needed. This can be done with an XML parser. Second method: use an XML marshaller. For Java there are plenty and I assume they will also exist for C++. Basically an XML marshaller takes an object and automatically creates an XML structure or file for it. You have less control over the XML layout but it just works.
Reading in an XML exporter file.
When creating mods in MO the ASI plugin reads in an XML file which contains all information. With an XML unmarshaller you point it to an XML file and it automatically creates an object for it which you can access. This is the same object as when the XML file was created. With an XML parser you can also process an XML file but you have to do more yourself. There are quite a few XML parsers. There are two most popular APIs for XML parsers:
- DOM - Tree-based parser that reads entire document into the tree structure.
- SAX - Event-based parser that operates on each piece of document sequentially.
Tannin is already using an XML parser for the NMM importer. Perhaps he is even using an XML (un)marshaller. So he can help you out further with additional information.
Posted 21 November 2013 - 01:27 PM
Posted 21 November 2013 - 01:47 PM
Posted 21 November 2013 - 03:41 PM
Hey wolverine thanks for the information. Given me a lot to think about.
For my part I will be busy up until about the 16th of December. I have to prioritize my classes over my side projects, and my classes are getting into finals. After that I will be on break for about two weeks. I will give it my best effort for at least that long and see what I can do. If anybody else wants to run ahead with this thing by all means, I will try and catch up when I have the time.
Your welcome. I wish you all the luck with your finals which of course takes priority. I have finished the part about the XML exporter. Tannin is already using similar stuff in the NMM importer to read the XML file created by NMM. I know that looking at someone else code and trying to figure out what is going and then use it, it can be very time consuming. Especially if they are not using libraries to make things easier. I hope I was able to explain that using the right libraries you can start fresh with your own code and that it probably is less difficult then you first thought. Of course I've simplified things and things like a good UI will give you some troubles. But remember that the current MO plugins also use an UI, at least the manual installer and the NMM importer so you can have a look there. If in the future you need more information I will be glad to help you further, but I'm not a c/C++ programmer (Java guy).
I think you (and for that matter S.T.E.P. and we all) have come a long way. It looks as if the blueprint for an Automatic Skyrim Installer which is maintainable by S.T.E.P. staff and also usable for friends who want to share their setup with other is in place. Again success with the finals and enjoy your future coding efforts.
1 user(s) are reading this topic
0 members, 1 guests, 0 anonymous users