All right, this took a lot out of me (and I took a bit of a "break" partway through because work leaked into my free time), but I've finally got what appears to be a pretty good automated plugin cleaning solution. On my box, it cleans Update.esm, Dawnguard.esm, HearthFires.esm, and Dragonborn.esm all in about 7 seconds total. It can be this much faster than xEdit mainly because we have to tell it exactly which records are ITMs and which are UDRs instead of having xEdit do its filter thing to figure this out on its own, but also because we can combine a lot of work that's common to all plugins into one. Both xEdit and this tool need to catalog the records present in Skyrim.esm and Update.esm so it can properly do the UDR step (for those unfamiliar, UDR means "take references that were deleted in the plugin I'm cleaning, restore their contents and location from the previous version, and make a few well-known changes to them", so unlike for deleting ITMs which can be done independently as long as you have a list of what to delete, we need to have the previous version of everything). However, since all 4 plugins need to catalog Skyrim.esm records, and the latter 3 need to catalog Update.esm on top of it, we can get savings by reusing the same catalog across all the plugins we need to clean. It's also kinda cool that as soon as we "clean" Update.esm, we can add it straight to the catalog for cleaning the other plugins.
Now that I've got what I wanted out of that incredibly long detour, here's what I see is on the roadmap:
- Wrap the new automated plugin cleaning process so that it can be invoked via the XML file.
- Figure out the ITMs / UDRs for third-party plugins that also need cleaning and get their data into the XML file as well.
- Finish up the last one or two groups from STEP Core.
- Investigate automated SkyProc stuff.
- This is PROBABLY something I can leverage my new tool to do.
- Alternatively, maybe the output files can be distributed, since everything winds up being "static" anyway. I have to figure out what's appropriate from a non-technical standpoint.
- "Automate" Bashed Patch creation.
- Almost certainly, this new tool can do this. I'm probably just going to do a cop-out and define it in the XML file.
- Figure out DynDOLOD.
- I'm not sure if I'm ready to spend the couple of weeks that this would take to automate "nicely". Maybe this is where I just give up and write something for AutoHotkey.
- Testing, testing, testing.
- Especially for the automated plugin cleaning stuff, I had to give up on making the output exactly byte-for-byte identical with xEdit's output. xEdit has about a decade of history behind it, and I'm just the latest in a very long line of attempts to automate STEP, who's never even written a regular mod for Skyrim. I want to make sure I prove to myself, at least, that the differences are fine.
- Package and distribute.
- Keep up-to-date with newer versions of mods.
- Implement "Pack" support for things that build on top of STEP Core, starting with a "Pack" version of STEP Extended.
So that's what the future looks like. Right now, it's actually somewhat usable: you can just run the application with the right command-line args and if you have all the files downloaded and certain versions of the official DLC plugins pre-cooked, it'll set up Mod Organizer with a profile exactly as you'd expect by looking at the XML file, and it'll also create a batch file that you can double-click to start Skyrim without having to think about it. It's just that there's more that needs to be done behind-the-scenes to make it correct and complete.