Jump to content


Photo

ReLinker


  • Please log in to reply
135 replies to this topic

#16 dunc001

dunc001

    Thane

  • Members
  • PipPipPipPipPipPip
  • 434 posts

Posted 16 August 2016 - 11:45 AM

In a nutshell, this "finishes" the merge by merging what MergePlugins can't merge (the scripts and LODGen)

I'm not sure this entirely explains it.  Take the example of the Treasure Hunts merge for the five treasure hunt mods which integrate into Legacy of the Dragonborn.  For the displays to be enabled the LotD enabler script looks for the original esp names.  If all five are merged into one new esp then obviously when LotD looks it can't see them and the displays remain disabled.  What ReLinker does is not simply an extension of Merge Plugins.  It scans your load order for merges, then reads the plugins.txt and map.txt in each merge. Then it looks at ALL of the other scripts in your load order looking for any instance of any of the original esp names for mods now combined in any of the merges.  When it finds an instance it then changes it to point at the new merged plugin esp and edits any related formIDs based on the relevant map.txt.  This way LotD now knows that Treasure Hunts Merged.esp now contains all five original esps and the enabler script is updated accordingly allowing the displays all to enable.

 

TLDR - ReLinker edits any script in your load order (not just those which are part of a merge) which is looking for an esp now contained in a merged plugin to point at the merge instead.


  • 1

#17 Ganda

Ganda

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 375 posts

Posted 16 August 2016 - 11:48 AM

I'm not sure this entirely explains it.  Take the example of the Treasure Hunts merge for the five treasure hunt mods which integrate into Legacy of the Dragonborn.  For the displays to be enabled the LotD enabler script looks for the original esp names.  If all five are merged into one new esp then obviously when LotD looks it can't see them and the displays remain disabled.  What ReLinker does is not simply an extension of Merge Plugins.  It scans your load order for merges, then reads the plugins.txt and map.txt in each merge. Then it looks at ALL of the other scripts in your load order looking for any instance of any of the original esp names for mods now combined in any of the merges.  When it finds an instance it then changes it to point at the new merged plugin esp and edits any related formIDs based on the relevant map.txt.  This way LotD now knows that Treasure Hunts Merged.esp now contains all five original esps and the enabler script is updated accordingly allowing the displays all to enable.

 

TLDR - ReLinker edits any script in your load order (not just those which are part of a merge) which is looking for an esp now contained in a merged plugin to point at the merge instead.

Dunc, do you mind making a "tiny" explanation to add to the OP and to the nexus?


Edited by Ganda, 16 August 2016 - 11:49 AM.

  • 0

#18 Darth_mathias

Darth_mathias

    Dragon King

  • Pack Authors
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 3,611 posts

Posted 16 August 2016 - 12:14 PM

nice to see this released guess I will have to finally get up off my ass and give it a try.


  • 0

#19 hishutup

hishutup

    Daedric Prince

  • Super Moderators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2,554 posts

Posted 16 August 2016 - 02:08 PM

Why I think MP should not handle this.

The billboards, yes but for the scripts, probably not. Its kind of hard to explain why but treat it like bash but instead of records, its scripts. Let's say you have an interesting merge where plugins a, b, c, d and e reference each other. If someone merges b and c together(BC), then merges d and e(DE). Also, let every formid be altered. If mp handled this in the merge, only a and BC will work, BC would know nothing of DE and vice-versa. However, treating it like a "autopatcher" avoids this issue.

As for smash, why not that's up to mator, Ganda is pretty open.

When you say "scripts" that MergePlugins doesn't handle, do you mean scripts that use the GetFromForm and/or GetFromFile functions (the functions that hardcode the ESP/ESM filename into the script)?
If so, that is awesome, because that is the last piece that Mator's MergePlugins standalone doesn't handle (since he added script fragment handling).

Yup, it handles those and a few custom functions. sometimes the compiler fails to compiler, I found a few condition because of skyUI which I should add to the op.

#20 Ganda

Ganda

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 375 posts

Posted 16 August 2016 - 02:17 PM

Ganda is pretty open.


Phrasing!

Yup, it handles those and a few custom functions. sometimes the compiler fails to compiler, I found a few condition because of skyUI which I should add to the op.


Read the op again hishy. Also, comments are waiting for you in the nexus
  • 0

#21 hishutup

hishutup

    Daedric Prince

  • Super Moderators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2,554 posts

Posted 16 August 2016 - 02:26 PM

Phrasing!

Nah...

 

 

Read the op again hishy. Also, comments are waiting for you in the nexus

Nah...

No, comments are waiting for you.



#22 dunc001

dunc001

    Thane

  • Members
  • PipPipPipPipPipPip
  • 434 posts

Posted 16 August 2016 - 02:33 PM

Dunc, do you mind making a "tiny" explanation to add to the OP and to the nexus?


Sure, np as long as you don't need it yesterday...
  • 0

#23 Mator

Mator

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 417 posts

Posted 16 August 2016 - 03:46 PM

In a nutshell, this "finishes" the merge by merging what MergePlugins can't merge (the scripts and LODGen)

 

EDIT: As I said before, I made this on hishy's instructions. See this post.

Slightly inaccurate.  The only scripts Merge Plugins "can't merge" are ones with GetFormFromFile calls (also GetModByName, checkModPresence, and IsPluginLoaded calls).  That's what you're handling here.  Merge Plugins handles script fragments and ordinary scripts don't need any special handling.

 

EDIT: Seems other people covered that already.

 

I explained all of this to hishy in depth at some point.  It's cool that he managed to remember the details enough to get someone to make a solution.

 

Merge Plugins COULD handle it, but it would have to function similar to how Ganda has this programmed in that it'd have to check all scripts after each merge because each merge can require the alteration of any number of scripts from any number of other mods/merges.

 

ANYWHO...

 

Really awesome to see this!  I wanted to program it but I've been programming my ass off on Mod Picker so I haven't had the time.  Nice work Ganda, I'll check out the code sometime soon-ish.  :)

 

EDIT 2:

 

The patch is only needed for compiling when running the CK through MO. Else the compiler in the CK will fail. It's not likely needed in any other situations such as this.

 

I kinda see this being absorbed into Mator's tool since that is just local. Would you be okay if he added this into Mator Smash? Though I don't think he's worked on that tool in a long time. Not since he started his website project.

I think it'd be better to absorb it into Merge Plugins because Smash doesn't have the hooks into the Papyrus Compiler/Decompiler integrations yet.  It'd be a "post-merge fix things patch".  The LODGen stuff can be handled immediately during the merging process like other file specific assets though.
 

 

EDIT 3:
Line 278 LMAO.

This feels a bit inefficient.  The way I'd do this is to process all the script files once and build a dictionary of all the GetFormFromFile, GetModByName, checkModPresence, and IsPluginLoaded calls in all scripts, then iterate over that dictionary and for each dictionary item iterate across all the maps.  Or just combine all the maps into a single map and use a simple indexOf call on a single map to see if a plugin is merged.  The main reason this would be faster is you only have to load the script files once to build the dictionary of the calls, then load them once more to actually replace the calls as necessary.  Right now you're loading the files n times, where n is the number of maps in merge_data_list, which is a lot of Disk I/O which is slow (even with tiny files).

 

Alternatively you could also just reverse the loop order (so iterate over the files and per file iterate over the maps).  This would force the LODGEN to be a separate loop as well.

 

Looks pretty good overall though.  Only other thing I'd suggest is breaking things into more functions so you don't have quite as much nesting going on and so the main procedure is abstracted a bit.


Edited by Mator, 16 August 2016 - 04:16 PM.

  • 0

#24 Ganda

Ganda

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 375 posts

Posted 16 August 2016 - 04:14 PM

Line 278 LMAO.


Just for you <3
  • 1

#25 Mator

Mator

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 417 posts

Posted 16 August 2016 - 04:34 PM

Are you working on the Nexus page still, Ganda?  It came up on google search results but said the author has hidden it.  :'(

 

Let me know when it's ready so I can pop in and give my thumbs up/endorsement.  ^_^

 

 

EDIT: Also, are you planning on posting this to r/skyrimmods?


Edited by Mator, 16 August 2016 - 04:37 PM.

  • 0

#26 Ganda

Ganda

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 375 posts

Posted 16 August 2016 - 05:24 PM

This feels a bit inefficient.  The way I'd do this is to process all the script files once and build a dictionary of all the GetFormFromFile, GetModByName, checkModPresence, and IsPluginLoaded calls in all scripts, then iterate over that dictionary and for each dictionary item iterate across all the maps.  Or just combine all the maps into a single map and use a simple indexOf call on a single map to see if a plugin is merged.  The main reason this would be faster is you only have to load the script files once to build the dictionary of the calls, then load them once more to actually replace the calls as necessary.  Right now you're loading the files n times, where n is the number of maps in merge_data_list, which is a lot of Disk I/O which is slow (even with tiny files).
 
Alternatively you could also just reverse the loop order (so iterate over the files and per file iterate over the maps).  This would force the LODGEN to be a separate loop as well.
 
Looks pretty good overall though.  Only other thing I'd suggest is breaking things into more functions so you don't have quite as much nesting going on and so the main procedure is abstracted a bit.

On my defense, I did that at 3 am before a 12 hour trip :p But yes, it could be optimized a lot more, the code is SUPER fugly and it's a bunch of spaghetti, I'll get to it after some other things I need to post.

The combine merge maps thing, python's dictionaries are unordered (and I don't like using ordereddicts on these cases) so merges of merges would get burnt :/ I'll find a way
 

Are you working on the Nexus page still, Ganda?  It came up on google search results but said the author has hidden it.  :'(
 
Let me know when it's ready so I can pop in and give my thumbs up/endorsement.  ^_^
 
 
EDIT: Also, are you planning on posting this to r/skyrimmods?

I somehow deleted the page o.0 Hishy will make a new one, since he's the brains on this one. and I never really posted anything on skyrimmods or ever been there tbh :/ All I've heard about was a couple of years ago that the community over there was worse than nexus, is it better now?
  • 0

#27 Mator

Mator

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 417 posts

Posted 16 August 2016 - 07:12 PM

On my defense, I did that at 3 am before a 12 hour trip :p But yes, it could be optimized a lot more, the code is SUPER fugly and it's a bunch of spaghetti, I'll get to it after some other things I need to post.


For sure man. ^_^
 

The combine merge maps thing, python's dictionaries are unordered (and I don't like using ordereddicts on these cases) so merges of merges would get burnt :/ I'll find a way


I don't support merging merges in merge plugins. I feel like saying "if someone does that, **** them", but building your code for the lowest common denominator is always a good idea.
 

I somehow deleted the page o.0 Hishy will make a new one, since he's the brains on this one. and I never really posted anything on skyrimmods or ever been there tbh :/ All I've heard about was a couple of years ago that the community over there was worse than nexus, is it better now?


Looking forward to seeing it back up! Silly Ganda deletes his own mods. XD

r/skyrimmods is arguably the best part of the Skyrim Modding community right now, in my opinion. I love hanging out there and interacting with the community there. I've also heard it was bad at some point, though it's been nothing but awesome while I've been there.  With regulars like Chesko, Sheson, Enai, myself, and others you should fit right in. :)


Edited by Mator, 16 August 2016 - 07:31 PM.

  • 0

#28 Ganda

Ganda

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 375 posts

Posted 17 August 2016 - 05:06 PM

Sorry mator (and whoever else this applies to) that I haven't answered yet, because hishy made me learn how to edit pex files directly which I dreaded (BUT IT WAS SOOO FUUUUUUUUN) so I spent the last day just doing that. I'll see to applying it to the relinker and I'll answer everyone whenever I get back from the very deep hole I'm about to put myself into. And thanks to the random nexus user, I have no idea who it is since that page was deleted, for suggesting this and sorry for thinking you were crazy.


  • 0

#29 hishutup

hishutup

    Daedric Prince

  • Super Moderators
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2,554 posts

Posted 17 August 2016 - 05:17 PM

Sorry mator (and whoever else this applies to) that I haven't answered yet, because hishy made me learn how to edit pex files directly which I dreaded (BUT IT WAS SOOO FUUUUUUUUN) so I spent the last day just doing that. I'll see to applying it to the relinker and I'll answer everyone whenever I get back from the very deep hole I'm about to put myself into. And thanks to the random nexus user, I have no idea who it is since that page was deleted, for suggesting this and sorry for thinking you were crazy.


It was the guy that made the restringer.

#30 Ganda

Ganda

    Thane

  • Mod Authors
  • PipPipPipPipPipPip
  • 375 posts

Posted 18 August 2016 - 12:47 PM

Ok guys, I need heeelp! I'd really appreciate it if someone could get me some scripts, both compiled and source files, that contain the following functions:
  • GetModByName but without Game. before it;
  • GetFormFromFile but without Game. before it;
  • IsPluginLoaded;
  • checkModPresence.
I need to check how these are stored. Thanks! :D
 

r/skyrimmods is arguably the best part of the Skyrim Modding community right now, in my opinion. I love hanging out there and interacting with the community there. I've also heard it was bad at some point, though it's been nothing but awesome while I've been there.  With regulars like Chesko, Sheson, Enai, myself, and others you should fit right in. :)

 
I'll give it a shot once I finish this! :p
 

It was the guy that made the restringer.


Wooow...
  • 0


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users