1. If the plugin changes landscape/terrain, especially the terrain height, then you should generate terrain LOD with
either CK or OscapexLODGen. Oscape may produce buggy terrain meshes with sheer drops in height. If that is the case try to EMSify the plugin just for generation.
Once terrain LOD is generated, just pick the Data\Meshes\terrain\tamriel\tamriel.*.btr and Data\Textures\terrain\tamriel\tamriel.*.dds from the area of your house (CK offers generating specific chunks I believe to save some time). For terrain LOD, I would only include LOD level 4 and maybe 8, as 16/32 affect a much larger area and the map, which is probably not needed.
To include object LOD, either use CK or xLODGen. DynDOLOD is intended to generate LOD for entire worldspaces. I will talk about both xLODGen and CK, but first we need to talk about the object LOD texture atlas.
Before generating object LOD with CK/xLODGen I suggest to install the fixed vanilla "data" from Skyrim Vanilla LOD Source Files v2.5 - no need for the "source" folder if generating LOD with xLODGen/DynDOLOD. DynDOLOD has the same fixes included, so there is no need for it, but in case it is installed, DynDOLOD Resources should overwrite it.
The problem with including object LOD for only a small area is, that it needs to either use the vanilla object LOD texture atlas, or its own atlas with a unique file name in order to be compatible with other mods. The object LOD BTO meshes support pointing to a different atlas texture. xLODGen/DynDOLOD both use a distinctive file name, which is different from vanilla. So, mixing only parts of their object LOD with vanilla object LOD is possible. Though, again, if you have another mod that tries to do the same, problems occur, as it would use the same unique filename from xLODGen, but the content will most likely be different.
xLODGen tries to use a custom atlas definition file (Edit Scripts\[game mode]-AtlasMap-[worldspace].txt, so for example Edit Scripts\TES5-AtlasMap-Tamriel.txt), when the option to build the texture atlas is not checked. This can be used to either generate object LOD that uses the vanilla atlas (I have yet to make the file for the vanilla atlas for TES5, for SSE they are already included in latest xEdit), or generate object LOD with a unique atlas texture file name of your choosing.
When you generate LOD with xLODGen and the atlas build option is checked, afterwards there will be a Edit Scripts\LODGenAtlasMap.txt for the last generation. In order for future LOD generations to reuse the same atlas map file, rename LODGenAtlasMap.txt to TES5-AtlasMap-Tamriel.txt. To generate object LOD that use a unique atlas texture file name that only belongs to your mod, open the file in notepad and search and replace all occurrences of textures\terrain\[worldspace]\Objects\[worldspace]ObjectsLOD.dds, so for example from textures\terrain\Tamriel\Objects\TamrielObjectsLOD.dds to textures\terrain\Tamriel\Objects\MyAwesome-MOD-LOD-Atlas-Tamriel.dds. Then also rename the files in textures\terrain\[worldsapce]\objects\*.dds to match of course.
Now each new generation with build atlas not checked will use that atlas map file. I will try to create the vanilla atlas map files some time soon.
If using CK, you are forced to only create LOD that uses the vanilla texture atlas (which happens automatically if no new LOD models define new LOD textures), unless you want to replace the object LOD of the entire worldspace. In order for CK object LOD generation to create compatible object LOD, you always need to create object LOD for the entire worldspace. If you generate only specific chunks with CK, the generated texture atlas will only contain LOD textures that are used in these chunks. In any case, visually compare the generated atlas texture with the vanilla atlas texture, to make sure they match exactly. If any texture position is off or missing, the texture atlas does not match and you will not be able to mix your LOD BTO files with vanilla LOD BTO files without either LOD having the wrong textures.
For CK to be able to generate object LOD, you need to install the "source" folder with the TGA textures from Skyrim Vanilla LOD Source Files v2.5.
Once you generated object LOD with CK/xLODGen just include the BTO LOD meshes of the area the mod changes and discard the others.
2. If the base record for the house already has LOD models defined, then nothing else needs to be done for xLODGen/DynDOLOD compatibility.
Disclaimer, there is a chance I am describing features of xEdit/xLODGen not available in the latest download. In case something doesn't work as described let me know.
If you need more detailed instructions how to actually use the tools, let me know.
Edited by sheson, 13 March 2018 - 04:38 AM.