Dynamic Distant Objects LOD - DynDOLOD 2.27

[sheson]

A newer version of DynDOLOD is available

 

DynDOLOD 2.22/2.23/2.24/2.25/2.26/2.27 minor version update post | Major version update Post | Skyrim Special Edition

 

What’s New?

 

This update includes general performance and stability improvements for DynDOLOD.exe, TexGen.exe and LODGen.exe. See change log below for more info.

 

There is no need to update LOD generated by earlier versions just because the tools have been updated. See Full update post for 2.xx update instructions

 

---

 

TexGen.exe now offers changing the texture output formats for advanced users. Leave at default for typical use. To change the texture format of the texture atlases created by DynDOLOD.exe edit its INIs. See docs\DynDOLOD_TexGen.html for more info.

 

DynDOLOD.exe and LODGen.exe now offer to generate LOD Level 32 static object LOD without billboard trees, which can be used for the map. See docs\trees.ultra\DynDOLOD-Trees.html for more info.

 

All tools have been updated to support Skyrim SE. This is in beta and limited for obvious and well known reasons. It should only be used by experienced DynDOLOD users. See docs\DynDOLOD_Manual_SSE.html for more info. It explains the key differences and similarities, as well as the new features introduced by Skyrim SE and its bugs/problems.

 

---

 

Skyrim

 

Mods Requiring DynDOLOD

Enhanced Landscapes - Winter Edition - Includes billboards and LOD textures that need to overwrite vanilla billboards and DynDOLOD Resources - Core Files textures. If LOD textures are generated with TexGen.exe, they should always overwrite everything else.

Winter Aspen Billboards

 

Verified working out of the box - added/updated meshes and/or textures

Helarchen Creek

Realistic Boat Bobbing

Skysight - Simply Bigger Trees - Billboards

Dragon Bridge

 

---

 

DynDOLOD 2.26 - Download from Mega | DynDOLOD 2.26 - Download from Mega

 

DynDOLOD Resources 2.22 for Skyrim - Download from Mega

 

DynDOLOD Resources SE 2.22 for Skyrim SE - Download from Mega

 

DynDOLOD Patches 2.23 for Skyrim - Download from Mega

 

---

 

[spoiler=Changelog DynDOLOD 2.22]

General performance and stability improvements - for real

DynDOLOD.exe - fixed sometimes using wrong glow LOD when using original LOD assignments

DynDOLOD.exe - added RemoveUnseenZShift to ini for same setting in LODGen.exe export file

DynDOLOD.exe - added Level32 switch to ini for same switch in LODGen.exe export file, see docs\trees.ultra\DynDOLOD-Trees.html

DynDOLOD.exe - added FlatLODVertexColor to ini to set vertex color of flat LOD, see docs\trees.ultra\DynDOLOD-Trees.html

DynDOLOD.exe - added FlatLODColorVariance to ini to randomly change vertex color of flat LOD, see docs\trees.ultra\DynDOLOD-Trees.html

DynDOLOD.exe - added StaticLODDiffuseFormat, StaticLODNormalFormat, TreeLODDiffuseFormat and TreeLODNormalFormat to ini to set texture atlas formats

DynDOLOD.exe - added TreeLODAverageColor to ini to replace black transparent parts of billboards with the average tree color - works only with alpha channel output formats

DynDOLOD.exe - added TreeLODBackgroundTexture to ini to replace black transparent parts of billboards with a texture - works only with alpha channel output formats

DynDOLOD.exe - shrink normal map texture atlas to 4x4 pixels if all pixels are identical

DynDOLOD.exe - added beta support for SSE, start with -SSE command line argument

DynDOLOD.exe - added a check to verify DynDOLOD Resources for the current game mode are installed

TexGen.exe - sync updates with DynDOLOD.exe

TexGen.exe - added output format options for diffuse and normal textures

TexGen.exe - fixed cases of not consequently using lowercase for comparisons

TexGen.exe - added generation for riftenstonewall02lod.dds

TexGen.exe - added generation for tundradriftwoodbark01lod.dds

TexGen.exe - fixed generation of rubblepilelod.dds

TexGen.exe - fixed generation of fxdirtmound01lod.dds

TexGen.exe - removed the need for temp files in game data folder

TexGen.exe - added beta support for SSE, start with -SSE command line argument

TexGen.exe - added fallback to Texconv.exe for reading in case DDS format is not natively supported

Texconv.exe - new - custom silent version, prints errors only

LODGen.exe - added support reading SSE LOD nif

LODGen.exe - added export Mode=SSE

LODGen.exe - added support for BSA version 0x69

LODGen.exe - added lz4 decompression

LODGen.exe - added GameMode=textureslist to generate a list of textures with UV in between atlas tolerance

LODGen.exe - added command line parameter gamemode to overwrite gamemode in export file

LODGen.exe - updated lz4 package to latest version

LODGen.exe - better error message in case required Redistributable is missing

LODGen.exe - added Level32=True, uses *.nif from LOD level 16 if 4th entry is not set to create LOD Level 32 bto

LODGen.exe - added RemoveUnseenZShift=[float] to adjust terrain height when removing unseen faces

LODGen.exe - added vertex color support for flat LOD, column 8 and 9 in FlatTextures file

LODGen.exe - some minor speed improvements and clean up

LODGen.exe - use assembly version in log

DynDOLOD_Manual.html - all docs updated for new game mode

DynDOLOD_Manual_SSE.html - new

DynDOLOD-Shortcut.txt - new

DynDOLOD-Trees.html - added info about static object LOD level 32 for map

DynDOLOD-Trees.html - added info for vertex colors for 2D tree LOD in static LOD for Skyrim SE

DynDOLOD Resources - updated/added meshes/textures for better compatibility with mods

DynDOLOD Resources SE - new - meshes in new Skyrim SE format - removed options/assets that currently do not apply to Skyrim SE

DynDOLOD Patches - fixed a typo in Wizard.txt

 

 

[spoiler=Changelog DynDOLOD 2.23]

DynDOLOD.exe - fixed some default INI settings

DynDOLOD Patches - update Beyond Reach billboards to be also for arnima.esm

 

 

[spoiler=Changelog DynDOLOD 2.24]

DynDOLOD.exe - check LODGen log and write result to DynDOLOD log, so it is more obvious if something went wrong

LODGen.exe - fixed error reading BSTriShapes with no vertices/triangles

LODGen.exe - apply UVScale and UVOffset shader values to model before merging

 

 

[spoiler=Changelog DynDOLOD 2.25]

DynDOLOD.exe - fixed a memory corruption sometimes causing nasty things

DynDOLOD.exe - fixed an error trying to check a non existing LODGen log

LODGen.exe - fixed FO4 export mode not working correctly

LODGen.exe - updated lz4 package and supporting libraries to latest version

 

 

 

[spoiler=Changelog DynDOLOD 2.26]

LODGen.exe - fixed a crash caused by a missing serialization

 

 

[spoiler=Changelog DynDOLOD 2.27]

DynDOLOD.exe - fixed trying to check LODGen logfile after wait times out

DynDOLOD.exe - fixed not reading billboard txt file from BSA

LODGen.exe - updated lz4 package and supporting libraries to latest version

LODGen.exe - fixed sometimes not reading BA2 files correctly

 

 

 

 

 

See Full update post for full changelog, fully detailed update instructions and updated compatibility information for Skyrim mods.

 

DynDOLOD 2.21

 

Skryim typos may be intentional as long as Skyrim SE has severe visual and game breaking bugs

[NordPride_WorldWide]

Hello, I'm getting this error several minutes into running DDL

 

"Exception in unit userscript line 322: Assertion failure (C:\Delphi\projects\DynDOLOD\wbImplementation.pas, line 4201)"

 

Any clue whats happening?

 

Thanks in advance

[NordPride_WorldWide]

furthermore, it happens everytime it tries to add complete crafting overhaul remade.esp as a master

[NordPride_WorldWide]

Scratch the last bit, doesnt matter if i remove CCOR or not, it just hits that line in the script and stops dead

 

EDIT: I even tried the old 2.21 version and got the same error...

Edited April 15, 2017 by NordPride_WorldWide

[sheson]

Hello, I'm getting this error several minutes into running DDL

 

"Exception in unit userscript line 322: Assertion failure (C:\Delphi\projects\DynDOLOD\wbImplementation.pas, line 4201)"

 

Any clue whats happening?

 

Thanks in advance

Check all plugins for errors in xEdit and fix them.

 

Post/upload log.

[NordPride_WorldWide]

Check all plugins for errors in xEdit and fix them.

 

Post/upload log.

 

do you mean the xEdit log?

https://www.dropbox.com/s/bh7pnw8iu311n1g/Log.txt?dl=0

it isnt giving me a dyndolod log, only the texgen

 

so i fixed everything i could and tried to run it again before coming back to post, but now it doesnt give me any error at all, and just freezes

Edited April 16, 2017 by NordPride_WorldWide

[NordPride_WorldWide]

UPDATE:

 

So i was originally trying to generate with TreeLOD=0 because I'm using enhanced vanilla trees and wanted to use the 3d LOD, but once i switched it to =1 it completed successfully.

 

Wonder what I'm doing wrong the other way

[sheson]

do you mean the xEdit log?

https://www.dropbox.com/s/bh7pnw8iu311n1g/Log.txt?dl=0

it isnt giving me a dyndolod log, only the texgen

 

so i fixed everything i could and tried to run it again before coming back to post, but now it doesnt give me any error at all, and just freezes

When it stopped with the exception and you closed it afterwards it wrote a DynDOLOD log.

 

If it freezes, make a screenshot. Edit DynDOLOD\Edit Scripts\DynDOLOD\DynDOLOD_SSE.ini with notepad and change FasterBase=0 (maybe also FasterCreate=0 but that shouldn't make a difference) then do the same generation without tree LOD again, see if it goes through. Try this test version

[elwaps]

I finally managed to catch the little brat while crashing.

 

 

Seems like I'd be running out of memory. Would fit the inconsistent timing and reproducability.

I found out the crashing isn't directly connected to using the fallback option or not but while creating non-fallback LODs 'only' took me 3 shots to complete, I've never successfully created fallback LODs. Can be randomness due to few tries tho.

 

I am using MO which doesn't support the x64 version running through it, I assume going for x64 would solve my problem?

Do you know any way to bypass the x86 restriction of MO? Or a way to run the x64 version standalone while using MO's folder structure?

 

Gonna retry now with performance monitor on now...

[elwaps]

Allright, here we go

 

> Rebooted computer

> Deactivated Microsoft Security Essentials

> Deleted everything in DynDOLOD folder except for the two LODGen-files you gave me earlier

> Reinstalled it from the 2.4 Standalone zip

> Deleted DynDOLOD and TexGen output

> Ran LOOT

> Deactivated BirdsHFClean.esp, Bashed Patch, 0.esp, NoLeveledItemRewards.esp, WTF.esp and Dual Sheath Redux Patch.esp in MO

> Edited DynDOLOD_TES5.ini: TreeLOD=0, TreeFullFallBack=1

> Started DynDOLOD through MO

> Choosing options: Chose all Worlds, Ticket Candles, Set Output Path to C:\OUTPUT DynDOLOD\, Loaded high Rules, Set max Tile Size to 2048, Set LOD Brightness to 5

> Clicked OK...

 

Screencapture of the crash itself plus a few seconds before and after

https://www.youtube.com/watch?v=8Mcx3ckXm4A

 

Screencapture of several minutes before the crash till the final error after the crash*

https://www.youtube.com/watch?v=3jqmfhN3MEU

*uploading and processing not finished yet, will take a while

 

I thought I give it a shot with EVT 3D trees combined with SFO 2.5b fallback trees, just to see where I would end up framerate-wise. However, I haven't managed to finish even one single successful LOD generation.

Edited April 17, 2017 by elwaps

[sheson]

I finally managed to catch the little brat while crashing.

 

 

Seems like I'd be running out of memory. Would fit the inconsistent timing and reproducability.

I found out the crashing isn't directly connected to using the fallback option or not but while creating non-fallback LODs 'only' took me 3 shots to complete, I've never successfully created fallback LODs. Can be randomness due to few tries tho.

 

I am using MO which doesn't support the x64 version running through it, I assume going for x64 would solve my problem?

Do you know any way to bypass the x86 restriction of MO? Or a way to run the x64 version standalone while using MO's folder structure?

 

Gonna retry now with performance monitor on now...

 

If the problem only happens sometimes for the same setup, then it is likely a hardware problem like CPU overclocking / heat problem, unstable memory timings. Windows UAC or antivius maybe.

 

Only if you can reproduce the same the error every time for the same setup and about the same time in LODGen.exe is there a problem with data processing that I could fix.

 

It can happen that it reads nif data wrong which can cause an out of memory error without it actually being really out of memory.

 

For now I would assume the same thing to happen with the 64 bit version. You either need to start DynDOLOD64 directly without MO (but then it will miss all the mods of course) or use MO2 64 bit (not sure how well that works with Skyrim). Leave that idea for later if we exhausted other the options without a solution.

 

To test if this a reproducible programming problem, use DynDOLOD expert mode by setting Expert=1 in \DynDOLOD\Edit Scripts\DynDOLOD\DynDOLOD_TES5.ini

 

Then change the world drop down top left to a worldspace to test. Then the "Execute LODGen.exe" button becomes available. Click it to start LODGen.exe directly with the already existing LOD data gathered earlier.

 

For troubleshooting it would be best to find the smallest (least amount of objects) worldspace that can causes this. Keep load order and other INI the same as when the error happened.

Edited April 17, 2017 by sheson

[elwaps]

Its not totally random. The CPU isn't overclocked and just like the GPU does not overheat. The 2133MHz RAM is currently running at 1600 with pretty conserative default settings because I was too lazy to fiddle with the XMP profiles after a BIOS reset so far. The environment is a freshly installed W7x64 on a separate SSD solely intended for Skyrim (see W10 DX9 VRAM limit), still with pretty default settings (UAC is deactivated tho). MSE is the most basic antivirus you could possibly imagine AND is deactivated

 

Until now I could observe

> the process was more stable with less mods (you don't say!)

> generating 'regular' LODs with TreeLOD=1 and TreeFullFallBack=0 works almost every single time (just did it without errors)

> generating LODs with TreeLOD=0 and TreeFullFallBack=0 often needs 2-3 runs to finish properly but then works fine

> generating LODs with TreeLOD=0 and TreeFullFallBack=1 never finished properly but always crashed so far

> the time it runs before crashing is always roughly the same - after 7 to 10 minutes, depending on overall duration and thereby settings, but never before 7 minutes

> my crashes are always memory crashes but the stage of the process in which the crash occurs seems to be 'random'*

 

*you can see this when comparing the screenshot with the short video: in the former it crashed while creataing atlas textures, in the latter it crashed during the 'Waiting for LODGen.exe to finish...' stage. This needs further experimenting, I will have an eye on the RAM usage of the first LODGen process to start, on the combined RAM usage of DynDOLOD and the LODGen processes together and if there's any number (around 1.8GB) correlating to crashes

 

Testing your suggestions now...

[elwaps]

 

Then change the world drop down top left to a worldspace to test. Then the "Execute LODGen.exe" button becomes available.

Unfortunately for me it doesn't become available but keeps being greyed out.

Perhaps I just need the explanation for dummies... I start DynDOLOD in expert mode, select a random worldspace from the dropdown - nothing happens to the Execute button. Load the high preset - nothing happens. Change random options like Tile Size or LOD Brightness - nothing happens. Change output path - nothing happens. Change the TreeLOD and Fallback options in the ini - nothing happens.

Am I doing sth. wrong?

 

/edit

Ran the regular tool again with TreeLOD=0 and TreeFullFallBack=1, crashed again after crossing the 1.7GB RAM line. At that moment the DynDOLOD process and the LODGen one both used about 1.75GB.

Edited April 17, 2017 by elwaps

[elwaps]

Ran it a second time, crashed again after roughly 8 minutes with about 1.75GB RAM usage. Correlation or causation...?

[sheson]

Its not totally random. The CPU isn't overclocked and just like the GPU does not overheat. The 2133MHz RAM is currently running at 1600 with pretty conserative default settings because I was too lazy to fiddle with the XMP profiles after a BIOS reset so far. The environment is a freshly installed W7x64 on a separate SSD solely intended for Skyrim (see W10 DX9 VRAM limit), still with pretty default settings (UAC is deactivated tho). MSE is the most basic antivirus you could possibly imagine AND is deactivated

 

Until now I could observe

> the process was more stable with less mods (you don't say!)

> generating 'regular' LODs with TreeLOD=1 and TreeFullFallBack=0 works almost every single time (just did it without errors)

> generating LODs with TreeLOD=0 and TreeFullFallBack=0 often needs 2-3 runs to finish properly but then works fine

> generating LODs with TreeLOD=0 and TreeFullFallBack=1 never finished properly but always crashed so far

> the time it runs before crashing is always roughly the same - after 7 to 10 minutes, depending on overall duration and thereby settings, but never before 7 minutes

> my crashes are always memory crashes but the stage of the process in which the crash occurs seems to be 'random'*

 

*you can see this when comparing the screenshot with the short video: in the former it crashed while creataing atlas textures, in the latter it crashed during the 'Waiting for LODGen.exe to finish...' stage. This needs further experimenting, I will have an eye on the RAM usage of the first LODGen process to start, on the combined RAM usage of DynDOLOD and the LODGen processes together and if there's any number (around 1.8GB) correlating to crashes

 

Testing your suggestions now...

We really don't care what DynDOLOD.exe is doing (creating textures or just waiting for LODgen.exe to finish), since we are discussing a crash of LODGen.exe.

 

If the crash is not 100% reproducible with the exact same settings, then the problem is caused by something else. If you are sure the system is stable, make sure the game is not installed to special windows folders. Make sure DynDOLOD is not installed to special windows folders, game folders or folder under control of MO.

 

If the Execute LODGen.exe button does not become available after changing the world drop down, then the export files in DynDOLOD\Edit Scripts\Export\ from earlier generations are missing. In that case, just hit OK to generate LOD and leave those files for later. Next time the Execute LODgen.exe button will becomes available and you can just run LODGen.exe without anything else.

 

The important part is to generate only one worldspace at a time and find the smallest one that consistently causes a problem with the same settings. 

Edited April 18, 2017 by sheson

[elwaps]

If the crash is not 100% reproducible with the exact same settings, then the problem is caused by something else.

As today I'm not really in the mood for play(test)ing or reading about and experimenting with merging plugins (unavoidable next step for my build) I'm gonna run a few more generations with the exact same settings while monitoring the RAM usage of the first LODGen process to test my hypothesis. While doing that I will record every single run to be able to analyze stuff afterwards I haven't noticed during the run itself.

 

If you are sure the system is stable, make sure the game is not installed to special windows folders. Make sure DynDOLOD is not installed to special windows folders, game folders or folder under control of MO

I'm as sure as one can be that my system is stable. Nonetheless I'm gonna run Prime95 and Furmark for a few hours tomorrow during work while logging temperatures plus a few passes of Memtest during dinner. However, I don't want to rape my SSD with write-testing all day, so I'll have to stick with perfectly fine SMART values.

 

> Skyrim is not installed to the programs or even Windows folder but to C:\Steam\steamapps\common\skyrim

> DynDOLOD is installed to ..\skyrim\DynDOLOD and thereby not under the influence of MO, which is located in ..\skyrim\ModOrganizer

> UAC is disabled, MO is running as administrator and full access to the skyrim folder is given to everyone (just to be sure)

> Microsoft Security Essentials is disabled all the time except for when I go online

> All drivers except the nVidia one are installed manually (pointing driver installation to inf-files instead of using the setups) to avoid unneccessary software installations

> Since getting a new board the system hasn't crashed once for about a year, neither my pretty cluttered Win10 or my freshly installed Skyrim-Win7 on its separate SSD

 

If the Execute LODGen.exe button does not become available after changing the world drop down, then the export files in DynDOLOD\Edit Scripts\Export\ from earlier generations are missing. In that case, just hit OK to generate LOD and leave those files for later. Next time the Execute LODgen.exe button will becomes available and you can just run LODGen.exe without anything else.

Gonna check this, perhaps I accidentially deleted the output without thinking about it. But first I'm gonna replicate the crashes a few more times.

 

The important part is to generate only one worldspace at a time and find the smallest one that consistently causes a problem with the same settings.

Got your intention, thanks so much for help and your top notch support in general. I'll be back soon... ;)

 

 

/edit

Should I still test with your modified LODGen files (the 1041kB ones) or the original ones that come with 2.24? Tested with the modified ones so far.

Edited April 18, 2017 by elwaps