Jump to content


Photo

Anti-Aliasing/Anisotropic Filtering not working properly


  • Please log in to reply
109 replies to this topic

#91 torminater

torminater

    High King

  • VIP-Supporter
  • PipPipPipPipPipPip
  • 2,108 posts

Posted 16 July 2012 - 11:15 AM

usually ini tweaks do more damage than they help. tweaking the havok settings correctly takes a really long time and a lot of testing.
  • 0

#92 LeetMiniWheat

LeetMiniWheat

    Knight

  • Citizen
  • Pip
  • 250 posts

Posted 18 July 2012 - 01:18 PM

this is odd, now I'm getting the same crashes on a new game that you got on the first post right before riverwood. google'd for "DialogueRiverwoodIntroScene (000954D9): attempting to start event scoped quest outside of story manager." and your post came up lol
  • 0

#93 torminater

torminater

    High King

  • VIP-Supporter
  • PipPipPipPipPipPip
  • 2,108 posts

Posted 18 July 2012 - 03:35 PM

then it's probably a vanilla bug... ;-) well, actually my rig crashed on the lvl predator script just above that:
the dialogue script you refer to and mentioned in the spoiler only came up after installing USKP 1.05 c beta
  • 0

#94 Neovalen

Neovalen

    Talos

  • Mod Author
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 5,593 posts

Posted 18 July 2012 - 05:01 PM

Maybe not the right place to ask this, but since we're discussing various Nvidia Inspector settings... I've set up 4x4SSAA/4xTrSGSSAA etc as noted in the first few posts of this thread. However, my question for everyone is... Nvidia SSAO vs ENB SSAO vs Both? I'm confused as to which I should use.
  • 0

#95 LeetMiniWheat

LeetMiniWheat

    Knight

  • Citizen
  • Pip
  • 250 posts

Posted 18 July 2012 - 07:34 PM

Maybe not the right place to ask this, but since we're discussing various Nvidia Inspector settings... I've set up 4x4SSAA/4xTrSGSSAA etc as noted in the first few posts of this thread.

However, my question for everyone is... Nvidia SSAO vs ENB SSAO vs Both? I'm confused as to which I should use.

I too am confused, but I saw no difference between off and on when using ENB. I think ENB uses it's own type of it, since it's definitely a lot better than regular. and it seems to switch off the regular when you enable it (I think, anyways) so I leave it off in nvidia inspector. if you find different results, let me know :D

Edit: also, don't use 4x4 supersample. it doesn't work, and even if it did your framerate would be like a slideshow. especially with your res, it would be rendering at 23040x4320. the game's setting overrides nvidia inspector anyways, but I set 4x MSAA override with 4x in-game just for good measure. (I saw some kind of fps boost when doing so, but could have been within margin of error)

Edit2: if your cards can handle it (which they should) I highly recommend 4x SGSSAA instead of 4x TrSSAA, it produces a much better image quality - gets rid of most of the shimmering from lizard skins, doors, leather helmets, standing stones, and other specular objects, etc etc

Edit3: and I think the jury's still out on matching MSAA with TrSSAA/SGSSAA but I do it for good measure. (like 4x msaa + 4x trssaa/sgssaa, and 8x msaa + 8x trssaa/sgssaa). someone else had mentioned it only matters with supersample AA (2x2, 3x3, 4x4) and not MSAA but I'm not entirely sure. I don't usually like to use more than 4x MSAA anyways since 8x and above are a much bigger performance hit.
  • 0

#96 Neovalen

Neovalen

    Talos

  • Mod Author
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 5,593 posts

Posted 18 July 2012 - 08:06 PM

Maybe not the right place to ask this, but since we're discussing various Nvidia Inspector settings... I've set up 4x4SSAA/4xTrSGSSAA etc as noted in the first few posts of this thread.

However, my question for everyone is... Nvidia SSAO vs ENB SSAO vs Both? I'm confused as to which I should use.

I too am confused, but I saw no difference between off and on when using ENB. I think ENB uses it's own type of it, since it's definitely a lot better than regular. and it seems to switch off the regular when you enable it (I think, anyways) so I leave it off in nvidia inspector. if you find different results, let me know :D

Edit: also, don't use 4x4 supersample. it doesn't work, and even if it did your framerate would be like a slideshow. especially with your res, it would be rendering at 23040x4320. the game's setting overrides nvidia inspector anyways, but I set 4x MSAA override with 4x in-game just for good measure. (I saw some kind of fps boost when doing so, but could have been within margin of error)

Edit2: if your cards can handle it (which they should) I highly recommend 4x SGSSAA instead of 4x TrSSAA, it produces a much better image quality - gets rid of most of the shimmering from lizard skins, doors, leather helmets, standing stones, and other specular objects, etc etc

Edit3: and I think the jury's still out on matching MSAA with TrSSAA/SGSSAA but I do it for good measure. (like 4x msaa + 4x trssaa/sgssaa, and 8x msaa + 8x trssaa/sgssaa). someone else had mentioned it only matters with supersample AA (2x2, 3x3, 4x4) and not MSAA but I'm not entirely sure. I don't usually like to use more than 4x MSAA anyways since 8x and above are a much bigger performance hit.

Alright... 4xMSAA doesn't work with Stereoscopic 3d enabled. I get a big red message from Nvidia saying it's been downgraded to 2xAA.... without 3d enabled it works fine.
Of course I don't get HDR then due to SLI.... gotta love it. Also noted that SSAO from Nvidia DOES NOT WORK since patch 1.5... Beth screwed something up when they implemented grass shadows.

Is it bad I'm actually HOPING for a patch from Bethesda? All of this is reminding me why I stopped playing Oblivion after too long even though I had fun with the game itself.
  • 0

#97 LeetMiniWheat

LeetMiniWheat

    Knight

  • Citizen
  • Pip
  • 250 posts

Posted 18 July 2012 - 08:45 PM

ENB's SSAO is pretty nice, there's no need for regular SSAO really, though it's good to know that it's broken. there's very vanilla-ish ENB's with minimal performance hit (2-3) like Enhanced Vanilla ENB. (just disable depth of field if you don't like that). it even works with RCRN as a dll proxy personally I'm using Vibrant ENB with the vibrancy turned off, combined with rcrnshaders.esp for darker nights
  • 0

#98 jmad75

jmad75

    Citizen

  • Citizen
  • 21 posts

Posted 18 July 2012 - 08:56 PM

Much of this discussion is lost on me. At this point, I don't know what my nvidia Inspector settings should be. GeForce GTX 560M is my card and I don't use SkyrimHD2K textures but I do have some shimmering issues.
  • 0

#99 LeetMiniWheat

LeetMiniWheat

    Knight

  • Citizen
  • Pip
  • 250 posts

Posted 19 July 2012 - 12:12 AM

Much of this discussion is lost on me. At this point I don't know what my nvidia Inspector settings should be. GeForce GTX 560M is my card and I don't use SkyrimHD2K textures but I do have some shimmering issues.

since your card is probably not fast enough for SGSSAA (sparse grid super sampling anti-aliasing), I would recommend using SMAA or FXAA, assuming you mean the shimmering on lizard skins, leather helmets, doors, standing stones, etc. SMAA does a bit better of a job than FXAA but not entirely a replacement for real transparency antialiasing.

there's a stand-alone SMAA injector here, but if you're using RCRN then that SMAA injector would replace RCRN's d3d9.dll file, so I would recommend uninstalling RCRN entirely else your contrast would be screwed up (or using an ENB that enhances contrast enough to replace RCRN's d3d9.dll). there's also a few ENB's that already include SMAA, like Vibrant ENB (though that ENB gave me a 10fps drop), but again these would replace RCRN. (though Vibrant ENB works very well with rcrnshaders.esp and textures)

as for FXAA, RCRN includes it by default (you can use rcrnconfigurator.exe to make sure it's on) and most ENB's include it. and then there's the in-game one. it will reduce the shimmering maybe halfway at the cost of extra blur, but not entirely get rid of it.

but if you really want to try SGSSAA, try 2x first. 4x will probably give you a massive drop in framerate.

Edit: you could also try the recommended 2x TrSSAA (Transparency Supersampling) to reduce tree leaf flickering, but it wont' do anything for specular highlights and it comes at a performance hit, though not as much as SGSSAA. I still recommend SMAA/FXAA on a laptop though, as the framerate will be much more playable


  • 0

#100 Neovalen

Neovalen

    Talos

  • Mod Author
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 5,593 posts

Posted 23 July 2012 - 04:23 PM

Alright, been a lot of speculation on this page on what does or does not work.... I'm just getting around now to re-installing Skyrim (moved drives and decided to make a clean run of it).

Things I have confirmed myself (Nvidia System):
1. HDR does not work for Skyrim in SLI unless Stereoscopic 3d is enabled. Stereoscopic 3d limits standard MSAA to 2x or lower.
2. Nvidia SSAO for Skyrim is broken as of patch 1.5+, so there is no need to enable it via Nvidia Inspector.

Things I am confused on:
1. What modes of AA actually work for Skyrim? Ansitropic Filtering = Clamp actually do anything? etc. Perhaps someone with some answers can post a definitive "best" setting?
  • 0

#101 LeetMiniWheat

LeetMiniWheat

    Knight

  • Citizen
  • Pip
  • 250 posts

Posted 23 July 2012 - 09:35 PM

Regular AA:
I think it's safe to say that 2x2, 3x3, and 4x4 supersampling doesn't work (without a special comptibility flag). It also appears that forcing MSAA (like 4x) in nvidia inspector, while the in-game antialiasing is set to 0 or 1 doesn't work either. (though I only tried Override, not Enhance). So AA needs to be set in-game/INI. I set mine to 4x in-game and Override to 4x in nvidia inspector just for good measure. (I seemed to get a little more fps with override+4x too but unsure about that)

Transparency AA:
TrSSAA and SGSSAA both work, but I'm uncertain if 8x works as I got no performance hit over 4x. SGSSAA offers superior image quality since it reduces shimmering/flickering/aliasing on specular highlights like lizard skins, bumpy doors, and standing stones.

LOD Bias:
Clamp LOD bias shouldn't do anything over Allow really, since I don't think the game changes LOD bias natively (not 100% sure here), you'd only need Allow on if you intend to lower the LOD bias for sharpening. I just set Clamp for good measure since I don't want any sharpening (it can cause pixelation and further exacerbate aliasing). Plus, I believe the ENB's have a higher quality sharpening at the cost of a little performance (but don't quote me on that).

Anisotropic Filtering:
Definitely set to 16x in nvidia inspector or nvidia CP, it improves quality on water. Disable in-game/INI anisotropic for good measure and increased performance.



This is from my own testing, but I think torminater will agree here. Please correct me if I'm wrong on anything though.
  • 0

#102 Neovalen

Neovalen

    Talos

  • Mod Author
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 5,593 posts

Posted 23 July 2012 - 10:39 PM

According to the RCRN page, "mirror" values have to be set in the game itself for all the effects to work from the CP.

After tuning the Driver Settings as listed in the Nvidia Tweak Guide, you have to do some things not reported in it, you have to set the right options for Skyrim.
In order to set them you have to start Skyrim via the Launcher and click on Options.
Here you have to set the “mirror” values of what you set on the driver panel, for example, if you set AF 16x and AA 8x forced by the Nvidia Control Panel, you will have to set the same values also into the in-game options, under the Antialiasing and Anisotropic Filter tabs.This thing is really important, and is necessary, because with the latest Skyrim’s patch only using the forced values without also having set the same values in the in-game options won’t work perperly.About the Nvidia FXAA, it’s really well done, so if you have a good rig we suggest to force it via the driver panel and use it along with the ones included in Skyrim (required, also if you force the Fxaa via panel) and RCRN.

I cannot confirm or deny on that, but their suggested settings for "Ultra" are as follows:

NVIDIA Control Panel:
FXAA: ON
Texture Filtering: High Quality
Anisotropic Filter: AF 16x
Transparency Antialiasing (TRAA): 8x Supersample
Skyrim:
Antialiasing: AA 8x
Anisotropic Filter: AF 16x
Detail: Ultra
- Advanced: FXAA ON
RCRN:
FXAA: ON

Link: https://www.rcrncommunity.com/nvidia
  • 0

#103 Neovalen

Neovalen

    Talos

  • Mod Author
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 5,593 posts

Posted 24 July 2012 - 12:24 AM

After reading the notes from RCRN and doing some internet research I managed to force 4x4 Supersampling. Unfortunately, Skyrim does NOT like it and has artifact issues (white shafts of light/flickering). FPS was ~30 with no lighting mod @ 1080p. Here was the settings to get it working: IN GAME: 2xAA, disabled AF, FXAA enabled. NVIDIA INSPECTOR settings attached. Screenshot of artifacts also attached (sorry for the poor quality, it only allowed me to attach 500kb). That being said, setting the Nvidia Inspector AA to 8x all the artifacts went away and I didn't see any jaggies ANYWHERE. Trying to use CSAA was also fruitless (game crashed / black screen). I also didn't see any FXAA blurring as I think the latest Nvidia drivers took pretty much all of it out. (Using 304.79) Also note: My VRAM usage is ~2500 even in just the opening sequence in Helgen. I strongly suggest not turning it all the way up without having a 4GB card.
  • 0

#104 Neovalen

Neovalen

    Talos

  • Mod Author
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 5,593 posts

Posted 24 July 2012 - 01:32 AM

Wanted to come back and note, disabling FXAA in both game and Nvidia Inspector had no visual difference than standard 8x MSAA forced in the Inspector. So I left it disabled from now on. This way of using AA also bypassed Nvidia's check for >2xAA in Stereo3d.
  • 0

#105 LeetMiniWheat

LeetMiniWheat

    Knight

  • Citizen
  • Pip
  • 250 posts

Posted 24 July 2012 - 04:22 AM

Hmm, interesting about the Anisotropic filtering... wonder what the difference is. Anisotropic 16x with 1x set in-game has been working for me (I think), else I would have noticed blurry roads. I still have my doubts about 4x4 supersampling, I just don't see how it could possibly be rendering a scene at your resolution TIMES FOUR. Was there a massive FPS hit after enabling it? In most games that it DID work in, I could barely even run 3x3 without a massive fps hit (even really old games). So, It could just be ignoring the setting... If it *is* working, you shouldn't need any TrSSAA/SGSSAA at all, or only 2x to get rid of the remaining minor jaggies on specular highlights. I still think for 2x2, 3x3, or 4x4 supersampling it's gonna need one of those generic compatibility flags that causes artifacts and other issues in most games. Did you have to change the "Antialiasing - Behavior Flags"? because mine is set differently and I haven't touched it. But I doubt that would enable 2x2/3x3/4x4 usage, though in some games it "just works". FXAA had major blurring last time I tried it. I believe the reason you didn't see a difference was because you had FXAA usage disallowed in nvidia inspector (but not totally sure here). And regarding FPS, I'm trying to stay at 60fps/vsync all the time, 30 feels too sluggish for me. I'm strongly considering trading my 690 for two 4GB 670's though after seeing my VRAM usage. (when EVGA releases the 4GB FTW)
  • 0


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users