Issue with software anti aliasing

[TirigonX]

Ever since using ENB I wasn't really happy with the anti aliasing. It does work, but it causes an annoying flickering in mid to far distance on certain objects. Check this short video to see what I mean. You may have to watch in HD and fullscreen to see it properly. Watch the planks, barrels and crates. It's actually even worse in game but youtube's compression seems to filter some of it.

It's most noticeable on wooden objects. But also thin objects like ropes or metal bars are affected. I had another video for that but youtube made it look like everything was ok lol.

 

It doesn't matter whether I use ENB's Edge and/or Subpixel AA or SMAA with highest preset. Or even all of them together. For this video I enabled Edge and Subpixel AA in enblocal.ini

I don't see this flickering with hardware AA, but of course that's not an option with ENB.

 

Does anyone else have this problem too? Or am I expecting too much and this is just how software AA looks?

 

 

System specs:
i5 3570K, 8 GB RAM, Radeon 270X, Win 7 64bit

[Pretendeavor]

Maybe you're expecting too much.. I'm not sure. I recognize it but it seems more extreme than I can remember. A lot of people swear by SMAA but I actually dislike it because imo the performance hit (when set to über) isn't worth using it over subpixel+edge AA imo. It also completely ruins my font :/

 

Perhaps it differs between enb presets. I usually dislike extreme amounts of sharpening (K enb-esque sharpening with SweetFX Lumasharpen and negative lod bias) for normal gameplay so you might want to check that out for your preset. I guess you could try downsampling if that is an option with your graphics card or see if changing the option FixAliasedTextures makes any difference. ENB's temporal AA has pretty good results too but creates artifacts when moving and is really performance heavy.

[TirigonX]

Damn youtube is really hiding a lot. When I watch the original video on my pc it's much more visible, just like in game. Maybe I can upload a video that makes it more obvious.

It's like AA is only working properly on close distance. Everthing further away than 10-20 meters seems to shimmer and thin objects like ropes get very jaggy.

 

I'm currently using Vividian ENB 7.02 which is pretty smooth. It has an optional version, which is even smoother but it doesn't improve this situation. FixAliasedTextures doesn't fix it either and downsampling is a bit too much for my card. I also tried Skylight and Phinix Naural ENB with the same result. TemporalAA seems to be a bit better but not much. And with Vividian enb it's causing a ghosting effect on weaopns in 1st person.

 

It just feels wrong to have all these eyecandy enb effects and then be disturbed by permanent flickering. I'm actually thinking about trying enb version 119, the last one that supports hardware AA.

[TechAngel85]

I'd make sure your INIs are default ones to eliminate the possibly of a rogue tweak. I don't have any issues and I use ENB AA combined with SMAA on high. My Skyrim is very smooth with this combination.

[hishutup]

That appears to be normal to me. I took some images awhile back from a different game, MSAA vs ENB edge AA (No AA?).

 

lowering LOD bias in ENBlocal.ini may help but you'll eventually get shimmering.

I use -0.33 to -0.75

-0.66 seems like a good balance.

[TirigonX]

Do you guys really get the same visual experience out of SMAA or enb's AA options as opposed to "true" hardware AA? Or is there some kind of degradation?

For me there is a very noticeable difference between 4x MSAA and uber SMAA/enb AA. 

It has nothing to do with enb itself or any ini settings. I double and triple checked that. These other AA methods just don't look as good as traditional driver AA on my system.

I admit I'm probably too focused on this right now. I can't stop looking at it and pay too much attention to it. Either that or my graphics card doesn't handle this alternative form of AA as good as other cards.

Anyway thanks for the help. I guess I'll have to accept it as it is for now. I could try installing older drivers but I doubt that will change anything.

[TechAngel85]

I really have no AA issues with SMAA. Lines are smooth. I've never known anyone to not have a good experience with it. Did you set up the proxy in your enblocal.ini and enable it?

 

With my GTX 760, I use ENB AA and SMAA on high and get 40s to 60 fps (depends on area). Very smooth and clean lines.

[Aiyen]

Software based AA solutions depend almost entirely upon your resolution. SMAA at higher resolutions will always be the better solution over MSAA due to the performance difference relative to visuals. 

 

If you indeed have jagged edges try to downsample at higher resolutions. Ofc. make sure that it is an AA issue first and not a sharpening issue, since lines can easily be because of that. 

 

The reason MSAA does look better than SMAA, is that MSAA 4x is more or less the same as saying you play at 4x the resolution.. if you play at 2x the resolution and then use SMAA you will probably have about the same quality overall. At least SMAA with high settings at 1440p will have removed just about all sort of jagged edges... if you go to 1600p they will have all gone. 

 

Downsampling is not that difficult to do, and for any who really just do not want jagged edges it is pretty much the best and easiest go to solution. 

[DoubleYou]

I use and prefer the sweetfx smaa. I never could see any effect from the standalone version on my system. Fxaa injector works good too.

[TirigonX]

I did a lot of testing during the last few days and I've come to the conclusion that SMAA, SweetFX and FXAA Injector are pretty much useless. Sure they fix some jagged lines. I can clearly see it when I move close to objects like lampposts, but none of these AA techniques is able to fix the horrible jittering I get on textures that are further away. These are my observations from using either post processor on it's own without ENB.

 

Using any of these on top of enb edge and subpixel AA does not seem to have ANY effect any at all. Those jags that aren't fixed by the enb AA, aren't fixed by the additional AA tools either. And yes, I made sure that everything's set up properly. I think I know all skyrim and enb .ini files by heart now.

 

The only way to get a result that comes close to hardware based AA is enabling all three enb AA: edge, subpixel and temporal. However temporal AA is incompatible with my favourite enb presets, Skylight and Vividian, as it causes a weird ghosting effect when stepping into shadows. Maybe I can find an enb that works with temporal AA and is also compatible with both Climates of Tamriel and Pure Weather (I'm using WAO)

 

Downsampling works great on it's own too, but combinded with an enb I get less than 20 fps so that's not really an option for me.

 

For now I have disabled deferred rendering in enblocal.ini and use 4x MSAA. This means a couple of enb effects are disabled, including ambient occlusion. I think the game still looks great though and most imortantly I'm not disturbed by flickering textures anymore.

 

I still don't know why I have these problems in first place. I tried all driver versions that are available for my card and played around with RadeonPro and nothing could fix it.

It's also not related to mods, I tested everything in an unmodded vanilla game. Although I noticed that adding SMIM makes the jittering even worse for some objects like shack roofs, static barrels and 3D ropes. Fences are an absolute nightmare regardless of using vanilla or SMIM. That's why I use Fences of Skyrim. But without proper AA even the fences from that mod flicker horribly at distance.

 

Anyway thanks everyone for the tips. I have the suspicion that it has something to do with my graphics card. So there might be nothing what you can do for me unless someone has the same card.

[Pretendeavor]

I think you're overestimating how Skyrim is supposed to look. I checked the flickering on woodposts, wood walkways and such myself a bit in whiterun when I last played and it happens a lot. You could try downsampling to 1440p and mess with the following settings:

 

In enbseries.ini:

 

SizeScale & SourceTexturesScale under both [REFLECTION] and [sSAO_SSIL]: try setting both to 0.5

EnableSupersampling under [REFLECTION] and [sSAO_SSIL]: disable both.

You could disable exterior reflections with ExteriorEnable=false or disable them altogether.

Set the [DEPTHOFFIELD] Quality to like 1.

Check enbeffectprepass.fx(.ini) and look for DoF Quality or Quality; set to 2-3 if set higher.

 

Set your shadow settings to medium/high instead of ultra in the skyrim inis.

Edited July 14, 2015 by Pretendeavor

[TechAngel85]

This is the best pic in my current gallery that I could find which shows the smoothness on my system:

 

https://copy.com/czBH0pDP9IiEaAOk

 

No idea why you wouldn't be getting good results with SMAA. The little bit of pixelation you see in the distance with the smoke in the air is simple JPG artifacting from saving the image. It doesn't look like that in-game. Clear and smooth.

[Spock]

Well, post processing AA other then Temporal AA cannot truly get rid of the shimmering when moving. They also have disadvantages and reduce image quality. SMAA is quite reasonable while FXAA is really bad in that regard (see my post in the Witcher 3 modding guide). Temporal AA always has some ghosting but I found the ENB TAA not too distracting last time I checked.

 

The shimmer your scene produces is quite bad though, I usually only get it with fences and to some extend with foliage. Maybe something else is wrong? Check your CCC to see if everything is set to the first option except for texture filtering to max and vsync to what works with ENB. Try different AF handling (CCC or ENB), if nothing helps maybe try a different texture. It could also be an ENB setting exacerbating the problem, did you try different presets?

Edited July 14, 2015 by Spock

[TechAngel85]

I have Nvidia so that might be a difference. The more I see in the modding/gaming world, the more I'm team Nvidia. AMD just seems to be playing catchup all of the time, whereas, Nvidia is constantly innovating.

[Spock]

I have Nvidia so that might be a difference. The more I see in the modding/gaming world, the more I'm team Nvidia. AMD just seems to be playing catchup all of the time, whereas, Nvidia is constantly innovating.

I'm currently on team red and have to problems whatsoever. I know there is a lot of NVidia love here but tbh, they make bad stuff too: gtx 970 and the whole Kepler generation with it's shitty shader architecture. I too prefer NVidia if their current architecture is good and affordable for their bigger driver staff, but currently it's not :/

Edited July 14, 2015 by Spock

[Audley]

Perhaps Skyrim has better support (or maybe it just works better in general) for things like SMAA. But I have spent ages trying to get SMAA to look good in Fallout. It just doesn't. Even 2x MSAA looks better than the strongest SMAA settings. In Fallout, SMAA does a good job with small things nearby, and large things at a distance. But with medium sized objects or anything with a transparency layer (fences, windows, grass etc.) it just doesn't compete with MSAA+Transparency Multisampling. I really wish that hardware AA worked with ENB. At this point it is really down to picking your poison. I personally can live with aliasing but it takes a good ten minutes of me playing a game before I start to ignore it.