I did a lot of testing during the last few days and I've come to the conclusion that SMAA, SweetFX and FXAA Injector are pretty much useless. Sure they fix some jagged lines. I can clearly see it when I move close to objects like lampposts, but none of these AA techniques is able to fix the horrible jittering I get on textures that are further away. These are my observations from using either post processor on it's own without ENB.
Using any of these on top of enb edge and subpixel AA does not seem to have ANY effect any at all. Those jags that aren't fixed by the enb AA, aren't fixed by the additional AA tools either. And yes, I made sure that everything's set up properly. I think I know all skyrim and enb .ini files by heart now.
The only way to get a result that comes close to hardware based AA is enabling all three enb AA: edge, subpixel and temporal. However temporal AA is incompatible with my favourite enb presets, Skylight and Vividian, as it causes a weird ghosting effect when stepping into shadows. Maybe I can find an enb that works with temporal AA and is also compatible with both Climates of Tamriel and Pure Weather (I'm using WAO)
Downsampling works great on it's own too, but combinded with an enb I get less than 20 fps so that's not really an option for me.
For now I have disabled deferred rendering in enblocal.ini and use 4x MSAA. This means a couple of enb effects are disabled, including ambient occlusion. I think the game still looks great though and most imortantly I'm not disturbed by flickering textures anymore.
I still don't know why I have these problems in first place. I tried all driver versions that are available for my card and played around with RadeonPro and nothing could fix it.
It's also not related to mods, I tested everything in an unmodded vanilla game. Although I noticed that adding SMIM makes the jittering even worse for some objects like shack roofs, static barrels and 3D ropes. Fences are an absolute nightmare regardless of using vanilla or SMIM. That's why I use Fences of Skyrim. But without proper AA even the fences from that mod flicker horribly at distance.
Anyway thanks everyone for the tips. I have the suspicion that it has something to do with my graphics card. So there might be nothing what you can do for me unless someone has the same card.