I'm just happy it runs more stable. No framerate drops, even at 1440p or higher. Shadows look considerably better. Textures are a minor improvement, but one nonetheless. I really like the new lighting so far.
I don't tend to mod my game too much. Most mods just don't seem worth it. Weapons that are either already comparable to existing in game weapons or entirely overpowered. Armours that are the same way. Some NPC and character design changes are nice, while others are creepy. I'm not too concerned with downloading 30 different mods just to make sure clutter based graphics look better.
The only major mods I usually download are a all-in-one texture mod, iNeed, first person mods (animations, models, etc.), a water mod, and maybe an ENB if I don't mind even more framerate drops. However in this new version the ENB is basically already in place, just needs to be tweaked a bit and runs pretty well.
However I haven't actually played through Skyrim since about 2012, so I'm pretty much the target audience for this remaster; Those who haven't played in a while/at all.
I can understand why people wouldn't want to completely abandon their hours of modding though for a lesser version. Basically, until most of these mods get transferred over, I don't think many people will be playing this version outside of seeing what's changed since they got it for free anyway.
EDIT: Wait, people are experiencing worse performance than the original?
Can anyone confirm this? Is this in comparison with original unmodded Skyrim and SE Skyrim, or are people actually applying an ENB to original Skyrim to fairly compare it to SE Skyrim?
Because at least in my experience, with the increased draw distance, enhanced lighting (ENB) effects, and foliage coverage, Skyrim Special Edition is running quite a bit better than simply modding these graphical increases to standard Skyrim.