I found it rather fun to suddenly realize the sniper rifle I'd been icing fools with had gone worryingly quiet. Taking note if any enemies had been holding anything particularly juicy before their demise, or using a vigor as a quick distraction while making a mad dash to a convenient tear was thrilling. The game is clearly designed around the choice to only have 2 guns immediately on hand. Never found myself wanting for a particular class of gun (light, medium, heavy/special) when I felt I needed it. I suppose I just prefer interacting with the guns in the environment (includes dead baddies) when I need some more punch, rather than playing "pick the gun that is the clear winner for this situation right away, then go to the next area and do it again." Having to improvise a bit can be fun.
You may notice I'm avoiding saying 'weapon' re: the 2 gun limit.
You may notice you'll have 10 weapons on you at any time by the end of the game. Vigors are absolutely weapons.
Plus, ALL the guns interact well with Vigors (discounting the Burst Gun, which can go interact with anything it likes far away from me, piece of shit that it is). Were Vigors not a thing that you could use in this game, the 2 gun limit would get pretty stale. Once I figured out that maxing out salts first is the infusion route for real men with hairy chests and deep voices, I found myself hardly using guns at all except to start off my exploding Overkill/Upgraded Vigor Chains of doom.
Really though, if you're getting hung up on not having all the guns all the time, try using Vigors a little bit more gratuitously. I was very rarely low on salts past the first few chapters thanks to "Endless Pockets" Elizabeth, and you'd be hard pressed to deny the appeal of turning every enemy in the room into a mass fireworks display instead of dully popping at them with the guns.
Hint for anyone unsure how to use Vigors effectively: As soon as you can, start using the Upgrade/Gear (my sleepy brain can't remember which it is) that allows fireballs/bucking bronco/shock jockey to chain onto nearby enemies once the main target has died, along with Overkill if you have it. And upgrade vigors over guns when at all possible.