You have a man with a spear, axe, mace or short sword. This is your basic unit.
They fight better together. You have a tactic!
You can give them a longer, two-handed spear or a shorter spear and a shield. You can also form a line of men with shields and short spears and put a line of men with longer spears behind them. You have a unit tactic.
Putting the chap with the short sword on a horse lets him get around more quickly to see what's going on over there or deliver a message. You have scout cavalry and despatch (or dispatch) riders.
You also have archers, before or after you get cavalry. Archers can work in two ways. They can lurk in the woods and pop out to shoot then leg it so the enemy keeps getting disoganised in trying to chase them down or they can form ranks and shoot at an area to cause damage and confusion among the enemy in that area. For examples of the second approach, see Braveheart or anything about Agincourt, although Agincourt's a bit further along the tech story.
They have archers, so you want a way to do something about their archers. An archer isn't much good against a spearman or swordsman at close range, but they're all the way over there. What you need is a way to get your chap with the short sword over there where they are really quickly. See scout cavalry. See archers. Get more horses, more riders and light armour and send them over together. You now have light cavalry.
Walls and castles are an interesting problem. So far, you have nothing much to use against them except "lots of arrows and a ladder" so you need something that'll damage big walls or at least big doors. This is where siege engines come in.
Siege engine one: the ram. Get a garden shed with a door in one end and a window in the other. Strengthen the frame and remove the floor. Put it on wheels. Hang a tree trunk from the frame with one end swinging out through the window. Put people inside it. Have them push it up to the door and swing the tree trunk. The roof will stop arrows. It works pretty well until some sod gets behind it with a long spear and pokes the crew in the kidneys or the guys on the wall set it on fire.
Siege engine two: the catapult. Spoon, spring, frame, big rock. Sprong. Crunch. Stick, sling, pivot point, frame, big rock. Fwip. Crunch. Works pretty well, and if you use a lot of smaller rocks rather than one big one it's a lot like a mass of archers only better at hurting people who have shields. Not so easy to get through the woods, but probably better on the battlefield.
Between the usefulness of horses and the usefulness of long spears, the idea of a really big spear carried by a man in heavy armour on a really big horse in its own heavy armour is kind of tempting, right? Big enough and heavy enough, it'll smash that garden shed just by stumbling into it. You can also use these guys to smash up the enemy's neat squares of spearmen and cause chaos and dismay to make it easier for your spearmen. This is the birth of the heavy cavalry unit, and its purpose is to charge and inflict damage. Heavy cavalry standing still has a lot of weak points where some little git with a dagger is going to gut the horse, cut the saddle strap, hamstring the rider or whatever. It works best charging.
Then someone comes up with gunpowder, and the big spoon is replaced with a cannon and all the spears now go bang once at the start of each battle. No, really, early muskets were spears that could fire one shot at the start of the battle. They're nothing like as good as archery but that bang's great for intimidating the enemy.
As guns got better and better, they got more and more useful as a means of fighting the battle rather than just a way to start it. The invention of the revolver, the brass cartridge, the revolver that used brass cartridges and the tube magazine or box magazine rifle made them valid as weapons that could also be used as spears, rather than spears with firepower, but the basic approach to things remained the same.
The big spanner in the works was the belt-fed machine-gun. Once a gun could reload itself in one tenth of a second and, with the aid of a water jacket to keep it cool, keep that up through a thousand rounds, cavalry stopped working and advancing in line abreast stopped working. See Ypres, the Somme, Gallipoli, Verdun and all that. Suddenly, there was a weapon that could be carried around and positioned by footsoldiers but not really used by one man as his personal weapon that could stop any advance. Suddenly, the defensive positions were almost unassailable. Understanding ballistics and using explosive cannon balls helped but not a right lot. Horses were useful for moving the artillery pieces, the replacements for the old catapults, around, but not for much else. The despatch riders had motorbikes.
The whole war got stalemated by the fact neither side could advance any further than within MG range of an enemy position. Then someone had a very clever idea: put soldiers with these guns inside an armoured box, mount it on wheels like a ram and have the wheels lay down a road surface at the front and pick it up again at the back to re-use it in a continuous loop. With a big enough motor inside to move it at a respectable speed, it could roll right over craters, barbed wire and enemy trenches and shoot along the trenches and into the backs of enemy gunners. It sounded crazy, but they thought it just might work, so they developed it. Of course, they didn't want anyone else to know what they were up to, so they pretended to be working on something innocuous like ... oh ... a new kind of storage tank for the soldiers' drinking water? That'll do. Call 'em experimental water tanks.
Well, one day these experimental tanks showed up at the front line and they didn't hold water but they made one heck of an impression. Suddenly, one side had an armoured thing that could charge into the enemy's formation and inflict damage. Heavy cavalry, in a rather different shape, was back on the scene!
First thoughts a soldier on the other side had that day: "What's that? What does it do? How do I kill it?" Next thought, shared by everyone on both sides: "Want!"
From the "How do I kill it?" we get mortars, tanks with anti-tank guns, recoilless anti-tank rifles, trucks with rocket launchers, dedicated tank-killer trucks, dedicated tank-killer tanks, thicker front armour, mortars hitting the roof, thicker top armour, artillery dropping their shells under the front ends, the driver sitting in a sling hanging from the roof instead of sitting on the floor, enemy tanks shooting diagonally to hit the sides, thicker side armour, soft-headed shells and rockets, reactive armour, tandem charges, laser detection, rockets designed to cruise along above roof-height with a range-finder pointing down and forwards so they can spot the far side of the tank and fire a shaped charge downwards against the roof aaaaaaaand so on.
From the "Want!" we get everything else on tracks and a side-order of chips.
Light cavalry and scout cavalry got in on the act with the Scimitar [http://www.armedforces.co.uk/army/listings/l0024.html] and Sabre for recent examples. The Sun likes to call them tanks or "light tanks" for its readers, but the army doesn't call them tanks.
The infantry got their Warrior IFVs (Infantry Fighting Vehicles) to carry them around at high speeds with a bit more protection than is offered by a soft-skinned truck and fight alongside them because every eight soldiers should have a friend with a 30mm cannon, yes. Again, the Sun's likely to call them tanks although APC (armoured personnel carrier) is widely-known so they might use that instead.
The signals and engineers have their tracked vehicles and the engineers' tractor is pretty awesome [http://www.armedforces.co.uk/army/listings/l0060.html].
The artillery also got into it, even though they're supposed to be well back from the action and lobbing shells over hills, trusting someone somewhere with a radio to tell them where they boom should happen. There have been a lot of instances of the artillery finding out the enemy's a lot closer than they'd been told and having to shoot them in the face and scarper, and they quite like being inside the tank-like thingy rather than stood next to the field gun when that happens.
The AS90 [http://www.army.mod.uk/equipment/artillery-air-defence/1510.aspx] is a self-propelled gun, the tracked and armoured part of the artillery. It looks a bit like a tank, but if you compare one to a Challenger 2 Main Battle Tank [http://www.army.mod.uk/equipment/fighting-vehicles/1475.aspx] you can see some differences. The tank's got better armour and a gun designed to punch shells through armour at shorter ranges. The self-propelled gun's got a big gun that elevates to lob shells a long way, a big turret housing it and rather less armour because it was never meant to get into dirty dancing with Soviet tanks and the Challenger was designed to do that and win. The tank's also got 1,200 bhp to the AS90's 660 bhp.
Of course, not everything's in tracked vehicles. They're expensive, they're noisy, they're slow, the chew up the roads, they can't use any but the strongest bridges (62.5 tonnes is a lot) and they DRINK fuel. We still use light trucks aka Land Rovers and Bedford 3-tonners for a lot of stuff.