You gain a level when you get ten increases in any, some or all of your Major skills. In and of itself, gaining levels doesn't do much. One of the obvious benefits is that you will gain extra Health points (1/10 of your Endurance). Another fairly obvious benefit is that you get to increase three of your attributes by one to five points each. The way the game decide how many attribute multipliers is fairly straightforward: take the number of increases for skills under any given attribute and divide by 2, dropping any fractions:
The upshot of all of this is that if you want to maximize the attribute multiplier, you have to be fairly careful about developing ALL of your skills. Or, to put it a bit more simply, don't forget to bring those Minor skills up, too. If you'd like to maximize your Strength multiplier, make sure that you work on all three offensive combat skills: Blade, Blunt and Hand-to-hand. It doesn't have to be done all of the time, though. For example, you can swing a sword or dagger for the first part of the game, pick up a mace or an axe later, and be Jackie Chan at another time. Once you hit 100 in any skill, you're not going to be able to develop it further and will have to use a different skill to get your multipliers.
The next problem you're going to run into is when you've got more than three attribute multipliers to choose from. You might be tempted to take the three highest, but that may not be the best route to go. So here are a few things to consider on each of your attributes:
As you gain levels, you're also going to be getting better loot (like 20 gold at first level, as compared to 745 gold at 30th level). This includes the materials of the weapons and armor.
This does not mean that these will be common at these levels (well, aside from 1st level, obviously). Even though Dwarven stuff might start appearing at about 5th level, I wasn't finding it frequently until 9th or 10th level.
The flip side of this equation, though, is that almost every hostile encounter you have is controlled by your level. And even for those that are pre-placed, your opponent's level is going to be tied to yours. So the higher your level, the more dangerous the enemies you encounter. For the most part, this isn't going to be a problem with the normal dungeon exploration and side-quest kind of stuff. But be advised that Oblivion (as in "the place behind the Oblivion Gates") becomes much more difficult at higher levels. Not impossible, mind you, but more difficult. Instead of encountering 1 or 2 critters, it might be closer to something like 5 or 6. And leveled NPCs seem to always get the +5 attribute multipliers as they go up. So if you're going to put off the Main Quest for a while, be prepared to proceed cautiously, take the proper equipment (hammers, potions, spells, etc.) and don't be surprised if you get spanked hard and often.