One of your first purchases should be a wagon. You can buy (or steal) one at any General Store. If you buy it, it will cost a few hundred GP, but it will pay for itself several times over. In the original (unpatched) version of the game, wagons had a carrying capacity of 1000 kg. One of the earlier patches reduced that capacity to 500 kg and it was later bumped up to 750 kg, where it remains to this day. 750 kg is about 10 times what you could carry on your own, so its nothing to sneer at. If you have the money, go ahead and buy a horse. It makes traveling a lot faster without having to fatigue yourself. You can live without a horse, though. Your wagon (and horse and ship, if you buy them) remain outside when you enter a dungeon or one of the three castles (Daggerfall, Wayrest and Sentinel), but are otherwise accessible. You may not access your ship while you are inside a building.
There is a hole in the program (I dont call it a bug because Bethesda has been aware of it for some time and hasnt fixed it) that will allow you to bring your wagon into the dungeon. Not physically, but you can have access to its carrying capacity. To take advantage of it, visit any merchant, make like youre going to sell something (which will bring up your inventory screen), click on the Wagon button, and exit. When you open your inventory again, the contents of your wagon are showing in the left column instead of the right column. It will remain this way until you click on one of your personal inventory buttons (Weapons, Magic Items, Miscellaneous, or Ingredients) or load a saved game. The same trick works if you try to cast an "Identify" spell and look in your wagon (thanks to Jackie Eason for pointing this out to me).
The next time you kill something and take the stuff off the body, the stuff you take will go into your wagon rather than your personal inventory (except arrows and gold). As long as you dont try to access your personal inventory, your wagon will remain available to you without having to return to the entrance.