Like your ship, you may only own one house at a time. They come in a variety of styles and price ranges. You buy a house from the bank or you can get one from a Knights Guild upon reaching 9th level. If you acquire it from the bank, the house will be in the same town youre in when you buy it. If you reach the rank of Paladin in one of the knightly orders you will be given a house somewhere in that province. Most commonly the house will be in whichever town you were in when you are awarded the rank of "Paladin". There is a guy in the back of the guild that has "Receive House" on his menu. He will mark it on your map for you. Save the game before buying a house. Reload and buy a different one if you don't like the location.
You may sell your house back to the bank and this includes your Paladin House. Like your ship, any items stored inside are lost. You may also rest safely in your house and store items in it. An early patch fixed the bug that would not let your rest in your house.
In your house you may find boxes and other pieces of furniture which appear to function as shelves in a store. DO NOT STORE ITEMS IN THESE BOXES OR SHELVES!!!! The next time you leave town, they will disappear. One of the later patches has supposedly made these boxes and shelves inaccessible, but don't believe it. The major drawback to a house is that it stays in one place while you do not.