Kaye is the Reguard right inside the door of the Imperial Cult shrine in Ebonheart, so he's not difficult to find.
Kaye's quests tend to be more of the more typical "adventurer" kind of quests. You'll need to be prepared to do some combat, sneaking, dungeon-delving and the like. Because of this, they can be somewhat dangerous for lower-level characters, depending on how your skills and attributes look.
You always have the option of reporting back that a quest is too tough for you, but that will terminate the quest and drop Kaye's disposition by 10 points. Just as if you had declined to take the quest in the first place, you'll have to wait a day before he'll have another quest for you. Since the quests get a little harder as you go along, if you're having problems with a particular quest, it's probable that you'll have problems with the next one. So don't wimp out.
If you're having a hard time and need to talk to Kaye, just tell him that you're still working on it and he'll let you access his other topics. Don't try to terminate the quest until you're successful. If that means working on other stuff for a while and then coming back to it, so be it.
Like the other Imperial Cult quest-givers, quests are handled through the greeting rather than through normal dialogue, so be careful what you click on.
An altmer named Caryarel, who was being treated for Swamp Fever, left and a Limeware Platter seems to have left with him. Kaye would like for you to recover the platter. Since Caryarel was being treated for Swamp Fever, this suggests that you might find him in one of the towns on the Bitter Coast, such as Hla Oad or Gnaar Mok. The Altmer are also a rather tight community, so perhaps another Altmer might be able to tell you where to find him.
As it just so happens, there is an Altmer in the Hawkmoth garrison, right outside and he'll be happy to tell you that Caryarel lives in Gnaar Mok. You can get to Gnaar Mok by taking the boat from Ebonheart to Hla Oad and then from Hla Oad to Gnaar Mok.
Caryarel is walking around in town. If you ask him about the platter, he says he doesn't know anything about it. His house, right next to the water on the right as you face town from the boat, is locked (a level 20 lock), which you can pick, "Open" or nick the key from Caryarel if all else fails. You'll find the platter in a barrel behind the table.
Return the platter to Kaye and receive 200 gold and three "Divine Intervention" scrolls.
A Bosmer in Caldera, named Nedhelas, is complaining about his house being haunted. Kaye wants you to investigate and solve the problem. He gives you 200 gold for expenses.
You can get to Caldera by either taking the road north from Balmora, taking the road east from Gnaar Mok, or by by using a Guild Guide. You'll find Nedhelas wandering in the street.
He confesses to messing with a trapdoor that his landlord said he should leave alone. And now he has a ghost in his basement. This is not "a good thing". Not good for Nedhelas because he can't use his own house and not good for you (at least if you're low-level) because you need silver weapons on better to hit a ghost. There are a couple of solutions.
The first and probably easiest is to go learn a Bound Weapon spell that matches up with your skills (Bound Dagger if you're good with Short Blade, Bound Longsword if you're good with Long Blade, etc.). You can find each spell at the Balmora Mages Guild. The second would be to go find a silver weapon. Many Skeletons carry them (Longsword, Short Sword and Claymore). The weaponsmith in Balmora has a restocking supply of Silver Arrows (base cost is 3 gold per arrow). The third would be to come up with an enchanted weapon. You can check out the Enchanter in the Waistworks of the Foreign Quarter or find one as a loot drop.
The next problem is that ghosts use attribute-draining spells. Go prepared to deal with it.
Nedhelas will give you the key to his basement (it also opens the front door). The trapdoor is in the rear and says that it leads to the Heleran Ancestral Tomb. There's the cause of the problem. Down you go.
You'll find two critters down there: a Skeleton two levels below yours and the ghost of Galor Heleran. Take 'em down. While you're at it, you'll find a few crates and barrels that are probably worth investigating, but not much else special.
Report back to Kaye when you're done. Keep the change on the expense money and he gives you a couple of "Golnara's Eye Maze" scrolls (10%-40% Chameleon for 10 seconds).
For your next mission, Kaye sends you out against a Witch named Terlsa Dral. He thinks she's somewhere in the Asha-Ahe eggmine, which is near Khuul. He gives you 300 gold for expenses.
Khuul is one of the handful of places that is accessible by both boat and by Silt Strider. The eggmine is a short distance, almost directly south, from the Silt Strider platform.
When you get to the mine, talk to the miners to find out that there is something strange going on down by the lake. That something strange is Dral and her Barbarian and Healer buddies. They are, unfortunately, standing almost right on top of each other, so taking them down one at a time is going to be a problem. If you can't do ranged combat or magic, you're going to have to do this one-on-three. Since she's the spellcaster, I'd try to take down Dral first. Wounds can be healed fairly easily, but she can summon help(Scamps and Dremora, actually) are something you don't want to take lightly. The Healer, while a spellcaster, too, doesn't pack much in the way of offensive power aside from her Dark Elf Ancestral Guardian Power(which NPCs almost never use).
None of them have much in the way of interesting loot except for Dral's poisoned Steel Lonsword. When you're ready, report back to Kaye (keep the change) and get a Firestone Ring (1-25 Fire Damage on Touch).
An Acolyte named Linus Iulus disappeared some years back. He left Suran headed east and has not been heard from since. The Oracle has had a vision of a Silver Staff of Shaming (which Linus had) in the land of fire and in the shadow of the Mountain of Fear (an old name for Mount Kand). This means that you should be looking either east or west of Mount Kand. To help you find the Staff, Kaye gives you a Ring of Detect Enchantment.
You can get to Mount Kand by going north from the Silt Strider platform outside of Molag Mar and bearing left (uphill) when the path forks. The door to Mount Kand will be on your left near the top of the trail. Since the staff is supposed to be in the shadow of the mountain, you're looking for something on the overland map, rather than something inside of Mount Kand.
Kaye won't tell you (but I will) that you'll need a Levitate effect to go east from Mount Kand (the mountain is awfully steep). A very short linear distance from the door, but significantly higher on the slope of the mountain than the door, is a skeleton. On that skeleton is Linus' Staff of Shaming, Maran Amulet and Stendaran Belt. The Ring of Detect Enchantment should pinpoint it for you once it's inside your little overland map window.
You might be tempted to keep all of those for yourself. Don't. The Amulet is worth a little chunk of change, but the Belt isn't and you'll get your own Amulet and Belt as you get promoted in the Imperial Cult, anyway. If you turn over all three items, Kaye will let you keep the staff and give you a Restoration Shirt (Fortify Fatigue 1-10 points for 5 seconds) on top of it. If you try to keep the other two items, he'll take the staff and just give you the shirt.
Okur is an Argonian living in Hla Oad. She says she is being haunted by the ghost of someone who was murdered. Kaye would like you to talk to her and see what you can do about dealing with the ghost. You can act on things that the guards can't, so go see that justice is done and the ghost is laid to rest. He gives you 100 gold for expenses.
Okur lives in the house closest to the road to Seyda Neen. She tells you that the ghost is that of Julie Aumine and that she was killed by four smugglers, who took her amulet. Okur points you to Yasami, a bandit cave right off the road to Seyda Neen. She'd like you to deal with the four smugglers and return the amulet to her (it's just a common, unenchanted amulet) so that she can give the amulet to Julie's husband and finally put the spirit to rest.
Follow the road south from Hla Oad toward Seyda Neen and, sure enough, Yasami will be on your left a little ways down the road. Stealth before you enter. Two of the four smugglers will be near the door. They are close enough to each other that you probably won't be able to backstab both of them unless you have ungodly Sneak skills.
Dudley, a Level 12 Reguard Warrior, is closest to the door. He doesn't have enchanted anything and is wearing a Nordic Ringmail cuirass and the rest is Netch Leather, so shouldn't be too hard to take down. Tervan Drelas is a Level 13 Dunmer Battlemage and is just a little way down from Dudley. He, too, doesn't have anything enchanted and is wearing a Trollbone Cuirass and shield and the rest is Netch Leather. Again, not too hard to take down, but taking both of them could be a problem.
Down on the dock-looking structure is Larisus Dergius, a Level 13 Imperial Warrior. Again, nothing special, but you might be able to take him out without alerting the last guy.
Ralos Othrenim is a Level 16 Dunmer Assassin. He might have an enchanted weapon, but the rest of his stuff is unenchanted and he's wearing Chitin Armor. He also has Julie's amulet.
Now for the fun part. There are barrels and chests of loot around to the point where you probably can't carry it all and still move. So what you'll want to do is either have a "Mark" set somewhere or use one of the Intervention spells unless you feel like tromping back here a couple of times to get it all. Gather up as much loot as you can carry and dump it where you want to be when you get the last of it. Once you have everything into your inventory, cast your spell and then dump whatever at your destination (no one will run off with it).
Now head back to Okur and deliver Julie's amulet. She'll tell you that she can put the spirit to rest now. You should report back to Kaye to complete the quest and collect your reward: one scroll each of Hellfire, Summon Flame Atronach, and Fifth Barrier.
Kaye will have no more Shrine Sergeant quests for you and suggest that you go talk to the Healers or Almoners.