A kayak would be a great way to fish some of the more out of the way waters of around Joppa, Woodbridge Island, and the Plum Island River. I would rule the areas toward the mouth of the river out. One because of the currents(you can fish or you can paddle but not both) and two the boat traffic. While you might work your way out toward the Badgers, Bar, jetties and mouth before the boat traffic comes up, you'll take your life in your hands getting back up river. Friday night until Sunday late afternoon can be ridiculous.
However, the Plum Island River and Woodbridge Island are a different story. Alot of guided kayak trips frequent these areas so I assume the boat traffic isn't too much for even novice kayakers. These are bony waters which can require a stealthy approach afforded by a small boat, canoe, or kayak. I often fish the Woodbridge area early in the season out of a 12' aluminum boat which I let bounce along the creek banks on the incoming tides. The fishing is very much like trout fishing in that you focus on undercut banks and any current obstructions in the creek channels.
Another approach would be to put in at Cashman's Park and check out some of the sheltered upriver spots around Ram's and Carr Islands. This time of year the fishing will be spotty but you may snag a big resident fish. This trip would probably be more paddling than fishing.
The more I think about it, the more possiblities I can think of. Some of them might get you into the fall fishing on the oceanside. email for details.