The coastal waters have given many people a profession, and in years gone by, fish always played an important role in everyday food. Today fresh fish is quite expensive and exclusive; nowadays it is considered a luxury to eat fish.
A typical summer "dish" is shrimps. Do like the locals, buy a bag of fresh shrimps and enjoy them sitting beside the sea. You eat the shrimps with white bread, mayonnaise and a squeeze of lemon. However, the shrimps can be quite expensive due to the massive demand.
Sørlandet has a number of great fish restaurants; you will find many of them down by the fishmarket/qayside in Kristiansand.