Character leveling (The Witcher 3: Wild Hunt)

After moving to the second location, I decided to run around the "question marks" on the map before the main plot. Basically, there were opponents from lvl 9 and above (I have the 4th). Am I so unlucky with the choice of these interesting places on the map, or do I really need to start pumping up a little on the plot, so that later these places would coincide approximately in level? You can, of course, go through these opponents at this lvl, but I would like to consistently pump. I hope I was able to convey the essence of the issue, given the "twitching" at work)