Of course we can't know for sure what may be causing the problem in your case, but we can try to mention a few things that might typically cause problems maintaining a steady temperature in the styrofoam incubators.
First, if you are reliant on manual turning, opening the incubator to turn your eggs does drop the temperature... but that is pretty temporary and shouldn't cause problems if you are quick. It can help to mark one side of the egg so you can see which have been turned and which haven't. (Use something like soft pencil, that won't soak into the shell like, say, a marker would. And be careful, don't push too hard!)
Because a hen has to get up to eat, drink, and poo, opening the incubator briefly to turn your eggs manually really shouldn't cause a temperature problem. Of course, don't open the incubator more often than it takes to turn the eggs. However, if you don't put the lid on correctly when you are done, you could see problems with maintaining your temperature. Do be sure to replace the lid securely so there are no gaps and it is not sitting wrong.
More likely, though, is that your incubator isn't sited correctly. If your incubator is sitting in a place where it gets direct sunlight, that could heat it up when the sun shines in. If it's in a place where it gets breezes, then that can cause problems, too. If it's in a place where it gets both, the temperature could be going up and down depending on what's happening around it. As another example, it may get hot if one of your home's heating vents blows right on it, or the incubator may cool down if it's in a place where an outside door lets in a blast of winter air. If it's beside a drafty window or door the temperature of your incubator may vary more than you'd like, too. Plus, breeziness might also make it difficult for you to maintain proper humidity, if all that moist air is being blown away!
In addition, in that type of incubator, very small adjustments can drastically change the temperature, so if it's a little low and you give it a twist to raise the temperature, it's easy to over-adjust so the temperature gets way high by mistake! So, it's possible that you are having some problems just getting the incubator set, too. To get your incubator set correctly from the get-go, what you usually want to do is set your incubator up several days beforehand and get it going. (Make sure it is in a good spot, away from heat, cold, and direct sunlight, as described above.) After a day or so, start making (small!) adjustments to get it to hold the right temperature. Every time you make an adjustment, give it a couple of hours before you make another one, so you can see for sure what your first adjustment has done. Eventually--presuming it's located in a spot away from direct sunlight or any breezes--your incubator will be maintaining the proper temperature, with no adjustments needing to be made. When that happens, you know it's ready for eggs to be added.
Then, after you put the eggs in, don't make any adjustments for many hours. Obviously, the temperature will drop when you open the incubator to put the eggs in, and it may stay low for a while since the eggs go in cold! It takes a while for them to get heated up. If you know that your incubator temperature is set right, then you shouldn't need to make adjustments at this point; just wait until they get heated.
Sometimes at around day 10 or so, you may see that the temperature is starting to edge too high, and so you may need to slightly adjust the temperature down. This is because your eggs may have started producing a little bit of heat on their own as the chicks develop. If you have to adjust the temperature any--and you may not have to if you are incubating very few eggs--be sure to make VERY small adjustments because it's so easy to overcompensate and get it WAY too hot or WAY too cold by mistake.
A good tip that we share in our free guide to incubation and hatching is that you can add some (clean, sanitized) river stones or rocks to your incubator when you are hatching just a few eggs at a time, and they act as heat exchangers to help stabilize the temperature in your incubator. That means that when you open the incubator to turn your eggs, the heat will return to normal much more quickly when you're done.