It depends on the bug, and it depends on where the bugs live. For example, winter in Florida is a lot different from winter in Alaska.
Let's take the praying mantis as an example. Praying mantises create these really interesting egg cases on plants. The egg cases are brown and spongy, sort of like the foam you would find in a sofa cushion. An egg case is about the size of a big marble. The adult praying mantis puts the egg case on a branch of a bush or a tree. The eggs survive through the winter in the egg case and then hatch when the weather gets warm in the spring. Many praying mantises die in the winter, but the eggs make it through and create all the new praying mantises each year.
Some bugs hibernate through the winter by hiding in places under tree bark, fallen logs or stones. Since they are cold-blooded, these bugs go dormant in the winter and wake up in the spring. Honey bees camp out in their hives eating honey that they stored in the summer. Ants camp out in their nests underground. Each bug is a little different, but they all have strategies to make it through the winter.