I've had the same. I was getting panic attacks also and extreme anxiety. Strange fears that made no sense and I decided I was probably going crazy. I never figured it had anything to do with the high carbs or sugar I was still eating. Even though I wasn't eating a great deal of sugar, I was eating just enough to keep this happening.

I think it can appear to hit out of the blue, but the problem may have been there for sometime before the final straw. Once you hit that point, then it doesn't take much to set off those reactions.

I had to make an entire clean up of my diet to reduce the yeast and then the anxiety/panic problem went away (almost completely). Because the upset blood sugar levels appeared to be both from the diet, but also side effects from the yeast (which give off alcohols).

Took a few weeks. I had to go through the die off process though and it's not fun, but worth it. You know if it's yeast, when you start feeling the impact of them dying off. Once your body has reduced the excess yeast, you can start to notice real differences. In energy and mental outlook and many other things can improve.

I had to cut out sugar, wheat/gluten, yeast, and reduce or eliminate high carb foods like fruit, grains, milk (lactose), high carb vegetables like potatoes/corn/pea/beans.

Also coffee.

So though my diet was limited, at least I felt better and more normal. I just basically consume meat, chicken/liver, eggs (free range), low carb vegetables, yoghurt (sugar free, acidophilus), almonds/brazil nuts and butter, extra virgin olive oil, mercury free fish oil and take supplements.

Liquids - just herbal or black tea (no milk) and vitamin powders in water.

That's about it. If I go off this, I pay for it big time. It's not worth cheating at all.

This maybe what is happening to you. Also, when I start getting hypoglycemia symptoms many years ago, I thought I should eat more sugar to help, yet instead, it was making matters worse in the long run. once I got on a sugar free/gluten free diet, I had dramatic improvements (though at that time, I had mercury toxicity, but not the viral/infection issues I have now).