Most candida sufferers feel alone, and nobody really understands what they have or what they are going through. Because they do not experience these same symptoms they sometimes think that we are making it up or are crazy. This is not unusual for doctors to feel that way, and I absolutely feel your pain. I suffered with candida alone for many years before I was finally able to get the treatment that I needed.
Candida Die-Off is usually accompanied by being on a strict candida diet which will kill the candida. It's possible that your candida is reacting to the acohol in your body, but it's also possible that there is something else going on.
If you believe it to be your liver it would be best to get tests done. If you explain the symptoms to your doctor you might be able to get insurance to cover it. At least get liver enzyme tests done. It is odd that you would experience hangover like effects when you have small amounts of alcohol because the real cause of a hangover is dehydration and just 1 sip should not give someone a hangover for 2 days. There could be several different reasons such as you're allergic to the product the alcohol was made with. It doesn't always have to be candida, die-off, etc.
If anything does bug you though, completely avoid it. It cannot be good.
For candida die-off, I've had candidaoff which is sold by probacto with good results, but am not sure it will help you in your situation. You can try it, but it all depends on what your symptoms are caused by. Are they caused by the alcohol or candida? That is what you have to find out.