I could add a /setting to close menus when attacked, although I suppose you also might want to quickly exit menus before an attack. I could make a command to close dialogs. You'd have to press F6 to open chat and then use /close. I'll add it for the next update.