Al-Imaam Muhammad Ibn Saalih al-Uthaymeen (may Allaah be pleased with him) answered this question:
"This is a matter in which the scholars have differed. From amongst them, there are those who say that it is Haraam to talk while the Imaam is giving the Eid khutbah. Others have said that there's no harm in talking because it is not obligatory to stay and listen to the khutbah. There is no doubt, that from good mannerisms, is that one shouldn't talk. That's because when the person talks, he busy/distracts himself or other than himself from those whom he is addressing, or he busy/distracts the one who hears him or witnesses him."
Source: Fatawaa Al-Mar'ah Al-Muslimah pg. 260
Sisters Upon Al-Istiqaamah