Nitisha - we have chosen to disable Associate Contacts because it allows me the ability to search the entire contacts database - meaning, I cannot restrict it to *just* searching for contacts associated with the present account. It's just too risky that someone could inadvertently associate a contact from the wrong customer and send the email.
Setting that aside for a moment - when I tested the Associated Contacts feature, I don't see all fields available to select as one of my filters. We have a custom Contact Type field on the Contact object as well as on the Account Relationship Contact object (which provides the related contacts function) that would be crucial in ensuring we select the right "type" of contact. The Contact Type field could be set to VALUE A on the direct Contact object, the Contact Type field could be set to VALUE B on the Associated Relationship Contact object - but both for the same person.
Without these capabilities, Email Assist (and I guess also External Attendees) is "broken" for us. We really want to use Email Assist, but in its current state we cannot.