If by “Semite” you mean “Semitic language speaking” then yes. But “Semite” isn’t actually a term that’s used, it’s an invalid racial categorization. “Anti-semitic” in standard and common language specifically refers to bigotry against Jews