Genocide is the act of deliberately terminating the lives of an entire racial, national, ethnic, or religious group of sentient people. Throughout the galaxy there have been numerous examples of genocide or attempted genocide.
In the alternate reality created by Nero's temporal incursion, Nyota Uhura grew up hearing terrible stories from older members of her family about the wars of genocide in Africa's recent history. (TOS - Starfleet Academy novel: The Gemini Agent) In the same alternate reality, Nero later committed genocide by destroying Vulcan in 2258. He claimed that by doing so he was preventing genocide, since he blamed the Vulcans for the destruction of Romulus in the primary universe. (TOS movie, novelization & comic adaptation: Star Trek)
In the mid-2260s, the being Quetzalcoatl - who had visited Earth several hundred years earlier - traveled to the Klingon planet, Hrakkour, and influenced its population to adopt a more peaceful lifestyle. In response the Klingon Admiral Vlict Kenka committed genocide, killing the entire population of Hrakkour. (TOS video game: 25th Anniversary)
Section 31 attempted to commit genocide against the Founders of the Dominion. While Odo was being examined at Starfleet Command, an operative managed to infect him with a morphogenic virus, which Odo spread to other Changelings when he joined with them. Doctor Bashir discovered this, and managed to retrieve the cure from Luther Sloan's mind just before he died. (DS9 episodes: "When It Rains...", "Extreme Measures"; DS9 novel: Abyss)
At the conclusion of the Dominion War, nearly the entire Cardassian population turned against the Dominion during the Battle of Cardassia - the Cardassian fleet began attacking the Jem'Hadar and Breen vessels. When informed of this, the Female Changeling attempted to commit genocide when she ordered the Jem'Hadar to kill all the Cardassians still living on Cardassia Prime. The genocide was stopped when Odo convinced her to surrender to the Federation and their allies. (DS9 episode: "What You Leave Behind")