The Flying Dutchman was a ghost ship in old-Earth folklore, a ship that could never return home, and was doomed to sail the seven seas for eternity.
In the 22nd century several of the MACOs on the starship Columbia (NX-02) nicknamed their ship the flying dutchman in reference to the folktale after Captain Hernandez made the decision to travel at sub-warp speeds using the time dilation effect to get to the planet Erigol. (ST - Destiny novel: Gods of Night)
Later the nickname was also applied to the USS Enterprise-B by some members of Starfleet, as a reference to the starship's maiden-voyage in which Captain Kirk was believed killed. (TOS novel: The Captain's Daughter)