Worst of It

Fighting the odds; usually preceded by take the or have the; a situation in which a wager has an unfavorable return. Opposite of best of it.