to be down on one’s luck

Example: I was down on my luck And I thought of the good times.