drag down

to cause someone’s social position or standards of behavior to become worse, so that other people lose respect for them

to make someone feel unhappy or less hopeful