Two men on trial for killing an elderly man claim that he had abused his family for decades.