The Justice Department signaled that it isn't done scrutinizing how real-estate agents are paid, as the industry prepare to abandon its longstanding commission structure next month.