The HMRC is failing to "discriminate effectively" any activities it flags as tax avoidance, the Lords Economic Affairs Committee said in a report published on Tuesday.
"There is a clear difference in culpability, for example, between deliberate and contrived tax avoidance by sophisticated, high-income individuals, and uninformed or naive decisions by unrepresented taxpayers," the Lords committee said.
The report also noted loan charges which could potentially slap roughly 50,000 people with massive tax bills due to payment schemes they had previously used as contractors, adding that the HMRC failed to communicate effectively with affected parties.
"HMRC is right to tackle tax evasion and aggressive tax avoidance," committee chair Lord Forsyth of Drumlean said. "However, a careful balance must be struck between clamping down and treating taxpayers fairly."
"Our evidence has convinced us that this balance has tipped too far in favour of HMRC and against the fundamental protections every taxpayer should expect."
READ MORE: Yellow Vests: No Coincidence Macron, Merkel and May are in Dire Straits – Journo
"We are encouraged to see some parliamentarians recognise that HMRC has gone too far in many instances, and that the balance is tipped against ordinary taxpayers," TaxPayers' Alliance chief executive John O'Connell said.
"It's important to ensure that taxpayers who have been unfairly treated get the justice they deserve. Britain's hugely complex tax code places an enormous burden on ordinary taxpayers, and it's time for the government to take bold action to simplify the system."
The HMRC responded to the allegations on Tuesday, stating it had taken "unprecedented" actions to stop tax avoidance while using its powers "responsibly".
"We've taken unprecedented action to crack down on avoidance and evasion, making sure people pay their fair share of tax and securing funding for our vital public services," an HMRC spokesperson said in a statement. "Parliament has given HMRC powers it needs to tackle businesses and individuals who do not pay their fair share, and it uses them responsibly and subject to appropriate checks and balances."