"It is deeply regrettable that the Prime Minister dismisses the very people who helped secure his victory, and he should remember that no prime minister has a divine right to rule," the letter, published in The Telegraph newspaper, reads.
According to the letter, Cameron secured the majority he needed to form a government only through the dedication and hard work of Conservative Party members.
"We urge Mr Cameron to accept that loyal Conservative Party members want the opportunity for a balanced debate and that he and his colleagues should listen to the views expressed by all at the grassroots," the letter said.
As the nation prepares for the referendum, Cameron is facing a growing battle with eurosceptics over his deal with Brussels, while Conservative Party members traditionally hold eurosceptic views.