Religious freedom = Legal discrimination
We all have the freedom to believe what we want, and governments should allow us that. However, religions are filled with discrimination and restrictions in thought and action not allowed elsewhere.
Firstly, a caveat. While we have free will to choose whatever we want to believe, governments can shepherd our thoughts over time to be more amenable to their ideologies, and totalitarian governments can put so much psychological and physical pressure on people that they breakdown and forego their choice. Here we will deal with democracies that take some measures to care for their people. In these, there will be many laws protecting people from discrimination based upon things like gender, sexual orientation or colour.
However, while people can believe whatever they want, religions are far more prescriptive about who can belong, and what they can do. Most religions have discriminatory practices, particularly in relation to women and what they are allowed to do.
Of course, any club will have a focus that will tend to favour particular people or interests, but they don't tend to exclude other people by rule, probably because of legislation. That allows anyone to be involved, but only those interested will likely want to be involved. The point is that they are not legally discriminating, and so do not require special legal protections or controls other than for being for administration as a legal body conducting activities.
Religions have a special place in legal history because most societies were based around particular religions, and they tended to discriminate against other religions. As those societies matured and interacted with other societies, religions expanded and gathered adherants in other societies. To facilitate the peace, governments tended to frame laws allowing people to practice their religion in peace.
However, those framing laws regarding religions couldn't require fairness in other's religious practices if they wanted to keep their own discriminatory practices. Thus we have the situation in modern democracies where almost all institutions have to have be fair and non-dicriminatory, except for religions.
Humanity has evolved since most scriptures were written, so while those scriptures may contain elements of history and rudimentary legal structures and regulations, we now have much more sophisticated and expansive institutions for those, yet religions are given legal freedom to ignore aspects of these newer institutions.
Religions and lawmakers were once one and the same β and still are in many countries β but modern democracies have attempted to separate religion from direct involvement in political processes, mainly because of the real problems that brings. To serve modern pluralistic societies, religions need to be legally restricted to providing spiritual guidance and advising practices that enhance peoples' spiritual understanding. Of course, they can provide aid to the disadvantaged, but really governments should just take on their care of duty to their citizens instead of relying upon the charities.
The main area where religion can run counter to the aims of freedom in thought in society is in education, where care must be taken to ensure children are not subject to such a restriction of ideas that they are not able to properly exercise their free will as they grow older, just because they are not able to think outside a carefully constructed ideological framework that does not psychologically allow them to deviate from it.
With the long-term pattern of diminishing prevalence of people nominating a particular religion, we need to ensure that that trend is respected, and religions are not given preferential treatment in how they deal with people who otherwise should have more choices and opportunities. Religions are mainly based on texts that are essentially only opinions, and while some of them may hold some truth, people must still be free to choose what they believe, without undue pressure to conform to discriminatory practices that would otherwise be illegal elsewhere in society.