Every teenager discovers, at some point, the inherent weakness in the idea of freedom of speech: "if you support freedom of speech, then you support my right to loudly proclaim that you are racist/impotent/a paedophile/etc, in the terms and manner of my choosing, regardless of whether my accusations are based on any factual evidence.".
Those who went to private school, or who received their education back when children, like animals, had no say in their upbringing (and rightly so, for like animals, children have agency but no ability to perceive goals beyond their immediate self-interest), were quickly taught that what is colloquially referred to as "freedom of speech" is actually "freedom of opinion": an individual has the right to hold any opinion they desire (even and especially unpopular ones), and has the right to express their opinion according to the rules of civil discourse, without fear of legal reprisal.
A complete reading of the oft-invoked First Amendment (to the 1787 Constitution of the United States, it hardly need be said) specifically limits what can be considered a constraint on freedom of speech (ignoring, for the moment, interpretations by the Supreme Court which may or may not be rooted in political expediency):
Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.*
The First Amendment, therefore, only makes it illegal for Congress to pass a law diminishing freedom of speech. Since many State legislatures take their cues from Congress, and many local legislatures take their cues from the State legislature, this effectively translates to "No law shall be passed which abridges the freedom of speech". We can leave off the technical definitions of "abridge" and "freedom of speech" for now: what is important is that this is a restriction on the laws that can be passed. It is not a guarantee of freedom of speech. Private individuals, non-legislative organizations, companies, and so forth can all impose restrictions on what may or may not be said in their presence, using their facilities, or over their communication channels.
What about that "abridging", though, and that "freedom of speech"? It is illustrative to look at the laws that do apply to speech: namely slander, libel, and fraud.
Slander and fraud are similar: knowingly representing a falsity as truth. This includes the famous "falsely yelling 'Fire!' in a crowded theater": it is quite permissible to yell "Fire!" if there is, indeed, a fire, but to do so when there is not is to make a misrepresentation of fact -- and claiming that every individual in the theater could perform their own fact-checking to verify the claim of fire is not a valid defense, nor is the claim that the person yelling "Fire!" was not in the uniform of a Fire Department, and therefore should not have been considered an authority on the matter of the theater's flammability.
Unscrupulous individuals (cf. LBJ's "But let's make the bastard deny it!") will present slanderous accusations as opinion ("it is my opinion that he is a paedophile!"), and that is why we have libel. This gets a bit tricky, because there are matters of public interest ("does the head of the SEC have a gambling problem?") and public record ("this person was convicted of child molestation by a fair jury trial"). Simply put, if the matter is not of public interest or public record, it can be considered libelous: the publication (on any media, be it print, radio, video, twitter) of information, true or false, intended to damage the reputation of another. It may be both true and unsavory that your political opponent regularly masturbates to the AARP Magazine; but it is libelous to publish the claim that he does. That word "publish" is key: recording a damaging claim is legal,
but publishing that recording is libelous. This is regardless of intent: the intent of neither the speaker nor the publisher may have been to damage the target of the claim, but the act of publishing causes damage and is therefore libelous.
Of course, all of this leaves out the actual justice system, which requires that legal expenses be incurred and that damages be demonstrated and so forth. Again, the justice system is as erratic and self-serving of its personnel as any political institution, and is not to be entirely relied on as an arbiter of what should and should not be legal.
*Emphasis mine, of course, to extract the clauses relevant to freedom of speech. The same thing could be done with the Second Amendment, for example: those troublesome first two clauses amount to the simple justification "because a well-regulated militia is necessary to the security of a free state", and while they provide context, they are not part of the actual legal clause about the right of "the people" (but not, you will note, "every person") to bear arms.
