Pollster Richard Baris explains it all. The most important single factor is sampling errors. Most pollsters over sample urban areas and under sample rural areas and whites without college degrees. This produces polls that tend to overstate Democrat strength and understate Republican strength. Add in shy Trump voters and most polls are wildly inaccurate.
Everyone knows that there is a huge sample bias when it comes to phone polling, which is still the primary form of polling. The difficulties are immense: many people do not have land lines, and those that do are usually not willing to spend a long time talking to a pollster for long periods of time. This is partially due to (well founded) distrust in pollsters, but also because a lot of people don’t generally talk on the phone for long periods of time anymore (with texting, social media, and various forms of video communication being more widely used).
In a way this is advantageous to pollsters. Everyone knows that a raw sample can’t possibly be representative of the actual sentiment, so they have to reweight the data to have any chance of accuracy. But there are limitations to what can be known through reweighting (we’ll never get to that Asmiov story where the entire nation’s opinion is known through surveying one single man) and the pollsters usually aren’t concerned with increasing accuracy anyway.
Confirmation bias is also a factor I believe.
So go back to sleep Joe, Trump is doomed, doomed, doomed.
ssshhhh