Nobody, least of all referees, enjoys a paper which is badly written and full of errors.
Even if the result itself is first-class, a cavalier attitude to expository or accuracy issues (or the presumption that it is solely up to the editor and referee, rather than the author, to address these issues) will negatively impact both the referee reports on the paper, and the editor’s final decision on the paper. (Even if the paper does eventually appear, these issues may also discourage potential readers from working their way through the paper.)
In particular, having an error discovered after the submission process, which then necessitates sending a revised version onward to the referee, tends to tie up a lot of time and thus slow down an already sluggish process.
Even “trivial” issues, such as spelling errors, can unintentionally and needlessly convey a negative and unprofessional first impression of sloppiness and/or laziness, and can prejudice readers, referees, and editors against a manuscript.
One’ s use of citations should receive particular scrutiny; if an important and relevant reference in the subject is mentioned only tangentially, cited inaccurately, or omitted altogether, then this can cause all sorts of misunderstandings and unfavorable impressions which could have been avoided with a little bit more care. When in doubt, don’t hesitate to look up primary sources to double check that the citations are being handled properly.
Investing some time before submission to try to eradicate these sorts of problems (and to give the paper one last polish, in the meantime) is usually well worth it; it earns the goodwill of the editor and referee, thus speeding up the publishing process, and also improves the final product.