Smart Text Tools

Home › Guides

Cleaning up text you pasted from somewhere else

Invisible characters, broken line breaks and curly quotes — where they come from and how to get rid of them.

Pasted text arrives with passengers. Most of them are invisible, and all of them cause trouble later.

Line breaks from a PDF

A PDF has no paragraphs, only lines drawn in positions. Copying gives you a hard break at the end of every visual line, so the text reflows wrongly in anything you paste it into. Joining the lines while keeping blank lines as real paragraph breaks fixes it in one pass.

Non-breaking and zero-width characters

Web pages are full of non-breaking spaces, and messaging apps add zero-width joiners. They look identical to a normal space or to nothing at all. They break searching, sorting and exact matching, and they are the usual reason two identical-looking entries refuse to match in a spreadsheet.

Curly quotes and long dashes

Word processors replace straight quotes with curly ones and double hyphens with dashes. That is right for print and wrong for code, file names and plain-text forms, some of which reject the characters outright.

A sensible order of operations

  1. Remove invisible characters first, so later steps see the real text.
  2. Join broken lines if the text came from a PDF.
  3. Collapse repeated spaces and trim the line ends.
  4. Convert curly punctuation only if the destination needs plain characters.
  5. Count the words last — the figure before cleaning is usually wrong.

Devanagari legitimately uses the zero-width joiner in a few conjuncts, so the clean-up keeps those while removing the stray ones.

More guides