Remove duplicate lines
Drop repeated lines and keep the first copy, in the order you pasted them.
Paste a list. Near-duplicates that are not the same line are kept.
Processed in your browser. The text is not uploaded.Read the privacy policy.
Lines kept
No text yet. Paste, type, or load the sample.
The first trimmed copy of a non-empty line is kept. Later exact copies are dropped. Empty lines stay, so paragraph breaks are not treated as duplicates.
—
- Characters in
- —
- Characters out
- —
- Removed
- —
How to remove duplicate lines
- Paste the list or the draft. One item per line if you are cleaning a list.
- Leave “Ignore letter case” off when “Notes” and “notes” are different entries.
- Turn it on when the duplicate is the same words with different capitals, such as a heading that was pasted twice in different cases.
- Copy the result. If the lines are duplicated because of wrapping rather than repetition, remove line breaks first. Wrapping is not duplication.
The match is the trimmed line
Each non-empty line is trimmed, then compared with the lines already kept. The first time a key is seen, that trimmed line is kept. The next time, it is counted as removed and dropped. Empty lines are kept so a blank line between paragraphs is not “a duplicate of the blank line on page one.” After that, three or more newlines are collapsed to a single blank line, and blank lines at the very start or end are trimmed.
The sample contains two copies of “Introduction” and two copies of “The first finding,” plus “Notes” and “notes.” With case-sensitive matching it removes 2 lines. With case ignored it removes 3, because “notes” then matches “Notes” and the first spelling wins. Load the sample and toggle the checkbox. The removed count on the panel should follow those two numbers.
Spacing at the ends of a line does not make a unique entry. “Alpha” and “Alpha ” are the same key, and the kept line is the trimmed one. Spacing in the middle still matters: “Alpha beta” and “Alpha beta” do not match, because this page does not collapse internal spaces. If that is the duplicate you have, runRemove extra spaces first, then come back. Case conversion is also separate.Lowercase will not delete lines. It only changes letters, which is why the ignore-case option lives here instead of pretending to be a case tool.
What you should not expect
The tool will not merge “the first finding” with “first finding,” and it will not notice a sentence that was repeated inside a paragraph on the same line. Duplicate detection is line-based because that is the job this URL claims. A word count of the result, if you need one, belongs on theword counter after you copy. Sorting alphabetically is not included. Keeping the original order is the safer default for a log or a bibliography.
Limitations
- Only whole lines are compared, after trimming. Similar lines are kept.
- The first copy wins, even if a later copy is the one you edited.
- Ignoring case uses English (United States) lowercase. It is the wrong tool for a case distinction that matters in another language.
- There is no undo on a server. Use “Load sample text” or paste again if you toggled the wrong option.
Questions people ask
Which copy is kept?
The first one, from top to bottom. Later lines that match are dropped. The order of first appearances does not change. This is not a sort, and it is not a “keep the last edit” tool.
Does “Notes” match “notes”?
Only if you turn on “Ignore letter case.” The default is an exact match after trimming spaces at the ends of the line. The sample keeps both “Notes” and “notes” until you ignore case, and then it removes one more line.
Will blank lines disappear?
Empty lines are not treated as duplicate content, because a paragraph break would otherwise survive only once in the whole draft. Extra blank lines are still collapsed so you do not keep a stack of them. Non-empty lines are the ones that dedupe.
Why did two similar lines both stay?
Near-duplicates are kept. “The first finding” and “The first findings” are different lines. So are lines that differ by punctuation. The tool does not fuzzy-match, because a fuzzy match will delete a line you meant to keep.
Is the list uploaded?
No. Duplicate removal runs in the browser. Copy the result if you need it after you close the tab.