How to use the remove duplicate lines
- Paste your lines.
- Choose whether matching is case sensitive.
- Copy the unique lines. The repeated values are listed below the boxes.
About this tool
Paste a list and every repeated value is highlighted in the preview, with a count of how many times it appears. See the total items, unique items and duplicate copies at a glance, then remove duplicates with one click. The first occurrence is kept and the original order is preserved. Use Undo if you change your mind.
How the duplicate preview works
Paste your list on the left, one item per line. The preview on the right shows the same list with every repeated value highlighted in amber, the way Excel highlights duplicate cells. A badge such as ×3 tells you how many times that value appears in total. The cards above the boxes show total items, unique items, duplicate copies and how many different values are repeated.
Nothing is deleted until you click Remove duplicates. The first occurrence of each value stays, the original order is kept, and Undo restores your previous list if you change your mind.
A worked example
Take this list of six lines:
apple banana apple cherry banana apple
- Total items: 6. Unique items: 3. Duplicates (extra copies): 3. Values repeated: 2.
- apple is highlighted with ×3 and banana with ×2. cherry appears once, so it is not highlighted.
- After Remove duplicates the list becomes apple, banana, cherry, one per line.
Why two lines that look the same are not duplicates
The usual surprise is invisible differences. Three options control how lines are compared:
- Case sensitive: with it on, "Apple" and "apple" count as different values. Turn it off to treat them as the same.
- Trim spaces: removes spaces at the start and end of each line, so "apple " matches "apple". Leave it on unless the spaces matter.
- Ignore blank lines: empty lines are skipped, so they are not counted as one big duplicate.
Common uses
People use a duplicate remover for email and phone lists, order numbers and SKUs, URLs for a sitemap, keyword lists, log lines, student or employee IDs, and any column copied out of a spreadsheet. Finding duplicates is often the real goal, not removing them, for example to check whether the same invoice number was entered twice. The Show only duplicates option lists just the repeated values.
Doing it in Excel or Google Sheets
In Excel, select the column and use Data, then Remove Duplicates, or Home, Conditional Formatting, Highlight Cells Rules, Duplicate Values to colour them. In Google Sheets, use Data, then Data cleanup, then Remove duplicates, or the formula =UNIQUE(A:A). This page is faster when your list is not already in a spreadsheet, and it shows counts, which the built-in commands do not.
Very large lists
The preview draws up to 3,000 rows so the page stays quick. Remove duplicates and Copy always work on the full list, and the totals count every line.
Frequently asked questions
Which copy of a duplicate is kept?
The first occurrence stays and later ones are removed, so the original order is preserved.
Is whitespace ignored?
No. "apple" and "apple " count as different lines. Trim your list first if that matters.
Is my list uploaded?
No. Everything runs in your browser. Your input is never sent to a server.
What is the difference between duplicates and unique items?
Unique items is the number of distinct values. Duplicates is the number of extra copies that would be removed. A list with 6 lines and 3 distinct values has 3 unique items and 3 duplicates.
Does it keep the original order?
Yes. The first occurrence of each value stays where it was and later copies are removed.