Duplicate Line Remover

Remove duplicate lines from lists with keep-first or keep-last, optional case-insensitive matching, and whitespace trimming.

Files and content are processed locally in your browser and are not uploaded to our server.

About this tool

Deduplicate line-based lists such as emails, URLs, or log entries. Choose whether to keep the first or last occurrence of each duplicate, ignore letter case, and trim whitespace before comparing.

Features

Keep first or last duplicate occurrence
Optional case-insensitive comparison
Trim whitespace before matching
Show count of removed lines
Import and export text files

How to use

Paste lines: Enter one item per line or import a file.
Set rules: Pick keep-first/last, case, and trim options.
Remove duplicates: Click Remove duplicates and copy the unique list.

Examples

Clean mailing lists exported from multiple sources
Deduplicate server log lines before analysis
Merge keyword lists without repeats
Prepare unique URL lists for crawling

Limitations

Matching is line-based; partial duplicates inside a line are not detected
Order is preserved based on keep-first or keep-last setting
The text entered in the tool is processed directly in your browser and is not sent to a server.

FAQ

What is the difference between keep first and keep last?: Keep first retains the earliest occurrence of each duplicate line; keep last keeps the final one.
Are empty lines removed?: Empty lines are treated as lines. Enable trim if whitespace-only lines should collapse.
Does it sort the output?: No. Original order is kept except for removed duplicate lines.