Remove Duplicate Lines

Remove repeated text lines while keeping their first occurrence.

Your input is processed in this browser and is not uploaded.

Remove Duplicate Lines

Clean a pasted list by removing repeated lines. Deduplication preserves the first occurrence instead of sorting the list.

How to use it

  1. Paste one item per line.
  2. Choose Exact lines or Trim line edges first.
  3. Select Remove duplicate lines and inspect the remaining list before copying.

Worked example

Separate lines red, blue, red become red then blue. In exact mode, red and red remain distinct. Trim mode changes both to red and keeps one.

Limits and troubleshooting

Matching is case-sensitive: Red and red remain separate. Trim mode modifies retained lines by removing surrounding whitespace. Repeated empty lines collapse to one empty line, and output uses Unix newlines. This does not parse CSV records or normalize Unicode spelling. Input limit: 100,000 UTF-16 code units.

Frequently asked questions

Can I keep the last occurrence instead?

This version keeps the first occurrence in its original position. If the remaining list also needs ordering, continue with Sort Lines.

Why do visually identical lines remain?

Letter case, invisible characters or Unicode representations may differ. Inspect the source before applying additional normalization.