Languages
[Edit]
EN

JavaScript - remove duplicated lines

8 points
Created by:
JustMike
27850

In this short article, we would like to show how to remove duplicated lines from some text using JavaScript.

Quick solution:

// ONLINE-RUNNER:browser;

const newLineExpression = /\r\n|\n\r|\n|\r/g;

const removeDuplicatedLines = (text) => {
  	let result = '';
	const blocker = {}; // prevents lines dupplication
  	const lines = text.split(newLineExpression);
  	for (const line of lines) {
    	if (blocker.hasOwnProperty(line)) {
        	continue;
        }
      	blocker[line] = true;
      	result += line + '\n';
    }
  	return result;
};

// Usage example:

const text = `a
b
b
a
a
c
c`;

console.log(removeDuplicatedLines(text)); // a
                                          // b
                                          // c

filter() based example

This approach uses functional programming pattern.

// ONLINE-RUNNER:browser;

const newLineExpression = /\r\n|\n\r|\n|\r/g;

const removeDuplicatedLines = (text) => {
  	return text.split(newLineExpression)
  		.filter((item, index, array) => array.indexOf(item) === index)
    	.join('\n');
};

// Usage example:

const text = `a
b
b
a
a
c
c`;

console.log(removeDuplicatedLines(text)); // a
                                          // b
                                          // c

reduce() based example

This approach was created to show that it is possible to get the same effect with reduce() method.

// ONLINE-RUNNER:browser;

const newLineExpression = /\r\n|\n\r|\n|\r/g;

const removeDuplicatedLines = (text) => {
  	const blocker = {}; // prevents lines dupplication
  	return text.split(newLineExpression)
    	.reduce((result, line) => {
        	if (blocker.hasOwnProperty(line)) {
				return result;
            }
            blocker[line] = true;
            return result + line + '\n';
        }, '');
};

// Usage example:

const text = `a
b
b
a
a
c
c`;

console.log(removeDuplicatedLines(text)); // a
                                          // b
                                          // c

See also

  1. JavaScript - how to split string by newline?
  2. JavaScript - remove duplicates from array

Native Advertising
­čÜÇ
Get your tech brand or product in front of software developers.
For more information Contact us
Dirask - we help you to
solve coding problems.
Ask question.

ÔŁĄ´ŞĆ­čĺ╗ ­čÖé

Join