Languages
[Edit]
EN

JavaScript - remove duplicated lines

3 points
Created by:
Iona
415

Hello there! 👋 😊

In this article, I would like to show you two ways to remove duplicated lines from some text in JavaScript.

We're gonna use two methods: 

  • filter()
  • reduce()

1. filter() based example

This approach uses a functional programming pattern.

On the text I've used set of operations to remove duplicated lines:

  • split() method to split text into lines that take a newline character in 4 different variants which is a universal approach,
  • filter() method that creates new array of elements tested with provided function
  • join() method to merge items back.

Note:
You can read more about splitting text in this article.

Practical example:

// ONLINE-RUNNER:browser;

const newLineExpression = /\r\n|\n\r|\n|\r/g;

const removeDuplicatedLines = (text) => {
  	return text.split(newLineExpression)
  		.filter((item, index, array) => array.indexOf(item) === index)
    	.join('\n');
};

// Usage example:

const text = `a
b
b
a
a
c
c`;

console.log(removeDuplicatedLines(text)); // a
                                          // b
                                          // c

2. reduce() based example

This approach was created to show that it is possible to get the same effect as in the example above with reduce() method.

Practical example:

// ONLINE-RUNNER:browser;

const newLineExpression = /\r\n|\n\r|\n|\r/g;

const removeDuplicatedLines = (text) => {
  	const blocker = {}; // prevents lines dupplication
  	return text.split(newLineExpression)
    	.reduce((result, line) => {
        	if (blocker.hasOwnProperty(line)) {
				return result;
            }
            blocker[line] = true;
            return result + line + '\n';
        }, '');
};

// Usage example:

const text = `a
b
b
a
a
c
c`;

console.log(removeDuplicatedLines(text)); // a
                                          // b
                                          // c

References

Donate to Dirask
Our content is created by volunteers - like Wikipedia. If you think, the things we do are good, donate us. Thanks!
Join to our subscribers to be up to date with content, news and offers.

JavaScript - Blog posts

Native Advertising
🚀
Get your tech brand or product in front of software developers.
For more information Contact us
Dirask - we help you to
solve coding problems.
Ask question.

❤️💻 🙂

Join