EN
JavaScript - remove duplicated lines
3 points
Hello there! 👋 😊
In this article, I would like to show you two ways to remove duplicated lines from some text in JavaScript.
We're gonna use two methods:
filter()reduce()
This approach uses a functional programming pattern.
On the text I've used set of operations to remove duplicated lines:
split()method to split text into lines that take a newline character in 4 different variants which is a universal approach,filter()method that creates new array of elements tested with provided functionjoin()method to merge items back.
Note:
You can read more about splitting text in this article.
Practical example:
xxxxxxxxxx1
const newLineExpression = /\r\n|\n\r|\n|\r/g;2
3
const removeDuplicatedLines = (text) => {4
return text.split(newLineExpression)5
.filter((item, index, array) => array.indexOf(item) === index)6
.join('\n');7
};8
9
// Usage example:10
11
const text = `a12
b13
b14
a15
a16
c17
c`;18
19
console.log(removeDuplicatedLines(text)); // a20
// b21
// cThis approach was created to show that it is possible to get the same effect as in the example above with reduce() method.
Practical example:
xxxxxxxxxx1
const newLineExpression = /\r\n|\n\r|\n|\r/g;2
3
const removeDuplicatedLines = (text) => {4
const blocker = {}; // prevents lines dupplication5
return text.split(newLineExpression)6
.reduce((result, line) => {7
if (blocker.hasOwnProperty(line)) {8
return result;9
}10
blocker[line] = true;11
return result + line + '\n';12
}, '');13
};14
15
// Usage example:16
17
const text = `a18
b19
b20
a21
a22
c23
c`;24
25
console.log(removeDuplicatedLines(text)); // a26
// b27
// c