TypeScript - remove duplicated lines

In this short article, we would like to show how to remove duplicated lines from some text using TypeScript.

Quick solution:

const newLineExpression: RegExp = /\r\n|\n\r|\n|\r/g;

const removeDuplicatedLines = (text: string): string => {
  let result = '';
  const blocker = {}; // prevents lines dupplication
  const lines = text.split(newLineExpression);
  for (const line of lines) {
    if (blocker.hasOwnProperty(line)) {
      continue;
    }
    blocker[line] = true;
    result += line + '\n';
  }
  return result;
};


// Usage example:

const text: string = `a
b
b
a
a
c
c`;

console.log(removeDuplicatedLines(text)); // a
                                          // b
                                          // c

filter() based example

This approach uses a functional programming pattern.

const newLineExpression: RegExp = /\r\n|\n\r|\n|\r/g;

const removeDuplicatedLines = (text: string): string => {
  return text
    .split(newLineExpression)
    .filter((item, index, array) => array.indexOf(item) === index)
    .join('\n');
};


// Usage example:

const text: string = `a
b
b
a
a
c
c`;

console.log(removeDuplicatedLines(text)); // a
                                          // b
                                          // c

Output:

a
b
c

reduce() based example

This approach was created to show that it is possible to get the same effect with reduce() method.

const newLineExpression: RegExp = /\r\n|\n\r|\n|\r/g;

const removeDuplicatedLines = (text: string): string => {
  const blocker = {}; // prevents lines dupplication
  return text.split(newLineExpression).reduce((result, line) => {
    if (blocker.hasOwnProperty(line)) {
      return result;
    }
    blocker[line] = true;
    return result + line + '\n';
  }, '');
};


// Usage example:

const text: string = `a
b
b
a
a
c
c`;

console.log(removeDuplicatedLines(text)); // a
                                          // b
                                          // c

Output:

a
b
c