[javascript] Javascript array sort and unique

I have a JavaScript array like this:

var myData=['237','124','255','124','366','255'];

I need the array elements to be unique and sorted:

myData[0]='124';
myData[1]='237';
myData[2]='255';
myData[3]='366';

Even though the members of array look like integers, they're not integers, since I have already converted each to be string:

var myData[0]=num.toString();
//...and so on.

Is there any way to do all of these tasks in JavaScript?

How about:

array.sort().filter(function(elem, index, arr) {
  return index == arr.length - 1 || arr[index + 1] != elem
})

This is similar to @loostro answer but instead of using indexOf which will reiterate the array for each element to verify that is the first found, it just checks that the next element is different than the current.

// Another way, that does not rearrange the original Array 
// and spends a little less time handling duplicates.

function uniqueSort(arr, sortby){
    var A1= arr.slice();
    A1= typeof sortby== 'function'? A1.sort(sortby): A1.sort();

    var last= A1.shift(), next, A2= [last];
    while(A1.length){
        next= A1.shift();
        while(next=== last) next= A1.shift();
        if(next!=undefined){
            A2[A2.length]= next;
            last= next;
        }
    }
    return A2;
}
var myData= ['237','124','255','124','366','255','100','1000'];
uniqueSort(myData,function(a,b){return a-b})

// the ordinary sort() returns the same array as the number sort here,
// but some strings of digits do not sort so nicely numerical.

Here is a simple one liner with O(N), no complicated loops necessary.

> Object.keys(['a', 'b', 'a'].reduce((l, r) => l[r] = l, {})).sort()
[ 'a', 'b' ]

Explanation

Original data set, assume its coming in from an external function

const data = ['a', 'b', 'a']

We want to group all the values onto an object as keys as the method of deduplication. So we use reduce with an object as the default value:

[].reduce(fn, {})

The next step is to create a reduce function which will put the values in the array onto the object. The end result is an object with a unique set of keys.

const reduced = data.reduce((l, r) => l[r] = l, {})

We set l[r] = l because in javascript the value of the assignment expression is returned when an assignment statement is used as an expression. l is the accumulator object and r is the key value. You can also use Object.assign(l, { [r]: (l[r] || 0) + 1 }) or something similar to get the count of each value if that was important to you.

Next we want to get the keys of that object

const keys = Object.keys(reduced)

Then simply use the built-in sort

console.log(keys.sort())

Which is the set of unique values of the original array, sorted

['a', 'b']

O[N^2] solutions are bad, especially when the data is already sorted, there is no need to do two nested loops for removing duplicates. One loop and comparing to the previous element will work great.

A simple solution with O[] of sort() would suffice. My solution is:

function sortUnique(arr, compareFunction) {
  let sorted = arr.sort(compareFunction);
  let result = sorted.filter(compareFunction
    ? function(val, i, a) { return (i == 0 || compareFunction(a[i-1], val) != 0); }
    : function(val, i, a) { return (i == 0 || a[i-1] !== val); }
  );
  return result;
}

BTW, can do something like this to have Array.sortUnique() method:

Array.prototype.sortUnique = function(compareFunction) {return sortUnique(this, compareFunction); }

Furthermore, sort() could be modified to remove second element if compare() function returns 0 (equal elements), though that code can become messy (need to revise loop boundaries in the flight). Besides, I stay away from making my own sort() functions in interpreted languages, since it will most certainly degrade the performance. So this addition is for the ECMA 2019+ consideration.

I guess I'll post this answer for some variety. This technique for purging duplicates is something I picked up on for a project in Flash I'm currently working on about a month or so ago.

What you do is make an object and fill it with both a key and a value utilizing each array item. Since duplicate keys are discarded, duplicates are removed.

var nums = [1, 1, 2, 3, 3, 4, 5, 5, 6, 7, 7, 8, 9, 9, 10];
var newNums = purgeArray(nums);

function purgeArray(ar)
{
    var obj = {};
    var temp = [];
    for(var i=0;i<ar.length;i++)
    {
        obj[ar[i]] = ar[i];
    }
    for (var item in obj)
    {
        temp.push(obj[item]);
    }
    return temp;
}

There's already 5 other answers, so I don't see a need to post a sorting function.

This function doesn't fail for more than two duplicates values:

function unique(arr) {
    var a = [];
    var l = arr.length;
    for(var i=0; i<l; i++) {
        for(var j=i+1; j<l; j++) {
            // If a[i] is found later in the array
            if (arr[i] === arr[j])
              j = ++i;
        }
        a.push(arr[i]);
    }
    return a;
};

function sort() only is only good if your number has same digit, example:

var myData = ["3","11","1","2"]

will return;

var myData = ["1","11","2","3"]

and here improvement for function from mrmonkington

myData.sort().sort(function(a,b){return a - b;}).filter(function(el,i,a){if(i==a.indexOf(el) & el.length>0)return 1;return 0;})

the above function will also delete empty array and you can checkout the demo below

http://jsbin.com/ahojip/2/edit

I'm afraid you can't combine these functions, ie. you gotta do something like this:-

myData.unique().sort();

Alternatively you can implement a kind of sortedset (as available in other languages) - which carries both the notion of sorting and removing duplicates, as you require.

Hope this helps.

References:-

Array.sort

Array.unique

This might be adequate in circumstances where you can't define the function in advance (like in a bookmarklet):

myData.sort().filter(function(el,i,a){return i===a.indexOf(el)})

No redundant "return" array, no ECMA5 built-ins (I'm pretty sure!) and simple to read.

function removeDuplicates(target_array) {
    target_array.sort();
    var i = 0;

    while(i < target_array.length) {
        if(target_array[i] === target_array[i+1]) {
            target_array.splice(i+1,1);
        }
        else {
            i += 1;
        }
    }
    return target_array;
}

The fastest and simpleness way to do this task.

const N = Math.pow(8, 8)
let data = Array.from({length:  N}, () => Math.floor(Math.random() * N))
let newData = {}
let len = data.length

// the magic
while (len--) {
    newData[data[len]] = true
}

You can now achieve the result in just one line of code.

Using new Set to reduce the array to unique set of values. Apply the sort method after to order the string values.

var myData=['237','124','255','124','366','255']

var uniqueAndSorted = [...new Set(myData)].sort() 

UPDATED for newer methods introduced in JavaScript since time of question.

A way to use a custom sort function

//func has to return 0 in the case in which they are equal
sort_unique = function(arr,func) {
        func = func || function (a, b) {
            return a*1 - b*1;
        };
        arr = arr.sort(func);
        var ret = [arr[0]];
        for (var i = 1; i < arr.length; i++) {
            if (func(arr[i-1],arr[i]) != 0) 
                ret.push(arr[i]);
            }
        }
        return ret;
    }

Example: desc order for an array of objects

MyArray = sort_unique(MyArray , function(a,b){
            return  b.iterator_internal*1 - a.iterator_internal*1;
        });

Here's my (more modern) approach using Array.protoype.reduce():

[2, 1, 2, 3].reduce((a, x) => a.includes(x) ? a : [...a, x], []).sort()
// returns [1, 2, 3]

Edit: More performant version as pointed out in the comments:

arr.sort().filter((x, i, a) => !i || x != a[i-1])

Try using an external library like underscore

var f = _.compose(_.uniq, function(array) {
    return _.sortBy(array, _.identity);
});

var sortedUnique = f(array);

This relies on _.compose, _.uniq, _.sortBy, _.identity

See live example

What is it doing?

We want a function that takes an array and then returns a sorted array with the non-unique entries removed. This function needs to do two things, sorting and making the array unique.

This is a good job for composition, so we compose the unique & sort function together. _.uniq can just be applied on the array with one argument so it's just passed to _.compose

the _.sortBy function needs a sorting conditional functional. it expects a function that returns a value and the array will be sorted on that value. Since the value that we are ordering it by is the value in the array we can just pass the _.identity function.

We now have a composition of a function that (takes an array and returns a unique array) and a function that (takes an array and returns a sorted array, sorted by their values).

We simply apply the composition on the array and we have our uniquely sorted array.

The solution in a more elegant way.

var myData=['237','124','255','124','366','255'];

console.log(Array.from(new Set(myData)).sort());

I know the question is very old, but maybe someone will come in handy

function sort_unique(arr) {
    return arr.sort().filter(function(el,i,a) {
        return (i==a.indexOf(el));
    });
}