Remove duplicates from an array of objects in JavaScript

Question

I have an object that contains an array of objects   things   new Object     things thing   new Array     things thing push  place  here  name  stuff     things thing push  place  there  name  morestuff     things thing push  place  there  name  morestuff       I m wondering what is the best method to remove duplicate objects from an array   So for example  things thing would become      place  here  name  stuff     place  there  name  morestuff

User · Answer

x000D   x000D  var things   new Object    x000D   x000D  things thing   new Array    x000D   x000D  things thing push  place  here  name  stuff     x000D  things thing push  place  there  name  morestuff     x000D  things thing push  place  there  name  morestuff     x000D  console log things   x000D  function removeDuplicate result  id    x000D      let duplicate       x000D      return result filter ele   gt   duplicate ele id    amp  amp                     duplicate ele id     true    x000D    x000D  let resolverarray   removeDuplicate things thing  place   x000D  console log resolverarray   x000D   x000D   x000D

User · Answer

Another way would be to use reduce function and have a new array to be the accumulator  If there is already a thing with the same name in the accumulator array then don t add it there   let list   things thing  list   list reduce  accumulator  thing    gt        if   accumulator filter  duplicate    gt  thing name     duplicate name  0             accumulator push thing             return accumulator          thing things   list    I m adding this answer  because I couldn t find nice  readable es6 solution  I use babel to handle arrow functions  that s compatible with Internet Explorer 11  The problem is IE11 doesn t have Map values   or Set values   without polyfill  For the same reason I used filter   0  to get first element instead of find

User · Answer

I had this exact same requirement  to remove duplicate objects in a array  based on duplicates on a single field   I found the code here  Javascript  Remove Duplicates from Array of Objects  So in my example  I m removing any object from the array that has a duplicate licenseNum string value   var arrayWithDuplicates           type   LICENSE    licenseNum    12345   state  NV          type   LICENSE    licenseNum    A7846   state  CA          type   LICENSE    licenseNum    12345   state  OR          type   LICENSE    licenseNum    10849   state  CA          type   LICENSE    licenseNum    B7037   state  WA          type   LICENSE    licenseNum    12345   state  NM       function removeDuplicates originalArray  prop         var newArray            var lookupObject              for var i in originalArray            lookupObject originalArray i  prop     originalArray i                for i in lookupObject             newArray push lookupObject i                 return newArray      var uniqueArray   removeDuplicates arrayWithDuplicates   licenseNum    console log  uniqueArray is      JSON stringify uniqueArray      The results   uniqueArray is       type   LICENSE   licenseNum   10849   state   CA      type   LICENSE   licenseNum   12345   state   NM      type   LICENSE   licenseNum   A7846   state   CA      type   LICENSE   licenseNum   B7037   state   WA

User · Answer

If you don t want to specify a list of properties   function removeDuplicates myArr      var props   Object keys myArr 0     return myArr filter  item  index  self    gt      index     self findIndex  t    gt          props every prop   gt            return t prop      item prop                          OBS  Not compatible with IE11

User · Answer

A primitive method would be   var obj        for   var i 0  len things thing length  i  lt  len  i         obj things thing i   place      things thing i    things thing   new Array    for   var key in obj       things thing push obj key

User · Answer

You can convert the array objects into strings so they can be compared  add the strings to a Set so the comparable duplicates will be automatically removed and then convert each of the strings back into objects   It might not be as performant as other answers  but it s readable   const things        things thing       things thing push  place  here  name  stuff     things thing push  place  there  name  morestuff     things thing push  place  there  name  morestuff      const uniqueArray    arr    gt       const stringifiedArray   arr map  item    gt  JSON stringify item      const set   new Set stringifiedArray      return Array from set  map  item    gt  JSON parse item       const uniqueThings   uniqueArray things thing    console log uniqueThings

User · Answer

The simplest way is use filter    x000D   x000D  var uniq      x000D  var arr       id   1     id   1     id   2    x000D  var arrFiltered   arr filter obj   gt   uniq obj id   amp  amp   uniq obj id    true    x000D  console log  arrFiltered   arrFiltered  x000D   x000D   x000D

User · Answer

Generic for any array of objects         Remove duplicated values without losing information    const removeValues    items  key    gt      let tmp          items forEach item   gt        tmp item key       tmp item key      item   Object assign tmp item key    item           items         Object keys tmp  forEach key   gt  items push tmp key        return items      Hope it could help to anyone

User · Answer

If you want to de-duplicate your array based on all arguments and not just one  You can use the uniqBy function of lodash that can take a function as a second argument   You will have this one-liner      uniqBy array  e   gt    return e place  amp  amp  e name

User · Answer

This way works well for me  function arrayUnique arr  uniqueKey      const flagList   new Set     return arr filter function item        if   flagList has item uniqueKey            flagList add item uniqueKey         return true              const data             name   Kyle       occupation   Fashion Designer               name   Kyle       occupation   Fashion Designer               name   Emily       occupation   Web Designer               name   Melissa       occupation   Fashion Designer               name   Tom       occupation   Web Developer               name   Tom       occupation   Web Developer        console table arrayUnique data   name      work well  printout  ------------------------------------------      index       name           occupation         --------- ----------- --------------------         0         Kyle        Fashion Designer           1         Emily         Web Designer             2        Melissa      Fashion Designer           3          Tom         Web Developer        ------------------------------------------   ES5  function arrayUnique arr  uniqueKey      const flagList        return arr filter function item        if  flagList indexOf item uniqueKey       -1          flagList push item uniqueKey         return true               These two ways are simpler and more understandable

User · Answer

One liner using Set   x000D   x000D  var things   new Object    x000D   x000D  things thing   new Array    x000D   x000D  things thing push  place  here  name  stuff     x000D  things thing push  place  there  name  morestuff     x000D  things thing push  place  there  name  morestuff     x000D   x000D     assign things thing to myData for brevity x000D  var myData   things thing  x000D   x000D  things thing   Array from new Set myData map JSON stringify    map JSON parse   x000D   x000D  console log things thing  x000D   x000D   x000D    Explanation    new Set myData map JSON stringify   creates a Set object using the stringified myData elements   Set object will ensure that every element is unique   Then I create an array based on the elements of the created set using Array from   Finally  I use JSON parse to convert stringified element back to an object

User · Answer

function filterDuplicateQueries queries       let uniqueQueries            queries forEach  l  i   gt           let alreadyExist   false          if uniqueQueries length gt 0               uniqueQueries forEach  k  j   gt                   if k query    l query                       alreadyExist   true                                                      if  alreadyExist              uniqueQueries push l

User · Answer

removeDuplicates   takes in an array of objects and returns a new array without any duplicate objects  based on the id property   const allTests        name   Test1   id   1        name   Test3   id   3       name   Test2   id   2       name   Test2   id   2       name   Test3   id   3       function removeDuplicates array      let uniq         return array filter obj   gt   uniq obj id   amp  amp   uniq obj id    true      removeDuplicates allTests    Expected outcome       name   Test1   id   1        name   Test3   id   3       name   Test2   id   2       First  we set the value of variable uniq to an empty object  Next  we filter through the array of objects  Filter creates a new array with all elements that pass the test implemented by the provided function  return array filter obj   gt   uniq obj id   amp  amp   uniq obj id    true     Above  we use the short-circuiting functionality of  amp  amp   If the left side of the  amp  amp  evaluates to true  then it returns the value on the right of the  amp  amp   If the left side is false  it returns what is on the left side of the  amp  amp   For each object obj  we check uniq for a property named the value of obj id  In this case  on the first iteration it would check for the property  1    We want the opposite of what it returns  either true or false  which is why we use the   in  uniq obj id   If uniq has the id property already  it returns true which evaluates to false     telling the filter function NOT to add that obj  However  if it does not find the obj id property  it returns false which then evaluates to true     and returns everything to the right of the  amp  amp   or  uniq obj id    true   This is a truthy value  telling the filter method to add that obj to the returned array  and it also adds the property  1  true  to uniq  This ensures that any other obj instance with that same id will not be added again

User · Answer

Using ES6  in a single line you can get a unique list of objects by key   const unique       new Map arr map item   gt   item key   item    values      It can be put into a function   function getUniqueListBy arr  key        return     new Map arr map item   gt   item key   item    values        Here is a working example    x000D   x000D  const arr     x000D       place   here    name   x   other   other stuff1     x000D       place   there   name   x   other   other stuff2     x000D       place   here    name   y   other   other stuff4     x000D       place   here    name   z   other   other stuff5    x000D    x000D   x000D  function getUniqueListBy arr  key    x000D      return     new Map arr map item   gt   item key   item    values    x000D    x000D   x000D  const arr1   getUniqueListBy arr   place   x000D   x000D  console log  Unique by place   x000D  console log JSON stringify arr1   x000D   x000D  console log   nUnique by name   x000D  const arr2   getUniqueListBy arr   name   x000D   x000D  console log JSON stringify arr2   x000D   x000D   x000D    How does it work  First the array is remapped in a way that it can be used as an input for a  Map      arr map item     item key   item      which means each item of the array will be transformed in another array with 2 elements  the selected key as first element and the entire initial item as second element  this is called an entry  ex  array entries  map entries   And here is the official doc with an example showing how to add array entries in Map constructor   Example when key is place      here    place   here    name   x   other   other stuff1             Secondly  we pass this modified array to the Map constructor and here is the magic happening  Map will eliminate the duplicate keys values  keeping only last inserted value of the same key  Note  Map keeps the order of insertion   check difference between Map and object      new Map entry array just mapped above    Third we use the map values to retrieve the original items  but this time without duplicates      new Map mappedArr  values     And last one is to add those values into a fresh new array so that it can look as the initial structure and return that      return     new Map mappedArr  values

User · Answer

Shortest one liners for ES6   Find unique id s in an array   arr filter  v i a   gt a findIndex t  gt  t id     v id     i      Unique by multiple properties   place and name    arr filter  v i a   gt a findIndex t  gt  t place     v place  amp  amp  t name   v name     i    Unique by all properties  This will be slow for large arrays   arr filter  v i a   gt a findIndex t  gt  JSON stringify t      JSON stringify v      i    Keep the last occurrence   arr slice   reverse   filter  v i a   gt a findIndex t  gt  t id     v id     i  reverse

User · Answer

let data              name    Amir        surname    Rahnama                 name    Amir        surname    Stevens         let non duplicated data     uniqBy data   name

User · Answer

const objectsMap   new Map    const placesName         place   quot here quot   name   quot stuff quot         place   quot there quot   name   quot morestuff quot         place   quot there quot   name   quot morestuff quot        placesName forEach  object    gt      objectsMap set object place  object       console log objectsMap

User · Answer

npm i lodash   let non duplicated data     uniqBy pendingDeposits  v   gt   v stellarAccount  v externalTransactionId  join

User · Answer

x000D   x000D   const things        place  here  name  stuff       place  there  name  morestuff       place  there  name  morestuff      const filteredArr   things reduce  thing  current    gt      const x   thing find item   gt  item place     current place     if   x        return thing concat  current        else       return thing              console log filteredArr  x000D   x000D   x000D   Solution Via Set Object   According to the data type  x000D   x000D  const seen   new Set     const things        place  here  name  stuff       place  there  name  morestuff       place  there  name  morestuff       const filteredArr   things filter el   gt      const duplicate   seen has el place     seen add el place     return  duplicate      console log filteredArr  x000D   x000D   x000D   Set Object Feature Each value in the Set Object has to be unique  the value equality will be checked The Purpose of Set object  storing unique values according to the  Data type   whether primitive values or object references it has very useful four Instance methods add  clear   has  amp  delete  Unique  amp  data Type feature    addmethod it s  push unique data into collection by default also preserve data type    that means it prevent to  push duplicate item into collection also it will check data type by default    has method sometime needs to check data item exist into the collection and   it s handy  method for the collection to cheek unique id or item and data type   delete  method it will remove specific item from the collection by identifying data type   clear  method it will remove all collection items from one specific variable and set as empty object Set object has also Iteration methods  amp  more feature   Better Read from Here   Set - JavaScript   MDN

User · Answer

const uniqueElements    arr  fn    gt  arr reduce  acc  v    gt        if   acc some x   gt  fn v  x      acc push v         return acc           const stuff          place  here  name  stuff         place  there  name  morestuff         place  there  name  morestuff        const unique   uniqueElements stuff   a b    gt  a place     b place  amp  amp  a name     b name        console log  unique             place    here        name    stuff                place    there        name    morestuff

User · Answer

x000D   x000D  let myData     place  here  name  stuff     x000D    place  there  name  morestuff    x000D    place  there  name  morestuff     x000D   x000D   x000D  let q       new Map myData map obj   gt   JSON stringify obj   obj    values     x000D   x000D  console log q  x000D   x000D   x000D    One-liner using ES6 and new Map        assign things thing to myData let myData   things thing       new Map myData map obj   gt   JSON stringify obj   obj    values       Details -   Doing  map   on the data list and converting each individual object into a  key  value  pair array length  2   the first element key  would be the stringified version of the object and second value  would be an object itself  Adding above created array list to new Map   would have the key as stringified object and any same key addition would result in overriding the already existing key   Using  values   would give MapIterator with all values in a Map  obj in our case  Finally  spread     operator to give new Array with values from the above step

User · Answer

es6 magic in one line    readable at that     returns the union of two arrays where duplicate objects with the same  prop  are removed const removeDuplicatesWith    a  b  prop    gt      a filter x   gt   b find y   gt  x prop      y prop

User · Answer

This is a generic way of doing this  you pass in a function that tests whether two elements of an array are considered equal  In this case  it compares the values of the name and place properties of the two objects being compared   ES5 answer   x000D   x000D  function removeDuplicates arr  equals    x000D      var originalArr   arr slice 0   x000D      var i  len  val  x000D      arr length   0  x000D   x000D      for  i   0  len   originalArr length  i  lt  len    i    x000D          val   originalArr i   x000D          if   arr some function item    return equals item  val         x000D              arr push val   x000D            x000D        x000D    x000D   x000D  function thingsEqual thing1  thing2    x000D      return thing1 place     thing2 place x000D           amp  amp  thing1 name     thing2 name  x000D    x000D   x000D  var things     x000D     place  here  name  stuff    x000D     place  there  name  morestuff    x000D     place  there  name  morestuff   x000D     x000D   x000D  removeDuplicates things  thingsEqual   x000D  console log things   x000D   x000D   x000D    Original ES3 answer  function arrayContains arr  val  equals        var i   arr length      while  i--            if   equals arr i   val                  return true                      return false     function removeDuplicates arr  equals        var originalArr   arr slice 0       var i  len  j  val      arr length   0       for  i   0  len   originalArr length  i  lt  len    i            val   originalArr i           if   arrayContains arr  val  equals                 arr push val                      function thingsEqual thing1  thing2        return thing1 place     thing2 place          amp  amp  thing1 name     thing2 name     removeDuplicates things thing  thingsEqual

User · Answer

Here is a solution using new filter function  of JavaScript that is quite easy   Let s say you have an array like this   var duplicatesArray     AKASH   AKASH   NAVIN   HARISH   NAVIN   HARISH   AKASH   MANJULIKA   AKASH   TAPASWENI   MANJULIKA   HARISH   TAPASWENI   AKASH   MANISH   HARISH   TAPASWENI   MANJULIKA   MANISH      The filter function will allow you to create a new array  using a callback function once for each element in the array  So you could set up the unique array like this   var uniqueArray   duplicatesArray filter function elem  pos   return duplicatesArray indexOf elem     pos          In this scenario your unique array will run through all of the values in the duplicate array  The elem variable represents the value of the element in the array  mike james james alex   the position is it s 0-indexed position in the array  0 1 2 3      and the duplicatesArray indexOf elem  value is just the index of the first occurrence of that element in the original array  So  because the element  james  is duplicated  when we loop through all of the elements in the duplicatesArray and push them to the uniqueArray  the first time we hit james  our  pos  value is 1  and our indexOf elem  is 1 as well  so James gets pushed to the uniqueArray  The second time we hit James  our  pos  value is 2  and our indexOf elem  is still 1  because it only finds the first instance of an array element   so the duplicate is not pushed  Therefore  our uniqueArray contains only unique values    Here is the Demo of above function Click Here for the above function example

User · Answer

here is my solution  it searches for duplicates based on object prop and when it finds a duplicate object it replaces its value in array1 with array2 value  function mergeSecondArrayIntoFirstArrayByProperty array1  array2        for  var i   0  i  lt  array2 length  i              var found   false          for  var j   0  j  lt  array1 length  j                  if  array2 i  prop     array1 j  prop       if item exist in array1                 array1 j    array2 i      replace it in array1 with array2 value                 found   true                                  if   found     if item in array2 not found in array1  add it to array1             array1 push array2 i               return array1

User · Answer

If you can use Javascript libraries such as underscore or lodash  I recommend having a look at   uniq function in their libraries  From lodash     uniq array   isSorted false    callback   identity    thisArg     Basically  you pass in the array that in here is an object literal and you pass in the attribute that you want to remove duplicates with in the original data array  like this   var data      name    Amir    surname    Rahnama      name    Amir    surname    Stevens     var non duplidated data     uniq data   name       UPDATE  Lodash now has introduced a  uniqBy as well

User · Answer

This answer will probably not be found by anyone but here is a short ES6 way with a better runtime than the 60  answers that already exist  let ids   array map o   gt  o id  let filtered   array filter   id   index    gt   ids includes id  index 1     Example   x000D   x000D  let arr     id  1  name   one     id  2  name   two     id  1  name   one     let ids   arr map o   gt  o id  let filtered   arr filter   id   index    gt   ids includes id  index   1    console log filtered  x000D   x000D   x000D   How it works  Array filter   removes all duplicate objects by checking if the previously mapped id-array includes the current id   id  destructs the object into only its id   To only filter out actual duplicates  it is using Array includes   s second parameter fromIndex with index   1 which will ignore the current object and all previous  Since every iteration of the filter callback method will only search the array beginning at the current index   1  this also dramatically reduces the runtime because only objects not previously filtered get checked  This obviously also works for any other key that is not called id or even multiple or all keys

User · Answer

The problem can be simplified to removing duplicates from the thing array  You can implement a faster O n  solution  assuming native key lookup is negligible  by using an object to both maintain unique criteria as keys and storing associated values  Basically  the idea is to store all objects by their unique key  so that duplicates overwrite themselves   x000D   x000D  const thing      place   here   name  stuff       place   there   name  morestuff       place   there   name  morestuff       const uniques      for  const t of thing      const key   t place         t name     Or whatever string criteria you want  which can be generified as Object keys t  join        uniques key    t                       Last duplicate wins   const uniqueThing   Object values uniques  console log uniqueThing  x000D   x000D   x000D

User · Answer

Make Something simple  Fancy is good but unreadable code is useless  Enjoy  -    x000D   x000D  var a     x000D     x000D    executiveId  6873702  x000D    largePhotoCircle  null  x000D    name   John A  Cuomo   x000D    photoURL  null  x000D    primaryCompany   VSE CORP   x000D    primaryTitle   Chief Executive Officer  President and Director  x000D      x000D     x000D    executiveId  6873702  x000D    largePhotoCircle  null  x000D    name   John A  Cuomo   x000D    photoURL  null  x000D    primaryCompany   VSE CORP   x000D    primaryTitle   Chief Executive Officer  President and Director  x000D      x000D     x000D    executiveId  6873703  x000D    largePhotoCircle  null  x000D    name   John A  Cuomo   x000D    photoURL  null  x000D    primaryCompany   VSE CORP   x000D    primaryTitle   Chief Executive Officer  President and Director   x000D     x000D     x000D   x000D  function filterDuplicate myArr  prop    x000D           Format -  1  x000D   x000D           return myArr filter  obj  pos  arr    gt    x000D               return arr map mapObj   gt  mapObj prop   indexOf obj prop       pos  x000D               x000D   x000D           Format -  2  x000D        var res       x000D        var resArr       x000D        for  var elem of myArr    x000D          res elem executiveId    elem  x000D          x000D        for  let  index  elem  of Object entries res     x000D          resArr push elem   x000D          x000D        return resArr  x000D      x000D     x000D  let finalRes   filterDuplicate a  executiveId    x000D  console log  finalResults     finalRes   x000D   x000D   x000D

User · Answer

If you strictly want to remove duplicates based on one property  you can reduce the array into and object based on the place property  since the object can only have unique keys  you can then just get the values to get back to  an array   const unique   Object values things thing reduce  o  t    gt        o   t place   t

User · Answer

You could also use a Map   const dedupThings   Array from things thing reduce  m  t    gt  m set t place  t   new Map    values       Full sample   const things   new Object     things thing   new Array     things thing push  place  here  name  stuff     things thing push  place  there  name  morestuff     things thing push  place  there  name  morestuff      const dedupThings   Array from things thing reduce  m  t    gt  m set t place  t   new Map    values      console log JSON stringify dedupThings  null  4      Result                    place    here            name    stuff                        place    there            name    morestuff

User · Answer

Another option would be to create a custom indexOf function  which compares the values of your chosen property for each object and wrap this in a reduce function      var uniq   redundant array reduce function a b         function indexOfProperty  a  b             for  var i 0 i lt a length i                   if a i  property    b property                      return i                                        return -1                 if  indexOfProperty a b   lt  0   a push b           return a

User · Answer

A TypeScript solution  This will remove duplicate objects and also preserve the types of the objects   function removeDuplicateObjects array  any        return     new Set array map s   gt  JSON stringify s          map s   gt  JSON parse s

User · Answer

Here s another option to do it using Array iterating methods if you need comparison only by one field of an object       function uniq a  param           return a filter function item  pos  array               return array map function mapItem   return mapItem param      indexOf item param       pos                        uniq things thing   place

User · Answer

Continuing exploring ES6 ways of removing duplicates from array of objects  setting thisArg argument of Array prototype filter to new Set provides a decent alternative    x000D   x000D  const things     x000D     place  here  name  stuff    x000D     place  there  name  morestuff    x000D     place  there  name  morestuff   x000D     x000D   x000D  const filtered   things filter function  place  name     x000D   x000D    const key     place   name    x000D   x000D    return  this has key   amp  amp  this add key   x000D   x000D     new Set   x000D   x000D  console log filtered   x000D   x000D   x000D    However  it will not work with arrow functions      gt   as this is bound to their lexical scope

User · Answer

You can also create a generic function which will filter the array based on the object key you pass to the function  function getUnique arr  comp       return arr     map e   gt  e comp       map  e  i  final    gt  final indexOf e      i  amp  amp  i      store the keys of the unique objects     filter e   gt  arr e   map e   gt  arr e       eliminate the dead keys  amp  store unique objects       and you can call the function like this   getUnique things thing  name      to filter on basis of name  getUnique things thing  place      to filter on basis of place

User · Answer

How about with some es6 magic   things thing   things thing filter  thing  index  self    gt    index     self findIndex  t    gt        t place     thing place  amp  amp  t name     thing name          Reference URL  A more generic solution would be   const uniqueArray   things thing filter  thing  index    gt      const  thing   JSON stringify thing     return index     things thing findIndex obj   gt        return JSON stringify obj       thing              Stackblitz Example

User · Answer

This solution is generic for any kind of object and checks for every  key  value  of the Object in the array   Using an temporary object as a hash table to see if the entire Object was ever present as a key  If the string representation of the Object is found then that item is removed from the array     x000D   x000D  var arrOfDup      id  123   name   name    desc   some desc    x000D                    id  125   name   another name    desc   another desc    x000D                    id  123   name   name    desc   some desc    x000D                    id  125   name   another name    desc   another desc    x000D                    id  125   name   another name    desc   another desc     x000D   x000D  function removeDupes dupeArray   x000D    let temp       x000D    let tempArray   JSON parse JSON stringify dupeArray    x000D    dupeArray forEach  item  pos    gt    x000D      if temp JSON stringify item     x000D        tempArray pop    x000D       else  x000D        temp JSON stringify item     item  x000D        x000D        x000D   return tempArray  x000D    x000D   x000D  arrOfDup   removeDupes arrOfDup   x000D   x000D  arrOfDup forEach  item  pos    gt    x000D    console log  item in array at position   pos  is   JSON stringify item      x000D      x000D   x000D   x000D

User · Answer

str      item id  1     item id  2     item id  2     obj     for  x in str       if check str x  item id            obj push str x              function check id       flag 0     for  y in obj           if obj y  item id     id               flag  1                     if flag   0  return true     else return false    console log obj    str is an array of objects  There exists objects having same value  here a small example  there are two objects having same item id as 2   check id  is a function that checks if any object having same item id exists or not  if it exists return false otherwise return true  According to that result  push the object into a new array obj The output of the above code is    item id  1    item id  2

User · Answer

My two cents here  If you know the properties are in the same order  you can stringify the elements and remove dupes from the array and parse the array again  Something like this   x000D   x000D  var things   new Object     things thing   new Array     things thing push  place  here  name  stuff     things thing push  place  there  name  morestuff     things thing push  place  there  name  morestuff        let stringified   things thing map i  gt JSON stringify i    let unique    stringified filter  k  idx   gt  stringified indexOf k      idx                            map j  gt  JSON parse j   console log unique   x000D   x000D   x000D

User · Answer

I think the best approach is using reduce and Map object  This is a single line solution   x000D   x000D  const data        id  1  name   David       id  2  name   Mark       id  2  name   Lora       id  4  name   Tyler       id  4  name   Donald       id  5  name   Adrian       id  6  name   Michael      const uniqueData       data reduce  map  obj    gt  map set obj id  obj   new Map    values      console log uniqueData        in  map set obj id  obj         obj id  is key   don t worry  we ll get only values using the  values   method     obj  is whole object     x000D   x000D   x000D

User · Answer

Dang  kids  let s crush this thing down  why don t we    x000D   x000D  let uniqIds       source     id  a    id  b    id  c    id  b    id  a    id  d     x000D  let filtered   source filter obj   gt   uniqIds obj id   amp  amp   uniqIds obj id    true    x000D  console log filtered   x000D     EXPECTED    id  a    id  b    id  c    id  d     x000D   x000D   x000D

User · Answer

To add one more to the list  Using ES6 and Array reduce with Array find  In this example filtering objects based on a guid property     let filtered   array reduce  accumulator  current    gt      if    accumulator find   guid     gt  guid     current guid         accumulator push current         return accumulator            Extending this one to allow selection of a property and compress it into a one liner   const uniqify    array  key    gt  array reduce  prev  curr    gt  prev find a   gt  a key      curr key     prev   prev push curr   amp  amp  prev         To use it pass an array of objects and the name of the key you wish to de-dupe on as a string value   const result   uniqify myArrayOfObjects   guid

User · Answer

Have you heard of Lodash library   I recommend you this utility  when you don t really want to apply your logic to the code  and use already present code which is optimised and reliable    Consider making an array like this  things thing push  place  utopia  name  unicorn     things thing push  place  jade palace  name  po     things thing push  place  jade palace  name  tigress     things thing push  place  utopia  name  flying reindeer     things thing push  place  panda village  name  po       Note that if you want to keep one attribute unique  you may very well do that by using lodash library  Here  you may use   uniqBy      uniqBy array   iteratee  identity     This method is like   uniq  which returns a duplicate-free version of an array  in which only the first occurrence of each element is kept  except that it accepts iteratee which is invoked for each element in array to generate the criterion by which uniqueness is computed   So  for example  if you want to return an array having unique attribute of  place        uniqBy things thing   place     Similarly  if you want unique attribute as  name        uniqBy things thing   name     Hope this helps   Cheers

User · Answer

What about this   function dedupe arr  compFn       let res           if   compFn  compFn    a  b    gt    return a     b        arr map a   gt   if  res find b   gt  compFn a  b    res push a         return res

User · Answer

If you can wait to eliminate the duplicates until after all the additions  the typical approach is to first sort the array and then eliminate duplicates  The sorting avoids the N   N approach of scanning the array for each element as you walk through them   The  eliminate duplicates  function is usually called unique or uniq  Some existing implementations may combine the two steps  e g   prototype s uniq  This post has few ideas to try  and some to avoid  -    if your library doesn t already have one  Personally I find this one the most straight forward       function unique a           a sort            for var i   1  i  lt  a length                 if a i-1     a i                    a splice i  1                 else                   i                                    return a                  Provide your own comparison     function unique a  compareFunc           a sort  compareFunc            for var i   1  i  lt  a length                 if  compareFunc a i-1   a i       0                   a splice i  1                 else                   i                                    return a

User · Answer

Simple solution with ES6  reduce  and  find  array helper methods  Works efficiently and perfectly fine    use strict    var things   new Object    things thing   new Array    things thing push       place   here       name   stuff      things thing push       place   there       name   morestuff      things thing push       place   there       name   morestuff          the logic is here  function removeDup something        return something thing reduce function  prev  ele            var found   prev find function  fele                return ele place     fele place  amp  amp  ele name     fele name                      if   found                prev push ele                     return prev                console log removeDup things

User · Answer

var testArray    a   b   c   d   e   b   c   d      function removeDuplicatesFromArray arr     var obj      var uniqueArr      for var i 0 i lt arr length i          if  obj hasOwnProperty arr i             obj arr i     arr i           uniqueArr push arr i              return uniqueArr     var newArr   removeDuplicatesFromArray testArray   console log newArr    Output -    a    b    c    d    e

User · Answer

function genFilterData arr  key  key1          let data             data       new Map arr map  x    gt   x key     x key1   x    values                const makeData             for  let i   0  i  lt  data length  i    1            makeData push    key   data i  key    key1   data i  key1                         return makeData            const arr          make   quot here1 quot   makeText  hj  k 9 l 99        make   quot here quot   makeText  hj  k 9 l 9        make   quot here quot   makeText  hj  k 9 l 9          const finalData  genFilterData data   Make    MakeText                 console log finalData

User · Answer

I know there is a ton of answers in this question already  but bear with me    Some of the objects in your array may have additional properties that you are not interested in  or you simply want to find the unique objects considering only a subset of the properties  Consider the array below  Say you want to find the unique objects in this array considering only propOne and propTwo  and ignore any other properties that may be there  The expected result should include only the first and last objects  So here goes the code   x000D   x000D  const array          propOne   a       propTwo   b       propThree   I have no part in this              propOne   a       propTwo   b       someOtherProperty   no one cares about this              propOne   x       propTwo   y       yetAnotherJunk   I am valueless really       noOneHasThis   I have something no one has       const uniques       new Set      array map x   gt  JSON stringify     propOne  propTwo      gt     propOne  propTwo     x       map JSON parse    console log uniques   x000D   x000D   x000D

User · Answer

If you need an unique array based on multiple properties in the object you can do this with map and combining the properties of the object       var hash   array map function element           var string              for  var key in element               string    element key                    return string            array   array filter function element  index           var string              for  var key in element               string    element key                    return hash indexOf string     index

User · Answer

function dupData       var arr      comment    quot a quot    quot a quot    quot bbb quot    quot xyz quot    quot bbb quot         let newData         comment forEach function  val  index        if  comment indexOf val  index   1   gt  -1          if  newData indexOf val      -1    newData push val

User · Answer

I believe a combination of reduce with JSON stringify to perfectly compare Objects and selectively adding those who are not already in the accumulator is an elegant way   Keep in mind that JSON stringify might become a performance issue  in extreme cases where the array has many Objects and they are complex  BUT for majority of the time  this is the shortest way to go IMHO    x000D   x000D  var collection    a 1   a 2   a 1   a 3   x000D   x000D  var filtered   collection reduce  filtered  item    gt    x000D    if   filtered some filteredItem   gt  JSON stringify filteredItem     JSON stringify item     x000D      filtered push item  x000D    return filtered x000D         x000D   x000D  console log filtered  x000D   x000D   x000D    Another way of writing the same  but less efficient    collection reduce  filtered  item    gt     filtered some filteredItem   gt       JSON stringify filteredItem      JSON stringify item            filtered             filtered  item

User · Answer

You can use Object values   combined with Array prototype reduce      x000D   x000D  const things   new Object    x000D   x000D  things thing   new Array    x000D   x000D  things thing push  place  here  name  stuff     x000D  things thing push  place  there  name  morestuff     x000D  things thing push  place  there  name  morestuff     x000D   x000D  const result   Object values things thing reduce  a  c    gt   a    c place   c name      c  a          x000D   x000D  console log result   x000D   as-console-wrapper   max-height  100   important  top  0    x000D   x000D   x000D

User · Answer

If you find yourself needing to remove duplicate objects from arrays based on particular fields frequently  it might be worth creating a distinct array  predicate  function that you can import from anywhere in your project  This would look like  const things     place  here  name  stuff          const distinctThings   distinct things  thing   gt  thing place     The distinct function can use any of the implementations given in the many good answers above  The easiest one uses findIndex   const distinct    items  predicate    gt  items filter  uniqueItem  index    gt      items findIndex item   gt          predicate item      predicate uniqueItem       index

User · Answer

If you are using Lodash library you can use the below function as well  It should remove duplicate objects  var objects       x   1   y   2       x   2   y   1       x   1   y   2       uniqWith objects    isEqual

User · Answer

Considering lodash uniqWith  var objects       x   1   y   2       x   2   y   1       x   1   y   2        uniqWith objects    isEqual        gt      x   1   y   2       x   2   y   1

User · Answer

Source  JSFiddle  This will remove the duplicate object without passing any key    x000D   x000D  uniqueArray   a   gt      new Set a map o   gt  JSON stringify o     map s   gt  JSON parse s    x000D   x000D  var objects       x   1   y   2       x   2   y   1       x   1   y   2     x000D   x000D  var unique   uniqueArray objects   x000D  console log  Original Object  objects   x000D  console log  Unique  unique   x000D   x000D   x000D    uniqueArray   a   gt      new Set a map o   gt  JSON stringify o     map s   gt  JSON parse s         var objects       x   1   y   2       x   2   y   1       x   1   y   2          var unique   uniqueArray objects       console log objects       console log unique

User · Answer

If you don t mind your unique array being sorted afterwards  this would be an efficient solution   things thing    sort   a  b    gt  a place  lt  b place     filter  current  index  array    gt      index     0    current place     array index - 1  place    This way  you only have to compare the current element with the previous element in the array  Sorting once before filtering  O n log n    is cheaper than searching for a duplicate in the entire array for every array element  O n

User · Answer

For a readable and a simple solution searcher  her is my version       function removeDupplicationsFromArrayByProp originalArray  prop            let results               for let i 0  i lt originalArray length i                 results originalArray i  prop     originalArray i                     return Object values results

User · Answer

Here is another technique to find number of duplicate and and remove it easily from you data object   dupsCount  is number of duplicate files count  sort your data first then remove  it will gives you fastest duplication remove     dataArray sort function  a  b                var textA   a name toUpperCase                var textB   b name toUpperCase                return  textA  lt  textB    -1    textA  gt  textB    1   0                      for  var i   0  i  lt  dataArray length - 1                  if  dataArray i  name    dataArray i   1  name                    dupsCount                    dataArray splice i  1                 else                   i

User · Answer

ES6 one liner is here  x000D   x000D  let arr        id 1 name  sravan ganji       id 2 name  pinky       id 4 name  mammu       id 3 name  sanju       id 3 name  ram        console log Object values arr reduce  acc cur   gt Object assign acc   cur id  cur         x000D   x000D   x000D

User · Answer

Here is a solution for ES6 where you only want to keep the last item  This solution is functional and Airbnb style compliant  const things       thing          place   here   name   stuff           place   there   name   morestuff1           place   there   name   morestuff2               const removeDuplicates    array  key    gt      return array reduce  arr  item    gt        const removed   arr filter i   gt  i key      item key        return     removed  item                 console log removeDuplicates things thing   place         gt     place   here   name   stuff       place   there   name   morestuff2

User · Answer

This is simple way how to remove duplicity from array of objects   I work with data a lot and this is useful for me   const data     name   AAA     name   AAA     name   BBB     name   AAA     function removeDuplicity datas       return datas filter  item  index arr   gt       const c   arr map item  gt  item name       return  index     c indexOf item name          console log removeDuplicity data     will print into console      object Object    name   AAA      object Object    name   BBB

[javascript] Remove duplicates from an array of objects in JavaScript

Examples related to javascript

Examples related to arrays

Examples related to object

Examples related to duplicates