Removing duplicate values from a PowerShell array

Question

How can I remove duplicates from a PowerShell array    a     1 2 3 4 5 5 6 7 8 9 0 0

User · Answer

With my method you can completely remove duplicate values  leaving you with values from the array that only had a count of 1  It was not clear if this is what the OP actually wanted however I was unable to find an example of this solution online so here it is    array    Bananna Apple Carrot Pear Apricot Pear Bananna    -split   r n     array   Group-Object -NoElement        count -eq 1   Name

User · Answer

Whether you re using SORT -UNIQUE  SELECT -UNIQUE or GET-UNIQUE from Powershell 2 0 to 5 1  all the examples given are on single Column arrays   I have yet to get this to function across Arrays with multiple Columns to REMOVE Duplicate Rows to leave single occurrences of a Row across said Columns  or develop an alternative script solution   Instead these cmdlets have only returned Rows in an Array that occurred ONCE with singular occurrence and dumped everything that had a duplicate   Typically I have to Remove Duplicates manually from the final CSV output in Excel to finish the report  but sometimes I would like to continue working with said data within Powershell after removing the duplicates

User · Answer

a   sort -unique   This works with case-insensitive  therefore removing duplicates strings with differing cases  Solved my problem    ServerList           FS3        HQ2        hq2      sort -Unique   ServerList   The above outputs   FS3 HQ2

User · Answer

If the list is sorted  you can use the Get-Unique cmdlet     a   Get-Unique

User · Answer

In case you want to be fully bomb prove  this is what I would advice      Apples    Apples     APPLES    Banana          Sort-Object -Property   Expression     Trim     -Unique   Output   Apples Banana   This uses the Property parameter to first Trim   the strings  so extra spaces are removed and then selects only the -Unique values   More info on Sort-Object   Get-Help Sort-Object -ShowWindow

User · Answer

This is how you get unique from an array with two or more properties  The sort is vital and the key to getting it to work correctly  Otherwise you just get one item returned  PowerShell Script   objects           PSCustomObject     Message    quot 1 quot   MachineName    quot 1 quot         PSCustomObject     Message    quot 2 quot   MachineName    quot 1 quot         PSCustomObject     Message    quot 3 quot   MachineName    quot 1 quot         PSCustomObject     Message    quot 4 quot   MachineName    quot 1 quot         PSCustomObject     Message    quot 5 quot   MachineName    quot 1 quot         PSCustomObject     Message    quot 1 quot   MachineName    quot 2 quot         PSCustomObject     Message    quot 2 quot   MachineName    quot 2 quot         PSCustomObject     Message    quot 3 quot   MachineName    quot 2 quot         PSCustomObject     Message    quot 4 quot   MachineName    quot 2 quot         PSCustomObject     Message    quot 5 quot   MachineName    quot 2 quot         PSCustomObject     Message    quot 1 quot   MachineName    quot 1 quot         PSCustomObject     Message    quot 2 quot   MachineName    quot 1 quot         PSCustomObject     Message    quot 3 quot   MachineName    quot 1 quot         PSCustomObject     Message    quot 4 quot   MachineName    quot 1 quot         PSCustomObject     Message    quot 5 quot   MachineName    quot 1 quot         PSCustomObject     Message    quot 1 quot   MachineName    quot 2 quot         PSCustomObject     Message    quot 2 quot   MachineName    quot 2 quot         PSCustomObject     Message    quot 3 quot   MachineName    quot 2 quot         PSCustomObject     Message    quot 4 quot   MachineName    quot 2 quot         PSCustomObject     Message    quot 5 quot   MachineName    quot 2 quot       Write-Host  quot Sorted on both properties with -Unique quot  -ForegroundColor Yellow  objects   Sort-Object -Property Message MachineName -Unique   Out-Host  Write-Host  quot Sorted on just Message with -Unique quot  -ForegroundColor Yellow  objects   Sort-Object -Property Message -Unique   Out-Host  Write-Host  quot Sorted on just MachineName with -Unique quot  -ForegroundColor Yellow  objects   Sort-Object -Property MachineName -Unique   Out-Host  Output  Sorted on both properties with -Unique  Message MachineName ------- ----------- 1       1           1       2           2       1           2       2           3       1           3       2           4       1           4       2           5       1           5       2             Sorted on just Message with -Unique  Message MachineName ------- ----------- 1       1           2       1           3       1           4       1           5       2             Sorted on just MachineName with -Unique  Message MachineName ------- ----------- 1       1           3       2    Source  https   powershell org forums topic need-to-unique-based-on-multiple-properties

User · Answer

To get unique items from an array and preserve their order  you can use  NET HashSet   Array     1  3  1  2   Set   New-Object -TypeName  System Collections Generic HashSet int   -ArgumentList    int    Array     PS gt  Set   1   3   2  Works best with string arrays that contain both uppercase and lowercase items where you need to preserve first occurrence of each item in case-insensitive manner   Array      quot B quot    quot b quot    quot a quot    quot A quot    Set   New-Object -TypeName  System Collections Generic HashSet string   -ArgumentList   string    Array   StringComparer   OrdinalIgnoreCase     PS gt  Set   B   a  Works as expected with other types  Shortened syntax  compatible with PowerShell 5 1 and newer   Array      quot B quot    quot b quot    quot a quot    quot A quot    Set    Collections Generic HashSet string    new  string    Array   StringComparer   OrdinalIgnoreCase    Array     1  3  1  2   Set    Collections Generic HashSet int    new  int    Array

User · Answer

Another option is to use Sort-Object  whose alias is sort  but only on Windows  with the -Unique switch  which combines sorting with removal of duplicates    a   sort -unique

User · Answer

Use Select-Object  whose alias is select  with the -Unique switch  e g     a     1 2 3 4 5 5 6 7 8 9 0 0   a    a   select -Unique

[arrays] Removing duplicate values from a PowerShell array

Examples related to arrays

Examples related to powershell