remove ALL duplicate elements from an array

Question

I have this code to remove duplicates (all occurrences) from an associative array, does PHP have methods to do this ? Or is there a way to improve the code ?

I looked for array_unique, array_search, array_map, array_reduce...

$articles = [
    [
        "id" => 0,
        "title" => "lorem",
        "reference" => "A"
    ],
    [
        "id" => 1,
        "title" => "ipsum",
        "reference" => "B"
    ],
    [
        "id" => 2,
        "title" => "dolor",
        "reference" => "C"
    ],
    [
        "id" => 3,
        "title" => "sit",
        "reference" => "A"
    ]
];

$references = array_column($articles, "reference");
$duplicates = array_values(array_unique(array_diff_assoc($references, array_unique($references))));

foreach($duplicates as $duplicate) {
    foreach($references as $index => $reference) {
        if($duplicate === $reference) {
            unset($articles[$index]);
        }
    }
}

/**
 * $articles = [
 *     [
 *         "id" => 1,
 *         "title" => "ipsum",
 *         "reference" => "B"    
 *     ],
 *     [
 *         "id" => 2,
 *         "title" => "dolor",
 *         "reference" => "C"  
 *     ]
 * ]
 */

Answer 1

This task can and should be done with a single loop with no pre-looping variable population and no inefficient in_array() calls. Searching keys in php is always more efficient than searching values.

Code: (Demo)

$found = [];
foreach ($articles as $index => ['reference' => $ref]) {
    if (!isset($found[$ref])) {
        $found[$ref] = $index;
    } else {
        unset($articles[$index], $articles[$found[$ref]]);
    }
}
var_export($articles);

Output:

array (
  1 => 
  array (
    'id' => 1,
    'title' => 'ipsum',
    'reference' => 'B',
  ),
  2 => 
  array (
    'id' => 2,
    'title' => 'dolor',
    'reference' => 'C',
  ),
)

I am using array destructuring syntax in my foreach() for brevity and because I don't need the other column values.

Finally, it doesn't matter if there are triplicates (or more instances of a reference value), the script will handle these in the same fashion. unset() will not generate any notices, warnings, or errors if it is given a non-existent element (as a parameter) -- this is why it is safe to unconditionally unset the first found reference potentially multiple times.

Answer 2

is there a way to improve the code ?

Instead of having two foreach loops:

foreach($duplicates as $duplicate) {
    foreach($references as $index => $reference) {
        if($duplicate === $reference) {
            unset($articles[$index]);
        }
    }
}

It can be simplified using in_array():

foreach($references as $index => $reference) {
    if (in_array($reference, $duplicates, true)) {
        unset($articles[$index]);
    }
}

While it would still technically have the same complexity (i.e. two loops) it would have one less indentation level, and utilize a built-in function to check if the reference is in the list of duplicate references.

Another solution would be to use array_flip() to map the last index to references, then loop through the articles and if the index of the current article does not match the index of the last reference (meaning its a duplicate) then remove both the article at the current index as well as the article at the last index that has the reference.

$references = array_column($articles, "reference");
$lastIndexes = array_flip($references);

foreach ($articles as $index => $article) {
    if ($lastIndexes[$article['reference']] !== $index) {
        unset($articles[$index], $articles[$lastIndexes[$article['reference']]]);
    }
}

Or to make it more readable, the last index can be assigned to a variable:

foreach($articles as $index => $article) {
    $lastIndex = $lastIndexes[$article['reference']];
    if ($lastIndex !== $index) {
        unset($articles[$index], $articles[$lastIndex]);
    }
}