I have the following code on k-Anonymity ('*' suppress values version) that I want to check its correctness. The code compiles and outputs as expected.
public class KAnonymity {
/**
* @param k_anonymity_constant k constant
* @param qidAtts list of Quasi-ID attributes
* @param initDataset the dataset to k-Anonymize
* @return k-Anonymized Datasets
*/
public static Dataset<Row> kAnonymizeBySuppression( Dataset<Row> initDataset, List<String> qidAtts, Integer k_anonymity_constant) {
List<Column> groupByQidAttributes = qidAtts.stream().map(functions::col).collect(Collectors.toList());
Dataset<Row> processedDataset = initDataset.withColumn("qidsFreqs", count("*").over(Window.partitionBy(groupByQidAttributes.toArray(new Column[groupByQidAttributes.size()]))));
for (String qidAtt : qidAtts)
processedDataset = processedDataset.withColumn(qidAtt, when(col("qidsFreqs").leq(lit(k_anonymity_constant)), lit('*')).otherwise(col(qidAtt)));
return processedDataset.drop("qidsFreqs");
}
}
PS: Unit test case (k=5)
//Read the csv dataset
Dataset<Row> dataset = spark.read().csv(pathToCSV);
// Anonymize it
List<String qidAttsList = new ArrayList<>();
qidAttsList.add("qidAtt1");
qidAttsList.add("qidAtt2");
...
Dataset<Row> anonymizedDF = KAnonymity.kAnonymizeBySuppression(dataset, qidAttsList, 5);
kAnonymizeBySuppression()
the only method in the class? If not please post the entire class. Please read How do I ask a good question?. and A guide to Code Review for Stack Overflow users. \$\endgroup\$