Record parsing to Scala object with some unknown fields

Question

I am getting some customer data but don't know the complete schema beforehand. I want to parse this data and create a Scala object from it. But due to unknown fields in the record, I fill only the important fields and remaining record data is dumped in a remData placeholder field. This remData is used so that we can still see the customer data in output and can add more fields in future.

---

The customer provides 3-4 types of such feeds with each feed having 15-20 fields. I want to generate a generic framework for such parsing. I tried this code and it works but I wonder if there are any common patterns for such type of parsing or a cleaner approach.

class Record{
    var imptField1 =""
    var imptField2 =""
    var remData =""
    def this(data:String){
      this()
      val fieldKeyValues = data.split(",")
      for( fieldKeyValue <- fieldKeyValues){
        val tokens = fieldKeyValue.split("=")
        val key = tokens(0) 
        key match{
          case "imptField1"=> this.imptField1 = tokens(1)
          case "imptField2"=> this.imptField2 = tokens(1)
          case _ => this.remData +=fieldKeyValue+","
        }
      }
   }
 }

I also tried using Jackson objectMapper for that but it doesn't provide the functionality of dumping unknown fields in a placeholder. Please suggest a more efficient/cleaner way of doing this.

Answer 1

I would parse the row into a Map[String,String] first (by collecting the key-value pairs that you already know how to split), and then do

this.imptField1 = map.get("imptField1").getOrElse("")
this.imptField2 = map.get("imptField2").getOrElse("")
this.remData = map  // just the whole Map, or maybe remove the above two keys

I would also change the class to be immutable and move the parsing code into the companion object (it would do the parsing, then pass the map to the constructor of the class which can directly assign to its val fields).

Answer 2

In case you don't need to modify the field after construction, you can make the fields vals and use a factory to construct your instance:

class Record (val imptField1: String, val imptField2: String, val remData: String)

object Record {
    def apply (data: String): Record2 = {
        val (field1, field2, remaining) = data.split(",").map{ fieldKeyValue =>
            fieldKeyValue.split("=")
        }.foldLeft(("", "", Vector.empty[String])){case ((f1, f2, rem), Array(k, v)) => k match {
            case "imptField1" => (v, f2, rem)
            case "imptField2" => (f1, v, rem)
            case _ => (f1, f2, rem :+ (k + "=" + v))
        }}
        new Record(field1, field2, remaining.mkString("\n"))
    }    
}