Skip to main content
5 of 5
edited tags
200_success
  • 144.2k
  • 22
  • 188
  • 473

Record parsing to Scala object with some unknown fields

I am getting some customer data but don't know the complete schema beforehand. I want to parse this data and create a Scala object from it. But due to unknown fields in the record, I fill only the important fields and remaining record data is dumped in a remData placeholder field. This remData is used so that we can still see the customer data in output and can add more fields in future.


The customer provides 3-4 types of such feeds with each feed having 15-20 fields. I want to generate a generic framework for such parsing. I tried this code and it works but I wonder if there are any common patterns for such type of parsing or a cleaner approach.

class Record{
    var imptField1 =""
    var imptField2 =""
    var remData =""
    def this(data:String){
      this()
      val fieldKeyValues = data.split(",")
      for( fieldKeyValue <- fieldKeyValues){
        val tokens = fieldKeyValue.split("=")
        val key = tokens(0) 
        key match{
          case "imptField1"=> this.imptField1 = tokens(1)
          case "imptField2"=> this.imptField2 = tokens(1)
          case _ => this.remData +=fieldKeyValue+","
        }
      }
   }
 }

I also tried using Jackson objectMapper for that but it doesn't provide the functionality of dumping unknown fields in a placeholder. Please suggest a more efficient/cleaner way of doing this.