4
\$\begingroup\$

I'm working on part of a service for data processing. The main idea of the service is to get the input data, process it and return the processed data to another service. Input and output data format are flat JSON documents (documents without nested objects and lists, only first level objects). The core entity of this service is Filter. Filter accepts parsed JSON data, processes it and returns processed data in its internal format (in my case this is Map<String, BaseValue). Filter has a context (current state of data) and a list of operations. Here is the code of the Filter class:

package com.rk.filter;

import com.rk.filter.context.FilterContext;
import com.rk.operations.Operation;
import com.rk.types.BaseValue;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.function.Consumer;

public class Filter {

    private List<Operation> ops = new ArrayList<>();

    public Map<String, BaseValue> apply(Map<String, BaseValue> inputs) {
        return apply(inputs, null);
    }

    public Map<String, BaseValue> apply(Map<String, BaseValue> inputs, Consumer<String> traceConsumer) {
        FilterContext context = new FilterContext(inputs);

        for (Operation block: ops) {
            block.apply(context);

            if (traceConsumer != null) {
                traceConsumer.accept(context.trace());
            }
        }

        return context.getData();
    }

    public void addOperation(Operation operation) {
        ops.add(operation);
    }
}

Code of FilterContext class:

package com.rk.filter.context;

import com.rk.filter.MissingValuesException;
import com.rk.types.BaseValue;
import com.rk.types.NullValue;

import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
import java.util.stream.Collectors;

public class FilterContext {

    private final Map<String, BaseValue> data;
    private final Map<String, BaseValue> variables;

    public Map<String, BaseValue> getData() {
        return data;
    }

    public FilterContext(Map<String, BaseValue> inputData) {
        this.data = new HashMap<>(inputData);
        this.variables = new HashMap<>();
    }

    public BaseValue getFromData(String key) {
        BaseValue value = data.get(key);

        assertReadValueIsNotNull(value, "data", key);

        return value;
    }

    public void setToData(String key, BaseValue value) {
        value = Objects.requireNonNull(value);

        data.put(key, value);
    }

    public BaseValue getFromVariables(String key) {
        BaseValue value = variables.get(key);

        assertReadValueIsNotNull(value, "variables", key);

        return value;
    }

    public void setToVariables(String key, BaseValue value) {
        value = Objects.requireNonNull(value);

        variables.put(key, value);
    }

    public String trace() {
        return "data: " + formatData(data) + "\nvariables: " + formatData(variables);
    }

    private static String formatData(Map<String, BaseValue> data) {
        return data.entrySet().stream()
                .map((Map.Entry e) -> e.getKey() + " = " + e.getValue())
                .collect(Collectors.joining(", "));
    }

    private static void assertReadValueIsNotNull(Object value, String storageName, String key) {
        if (value == null) {
            throw new MissingValuesException(storageName, key);
        }
    }
}

Operation is an entity that accepts data from given sources and writes the result to the given targets. Each Operation class defines its sources and targets, for example the string concatenation operation (A + B) takes 2 sources (A and B) and writes the result to the OUT target. My current implementation of operations knows about FilterContext:

package com.rk.operations;

import com.rk.filter.context.FilterContext;

public interface Operation {

    void apply(FilterContext context);
}

Example of operation implementation:

package com.rk.operations;

import com.rk.filter.context.FilterContext;
import com.rk.filter.context.Getter;
import com.rk.filter.context.Setter;
import com.rk.types.NumberValue;

import java.math.BigDecimal;

public class NumberAddOperation implements Operation {

    private final Getter a;
    private final Getter b;
    private final Setter out;

    public NumberAddOperation(Getter a, Getter b, Setter out) {
        this.a = a;
        this.b = b;
        this.out = out;
    }

    @Override
    public void apply(FilterContext context) {
        NumberValue aValue = a.getValue(context).toNumberValue();
        NumberValue bValue = b.getValue(context).toNumberValue();

        BigDecimal result = aValue.getValue().add(bValue.getValue());

        out.setValue(new NumberValue(result), context);
    }
}

As you can see, Operation has two getters and one setter. The Getter interface implementations says what kind of data we should read from FilterContext (the data source). The Setter interface implementations says where we should put our data (the data target).

Code of Getter interface:

package com.rk.filter.context;

import com.rk.types.BaseValue;

public interface Getter {

    BaseValue getValue(FilterContext context);
}

Code of Setter interface:

package com.rk.filter.context;

import com.rk.types.BaseValue;

public interface Setter {

    void setValue(BaseValue value, FilterContext context);
}

Now I have three implementations of these interfaces:

  • class DataAccess - both read and write to context data field;
  • class VariablesAccess - both read and write to context variables field;
  • class ConstGetter - special getter that provides const value and actually doesn't read or write context.

Here is some runnable code:

package com.rk;

import com.rk.filter.Filter;
import com.rk.filter.context.ConstGetter;
import com.rk.filter.context.DataAccess;
import com.rk.filter.context.VariablesAccess;
import com.rk.operations.AssignOperation;
import com.rk.operations.NumberAddOperation;
import com.rk.operations.Operation;
import com.rk.operations.StringTrimOperation;
import com.rk.types.BaseValue;
import com.rk.types.NumberValue;
import com.rk.types.StringValue;

import java.math.BigDecimal;
import java.util.HashMap;
import java.util.Map;

public class Main {

    public static void main(String[] args) {
        /* Setup test data */
        Map<String, BaseValue> data = new HashMap<>();
        data.put("foo", new StringValue("  hello  "));
        data.put("bar", new StringValue("500"));

        BigDecimal number = new BigDecimal(155);

        /* Setup operations */
        Operation op1 = new StringTrimOperation(new DataAccess("foo"), new DataAccess("foo"));
        Operation op2 = new AssignOperation(new DataAccess("bar"), new VariablesAccess("var_bar"));
        Operation op3 = new NumberAddOperation(
                new VariablesAccess("var_bar"),
                new ConstGetter(new NumberValue(number)),
                new DataAccess("bar"));

        /* Setup filters */
        Filter filter = new Filter();

        filter.addOperation(op1);
        filter.addOperation(op2);
        filter.addOperation(op3);

        /* Execute */
        Map<String, BaseValue> result = filter.apply(data, (trace -> {
            System.out.println(trace);
            System.out.println();
        }));

        System.out.println(result.get("foo").toStringValue().getValue()); // "hello"
        System.out.println(result.get("bar").toNumberValue().getValue()); // 655
    }
}

In reality, the service configurations of Filter, operations, getter and setter will all be taken from a database.

Note about BaseValue class: base value is a set of 4 basic types (null, string, number and boolean) that can be converted to each other:

package com.rk.types;

abstract public class BaseValue {

    abstract public StringValue toStringValue();
    abstract public BooleanValue toBooleanValue();
    abstract public NumberValue toNumberValue();

    public final NullValue toNullValue() {
        return NullValue.NULL;
    }
}

Example of implementation of NumberValue:

package com.rk.types;

import java.math.BigDecimal;
import java.util.Objects;

public class NumberValue extends BaseValue implements TypeAdapter<BigDecimal> {

    private static final BigDecimal DEFAULT_VALUE = BigDecimal.ZERO;

    private BigDecimal value;

    public NumberValue() {
        this.value = DEFAULT_VALUE;
    }

    public NumberValue(BigDecimal value) {
        this.value = Objects.requireNonNull(value);
    }

    @Override
    public BigDecimal getValue() {
        return this.value;
    }

    @Override
    public void setValue(BigDecimal value) {
        this.value = Objects.requireNonNull(value);
    }

    @Override
    public BooleanValue toBooleanValue() {
        return new BooleanValue(!isZero());
    }

    @Override
    public NumberValue toNumberValue() {
        return this;
    }

    @Override
    public StringValue toStringValue() {
        return new StringValue(value.toPlainString());
    }

    private boolean isZero() {
        return value.equals(BigDecimal.ZERO);
    }

    @Override
    public String toString() {
        return value.toString();
    }
}

Is it possible to make this code better? It seems that my mechanism of Getter and Setter is not too good, because the operations classes know about the Filter context, even though they don't work with the context interface directly.

Implementation note: I assume that my code should be null-safe. It means that my data storage can't have keys with null values. For null-like values, an instance of class NullValue should be used.

The whole code is published on GitHub.

Class diagram. Class diagram

\$\endgroup\$
  • \$\begingroup\$ If you want to remove boiler plate code like getter setter then use lombok. \$\endgroup\$ – dkb Dec 21 '18 at 7:50
  • \$\begingroup\$ Use Optional incase if you want your code to be null-check-safe. \$\endgroup\$ – dkb Dec 21 '18 at 7:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.