Questions tagged [etl]

ETL is an acronym for Extract-Transform-Load. This refers a database usage where data is extracted from a source, transformed into a different format, and loaded to a target database for analysis or reporting.

Filter by
Sorted by
Tagged with
1
vote
0answers
27 views

Relational join of two datasets

Front Matter I'm learning Scala and have not gotten used to functional programming and the language. I'm hoping a review of my naively implemented code can help me bridge my object-oriented ways to ...
0
votes
0answers
211 views

Dynamically configurable ZMQ filter with Flask API

I have a stream of ZMQ messages, I need to filter the right ones and save them in MongoDB. The trick is to have an option to set the filter conditions dynamically. I have come up with this solution ...
1
vote
0answers
1k views

Handling slow processing the Spring batch way

We are using spring-batch to extract data from a SAP system, during processing we pass the extracted full texts to a tagging service, in the end the enriched documents are stored in Solr to be served ...
1
vote
0answers
574 views

Snaplogic Script to group key-value pairs based on a certain ID field

Following is the script I'm using to group key-value pairs based on an ID field viz. interface_id: ...
2
votes
1answer
875 views

Extract data from one column of SQL Server table

I have an SQL Server table from which I want to extract data from one column (DD) and return as multiple columns. The data is in ...
3
votes
1answer
547 views

ETL code to synchronize external system to internal database

I'm looking to see if my current evolution of ETL logic is getting in the ball park of "how it's done in the real world". Specifically, do I use the metaphors correctly (...
5
votes
1answer
363 views

MergeUtility for Entity Framework

Here is a utility that supports ETL/merging in Entity Framework. If it's not appealing as a general purpose tool, why? If it is appealing as a general purpose tool, how might the design be made ...