Revisions to Read bunch of files in parallel and populate concurrent map

edited tags

Link

edited Mar 20, 2022 at 19:06

200_success

144.2k
22
188
473

edited tags

Link

edited Mar 19, 2022 at 16:19

AndyP

239
2
10

go concurrency

added 128 characters in body

Source Link

edited Mar 19, 2022 at 7:12

AndyP

239
2
10

I am trying to read bunch of S3 parquet files in parallel from a S3 bucket. After reading all these files, I am populating my products and productCatalog concurrent map. This happens during server startup and then I have getters method GetProductMap and GetProductCatalogMap to return these maps which be used by main application threads.

I am using this parquet library to read all these S3 files.

I am trying to read bunch of S3 files in parallel from a S3 bucket. After reading all these files, I am populating my products and productCatalog concurrent map. This happens during server startup and then I have getters method GetProductMap and GetProductCatalogMap to return these maps which be used by main application threads.

I am trying to read bunch of S3 parquet files in parallel from a S3 bucket. After reading all these files, I am populating my products and productCatalog concurrent map. This happens during server startup and then I have getters method GetProductMap and GetProductCatalogMap to return these maps which be used by main application threads.

I am using this parquet library to read all these S3 files.

Source Link

asked Mar 19, 2022 at 6:51

AndyP

239
2
10

Loading

Stack Exchange Network

Return to Question