Amazon Web Services (AWS) is a collection of commercial services centered around virtualized, commodity and clustered computing, networking, database and storage, as well as supporting tools that are offered through infrastructure as a service.

learn more… | top users | synonyms (1)

1
vote
0answers
34 views

AWS Redshift wrapper class that automates similar types of loads from S3

I have a AWS Redshift wrapper class that automates similar types of loads from S3 for me, and I have recently adapted it to work for Spark jobs, which don't require a manifest, and instead need a ...
1
vote
1answer
27 views

Node.js put records to Kinesis with infinite retry strategy.

My application is really easy. It's going to add 500 records at a time to AWS Kinesis and if there's an error occur I'll just add that back to the queue and it's going to be retry. It's going well but ...
5
votes
2answers
85 views

Rails controller to manage EC2 instances

I did a small sample application for an Interview. Since I didn't got accepted I would like your opinion on how to improve it. The aux methods are private. I have a ...
4
votes
1answer
36 views

Simple shell script to backup to Amazon over iSCSI

The script works but I know that it can be improved. Please note that my scripting skills are very basic :) (as it can be seen in my Frankenstein script – I took bits and pieces of the code from ...
1
vote
0answers
52 views

Unit tests for a Redshift wrapper class

I'm somewhat new to TDD and unit testing, but I've written a suite of unit tests to check my functionality. The basic classes automate simple load and unload operations from s3 and redshift, and ...
2
votes
1answer
105 views

Redshift Load and Unload Wrapper class

We do constant traffic with our Redshift tables, and so I created a wrapper class that will allow for custom sql to be ran (or a default generic stmt), and can run a ...
1
vote
1answer
97 views

Listing EC2 instances

I've got this massive nested for loop with a few if and else statements in nodeJs. The output of all of this is just string which is combined from different values out of JSon object. What I want to ...
5
votes
1answer
615 views

Convert zip to gzip and upload to S3 bucket

My code accesses an FTP server, downloads a .zip file, pushes the file contents as .gz to an AWS S3 bucket. ...
5
votes
1answer
748 views

Testing boto connection

I am writing a unit test to be sure the code can connect to aws via boto, since I will be putting the credentials in place via some other process than code deployment. So far, I've only figured out ...
5
votes
1answer
141 views

Bash script to backup from S3 to GC

I'm working on a script to backup S3 to GC everyday. I'm not sure if my script has any potential bug or error that might have destroyed anything from both sides? The data is quite sensitive and I don'...
2
votes
1answer
167 views

Generating URL and status for an S3 bucket

The following program generates the S3 URL and response (status code?) for the key, provided the bucket name and if the S3 URL is a mess (wrong url), then it generates empty string as URL and error as ...
5
votes
3answers
68 views

Structuring code that shells out to an external script

This code that shells out to an external script could use some input in order to read smoother, I think. Praise and critique are welcome. ...
1
vote
1answer
2k views

Gets AWS VPC subnet info

I was hoping to get a quick review of a script I wrote for retrieving AWS VPC Subnet information. Any and all ideas welcome. I am looking specifically for logic enhancements, and optimizations to ...
4
votes
1answer
550 views

Uploading a video in S3 Using Future

I've posted a similar program previously, but I have not made any major modifications in the code, so I am posting it again by deleting the previous question. I am afraid of the thread keyword and I ...
8
votes
1answer
819 views

Connecting to EC2 instances using the instance name instead of IP

I'm an intensive user of AWS EC2 instances, many instances are launched, stopped, repurposed, etc. To connect to any instance using SSH I must keep track of their ...
1
vote
1answer
794 views

Python Boto Script - Sorting based on date

I had a question regarding my code which downloads a file from S3 with the highest (most recent) timedated filename format: YYYYMMDDHHMMSS.zip ...
2
votes
1answer
530 views

Amazon S3 pass-through controller

I am building a website that uses Amazon S3 to host its image (and other) files. I require that images displayed on the website look like they come from my site and not from Amazon (so we are talking ...
8
votes
2answers
33k views

List objects in a Amazon S3 folder without also listing objects in sub folders

I'm using the Amazon S3 Java SDK to fetch a list of files in a (simulated) sub-folder. This code is rather standard (AWSConfiguration is a class that contains a ...