Questions tagged [network-file-transfer]

Questions relating to the copying of files from one filesystem, over a network, to a destination filesystem

Filter by
Sorted by
Tagged with
3
votes
0answers
32 views

Adapter pattern to support multiple file storage systems

I am just getting in to following design patterns. I have create a class that allows for performing a couple operations on different remote storage systems: drive, dropbox, .etc (I call these ...
2
votes
1answer
48 views

Java file transfer through sockets

I am making a program that transfers files from one client to another. Right now I am just working on sending the file to the server and then later I will work on transferring it to the other client. ...
2
votes
1answer
51 views

Upload a file asynchronously

I need your opinion about my practice. I want to improve it. The requirements are the following: RESTFUL + OWIN (SelfHosted) Endpoint - Upload a file asynchronously Make a request to the endpoint ...
3
votes
1answer
48 views

PHP remote upload security

I need to upload images to a remote HTTP server from a local script. The image name format is {32-char MD5}.jpg. Server directory format is ...
3
votes
0answers
74 views

Using Python threads to download images

I am using Python threads to download images. I have a JSON file that contains URLs to the images in the following structure: ...
0
votes
2answers
468 views

Laravel: Export customer record as CSV

Using Laravel, I need to export some value from the DB in a CSV and then upload the CSV via sFTP or return it in a Response. I'm trying to be use S.O.L.I.D. principles but with this scenario, I'm not ...
1
vote
0answers
396 views

Python requests downloading big files in slower speed than wget

This is a simple toy downloader using python's requests library. I’ve monitored the download process is slower on an ethernet connected box. The realtime speed is ...
1
vote
1answer
106 views

Laravel image upload

I'm new to the filesystem from Laravel, but I need an upload script that allows users to upload images to my server. However, i'm concerned about securety, so I thought why not ask for advice, because ...
1
vote
1answer
18 views

Saving a submitted Mongoose model, possibly with an uploaded image

I have some code which saves a mongoose model from the submitted form data. However the user might also upload an image along with the form, which needs to be saved on the server (moved from the ...
1
vote
1answer
1k views

Python rsync to copy the logs from multiple remote servers

I have the below code which i'm working to copy(using rsync) the contents From remote host labserver01 and dumps those into the directory ...
4
votes
1answer
359 views

Simple Java Download Manager

I have written a simple download accelerator in java which downloads the same file in multiple threads and then combines them all. I would love some feedback on my design. How the same implementation ...
1
vote
2answers
314 views

Python functions to fetch spreadsheets through SFTP and from Dropbox [closed]

I have written two functions- first one is to read files from SFTP and second one is to read files from Dropbox. Both functions have some similar line of code like validating extension and reading/...
2
votes
1answer
98 views

PHP Template: Image file upload handler

I'm trying to develop some templates for common PHP tasks I've been dealing with. One of which is a general image file upload handler. So far I'm using the following reusable code which seems to be ...
3
votes
1answer
2k views

Google Drive upload for large files

I'm using Google Drive API for uploading large disk image files ( >100 GB). My Code uses this wrapper to upload these large files in chunks of 40mb. The files are stored on NAS drive. The code works ...
11
votes
0answers
99 views

Download files in parallel

This module has a ensure_downloaded function which takes a slice of Urls. It downloads all of the urls to a local downloads ...
4
votes
0answers
335 views

Uploading captured video to Google Cloud Storage

My program intends to capture video streams (in mjpeg) with OpenCV and upload the captured frames into Google Cloud Storage for later processing. I am expecting to capture ~15-20 frames per second and ...
3
votes
1answer
35 views

Handling download requests using PM2

I Using Nodejs v9.10.1 for handle download operation from another application. When using PM2 in fork mode to make alive my code. After 50 minutes, this will show high memory usage. Server Config is ...
3
votes
0answers
563 views

Read remote and local directories and transfer data

The app is made to be able to send large amounts (1000+) of big data files (2GB>) to a online long term storage service. The application they themselves provide for this service is very buggy and ...
3
votes
1answer
5k views

File Transfer over TCP

I'm working on an Asynchronous TCP Client/Server using the old BeginXXX and EndXXXSocket API. The goal of this hobby project is ...
6
votes
1answer
170 views

Archives xkcd comics

This is a work in progress, as you may read I plan to revise to utilise JSON data rather than directly scraping. I am considering refactoring the try//...
3
votes
1answer
158 views

One to One network file transfer code C# socket based

I have been developing a piece of software which has a central computer talk to many smaller nodes with a star network configuration. Each smaller node sends a file around 5MB in size once the central ...
0
votes
1answer
1k views

File upload in AngularJS causes browser out of memory error

I have an AngularJS code to perform upload of files. At first I thought it works OK but when I've tried to upload bigger files (e.g. 5 files, 10 MB each) then I saw that this code has very poor ...
3
votes
1answer
1k views

Download and write ~100k images

Currently I will be downloading around ~100K images. This process may run on a weekly basis, I'm considering the following factors: HTTP Requests (Will be taken care by external LB, so same source IP ...
3
votes
2answers
77 views

Uploading photos of various sizes

In my solution I've implemented transaction-like methods of uploading posts media. For example I'm adding new post with photo. I want to have 700x700, 200x200, 70x70, 10x10 px sizes in my storage and ...
3
votes
2answers
2k views

Sending a file over a socket with AES encryption

I'm not sure what's the best way to deal with streams that need to be closed whenever I'm done using them? Usually the need to also catch the exception inside the ...
5
votes
1answer
1k views

User Avatar upload

I have been looking at some coding videos on Laracasts and the focus seems to be very readable code and keeping the controller clean. How can I clean this up? This code takes an uploaded user avatar, ...
3
votes
1answer
954 views

Resumable download with System.net.HTTPClient

This is a Delphi class, based on System.net.HTTPClient with a function for downloading a file from a URL and saving on a filename destination: ...
2
votes
0answers
1k views

Folder sharing with python sockets

I am just getting started with python. I wanted to make a folder transfer tool to get familiar with python sockets. ...
6
votes
2answers
7k views

Protocol implementation, TCP, sending images through sockets - follow-up

I would like to ask about your opinion about my code. The idea is simple: I designed my own protocol, where client asks server about the image, and the server sends the image, following the below ...
2
votes
1answer
810 views

Download pictures from Reddit

...
7
votes
1answer
77 views

A simple PyPI crawler

I had to make a program to crawl through a bunch of PyPI packages and see how many of them implement custom compare operators (i.e. grep for def __le__ etc). After ...
0
votes
1answer
99 views

Dynamic software update system using PHP

I have this code I have made to check after new release, when there is a new one it ask you for install it, and then installs it if you accept. The file is a zip file with the new folders and files ...
10
votes
1answer
1k views

Automating the download of a GitHub repo

This script is designed to locate and download the first repository in your sidebar on the GitHub repository page. It downloads the .zip file for that repo and saves it to the default location. I ...
14
votes
1answer
182 views

Download images from an image board based on tags and blacklists

This code is meant to download images from an image board to create a local backup. Images are grouped by tags selected by the user and then downloaded using multiprocessing. Normal use of this ...
7
votes
1answer
3k views

SFTP Ruby script for downloading files from server

I have written this Ruby script to download some file from server using SFTP. Help me improve this one. Find the original script at this link ...
2
votes
0answers
2k views

Sending ACK/NACK for packets

The code is working perfectly. I'm little bit sure this is a packet what I am sending from client to server in streams. Is this packet? Am I sending in a right way? Another thing is that I am sending ...
3
votes
0answers
207 views

Linux performance using multi-threads (vs one thread) and ssh message

My server receives requests in one thread and responds in another. I am using thread pools of pthread-s for both receiving and sending. The performance is not that bad, however... When I am doing the ...
4
votes
2answers
445 views

Basic PHP Cloud script

I would like to add basic Cloud system to my website project. I have my php login system already included. Every person must be logged in to view this site. There are several session variables like <...
3
votes
0answers
249 views

Uploading series of large files to API via Node

I wrote a node script to traverse a folder of hour-long mp3s and upload them to Mixcloud via their API. It works, but I suspect it's fairly inefficient - the computer it's going to run on at our radio ...
5
votes
3answers
334 views

Upload an image to a folder through a form

I am using this code to upload an image. Please review it and give your feedback regarding performance, security, and quality. This also works in PDO project. ...
5
votes
1answer
92 views

Tool to prepare custom screenshots for uploading to Steam Cloud

I've recently released my first ever Qt application, and would be glad if someone will look into it and tell me if there are some flagrant anti-patterns and likes. This grabs some screenshots (paths) ...
14
votes
2answers
796 views

VIM colors downloader in Python

Recently, I wanted to change my vim colors to something new. So I went to the vim colors website and then I decided that I wanted to download ALL the colors. So I ...
1
vote
1answer
2k views

FTP File Upload Queue

This code implements an FTPManager class and is designed to be a place to insert file data to be sent later, without having the program wait for a potentially very ...
1
vote
0answers
50 views

More testable file upload code

I am writing a backend with Node JS. It is my first Node JS application so I haven't learnt all the principles Node involves. I am writing the code for a media upload mechanism (to handle profile ...
6
votes
1answer
3k views

File downloader using Java, multithreading and HTTP Range request

I'm making a file downloader using Java. The project is on GitHub; I have included the most relevant excerpts here. I separate the file into parts and download them using HTTP Range request, with ...
3
votes
1answer
46 views

Restricted file upload to remote the Ruby way

I am using a Rails web app on the machine webapp, from which I call an action, that pushes a file to a remote server backend. ...
1
vote
0answers
104 views

File upload with Node streams

I'm implementing a file upload using NodeJS. I've got fully functional code, however, it feels overly complicated and I'm hoping there would be an easier way to implement the same thing? In ...
6
votes
1answer
79 views

Script to download sequentially named files, rename them, and delete smaller files

I've written a little script to download sequentially named files, rename them, and delete files smaller than an certain number of kilobytes. I came up with this but I'm not too happy. Any advice for ...
3
votes
1answer
114 views

Auto-upload tool

I created an automatic upload tool, which is working great (most of the time). As I want to improve the code I have several questions: Is it possible to make the iexplorer process smoother? I don't ...
4
votes
1answer
1k views

Get image from other site in PHP

I would like to get images from other websites. This is my current method but I'm not sure if it is alright. ...