Questions tagged [network-file-transfer]

Questions relating to the copying of files from one filesystem, over a network, to a destination filesystem

Filter by
Sorted by
Tagged with
17 votes
2 answers
21k views

Python Back-Up Script

I wrote this script to backup some important files. It backs up the files to a local folder as well as to an external hard drive. It creates a new subdirectory which has its name constructed with ...
asteri's user avatar
  • 4,889
15 votes
0 answers
241 views

Download files in parallel

This module has a ensure_downloaded function which takes a slice of Urls. It downloads all of the urls to a local downloads ...
Winston Ewert's user avatar
14 votes
2 answers
842 views

VIM colors downloader in Python

Recently, I wanted to change my vim colors to something new. So I went to the vim colors website and then I decided that I wanted to download ALL the colors. So I ...
Kernel.Panic's user avatar
14 votes
1 answer
227 views

Download images from an image board based on tags and blacklists

This code is meant to download images from an image board to create a local backup. Images are grouped by tags selected by the user and then downloaded using multiprocessing. Normal use of this ...
Sean's user avatar
  • 241
13 votes
1 answer
3k views

Object-oriented file upload handler for a CMS

I've started with a bunch of functions, made the code work, then I spent a day rewriting it in an OO way, as much as I can. I've tried to group up the task into objects, letting the relative data go ...
myNameIsTheo's user avatar
10 votes
5 answers
6k views

Resumable HTTP download class

Any suggestions, corrections and advice would be very nice and much appreciated. ...
Expenzor's user avatar
  • 227
10 votes
1 answer
51k views

Java multi-thread file server and client

My task was to build a multi-thread file server and a client that can upload or download a named file over sockets. It is assumed that the client will finish after its operation and there is no need ...
chris loughnane's user avatar
10 votes
1 answer
2k views

Automating the download of a GitHub repo

This script is designed to locate and download the first repository in your sidebar on the GitHub repository page. It downloads the .zip file for that repo and saves it to the default location. I ...
Der Kommissar's user avatar
9 votes
3 answers
17k views

Sending a file in servlet response, using a class inside a method

I am processing file inside my method - basically preparing it for the download. But I decided to split actions for setting headers and reading bytes. As Java does not support method declaration ...
lapots's user avatar
  • 347
8 votes
1 answer
190 views

Bad code to handle image upload

I'm using this form to submit articles into the database: ...
user avatar
7 votes
1 answer
6k views

SFTP Ruby script for downloading files from server

I have written this Ruby script to download some file from server using SFTP. Help me improve this one. Find the original script at this link ...
Ashish Wadekar's user avatar
7 votes
1 answer
7k views

Download a file from given URL and retry on connection errors

I wrote a script for Python 2.78 which is supposed to download and save a file from a given URL using the requests library. In case that a connection to the server can be established and a valid ...
Dirk's user avatar
  • 173
7 votes
2 answers
108 views

Multi-site backup utility with basic sanity checks

We have a large set of projects at work. It's very important we keep track of those and every so often we need to run a full back-up that's unrelated to the 'normal' back-ups IT is running (snapshots ...
Mast's user avatar
  • 13.4k
7 votes
1 answer
7k views

File downloader using Java, multithreading and HTTP Range request

I'm making a file downloader using Java. The project is on GitHub; I have included the most relevant excerpts here. I separate the file into parts and download them using HTTP Range request, with ...
Quan Tran's user avatar
7 votes
1 answer
132 views

A simple PyPI crawler

I had to make a program to crawl through a bunch of PyPI packages and see how many of them implement custom compare operators (i.e. grep for def __le__ etc). After ...
Elliot Gorokhovsky's user avatar
7 votes
2 answers
12k views

Protocol implementation, TCP, sending images through sockets - follow-up

I would like to ask about your opinion about my code. The idea is simple: I designed my own protocol, where client asks server about the image, and the server sends the image, following the below ...
yak's user avatar
  • 223
6 votes
2 answers
1k views

Servlet file uploader

I think this code looks ugly, especially with the multiple try-catch blocks, but I don't know how to rewrite it: ...
gstackoverflow's user avatar
6 votes
1 answer
266 views

Archives xkcd comics

This is a work in progress, as you may read I plan to revise to utilise JSON data rather than directly scraping. I am considering refactoring the try//...
toonarmycaptain's user avatar
6 votes
2 answers
353 views

PHP readfile() with 304 Not Modified and HTTP 206 Partial Content support: httpreadfile

I wanted a readfile with 304 Not Modified and 206 Partial Content support, I think i got right, but probably spent way more effort on error-checking than worthwhile.. Also I didn't develop it by ...
hanshenrik's user avatar
6 votes
1 answer
18k views

Downloading all files in a FTP folder and then deleting them

I've created two methods in a class that allow me to download the contents of an FTP folder and if specified then delete them. Although the code operates as intended I think it's vastly bloated and I'...
Michael A's user avatar
  • 1,641
6 votes
1 answer
3k views

Increase speed of Google Drive upload for large files

I'm using Google Drive API for uploading large disk image files ( >100 GB). My Code uses this wrapper to upload these large files in chunks of 40mb. The files are stored on NAS drive. The code works ...
Brendan Gooden's user avatar
6 votes
1 answer
1k views

Multiple file upload class

I've written this a few days back, which is not too big. Do you see anywhere I can improve my logic? ...
Jesse's user avatar
  • 205
6 votes
1 answer
156 views

Adapter pattern to support multiple file storage systems

I am just getting into the following design patterns. I have created a class that allows for performing a couple of operations on different remote storage systems: drive, dropbox, .etc (I call these <...
Alex's user avatar
  • 223
6 votes
1 answer
96 views

Script to download sequentially named files, rename them, and delete smaller files

I've written a little script to download sequentially named files, rename them, and delete files smaller than an certain number of kilobytes. I came up with this but I'm not too happy. Any advice for ...
BashN00b's user avatar
6 votes
1 answer
331 views

Java batch movie downloader

The idea is to batch download a list of movies (torrents) off a torrent site and add them to your server. I have a little bit of Java experience (sophomore in college), so I'm looking for things that ...
Ben's user avatar
  • 161
6 votes
1 answer
193 views

Copying files as fast as possible

I am running my shell script on machineA which copies the files from machineB and machineC ...
arsenal's user avatar
  • 2,167
6 votes
1 answer
2k views

A PowerShell script like one-way rsync but worse

This PowerShell script works a bit like a one-way rsync; it's good enough for my Surface RT needs, anyway. However, seeing as this is my first PS script, and I was ...
michaelb958--GoFundMonica's user avatar
5 votes
3 answers
699 views

Upload an image to a folder through a form

I am using this code to upload an image. Please review it and give your feedback regarding performance, security, and quality. This also works in PDO project. ...
USer345738380's user avatar
5 votes
2 answers
697 views

Uploading a file to the server

I wrote a simple PHP script which allows the user to upload a file to the server. It works, but I'm not very sure about security. Could you give me some hints about security mistakes? In a nutshell: ...
Rogach's user avatar
  • 397
5 votes
3 answers
204 views

Python 3 Multi-connection downloader

I have written a fully functional simple command line multi-connection downloader, written in Python 3, using just threading, ...
Ξένη Γήινος's user avatar
5 votes
2 answers
361 views

Bash script to download sequentially numbered images

This is a pretty basic bash script (3.2 on Mac). I am downloading 584 images from a site in order to create an album. ...
enderland's user avatar
  • 1,726
5 votes
1 answer
1k views

User Avatar upload

I have been looking at some coding videos on Laracasts and the focus seems to be very readable code and keeping the controller clean. How can I clean this up? This code takes an uploaded user avatar, ...
Jeffrey's user avatar
  • 181
5 votes
1 answer
2k views

Checking file size and sending it to the server

I am currently working on a crash log sender. As the application crashes, the FindLogFiles() will search for the different crash logs we are maintaining. It will ...
MX D's user avatar
  • 545
5 votes
1 answer
26k views

Java multithreaded file server and client - emulate TCP over UDP

My task was similar to my last assignment but this time I had to do it with UDP instead of TCP. This basically means I had to emulate TCP over UDP. Multithreading ...
chris loughnane's user avatar
5 votes
1 answer
3k views

Simple Java Download Manager

I have written a simple download accelerator in java which downloads the same file in multiple threads and then combines them all. I would love some feedback on my design. How the same implementation ...
Gnik's user avatar
  • 882
5 votes
1 answer
131 views

Tool to prepare custom screenshots for uploading to Steam Cloud

I've recently released my first ever Qt application, and would be glad if someone will look into it and tell me if there are some flagrant anti-patterns and likes. This grabs some screenshots (paths) ...
Neurotransmitter's user avatar
5 votes
2 answers
352 views

Image download handler app

I have a paid app on the app engine. At the moment, usually there is only one instance. The app has a download handler that sends an image after re-sizing it if it is too big. The problem is: if a web ...
user164483's user avatar
5 votes
0 answers
916 views

Uploading captured video to Google Cloud Storage

My program intends to capture video streams (in mjpeg) with OpenCV and upload the captured frames into Google Cloud Storage for later processing. I am expecting to capture ~15-20 frames per second and ...
Rex Low's user avatar
  • 151
4 votes
1 answer
1k views

Get image from other site in PHP

I would like to get images from other websites. This is my current method but I'm not sure if it is alright. ...
Tamás's user avatar
  • 141
4 votes
2 answers
7k views

Bash CGI Upload File

I'm using the following Bash CGI to upload a file: ...
Juicy's user avatar
  • 481
4 votes
1 answer
3k views

Sending an XML file to an SFTP site

I have a console application that basically retrieves XML content, writes to the XML file and sends the file to an SFTP site: ...
Sunny's user avatar
  • 369
4 votes
1 answer
486 views

Cloud storage using python Socket library

One year ago I learned some Python 3 in Highschool, but I haven't really programmed anything since then. I decided to start a new project to try and practice a bit before entering the first year of an ...
Lucas Labouret's user avatar
4 votes
2 answers
516 views

Basic PHP Cloud script

I would like to add basic Cloud system to my website project. I have my php login system already included. Every person must be logged in to view this site. There are several session variables like <...
allin0n3's user avatar
4 votes
1 answer
34k views

Program to transfer files from one server to another in java and also display a progress bar on each file transfer

I have written a java code to transfer files from one server to another using the concept of socket programming. Now, I also want to check for the network connection availability before each file is ...
trisha mehta's user avatar
4 votes
1 answer
1k views

Uploading multiple files for faculty nomination system

I created a few forms for faculty members to nominate colleagues for awards. Many of the nominations require documentation to be submitted in order to validate the nomination. I created one form then ...
aparker81's user avatar
4 votes
1 answer
89 views

PHP remote upload security

I need to upload images to a remote HTTP server from a local script. The image name format is {32-char MD5}.jpg. Server directory format is ...
user9593274's user avatar
4 votes
1 answer
2k views

Automatically backup website files and database

For each of my various websites, I back up the web site files and the associated database for into a single dated compressed archive file (one zip file per website) and download that to my local ...
User's user avatar
  • 275
4 votes
1 answer
82 views

Simple shell script to backup to Amazon over iSCSI

The script works but I know that it can be improved. Please note that my scripting skills are very basic :) (as it can be seen in my Frankenstein script – I took bits and pieces of the code from ...
Pete's user avatar
  • 43
4 votes
1 answer
779 views

Check if an image download from the Internet has already started

In the method getView() in my implementation of ArrayAdapter I have to download image from Internet using the image's ...
Leonid Semyonov's user avatar
4 votes
0 answers
393 views

Linux performance using multi-threads (vs one thread) and ssh message

My server receives requests in one thread and responds in another. I am using thread pools of pthread-s for both receiving and sending. The performance is not that bad, however... When I am doing the ...
Eliad Cohen's user avatar