3
\$\begingroup\$

I'm not familiar with how data transmission works, but I gave my best shot trying to code a server and client-side JSON packet transmitter via Python. Essentially client.py generates random JSON packets and sends them to server.py which binds a port using a socket and listens for a JSON packet, which then stores the JSON packet as a file to prove the transmission was successful.

One problem I am aware of is that within my client.py, the sock class is kept being created and closed through each cycle. Though, if I write it like

    while True:
        json_packet = generate_json_packet()
        send_json_packet(sock, json_packet)
        time.sleep(1)

Python emits the following: ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine after the first successful received packet.

Server.py

# Server-Side that receives json packets from client over the network using port 5000. Then it saves the json packet to a file with the filename of current time.
import socket
import json
import random
import string


# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)


# Bind the socket to the port
server_address = ('localhost', 5000)
print('starting up on {} port {}'.format(*server_address))
sock.bind(server_address)


# Listen for incoming connections
sock.listen(1)


# Function to receive json packets from client
def receive_json(conn):
    data = conn.recv(1024)
    data = data.decode('utf-8')
    data = json.loads(data)
    return data


# Saves json packets to file and name it with id


def save_json(data):
    filename = ''.join(random.choice(
        string.digits) for _ in range(3))
    filename = filename + '.json'
    with open(filename, 'w') as f:
        json.dump(data, f)


if __name__ == '__main__':
    while True:
        # Wait for a connection
        print('waiting for a connection')
        connection, client_address = sock.accept()
        try:
            print('connection from', client_address)

            # Receive the data in json format
            data = receive_json(connection)
            print('received {!r}'.format(data))

            # Save the json packet to a file
            filename = save_json(data)
            print('saved to', filename)

        finally:
            # Clean up the connection
            connection.close()
# Sends random json packets to server over port 5000

import socket
import json
import random
import time


def generate_json_packet():
    # Generate random json packet with hashed data bits
    return {
        "id": random.randint(1, 100),
        "timestamp": time.time(),
        "data": hash(str(random.randint(1, 100)))
    }


# Send json packet to server
def send_json_packet(sock, json_packet):
    sock.send(json.dumps(json_packet).encode())


ip = "127.0.0.1"
port = "5000"


if __name__ == "__main__":

    while True:

        # Create socket
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        # Connect to server
        sock.connect((ip, int(port)))

        # Generate random json packet
        json_packet = generate_json_packet()
        # Send json packet to server
        send_json_packet(sock, json_packet)
        time.sleep(1)
        sock.close()
\$\endgroup\$
2

1 Answer 1

6
\$\begingroup\$

Your implementation is fatally flawed, and this is mostly not your fault: sockets are non-intuitive. First, when you say

receives json packets

that's not in the slightest what you're doing. You're sending JSON messages that happen to usually fit in one packet, and this coincidence is unreliable and will break under many, many common conditions. This trips up nearly every beginner socket programmer. You need to almost forget about packets entirely, assume that your message may be segmented into an arbitrary number of packets returned to the listener in a series of arbitrarily-sized byte buffers. This in turn means that you cannot rely on packet boundaries to be message boundaries. You need to define your own message boundary. One easy message boundary is a newline, and Python makes it easy to use a StreamRequestHandler that listens for a newline regardless of how many packets were received. So, to stabilise your implementation,

  • Call them JSON messages, not JSON packets
  • Use sendall, not send
  • Don't use socket at all on the server side; instead use StreamRequestHandler and TCPServer
  • Use rfile.readline() to guarantee a stable message boundary

Other assorted points:

  • file with the filename of current time is a lie
  • The server should not need to go through a JSON round-trip, and can just write the bytes out to a file - if you don't care about validating the data
  • You should be using with context management to guarantee socket closure

Suggested

Server:

"""
Server-Side that receives json packets from client over the network using port
5000. Then it saves the json packet to a file with the filename of current time.
"""
import random
import string
from socketserver import StreamRequestHandler, TCPServer


def save_json(data: bytes) -> None:
    """Saves json packets to file and name it with id"""
    filename = ''.join(
        random.choice(string.digits)
        for _ in range(3)
    ) + '.json'
    with open(filename, 'wb') as f:
        f.write(data)
    print('saved to', filename)


class DumpHandler(StreamRequestHandler):
    def handle(self) -> None:
        """receive json packets from client"""
        print('connection from {}:{}'.format(*self.client_address))
        try:
            while True:
                data = self.rfile.readline()
                if not data:
                    break
                print('received', data.decode().rstrip())
                save_json(data)
        finally:
            print('disconnected from {}:{}'.format(*self.client_address))


def main() -> None:
    server_address = ('localhost', 5000)
    print('starting up on {}:{}'.format(*server_address))
    with TCPServer(server_address, DumpHandler) as server:
        print('waiting for a connection')
        server.serve_forever()


if __name__ == '__main__':
    try:
        main()
    except KeyboardInterrupt:
        pass

Client:

"""Sends random json packets to server over port 5000"""

import socket
import json
from random import randint
from time import time, sleep
from typing import Dict, Any

IP = "127.0.0.1"
PORT = 5000


def generate_json_message() -> Dict[str, Any]:
    # """Generate random json packet with hashed data bits"""
    return {
        "id": randint(1, 100),
        "timestamp": time(),
        "data": hash(str(randint(1, 100)))
    }


def send_json_message(
    sock: socket.socket,
    json_message: Dict[str, Any],
) -> None:
    """Send json packet to server"""
    message = (json.dumps(json_message) + '\n').encode()
    sock.sendall(message)
    print(f'{len(message)} bytes sent')


def main() -> None:
    with socket.socket() as sock:
        sock.connect((IP, PORT))
        while True:
            json_message = generate_json_message()
            send_json_message(sock, json_message)
            sleep(1)


if __name__ == "__main__":
    try:
        main()
    except KeyboardInterrupt:
        pass
\$\endgroup\$
5
  • \$\begingroup\$ Why should the program ignore keyboard interruptions? \$\endgroup\$ Oct 30, 2021 at 1:54
  • 1
    \$\begingroup\$ @MiguelAlorda Because that's the typical way to cancel a program - Ctrl+C. There's no value in showing the stack trace. \$\endgroup\$
    – Reinderien
    Oct 30, 2021 at 2:24
  • \$\begingroup\$ Thank you @Reinderien! I do have some convention question that I want to ask you: why have you enunciated -> Dict[str, Any] or -> None for specific functions? I also see you skipping parsing raw data to json and directly saved it into a file. Does this work on every filetype? \$\endgroup\$
    – Joe
    Oct 30, 2021 at 7:59
  • 1
    \$\begingroup\$ @Joe Those are type hints compliant with python.org/dev/peps/pep-0484. They are very useful to tell yourself and other programmers the types of your variables, method parameters and returns; and to verify the correctness of your program using tools like mypy. \$\endgroup\$
    – Reinderien
    Oct 30, 2021 at 17:17
  • 1
    \$\begingroup\$ @Joe Yes, skipping a parse & re-serialize is possible for all data formats so long as they can enter and exit in bytes format and so long as you don't care to validate the data. \$\endgroup\$
    – Reinderien
    Oct 30, 2021 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.