# Tag Info

11

Some suggestions: The trailing slash in the mkdir command is redundant. $(…) is preferred over backticks for command substitution. Why use seq in one command? They both do the same loop, so you might as well use {1..100} in both places. Semicolons are unnecessary in the vast majority of cases. Simply use a newline to achieve the same separation between ... 10 Just a few quick comments You can use a dictionary as mapping file First you check the length of the bytes and use an temp variable 1, 2, 3, 4, 5, 6 to know which hashing algo to use lenght = len(hash) if lenght == 32: # MD5 hashmethod_list.append(1) ... if hashmethod_list[hashline] == 1: hashedguess = hashlib.md5(bytes(word, "utf-8")).... 9 The obligatory other solution solving your life: def timer(f): def wrapper(job_args, *args, **kwargs): fn_args, timeout, timeout_callback = job_args[:3] q = Queue() p = Process(target=f, args=(q, fn_args), kwargs=kwargs) p.start() p.join(timeout=timeout) p.terminate() p.join() if not q.... 9 1. Introduction From a practical point of view, the most important points are that Python has batteries included: To run tasks in a pool of worker threads, use concurrent.futures.ThreadPoolExecutor. But if what you really want to do is run external programs via the shell, as suggested by the use of os.system, then you don't need threads at all! Use ... 9 This covers an interesting topic. Great work! Because I am unfamiliar with this area, I utilized your unit testing to ensure changes I make did not break functionality. If they do, I apologize and please let me know. Utilize implicit repetition and iota Rather than manually defining the type and value of nCode, aCode, etc. we an implicitly get the value ... 7 Are there any issues with the code that I should be aware of? (besides from what I already said) If you haven't, maybe read Everything you never wanted to know about file locking; I guess the takeaway is that you should make sure that you're actually getting what you want with the Python fcntl module. The code looks good to me in general, apart from the ... 7 1. Summary of the problem The problem, as described in the post, is to construct an undirected graph with numbered nodes, such that for every $0 \le k < 2^n$ there is a pair of nodes in the graph $a, b$ such that one of the shortest routes from $a$ to $b$ is $a=a_1, a_2, \ldots, a_n=b$, and$$a_i \equiv \left\lfloor {k \over 2^{n-i}} \right\... 6 Here's a condensed illustration of how to achieve your stated purpose, namely to compute the outputs of a generator in parallel. I offer it because I could not understand the purpose of most of the complexity in your current code. I suspect there are issues that you have not explained to us or that I failed to infer (if so, this answer might be off the mark).... 5 Algorithmically speaking, there is no better way to find a matching password without considering characteristics of the password itself. For instance, if you take into account the length, you could restrict the generated passwords to be only of that length. If you consider the actual characters, then well, you already have the password, so you're done. ... 5 I can see two reasons why your program might be slow. The first one is that for each row you allocate memory and then free it. If you move the allocation outside of the while loop and reuse the memory then this will get rid of a whole bunch of unnecessary memory allocation operations. The second one is probably more important: MPI stands for Message ... 5 Parallelization is not necessarily implemented nicely in R. However, it is far more ideal to use R's batch process than opening 10x Rstudio sessions as you saw (less of a resource drain per task). Cores, cores, where art thou? The first thing I would do is find out how many cores you have access to. Within this script 4 seem to be allocated. It is very ... 5 Prefer generators instead of lists build_queue creates a list of tasks up front. It's unnecessary to store all task details in memory up front. You could use a generator instead, and yield the task parameters. That will minimize the memory use, by generating the task details just before the execution of the individual tasks. Avoid busy loops This piece of ... 5 Use multiprocessing.Pool The multiprocessing library already has a worker pool implementation ready to be used. Your code could be rewritten as: import time from multiprocessing import Pool def f(x): time.sleep(random.uniform(0.01, 0.5)) return x * 12 if __name__ == "__main__": c = Pool(2) for i in range(5): n = random.... 4 Is there anything i can do in these methods to get any boost in performance? Yes, You can reimplement them, using the 'memory-mapped files' (MMf) to solve both problems of parallel concurrency and access sharing. using (MemoryMappedFile file = MemoryMappedFile.CreateFromFile("test.bin")) using (MemoryMappedViewStream stream = file.CreateViewStream()) { ... 4 Good job using ArgumentParser. It is a very, very nice tool that should be used more often in Python. Near the top of your code where you are doing a bunch of checks on parameters, you have a sys.exit call if the user did something wrong when entering parameters. However, you don't pass in any exit number. The method sys.exit takes a single parameter ... 4 Do what a real cracker would do. Cheat. Here's a list of the most common passwords. Load the list and cycle through them until you have a match, or have exhausted the list. If you don't find a match, then you can fall back to the brute force approach. This is called a dictionary attack, by the way. Another approach would be to make your program "... 4 Using GNU Parallel you code will look something like this: #!/usr/bin/env bash export script="path_to_python_script" doit() { i="$1" j="$2"$script -args_1 "$i"$script -args_1 "\$i" -args_2 value -args_3 value } export -f doit parallel --resume --results data/file_{1}-{2}.txt doit ::: {1..100} ::: {1..100} In your original code if one ...

4

Here are some observations and things to consider. Are you sure you need multiprocessing or threads? There isn't any information in the question to say why they may be needed. There is overhead for using them. Perhaps an input-compute-output loop is sufficient. Do you anticipate the program to have throughput limited by IO or by CPU processing. The general ...

3

Oh! My first impression is that you felt in love with dictionaries - your multilevel dictionary ClassDic (which name should be class_dic by PEP 8 - Style Guide for Python Code) is something horrible! Why? Mainly because it so contradict the DRY principle (Don't Repeat Yourself). The one set of the keys used completely again and again - so are they ...

3

private final String pingMessage = "A\n"; private final int timeOut = 250;// ms I don't expect these change anytime soon or that they can be different between different instances of your TCPClient. Make them static. private ScheduledThreadPoolExecutor tcpPool = new ScheduledThreadPoolExecutor(5); I'd expect: private final ScheduledExecutorService ...

3

def fileparser_worker(filename, start, end, c): with open(filename) as inFile, open(outName + str(c),'w') as outFile: inFile.seek(start) #lines = inFile.readlines(end-start) because readlines calls readline multiple times it can be replaced with read. If the text should be split by newline and each split line should be ...

3

At a glance, the MPI code specifically doesn't look too bad. The processes start and stop as needed, and you don't appear to accidentally have a process freeing the wrong memory. As I'm not familiar with this fractal, I'll just comment on the MPI and some general best practices. You have STOP as a global variable. Try to keep it local so that it won't be ...

3

Better organisation You program in its current state is one huge chunk of snippet with all logic inside it. You should consider splitting it into separate smaller functions. The limits on $x, y, z$ are preset. Consider putting them as a GLOBAL_CONSTANT. Ideal import order (according to PEP8) is standard library imports related third party imports local ...

3

First things first: while nloops<=llno: if __name__ == '__main__': would make more sense with the order reversed; and def f(l1,l2, q, r, s): ... q.put(distance) r.put((node1a, node1b)) s.put((node2a, node2b)) while nloops<=llno: if __name__ == '__main__': q = Queue() r = Queue() s = Queue() ...

3

To address your immediate concerns, the code is safe. Each worker operates on its own (sub)set of data. Nothing is really shared, so there is no data race. It is unclear why the worker function calls tonumpyarray on the objects which already are numpy arrays. I strongly advise against using random numbers in testing. It is pretty much impossible to say ...

3

Your docstrings look a bit weird to me. The first line is usually a summary of what the function does. In your first function that line comes after the parameters and in the second one it does not exist at all. In get_stats when describing the parameter symbols there is no need to describe what the function internally does with it (i.e. split it up). This is ...

3

Style Please read PEP8 and use a consistent style throughout your code: put a space after comas; use a linebreak after colons; use UPPERCASE names for constants… Using locks First off, locks support the context manager protocol and you can thus simplify your printer to: def printer(lock, data): with lock: print(data) which may not warant a ...

3

Mutable default arguments sil_opts:dict = default_sil_options This can create some nasty bugs, and some Python inspectors specifically recommend against doing this. The alternative is to set the default to None, and then write an if in your function to assign the default dictionary if the argument is None. Redundant return It seems like you intend for ...

3

Constant reuse Store your base URL, "https://www.example.com", in a constant for reuse. HTTP codes Some status codes, such as 200, are common and obvious, while others such as 429 are less so. Fundamentally these are all magic numbers, and requests.codes contains symbols - including too_many_requests. URL construction allposts = [f'https://...

3

here's a few comments in no particular order. Have a look over PEP8. It's Python's style guide which helps keep Python code consistent and readable. Things like naming functions which snake_case, constants as CAPS, proper spacing around operators, commenting style and placement, spacing between imports, etc. It's recommended to use a context manager for ...

Only top voted, non community-wiki answers of a minimum length are eligible