I'm working on a dataset from a MOOC. I have a lot of python3 code snippets that I need to run and get the results from. To do this I've written a python script that loops over each snippet. For each snippet I:
- Create new StringIO objects
- Set
sys.stdout
andsys.stderr
to my stringIO buffers - Execute the code snippet in a
threading.thread
object - Join the thread
- Log the results in the stringIO buffers
- Restore stdout and stderr
This works fine for "correct" code, but this has undesired side effects in other cases:
- When the code has an infinite loop, thread.join doesn't kill the thread. The thread itself is a daemon thread, so it runs quietly in the background until my loop finishes.
- When the code has an infinite loop with a
print()
, the thread starts overwriting my actual stdout when I set it back to the default (away from the StringIO buffer). This pollutes my reporting.
Here is my current code:
def execCode(code, testScript=None):
# create file-like string to capture output
codeOut = io.StringIO()
codeErr = io.StringIO()
# capture output and errors
sys.stdout = codeOut
sys.stderr = codeErr
def worker():
exec(code, globals())
if testScript:
# flush stdout/stderror
sys.stdout.truncate(0)
sys.stdout.seek(0)
# sys.stderr.truncate(0)
# sys.stderr.seek(0)
exec(testScript)
thread = threading.Thread(target=worker, daemon=True)
# thread = Process(target=worker) #, stdout=codeOut, stderr=codeErr)
thread.start()
thread.join(0.5) # 500ms
execError = codeErr.getvalue().strip()
execOutput = codeOut.getvalue().strip()
if thread.is_alive():
thread.terminate()
execError = "TimeError: run time exceeded"
codeOut.close()
codeErr.close()
# restore stdout and stderr
sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__
# restore any overridden functions
restoreBuiltinFunctions()
if execError:
return False, stripOuterException(execError)
else:
return True, execOutput
To handle the undesirable cases, I've tried to use multithreading.Process
and/or contextlib.redirect_stdout
to run the code in a process (then I can call process.terminate()
), but I'm not having any success capturing stdout/stderr.
So my question is: What can I do to make this better/more robust to handle bad code snippets?
(And yes, I know this is a bad idea in general; I'm running it in a virtual machine just in case there is malicious code in there somewhere)
Python version is 3.5.3
Update
It occurs to me that there is a little more flexibility in this situation. I have a function, preprocess(code)
that accepts a the code submission as a string and alters it. Mostly I've been using it to swap out the value of some variables using regular expressions.
Here is an example implementation:
def preprocess(code):
import re
rx = re.compile('earlier_date\s*=\s*.+')
code = re.sub(rx, "earlier_date = date(2016, 5, 3)", code)
rx = re.compile('later_date\s*=\s*.+')
code = re.sub(rx, "later_date = date(2016, 5, 24)", code)
return code
I could use the preprocess function to help redirect STDOUT