Python Threading Return Values

I am new to threading an I have existing application that I would like to make a little quicker using threading. I have several functions that return to a main Dict and would like

Solution 1:

You can push tuples of (worker, data) to queue to identify the source. Also please note that due to Global Interpreter Lock Python threading is not very useful. I suggest to take a look at multiprocessing module which offers interface very similiar to multithreading but will actually scale with number of workers.

Edit:

Code sample.

import multiprocessing as mp

# py 3 compatibilitytry:
    from future_builtins importrange, mapexcept ImportError:
    pass


data = [
    # input data# {split_config: ... }
]

defcrypto(split_config, asax, names):
    # your code herepassif __name__ == "__main__":
    terminate = mp.Event()
    input = mp.Queue()
    output = mp.Queue()


    defworker(id, terminate, input, output):
        # use event here to graciously exit# using Process.terminate would leave queues# in undefined statewhilenot terminate.is_set():
            try:
                x = input.get(True, timeout=1000)
                output.put((id, crypto(**x)))
            except Queue.Empty:
                pass

    workers = [mp.Process(target=worker, args=(i, )) for i inrange(0, mp.cpu_count())]
    for worker in workers:
        worker.start()

    for x in data:
        input.put(x)

    # terminate workers
    terminate.set()

    # process results# make sure that queues are emptied otherwise Process.join can deadlockfor worker in workers:
        worker.join()