I have a large object that I want to save to the disk because it takes a minute to generate. The OOM reaper kills the process while pickle.dump ing the object.

It's a tuple of dicts of tuple of array.array.

Can pickle dump in chunks? If not, is there another technique I can use?

Some neat tricks for using Makefiles with Python that I hadn't seen before. The one I have seen, having a help target, has been super useful in team environments and for debugging when used with variables.

SciPy 2023 is looking for reviewers

(there is a discussion about this post here https://news.ycombinator.com/item?id=34787092 ...)

Here is the paper: https://arxiv.org/abs/2209.06909

Here is tbe code: https://github.com/sebawild/powersort

what's new

Build amazing TUIs (Text User Interfaces) with this innovative Python framework

cross-posted from: https://feddit.de/post/248196

I have lots of multiprocessing processes which have to add to and search in a dict. Deletion of values is not needed.

Atm I am using multiprocessing.Manager() and

dict = manager.dict()

This works pretty well, but I think that the manager is a huge bottleneck here. Any ideas? It has to run on older Python 3 versions, otherwise I would use this cool thing I found: https://github.com/ronny-rentner/UltraDict

via https://news.ycombinator.com/item?id=31637428

via https://news.ycombinator.com/item?id=31566031

source: https://github.com/satwikkansal/wtfpython

via https://news.ycombinator.com/item?id=31476521