submitted 3 years ago by lntl@lemmy.ml to c/python@lemmy.ml

9 comments fedilink hide all child comments

I have a large object that I want to save to the disk because it takes a minute to generate. The OOM reaper kills the process while pickle.dump ing the object.

It's a tuple of dicts of tuple of array.array.

Can pickle dump in chunks? If not, is there another technique I can use?

you are viewing a single comment's thread
view the rest of the comments

[-] lntl@lemmy.ml 2 points 3 years ago

It's a tuple of dicts of tuples of array.array, no numpy or torch :(

permalink
fedilink
source
parent
hideshow 2 child comments

[-] alehc@lemmy.ml 3 points 3 years ago* (last edited 3 years ago)

So python standard library lists?

permalink
fedilink
source
parent

this post was submitted on 09 Jun 2023

6 points (87.5% liked)

Python

3639 readers

1 users here now

News and discussions about the programming language Python

founded 7 years ago

MODERATORS

MinutePhrase@lemmy.ml