Jason Bell
Jul 26, 2021

You use what is required for the job in hand. If you read CSV, expect to write to CSV. If you read XLS, then write out to XLS.

As soon as any compression format is compared with noncompressed then your scoring is immaterial. Compressed formats are uncompressed before processing so you'll always hit memory/time penalties.

And until today, thirty-three years into my career with data, I'd never heard of Pickle..... sounds like I don't need to either. CSV is fine.

Jason Bell

A polymath of ML/AI, expert in container deployments and engineering. Author of two machine learning books for Wiley Inc.