Jason Bell

You use what is required for the job in hand. If you read CSV, expect to write to CSV. If you read XLS, then write out to XLS.

As soon as any compression format is compared with noncompressed then your scoring is immaterial. Compressed formats are uncompressed before processing so you'll always hit memory/time penalties.

And until today, thirty-three years into my career with data, I'd never heard of Pickle..... sounds like I don't need to either. CSV is fine.

--

--

Thanks for writing this post, very informative. Nice to see Lisp mentioned as they code structure means, as you rightly point out, rapidly prototype. Personally I use a mixture of Clojure (a lisp for the JVM), Java and Python.

Clojure gives me access to the JVM and all the Java libraries and any external jar libs (on Maven etc). Added with the libpython-clj library you have a very powerful AI/ML coding framework that leans on Python's excellent tooling but the JVM for balanced execution. Best of all worlds!

So yes, Lisp doesn't have the libraries, but Clojure certainly does!

Cheers

Jase

--

--

Jason Bell

Jason Bell

Author of Machine Learning: Hands on for Devs/TechProfs for Wiley. An OpenUK Ambassador and a Confluent Kafka Community Catalyst.