I’ve been a long-time Pandas user, relying on it heavily since the start of my data science career. However, up until the last couple of years, I struggled with certain issues, such as not being able to work with very large DataFrames or efficiently run heavy data processing tasks. I’d also often find my Jupyter notebooks cluttered with intermediate DataFrames after applying transformations, making it harder to read the code and keep my notebook tidy. For a long time, I had thought that these issues were just endemic to Pandas and accepted there wasn’t a better way; however, there is!
If yo