Skip to content

Speed up tokenizing of a row in csv and xstrtod parsing #25784

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Mar 20, 2019
Merged

Speed up tokenizing of a row in csv and xstrtod parsing #25784

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3e0cd7f
Try to speed up COLITER_NEXT
vnlitvinov Feb 26, 2019
c157716
speed up parser's functions by register key word
anmyachev Feb 25, 2019
73edece
Speed up comparisons for special symbols during tokenizing csv row
vnlitvinov Mar 19, 2019
b4c2ca8
Speed up xstrtod by first computing integer part of a float number in…
vnlitvinov Mar 19, 2019
41d1747
Added whatsnew entry
vnlitvinov Mar 19, 2019
12a6df9
Fixed whatsnew entry per @jreback comment
vnlitvinov Mar 20, 2019
72c7570
Fine-tuning whatsnew entry for improved readability
vnlitvinov Mar 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added whatsnew entry
  • Loading branch information
@vnlitvinov
vnlitvinov committed Mar 20, 2019
commit 41d17472e21452f41811864f0003d96c79e0c8cc
3 changes: 2 additions & 1 deletion doc/source/whatsnew/v0.25.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,8 @@ Performance Improvements
- Improved performance of :meth:`Series.searchsorted`. The speedup is especially large when the dtype is
int8/int16/int32 and the searched key is within the integer bounds for the dtype (:issue:`22034`)
- Improved performance of :meth:`pandas.core.groupby.GroupBy.quantile` (:issue:`20405`)

- Improved performance of `tokenize_bytes` in `tokenizer.c`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also say :meth:`read_csv` a user has no idea what any of the other things are

- Improved performance of `xstrtod` on small floats (especially significant when integer part of it fits in int32)

.. _whatsnew_0250.bug_fixes:

Expand Down