Skip to content

Instantly share code, notes, and snippets.

@js8544
Created October 23, 2023 07:02
Show Gist options
  • Save js8544/d64f289313c814a2df6a87a945a0b382 to your computer and use it in GitHub Desktop.
Save js8544/d64f289313c814a2df6a87a945a0b382 to your computer and use it in GitHub Desktop.
ARROW PR 38394: Dictionary ChunkedArray hash kernels optimization benchmark
### Before
-----------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
-----------------------------------------------------------------------------------------
ValueCountsDictionaryChunks/0 8112938 ns 8112013 ns 87 bytes_per_second=4.81539Gi/s items_per_second=517.048M/s null_percent=0 num_unique=100
ValueCountsDictionaryChunks/1 8563428 ns 8563225 ns 82 bytes_per_second=4.56166Gi/s items_per_second=489.804M/s null_percent=0.1 num_unique=100
ValueCountsDictionaryChunks/2 9591335 ns 9590370 ns 72 bytes_per_second=4.0731Gi/s items_per_second=437.345M/s null_percent=1 num_unique=100
ValueCountsDictionaryChunks/3 12068706 ns 12066611 ns 55 bytes_per_second=3.23724Gi/s items_per_second=347.596M/s null_percent=10 num_unique=100
ValueCountsDictionaryChunks/4 22068866 ns 22068960 ns 32 bytes_per_second=1.77002Gi/s items_per_second=190.054M/s null_percent=50 num_unique=100
ValueCountsDictionaryChunks/5 10474824 ns 10474596 ns 66 bytes_per_second=3.72926Gi/s items_per_second=400.426M/s null_percent=99 num_unique=100
ValueCountsDictionaryChunks/6 8641591 ns 8641564 ns 81 bytes_per_second=4.5203Gi/s items_per_second=485.364M/s null_percent=100 num_unique=100
ValueCountsDictionaryChunks/7 537463756 ns 537447624 ns 1 bytes_per_second=74.4259Mi/s items_per_second=7.80412M/s null_percent=0 num_unique=100k
ValueCountsDictionaryChunks/8 506843630 ns 506775330 ns 1 bytes_per_second=78.9304Mi/s items_per_second=8.27646M/s null_percent=0.1 num_unique=100k
ValueCountsDictionaryChunks/9 526191657 ns 526113873 ns 1 bytes_per_second=76.0292Mi/s items_per_second=7.97224M/s null_percent=1 num_unique=100k
ValueCountsDictionaryChunks/10 533948000 ns 533932498 ns 1 bytes_per_second=74.9158Mi/s items_per_second=7.85549M/s null_percent=10 num_unique=100k
ValueCountsDictionaryChunks/11 485905496 ns 485851582 ns 2 bytes_per_second=82.3297Mi/s items_per_second=8.63289M/s null_percent=50 num_unique=100k
ValueCountsDictionaryChunks/12 54465167 ns 54459173 ns 12 bytes_per_second=734.495Mi/s items_per_second=77.0174M/s null_percent=99 num_unique=100k
ValueCountsDictionaryChunks/13 8587787 ns 8587147 ns 81 bytes_per_second=4.54895Gi/s items_per_second=488.44M/s null_percent=100 num_unique=100k
### After
-----------------------------------------------------------------------------------------
Benchmark Time CPU Iterations UserCounters...
-----------------------------------------------------------------------------------------
ValueCountsDictionaryChunks/0 7966614 ns 7966261 ns 91 bytes_per_second=4.90349Gi/s items_per_second=526.509M/s null_percent=0 num_unique=100
ValueCountsDictionaryChunks/1 8320302 ns 8320284 ns 85 bytes_per_second=4.69485Gi/s items_per_second=504.106M/s null_percent=0.1 num_unique=100
ValueCountsDictionaryChunks/2 9323569 ns 9323436 ns 75 bytes_per_second=4.18971Gi/s items_per_second=449.867M/s null_percent=1 num_unique=100
ValueCountsDictionaryChunks/3 12000566 ns 11998826 ns 59 bytes_per_second=3.25553Gi/s items_per_second=349.56M/s null_percent=10 num_unique=100
ValueCountsDictionaryChunks/4 21641714 ns 21640423 ns 32 bytes_per_second=1.80507Gi/s items_per_second=193.818M/s null_percent=50 num_unique=100
ValueCountsDictionaryChunks/5 10142759 ns 10142761 ns 67 bytes_per_second=3.85127Gi/s items_per_second=413.527M/s null_percent=99 num_unique=100
ValueCountsDictionaryChunks/6 8591046 ns 8590837 ns 81 bytes_per_second=4.547Gi/s items_per_second=488.23M/s null_percent=100 num_unique=100
ValueCountsDictionaryChunks/7 134303597 ns 134301041 ns 5 bytes_per_second=297.838Mi/s items_per_second=31.2306M/s null_percent=0 num_unique=100k
ValueCountsDictionaryChunks/8 137137643 ns 137135458 ns 5 bytes_per_second=291.682Mi/s items_per_second=30.5851M/s null_percent=0.1 num_unique=100k
ValueCountsDictionaryChunks/9 136837967 ns 136828316 ns 5 bytes_per_second=292.337Mi/s items_per_second=30.6538M/s null_percent=1 num_unique=100k
ValueCountsDictionaryChunks/10 140485423 ns 140485416 ns 5 bytes_per_second=284.727Mi/s items_per_second=29.8558M/s null_percent=10 num_unique=100k
ValueCountsDictionaryChunks/11 102110513 ns 102109071 ns 7 bytes_per_second=391.738Mi/s items_per_second=41.0767M/s null_percent=50 num_unique=100k
ValueCountsDictionaryChunks/12 13535453 ns 13534458 ns 52 bytes_per_second=2.88615Gi/s items_per_second=309.898M/s null_percent=99 num_unique=100k
ValueCountsDictionaryChunks/13 8598628 ns 8598517 ns 81 bytes_per_second=4.54293Gi/s items_per_second=487.794M/s null_percent=100 num_unique=100k
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment