178 billion? That's nothing. I did trillions just this morning. I went to the grocery store and picked an item off the shelf, effectively filtering out the trillions of other products that I could have picked but didn't.
They did not process 178 billion rows per second. They did a search that found something in a large data set by eliminating the parts of the data set that could not have contained the item. Same way I did by picking one grocery store and going straight to the shelf.
Hm, if I understand their product correctly they are building a DB and their filtering actually returns correct results.
So, the analogy doesn't really hold true unless you actually have these trillions of alternate products stored in your brain and manage to cite the matching subset on demand.
In the 80's or so when I thought I was being really clever I came up with another variation on this and I recall being quite annoyed when someone on HN pointed out (many years later) that this was a staple of computing science for longer than that I had been busy with computers. So much for having original thoughts...
It does go to show that a huge number of inventions we consider foundational are really just from a normal person being in the right place at the right time. When a field is emerging there is a lot of low hanging fruit you can get your name stamped upon.
178 billion? That's nothing. I did trillions just this morning. I went to the grocery store and picked an item off the shelf, effectively filtering out the trillions of other products that I could have picked but didn't.
They did not process 178 billion rows per second. They did a search that found something in a large data set by eliminating the parts of the data set that could not have contained the item. Same way I did by picking one grocery store and going straight to the shelf.
Hm, if I understand their product correctly they are building a DB and their filtering actually returns correct results.
So, the analogy doesn't really hold true unless you actually have these trillions of alternate products stored in your brain and manage to cite the matching subset on demand.
That looks like a variation on a Bloom filter to me.
https://en.wikipedia.org/wiki/Bloom_filter
In the 80's or so when I thought I was being really clever I came up with another variation on this and I recall being quite annoyed when someone on HN pointed out (many years later) that this was a staple of computing science for longer than that I had been busy with computers. So much for having original thoughts...
Author here, indeed a variation of bloom filters: https://x.com/lemire/status/1971279371131646063
It does go to show that a huge number of inventions we consider foundational are really just from a normal person being in the right place at the right time. When a field is emerging there is a lot of low hanging fruit you can get your name stamped upon.
Same EventDB as https://github.com/ahri/eventdb or proprietary?
Proprietary.
I usually just call it 178 billion
But only if your billion is 10^9, not 10^12.
178.6e9rows/s/30days = 66150rows/s^2
This kind of reads like an action or war novel
As edited by ChatGPT…
Yeah, it's very clearly LLM-edited, but it's fun to read. The LLM did a good job.
It's not just a tech blog post - it's a thriller. ;)