The Phaeno is a place I think would be perfect. They are well funded and have interest in everything remotely automotive related. Also, they like to blend the border between science and art. Make sure to add some silhouettes of modern and old cars when pitching to them.
That would be hard to source / use, as It's a bit dated. But the technic behind it could be replicated. It uses IR LED on the bezel, around a glass. With total internal reflection the light only shows up where you touch the glass. Then film that with a camera. The advantage to my method, you don't get interference of objects hovering above the screen. But getting the camera & display behind the screen, could be hard.
I've just done something similar with a Object tracking + IR Floodlights and one of VA Imaging's MER2 cameras. We have had some fantastic results with objects sat on these "Interactive Dance Floors" from Alibaba. Didn't even need the IR Absorbing film to get decent tracking results using OpenCV @ 300fps or so.
This sound intriguing. Is the camera placed above the scene? How do those interactive Dance Floors come into play, did you use them for the light? Do they not have triggers, when somebody stands on it?
Yep, we've got the camera mounted vertically above the screen. We use the dance floors to render an interactive Unity game. The pixel density is pretty good. These panels have an array of IR LEDs embedded in them for detection but the density isn't good enough for small objects we want to track. Great for detecting anything the size of a foot or bigger.
That’s very cool. There’s a science museum in Sudbury Ontario that has something pretty much identical to this, albeit with slightly less cool colouring. Beside it was a sand table that projected the topography of the sand. Both were so awesome to play with. My youngest kept wanting to bring the blocks to the sand table to make buildings.
A small thought: if your hand affects the flow and it seems cool, add a small fan to simulate the blowing and see if that has a meaningful impact on the sensation of the experience.
We had this at Clemson University about 10 years ago. I think the whole design came out of an MIT lab though. There's video of it still up here: https://www.clemsongis.org/clemson-sandbox
it's not the exact science museum experience mentioned, but a quick Google search for "sand table video augment" pulled up a few, and this was the first one I found:
https://share.google/89A6x4yfaw5a4hOuh
When the first Xbox were getting long in the tooth, I believe people were repurposing the motion tracking bar as the mechanism to measure the topography of the sand table. That, coupled with a video projector mounted over the top of the sand table provides the additional colors and elevation lines. (And of course a bit of cool software to process and produce the image.)
This is one of those things that's really not that hard, nor expensive, with one decent hacker who wants to set it up. Maybe $1500 of parts? Feels like the kind of donation lots of people here could make to a local lower budget kids science centre. And I bet would be the kind of donation these centres would love to have.
As a bonus: you can likely make it out of "100% recycled e-waste" and "100% recycled lumber" (if you're building the table, too), giving it an extra educational theme. not only is this cool, fun, and educational, but it's a demonstration of doing something good with a used depthsense, projector, and computer.
Reminds me of Brett Victor's demo of projected AR turbulence around a toy car at Dynamicland. Only a short clip, but you get the idea: https://youtu.be/5Q9r-AEzRMA?t=47
Virtually any LCD screen will produce polarized light. This is the core working principle of those screens. So if you use a polarized filter (sunglasses for example) it will completly block the image coming from the screen.
We use IR Floodlights + IR absorbing film + a high speed IR camera. Works great in most lighting conditions and the screen doesn't show up on the camera.
Like a regular 2D fluid sim, the projection is not the bottleneck. The sim can be slow or fast depending on the quality you want. This one runs fine on integrated graphics card on my laptop.
That's really nice. You should give it or sell it to science museum or 50 and make them really big!
Thanks! That is the plan.. I created the demo to pitch to interested people.
The Phaeno is a place I think would be perfect. They are well funded and have interest in everything remotely automotive related. Also, they like to blend the border between science and art. Make sure to add some silhouettes of modern and old cars when pitching to them.
you could get an old microsoft surface 40 inch screen, the entire display is a video stream under linux with the sur40 kernel module.
For me, when I said giant I meant a 8 by 40 foot wall projector that you could walk in front of and have the fluids trail off your silhouette
https://vimeo.com/27500054
https://www.youtube.com/watch?v=kfF3o2kLg0w
That would be hard to source / use, as It's a bit dated. But the technic behind it could be replicated. It uses IR LED on the bezel, around a glass. With total internal reflection the light only shows up where you touch the glass. Then film that with a camera. The advantage to my method, you don't get interference of objects hovering above the screen. But getting the camera & display behind the screen, could be hard.
I've just done something similar with a Object tracking + IR Floodlights and one of VA Imaging's MER2 cameras. We have had some fantastic results with objects sat on these "Interactive Dance Floors" from Alibaba. Didn't even need the IR Absorbing film to get decent tracking results using OpenCV @ 300fps or so.
This sound intriguing. Is the camera placed above the scene? How do those interactive Dance Floors come into play, did you use them for the light? Do they not have triggers, when somebody stands on it?
Yep, we've got the camera mounted vertically above the screen. We use the dance floors to render an interactive Unity game. The pixel density is pretty good. These panels have an array of IR LEDs embedded in them for detection but the density isn't good enough for small objects we want to track. Great for detecting anything the size of a foot or bigger.
That’s very cool. There’s a science museum in Sudbury Ontario that has something pretty much identical to this, albeit with slightly less cool colouring. Beside it was a sand table that projected the topography of the sand. Both were so awesome to play with. My youngest kept wanting to bring the blocks to the sand table to make buildings.
A small thought: if your hand affects the flow and it seems cool, add a small fan to simulate the blowing and see if that has a meaningful impact on the sensation of the experience.
Ha, I've thought about adding a small fan too. Definitely need to do that.
Do you have a picture of the installation at the science museum?
The color just rotates through the oklab space (change hue only).
We had this at Clemson University about 10 years ago. I think the whole design came out of an MIT lab though. There's video of it still up here: https://www.clemsongis.org/clemson-sandbox
Naturally I can only find a photo of the sand table (and a cool piece of halite in the background!).
It’s at the Dynamic Earth exhibit at the Sudbury Nickel Mine. This was a few years ago.
it's not the exact science museum experience mentioned, but a quick Google search for "sand table video augment" pulled up a few, and this was the first one I found: https://share.google/89A6x4yfaw5a4hOuh
When the first Xbox were getting long in the tooth, I believe people were repurposing the motion tracking bar as the mechanism to measure the topography of the sand table. That, coupled with a video projector mounted over the top of the sand table provides the additional colors and elevation lines. (And of course a bit of cool software to process and produce the image.)
This is one of those things that's really not that hard, nor expensive, with one decent hacker who wants to set it up. Maybe $1500 of parts? Feels like the kind of donation lots of people here could make to a local lower budget kids science centre. And I bet would be the kind of donation these centres would love to have.
As a bonus: you can likely make it out of "100% recycled e-waste" and "100% recycled lumber" (if you're building the table, too), giving it an extra educational theme. not only is this cool, fun, and educational, but it's a demonstration of doing something good with a used depthsense, projector, and computer.
I've seen the sand table installation before, pretty cool. I was interested in how they did the fluid in particular. Thanks anyway.
Reminds me of Brett Victor's demo of projected AR turbulence around a toy car at Dynamicland. Only a short clip, but you get the idea: https://youtu.be/5Q9r-AEzRMA?t=47
I know and like Brett Victor's work. Definitely a source of inspiration!
Really cool. I love things like this where the human in the loop is the point.
Hi @danybittel, could you explain a bit more about the polarization filter you used to filter our the content and only let through the obstacles?
I am interested to know how it works.
Thanks.
Virtually any LCD screen will produce polarized light. This is the core working principle of those screens. So if you use a polarized filter (sunglasses for example) it will completly block the image coming from the screen.
You can see the effect here: https://www.reddit.com/r/mildlyinteresting/comments/crvpil/t...
I believe this will not work with OLED screens though, but I do not have one to test this.
We use IR Floodlights + IR absorbing film + a high speed IR camera. Works great in most lighting conditions and the screen doesn't show up on the camera.
I would LOVE to have this as my physical desk top.
I would love to have this on a transparent TV.
The polarization filter was a great idea. Solving a seemingly complex problem in a very simple way.
How much computing resources does it take?
Like a regular 2D fluid sim, the projection is not the bottleneck. The sim can be slow or fast depending on the quality you want. This one runs fine on integrated graphics card on my laptop.
Very cool! No code or anything though?