> The joysticks are connected to a pilot hydraulic circuit, which proportionally moves the cylinders in the main hydraulic circuit which ultimately moves the excavator joints
I've actually spent a decent amount of time running an excavator, as my Dad owns a construction / road building company. It was a great summer job!
An important note about the pilot hydraulics is that they _provide feedback to the operator_. I would encourage any system that moves these controls on behalf of a remote human operator or AI to add strain gauges or some other way to measure this force feedback so that this data isn't lost.
The handful of "drive by wire" pieces of equipment that my Dad or other skilled operators in my family have ran were universally panned, because the operators are isolated from this feedback and have a harder time telling when the machine is struggling or when their inputs are not sufficiently smooth. In the automotive world, skilled drivers have similar complaints about fully electronic steering or braking systems, as opposed to traditional vacuum or hydraulic boosting approaches where your foot still has a direct hydraulic connection to the brake pads.
You're right! This is exactly why we like to do mechanical actuation - we are able to achieve bilateral telepresence, which essentially gives the torque (haptic) feedback over the internet! So on small excavators, you can absolutely feel the resistance. We also stream the engine audio, which tells you how hard the hydraulic pump is working. Operators like our system for these reasons :)
I'd like to get a chance to talk to you and your Dad to get feedback. How do I reach you?
My email is contact at useflywheel dot ai
Yes, and you also get feedback from your butt as the machine tips and wobbles, particularly on smaller machines. Hearing the engine straining helps also. Often you can not clearly see what you are digging, this feedback lets you know if you are running into a rock or something.
One big advantage would be cameras mounted on the boom and rear view cameras, as many machines have obstructed views.
We're indeed streaming the audio and have haptic feedback. My hypothesis is the seat vibration isn't as helpful. It's sub optimal and operators would be far more productive without it. We would do a paper when we have enough data on this.
We're also putting more cameras but streaming a lot of them at once is tricky.
Humbly, have you used excavators of varying sizes on uneven ground? I have and would suggest it's more important than you might think. But if you've operated them, you might know better than I.
My car with its drive-by-wire brakes has a brake feedback simulator that gives the driver the kind of feeling associated with power-boosted hydraulic brakes. This is by far the most expensive single component in the car. Arguably these are just expensive accommodations for human flaws. A self-driving car wouldn't need them. Can't the self-driving system act directly on data like pressure, flow, and displacement?
That's indeed what we're trying to test to the extreme - to see how far we could go with just vision. We haven't done heavy excavation workflows yet, but we have some early success with some excavation workflows with just vision input and joystick action output (even without joint angle feedback!).
We're betting on having really huge data with compact observation input and experiment to see if it holds water. If not we can always dial it down and add more sensors/feedback.
Maybe it doesn't matter for a car because feeling the car's motion tells you most of what you need to know. A car is not meant to touch anything but the road, in normal conditions. I think steering is the only case where force feedback is very important for a car - In the winters up here, I can feel the steering go loose when I hit a patch of ice.
I imagine an excavator, meant to touch and dig through things, and lift things, benefits from force feedback for the same reason VR would.
Have you played those VR sword games? BeatSaber works great because you're cutting through abstract blobs that offer no resistance. But the medieval sword-slashing games feel weird because your sword can't impact your opponent.
I saw a video recently of a quadcopter lifting heavy objects. When it's overloaded, it can't maneuver because all its spare power is spent generating lift to maintain altitude. If the controls had force feedback, the copter's computer could tell you "I'm overloaded, I can't move" by putting maximum resistance on the sticks.
I think force feedback is key for small excavators, but not really true for 25+ tons excavators. Hence how easy it is for operators to accidentally kill someone with it.
What a fantastic business. I've watched lots of digger (as we call them here) drivers in action and I'm in awe of what these machines can do and how incredibly skilled their operators are. We have several large rock walls on our property and watching these guys delicately picking and placing huge rocks to key them in just blows my mind.
There was a YouTube video recently of an AI-assisted digger making a wall out of the concrete rubble from a demolition.
I believe the applications for really smart excavators must be huge. Sounds like this might be a step on that voyage.
You're spot on! The experienced operators are quite skilled at moving these machines. The number of operators who could do this work are decreasing and it's also pretty dangerous which explains the reduction of new people joining in, especially with better ways to make money today. We're getting started with remote teleop and moving towards full autonomy!
This looks very interesting. I understand that hydraulic retrofit is compelling, but you may want to consider CANbus-based retrofits as well (similar to how CommaAI works, where it simply sends messages to the CANbus network to control steering or speed just like ADAS features do). Modern equipment is moving away from hydraulic controls toward a fully digital cockpit and being able to plug in and teleop (for less than the cost of the manufacturers expensive robotics suite) would be amazing.
I have a Cat 289D skid steer I would happily contribute to the effort if you guys move into the compact equipment space (compact being a relative term, as my machine is only 6 tons compared to your 38 ton machine)
Thanks! Yes we're testing both CAN spoofing and retrofits. CAN spoofing really limits compatibility to a few machines/brands. Would love to get in touch with you: my email is contact ai useflywheel dot ai
Would you be able to replicate this with the heavy equipment and movements needed to plug orphaned oil wells? In Texas alone, it's a TAM of ~$38B, and ~$150B for the entire US.
It is possible. One good thing about our retrofit is it works on any hydraulic equipment, and not just excavators, although we are focussed on excavators for now. We are able to retrofit any hydraulic machinery because we actuate the joystics connected to the valves, which are quite standardized and usually come from handful of companies like Bosch. Will definitely 'dig' more into this though.
Skate to where the puck is going. By the time they're ready for prod (2-4 years), regime change might have occurred. Be ready with a solution to this environmental problem at that time. Lots of oil and gas infra over the next few decades that will require remediation after the world transitions to low carbon energy and electrification, while declining fertility rates globally will mean a shrinking pool of workers to pick from for the human labor component.
Locales who don't want their aquifers contaminated will also be motivated in the near term to get this work done. They would be great test cases for proof of value of this automation imho.
I haven't worked in the industry, but a family member does and owns several wells (ie the mineral rights) in Texas, passed down from their deceased parent who was a rig worker.
If you would be so kind as to suggest resources so that I can better educate myself in this domain for any missing nuance, I am open to any and all resources, including me traveling on my own dime to speak with a subject matter expert in person (who I am willing to compensate at a reasonable hourly rate).
Talk to people working in oil and gas at the executive level, or even better, try selling stuff to them. Oil and gas operates on a very different mindset than the nibbling-at-margins product-led-growth models that are so prevalent on HN.
The only metric that matters at the end of the day is how to print more money i.e. drill more oil. Literally nothing else matters, and compliance goes to the lowest bidder to make the problem go away. Right now that means playing nice with the whitehouse and hopefully bypassing the rest of the pesky EPA. I assure you not a single person in oil and gas likes the EPA, nor the infamous Subpart W.
We have some autonomous safety layers (comprised of object/human detection like you mentioned and others) even for remote teleop. But we haven't deployed autonomy on real sites yet. We still have some work to do before we can reliably deploy/test it out on the actual sites!
> The joysticks are connected to a pilot hydraulic circuit, which proportionally moves the cylinders in the main hydraulic circuit which ultimately moves the excavator joints
I've actually spent a decent amount of time running an excavator, as my Dad owns a construction / road building company. It was a great summer job!
An important note about the pilot hydraulics is that they _provide feedback to the operator_. I would encourage any system that moves these controls on behalf of a remote human operator or AI to add strain gauges or some other way to measure this force feedback so that this data isn't lost.
The handful of "drive by wire" pieces of equipment that my Dad or other skilled operators in my family have ran were universally panned, because the operators are isolated from this feedback and have a harder time telling when the machine is struggling or when their inputs are not sufficiently smooth. In the automotive world, skilled drivers have similar complaints about fully electronic steering or braking systems, as opposed to traditional vacuum or hydraulic boosting approaches where your foot still has a direct hydraulic connection to the brake pads.
You're right! This is exactly why we like to do mechanical actuation - we are able to achieve bilateral telepresence, which essentially gives the torque (haptic) feedback over the internet! So on small excavators, you can absolutely feel the resistance. We also stream the engine audio, which tells you how hard the hydraulic pump is working. Operators like our system for these reasons :)
I'd like to get a chance to talk to you and your Dad to get feedback. How do I reach you? My email is contact at useflywheel dot ai
Yes, and you also get feedback from your butt as the machine tips and wobbles, particularly on smaller machines. Hearing the engine straining helps also. Often you can not clearly see what you are digging, this feedback lets you know if you are running into a rock or something.
One big advantage would be cameras mounted on the boom and rear view cameras, as many machines have obstructed views.
We're indeed streaming the audio and have haptic feedback. My hypothesis is the seat vibration isn't as helpful. It's sub optimal and operators would be far more productive without it. We would do a paper when we have enough data on this. We're also putting more cameras but streaming a lot of them at once is tricky.
Humbly, have you used excavators of varying sizes on uneven ground? I have and would suggest it's more important than you might think. But if you've operated them, you might know better than I.
My car with its drive-by-wire brakes has a brake feedback simulator that gives the driver the kind of feeling associated with power-boosted hydraulic brakes. This is by far the most expensive single component in the car. Arguably these are just expensive accommodations for human flaws. A self-driving car wouldn't need them. Can't the self-driving system act directly on data like pressure, flow, and displacement?
That's indeed what we're trying to test to the extreme - to see how far we could go with just vision. We haven't done heavy excavation workflows yet, but we have some early success with some excavation workflows with just vision input and joystick action output (even without joint angle feedback!). We're betting on having really huge data with compact observation input and experiment to see if it holds water. If not we can always dial it down and add more sensors/feedback.
Maybe it doesn't matter for a car because feeling the car's motion tells you most of what you need to know. A car is not meant to touch anything but the road, in normal conditions. I think steering is the only case where force feedback is very important for a car - In the winters up here, I can feel the steering go loose when I hit a patch of ice.
I imagine an excavator, meant to touch and dig through things, and lift things, benefits from force feedback for the same reason VR would.
Have you played those VR sword games? BeatSaber works great because you're cutting through abstract blobs that offer no resistance. But the medieval sword-slashing games feel weird because your sword can't impact your opponent.
I saw a video recently of a quadcopter lifting heavy objects. When it's overloaded, it can't maneuver because all its spare power is spent generating lift to maintain altitude. If the controls had force feedback, the copter's computer could tell you "I'm overloaded, I can't move" by putting maximum resistance on the sticks.
Interestingly, we had some people try out VR teleop: https://x.com/Scobleizer/status/1970245161306464667
https://x.com/jash_mota/status/1969091992140304703
I think force feedback is key for small excavators, but not really true for 25+ tons excavators. Hence how easy it is for operators to accidentally kill someone with it.
What a fantastic business. I've watched lots of digger (as we call them here) drivers in action and I'm in awe of what these machines can do and how incredibly skilled their operators are. We have several large rock walls on our property and watching these guys delicately picking and placing huge rocks to key them in just blows my mind.
There was a YouTube video recently of an AI-assisted digger making a wall out of the concrete rubble from a demolition.
I believe the applications for really smart excavators must be huge. Sounds like this might be a step on that voyage.
You're spot on! The experienced operators are quite skilled at moving these machines. The number of operators who could do this work are decreasing and it's also pretty dangerous which explains the reduction of new people joining in, especially with better ways to make money today. We're getting started with remote teleop and moving towards full autonomy!
This looks very interesting. I understand that hydraulic retrofit is compelling, but you may want to consider CANbus-based retrofits as well (similar to how CommaAI works, where it simply sends messages to the CANbus network to control steering or speed just like ADAS features do). Modern equipment is moving away from hydraulic controls toward a fully digital cockpit and being able to plug in and teleop (for less than the cost of the manufacturers expensive robotics suite) would be amazing.
I have a Cat 289D skid steer I would happily contribute to the effort if you guys move into the compact equipment space (compact being a relative term, as my machine is only 6 tons compared to your 38 ton machine)
Thanks! Yes we're testing both CAN spoofing and retrofits. CAN spoofing really limits compatibility to a few machines/brands. Would love to get in touch with you: my email is contact ai useflywheel dot ai
Would you be able to replicate this with the heavy equipment and movements needed to plug orphaned oil wells? In Texas alone, it's a TAM of ~$38B, and ~$150B for the entire US.
The Looming Disaster Under America's Biggest Oil Field [video] - https://news.ycombinator.com/item?id=45361022 - September 2025
Texas has thousands of abandoned oil and gas wells. Who is responsible for cleaning them up? - https://www.texastribune.org/2025/05/08/texas-orphan-wells-e... - May 8th, 2025
The Rising Cost of the Oil Industry’s Slow Death - https://www.propublica.org/article/the-rising-cost-of-the-oi... - February 22nd, 2024
Well plugging SOP:
https://www.epa.gov/natural-gas-star-program/well-plugging
https://www.osha.gov/etools/oil-and-gas/abandoning-well
It is possible. One good thing about our retrofit is it works on any hydraulic equipment, and not just excavators, although we are focussed on excavators for now. We are able to retrofit any hydraulic machinery because we actuate the joystics connected to the valves, which are quite standardized and usually come from handful of companies like Bosch. Will definitely 'dig' more into this though.
Your service will need to be cheaper than just lobbying the current federal government.
Skate to where the puck is going. By the time they're ready for prod (2-4 years), regime change might have occurred. Be ready with a solution to this environmental problem at that time. Lots of oil and gas infra over the next few decades that will require remediation after the world transitions to low carbon energy and electrification, while declining fertility rates globally will mean a shrinking pool of workers to pick from for the human labor component.
Locales who don't want their aquifers contaminated will also be motivated in the near term to get this work done. They would be great test cases for proof of value of this automation imho.
(think in systems)
Very optimistic thinking in your first 2 sentences there...
Says somebody who has clearly never worked in oil and gas.
I haven't worked in the industry, but a family member does and owns several wells (ie the mineral rights) in Texas, passed down from their deceased parent who was a rig worker.
If you would be so kind as to suggest resources so that I can better educate myself in this domain for any missing nuance, I am open to any and all resources, including me traveling on my own dime to speak with a subject matter expert in person (who I am willing to compensate at a reasonable hourly rate).
Talk to people working in oil and gas at the executive level, or even better, try selling stuff to them. Oil and gas operates on a very different mindset than the nibbling-at-margins product-led-growth models that are so prevalent on HN.
The only metric that matters at the end of the day is how to print more money i.e. drill more oil. Literally nothing else matters, and compliance goes to the lowest bidder to make the problem go away. Right now that means playing nice with the whitehouse and hopefully bypassing the rest of the pesky EPA. I assure you not a single person in oil and gas likes the EPA, nor the infamous Subpart W.
Could a mini tunnel boring machine plug a well, from the side?
That's definitely a thing.
https://en.wikipedia.org/wiki/Relief_well
Whats the budget for getting an MVP for heavy equipment ?
I ask in all seriousness since, for example, retrofitting regular semis to electric requires millions and millions just to get started
It's far cheaper than millions! We're making something equipment owners want :) Let's chat? My email is contact at useflywheel dot ai
I feel like the inherent bootstrap cost of hardware startups is usually reflected in fundraising amounts.
This scratches the itch I've had since playing with Tonka trucks in the back yard when I was 5... Are you hiring?
Thanks! Let's chat: contact @ useflywheel dot ai
How much safety training have you done with the models? i.e. does it know to stop if it's about to drive over a human?
We have some autonomous safety layers (comprised of object/human detection like you mentioned and others) even for remote teleop. But we haven't deployed autonomy on real sites yet. We still have some work to do before we can reliably deploy/test it out on the actual sites!
Congratulations! Looks like an interesting project!
Thanks! We enjoyed a lot while iterating the hardware and looking at hours of excavator data (mostly done by Mahimana).