Control Bot
Please note this project is a work in progress.
To learn practical control theory, I have been working on programming a robot. I designed this bot to have four mecanum wheels and an RGBD camera in the front. This way, I can learn how to program robot movement to avoid any obstacles it sees. The mecanum wheels would let me explore different drive setups, e.g., tank drive or swerve drive, and the camera would let me explore how to do object detection using depth images and color images. By combining these, I could learn how to control the robot at a high level, through path planning, localization, mapping, and more.
Component Layout
When I was designing this robot, I wanted to have components that would have few resource limitations. So, I chose a Jetson Orin Nano as an onboard computer and an Intel RealSense D435i as the front-facing camera. I also wanted a long runtime, so I soldered eight 18650 Li-ion batteries to make a 4s 5600 mAh battery pack. Finally, I decided to use components I had lying around to control the motors and wheels. With all this in mind, I designed a 3D-printed frame to hold all the electronics in place.
Design
I placed the wheels in a 110mm square as a starting point to make the robot somewhat compact. I placed the motors 45 degrees from horizontal and designed an angular shape around them. I put the camera just before the front wheels and placed the Jetson above it at an angle. I added an Arduino Uno with a motor shield kit I had lying around, flat on top. With the space below the Uno, I created a slot to fit in my battery pack. I also added a chamfer for three voltage regulators that’d supply 12V to the Jetson and 5V to the Uno. To manage my cables, I added small through holes for zipties to hold some wires, and a slot in the middle of the frame for the motor wires to connect to the Uno.
Control Scheme
[Last Updated 4.30.25]
My goal, for now, is to have the robot point to the closest object it sees. I do this by taking the camera’s depth image, averaging all the columns into a 1D array, identifying the peaks, and finding out how far the peaks are from the center of the camera frame. The percent distance between the array's lowest peak and the array's center is the loss value.
This is my first time revisiting ROS in a long time, and I have been experimenting with breaking down my ROS node network. I started with a high-level controller node that would take simplified input values and decide output values. At the moment, I have an input loss value between -1.0 and 1.0 and an output set of motor commands between 65 and 254. This controller node takes the loss value and applies a linear scaling function to turn the robot to the right, if the loss is negative, or to the left, if the loss is positive, at a variable speed. The bot does not navigate its environment yet.
The /realsense2_camera/depth/image_rect_raw topic is the starting point of my network and is a built-in ROS topic from Intel’s ROS wrapper. It is a 2D array of depth values that my /DepthProcessor node breaks down into a /loss_function. The /DepthProcessor node also uses a display_helper class I wrote to show a live plot of depth values, color values, and the condensed 1D array. The /Controller node takes the /loss_function and outputs a string of /motor_commands, e.g., “150_-150_150_-150”. The /hw_interface node takes this string and uses the pySerial library to send an array of instructions, value by value, to the Arduino Uno. In the Uno, I have a script that checks for the delimiter byte, 255, and starts reading values in order. For example, if the value that follows the starting delimiter is -1, the following motor instruction will be backwards, whereas a value of +1 will be forward. An example array of values that the /hw_interface node sends would be <255, 1, 125, -1, 125, 1, 125, -1, 125>, which would make the robot turn left at a speed of 125.
The Current State - A work in progress
At the time of this writing, the robot can’t point at objects, but can move in response to an object. In the video, we see the robot identify the object with a small orange marker in the plot. Adjacent to the plot, we see a live video feed of the depth camera. When the /hw_interface node runs, the robot begins moving to the right, away from the object. Then it course corrects and starts moving towards the object again, but overshoots it. When it overshoots it, it sees another object in its depth frame, which causes the process to crash for an unknown reason.
I think because I am updating the graph and camera displays in series with the control process, I am reducing my sample resolution. This could mean the robot is too slow at updating its movements. Additionally, there is an edge case I have not programmed for, which is causing the robot to crash as seen in the video. Furthermore, there could be something wrong with my model that I need to tweak.