Some people have eyes bigger than their stomach; I have eyes bigger than my alotted free time. This is a list of projects I would be fascinated or interested in working on. I don’t list them here as a promise of coming soon, nor do I intend for this to be some guide for me when I find a spare moment. Think of this as more of a “wouldn’t it be cool if?" list to spawn conversation.

🛠️ In Progress 🏗️

I’m actively working on the following projects in some manner.

LLM context engines for robotic task planning

There’s local path planning - the raw kinematic planning of near distance movements. Then there’s global path planning - high level grid search to generically creates future plans to accomplish tasks. And then there’s task and mission planning. But what if we could utilize the demonstrated capabilities of an LLM to act as a context engine to handle task planning? Can we create a robot that can convert “I’m thirsty” to a series of steps including having the basic contextual knowledge of “sodas can be found in a fridge, which should be in the kitchen”, and the resulting global and task plans from there? How resilient to failures can we make it?

This is the plan for my Master’s capstone, which has just finished up. Spoiler; it works! I’m doing a quick write up for the rushed project, and am inspired to continue working on this problem for more generic approaches.

Personal research paper tutor

Staying up to date on the latest reearch and techniques in reinforcement learning, robotics, deep learning, LLMs, AI - it’s exhausting and requires the dedication of far too much time. My current modus operandi is to collect papers as I come across them and try to clear a block of time to go through them. This results in an ever expanding curriculum of papers of which I’ll only ever be able to look into a tiny amount. Even when I do dedicate time to go over papers, I have to be specific in my application of time as it can take a significant investment of attention to digest these papers.

I’m considering multiple avenues for creating an LLM agent for absorbing these papers, and providing summarized key points from papers, with the ability to deep dive in conversational query form when I need to, complete with referencing back to the source of the information. A built in search and embedding engine, and maybe some fine tuning - would result in a personalized tutor to speed my absorption of cutting edge research.

🤔 Future Work 📅

These are the projects that are on my hopeful to-do list; things I’d love to be given a block of time to tackle.

LLM multi-agent task planning and cooperation

Given that my attempt to utilize LLMs as high level task planners worked, I now have to consider cooperative embodied agents. What if we had a set of robots, that unbeknownst to them, could not complete their tasks without cooperation from other robotic agents in the system? And what if those robots had no compatibility, other than the ability to communicate by the English language?

Deep Reinforcement Learning + Poker

I used to be heavy poker player, and have been wondering if I could create a self-play agent that could become decent at No Limit Texas Hold ‘Em. It’s an interesting space a continuous action in a huge search space with hidden information that would challenge my current experience with reinforcement learning, with the fun possibility of the thing maybe being able to generate a little beer money.

🤷 Stalled ⌛

These are projects that I kicked off and got some distance on, or finished up and wasn’t quite happy with the outcome, and would like to get back to.

coppermind

Like everyone else, my first major project with an LLM was a personalized LLM agents with the ability to integrate into other functions; or more colloquially, a chatbot. It had the standard features - injected personality to sass me at every opportunity, text messaging and web API, generative embedding and conversation self summarization to enhance context memory. The most unique aspect of it was writing it all in Go because of my preference for typed languages and desire to simply create a download-and-go executable for others.

Ultimately it’s on pause, with the desire to go back and make it more tuned towards my own needs and more flexible.

birdseye

This is a stalled project I hope to get back to. I’ve rigged CARLA to run a car with 4 surrounding cameras, and am trying to use deep learning to construct not only a birds eye image view of the vehicle and cityscape from above, but also a semantically segmented view. It’s my first fully custom GAN and I have been learning a lot working through the process.

Hivenote

I started building this in Go a few years back (where does time go?) and then got distracted. It’s a Raspberry Pi with RFID/NFC reader, where I embed NFC stickers into laser cut wooden markers. If you place the NFC marker into a slot, the Pi will immediately begin playing an album, playlist, or podcast. It’s meant to be a living room art piece and bring physicality back to my media.

PPO on a robotic arm

I completed building an RL PPO project where a custom robotic arm environment tried to pick and place different shapes into a set of bins. The project worked-ish, but I wasn’t too happy with the results. I also had a highly variable result; a trained model that would either immediately slide the shape correctly or just twitch for its run duration. I wanted to give it another go; at the very least I wanted to try my custom trainer against stable-baselines.