Generative AI’s Ability to Connect the Dots and Making Incumbent Tools More Dominant
Yesterday, I had two instances when interacting with GPT-4, which made me realize two things:
- Generative AI’s ability to connect the dots like humans is still not up to the mark.
- Generative AI’s ability to make incumbents more dominant is real.
Generative AI’s Ability to Connect the Dots Like Humans is Still Not Up to the Mark
I worked on two problems yesterday, and both involved tools not commonly discussed online. The first was with OpenRefine. I was trying to see if one column had multiple pairs with another column. I tried many different prompts to suggest what I wanted, but it seems it could not give me the answer I was looking for. I even tried inputting the documentation into the chat, but it still didn’t work.
I finally went through the OpenRefine documentation and found what I needed.
The second issue was trying to set up a small UI project on Github Workspace and wanting to add hot reload to it. ChatGPT did help me with setting up LiveReload, but since Github Workspace doesn’t expose the port, but instead creates a URL, LiveReload didn’t work. It kept going to the hardcoded port 35729. I read through the documentation for LiveReload, but it didn’t offer many options that I could configure. After wasting an hour, I asked GPT-4 if there was another library to use, to which it suggested BrowserSync. I included it, but it had the same kind of problem. I tried explaining to GPT-4 that the problem was that the port was not exposed and it was going to a URL instead of a port. It couldn’t come up with a working answer. I found an option in BrowserSync to use a URL for the socket connection, and that worked.
I went back to GPT-4, asking if there was an option to change the socket URL, and it came up with the same code that I had just used. I asked why it didn’t suggest it before, and it said that it was focusing on a different aspect.
Unfortunately, I deleted the chat, so I can’t show the exact conversation. But this showed me that if it were a human with this information, they would have been able to connect the dots and suggest the solution. I have done this several times in the past, once I have the information and understand it, I can use that information to come up with a solution. GPT-4 uses Reinforcement Learning from Human Feedback (RLHF), so if it hasn’t seen the solution before, it is hard for it to come up with the solution.
AI’s Ability to Make Incumbents More Dominant is Real
People have already been talking about how Generative AI is going to make incumbents more dominant. They mostly talk about companies and individuals. But I think it is also true for tools.
Since both of these tools in my example are not the most popular tools, with everyday use cases, I could see that Gen AI was not able to help me with the problems as effectively as it would have with more popular tools.
I think this is going to make popular tools more dominant. As people continue to use more Generative AI to find their solutions, they are likely to lean more toward the popular tools, as they are more likely to get the answer they are looking for.
Conclusion
While there is no doubt about the vast capabilities of Generative AI, it still has a gap when trying to come up with solutions that are not mainstream. It makes sense why it does so well with human language, even poetry, and other creative tasks, as it already has a lot of data to learn from. Languages have limited words, and the massive amount of data these models have access to is enough to learn from. But I think when it comes to novel solutions, it still has some way to go. It might already be solving some of the problems that humans were not able to solve before, but to me, it seems like for humans that was a problem of scale and not the ability to come up with a solution.
