Helping an AI to read... kinda

Building AI is like teaching a baby to walk, talk, speak, and understand. Everythings starts from zero till its a hero.

As Artificial Intelligence becomes a bigger and bigger part of our lives, it is important that we as Computer Engineers and developers to understand how they work. There are many ways to make a computer think and understand. While it is easier to have it understand words and how they interact with each other to form a sentence, it is harder to have it take in pictures or video input and have it interprept what is going on. This is what I hoped to learn to do in this project. The Goal is to make an AI using a neural network to take in an alpha-numerical character and have it return the correct symbol back.

The full project can be seen Here for download and expirementation.

The first step was choosing a project that covered all of my bases. I wanted to learn about nueral networks, have it written in python for ease of explaining my code, and also have the information stored in a database so I do not have to restart the training every single time.

An overview of the Project

What is presented to the user is a drawing pad. This drawing pad allows the user to draw a number then two buttons to either enter that number into the pool to help train the model or to test the model against the existing AI. When adding the information directly into the pool, the program takes in the information from every pixel that was colored in and links that to the information accociated with the number you specified.

An example of this could be drawing a six and then confirming that the number is indeed six. The computer takes the data of the pixels drawn and accosiates that with the number six so if in the future, another drawing uses those same pixels, it assumes that number is a six. Given enough of this information, the computer can learn from thousands of inputs and can correctly guess what number was chosen when drawn.

How does the AI Nural network choose between numbers like 6 and 8 that might be very similar? The answer is with a lot of testing and training. The more clean versions of 6 and 8 it has the easier it is to find that gap that the 6 leaves. Lets begin with the number 8 as the testing input as an example. The computer starts at node 1with just the image. Depending on the number of pixels used to draw the 8, the computer assumes it is not a 1 or a 7. As it goes onto the next loop, it notices that the test image has at least one loop. It then eliminates the paths going to 5, 3, 2, and 5. It then notices these loops are more closed then open and it has two of these, so it eliminates all but 6, 8, and 9. It then takes in the data that both loops are closed so it settles on 8 and returns that as the answer.

Pretty smart right? I am sure thats how most people learned their numbers, by saying 'well its not a 6 or a 9, and it connects so its an eight' when they were learning there numbers at a young age. A computer must do this each time that a new test is put in. It must continuously check against previously learned items and compare if it looks similar or very different. This is why it is important to have a varied amount of numbers input and also shapes of numbers as if a toddler cant understand the difference between a cursive letter and a standard case until taught, neither will the computer.

Taking out the Trash

Something that has to be brought to mind is trash data or junk. This can be in many forms as a user drawing a 7 and saying its a 0, or drawing a 6 that almost fully connects to make it look like an 8. This can confuse the neural network and make the computer hallucinate or make up the wrong answer. What is to be done with this. Well, as most people that work with Chatbots can tell you, it takes a lot of training, a lot of correction, and some insentives to the computer for it to ignore this data or have it be so irrelivant that it does not factor into the final decision. A neural network might not even be smart enough after taking in literally thousands of data points. It can still make things up from poisoned data like chatgpt and claude. Below is a youtube video from CodeBullet where he uses a nueral network to guide a bot through a jumping game and even then it takes thousands and even tens of thousands of tries before it can complete it once.

This is the importance of testing, of having data it can pull from. This is why it is needed to test and retest the code over and over to make sure the user, or customer, is able to have a correct output without all the extra junk.

As you can see, it took their AI 927 generations with hundreds of bots per generation in order to perform the task at hand, imagine how many it takes to do something like this. Probably less then their ~927,000 but still a massive number to do basic things. Even LLMs need extensive training over months doing nothing but training to reach the intellegence they had back in the early 2020s. Thats why I have put the project into this website. So that the AI can take in any data individuals here decide to dedicate to it and become smarter. Hopefully over time, the more people that use the OCR and visit the website, the smarter it becomes. This will be implimented later.

In Conclusion - What I Learned

Over the course of the project, I learned a few things. The most important things were how to work both frontend and backend parts of an application and how to tie them together. The other thing I learned was the importance of a clean database and sanitizing data when it comes in. Allowing any type of input can lead to issues and can ruin a database very quickly. The last thing that I learned was how Artificial Intelligence works in the background. It truly is lists linked to lists and prompts that are tied to other prompts. When asked to do something, all it has is what it has been trained on, whether large or small, detailed or simple, it can only give answers based on what it already knows and that is why it important to train any model you decide to make on as many things as possible in order to make a smart AI at the end of the day.