Generate yourself a program, or on the shoddiness of AI

In the era of widespread enthusiasm around artificial intelligence, more and more people – both developers and completely non-technical users – reach for AI to write code, solve problems or build simple applications. The trouble is that the end result is often the digital equivalent of a botch job.

An example of such a botch job could be a Calorie Calculator collecting people's data as a lead magnet for a trainer or dietitian. The app works correctly in the basic scenario, but it doesn't validate email addresses, doesn't encrypt data and doesn't protect the form against SQL Injection attacks. Everything looks good on the screen, but behind the scenes lurk serious oversights that can have catastrophic consequences.

It works, but only at first glance. And in the world of technology, where reliability, security and compliance with standards count – that's definitely not enough.

Code that only looks like it works

More and more often, applications and sites are created whose code is written by people without technical training. Thanks to AI, you can today instantly generate an entire project structure – forms, login functions, databases. Everything looks good at first glance, but the devil is in the details. A lack of knowledge means no one thinks about security, maintenance, quality, unit tests or updating dependencies. The result? Bugs that can open the door to attackers, data leaks, non-compliance with GDPR or other regulations.

Research from Stanford University and CSET reports show clearly: almost half of the code generated by AI contains serious security flaws. Among them, classic vulnerabilities like SQL Injection and XSS dominate – an example could be the case of a popular open-source project in which AI-generated login form code left an SQLi vulnerability. This bug was later used to take over user data and inject malicious code.

Worse still, AI often replicates the same mistakes, because it learns from public repositories, which also contain faulty code fragments.

This creates a situation in which bugs not only get replicated – they multiply and spread, like digital viruses. Code is created that looks elegant, works in basic cases, but fails at critical moments – where resistance to attacks and compliance with security practices are required.

AI as an advisor – and a hallucinating shaman

Non-technical people increasingly treat AI like an expert on everything – they ask questions about system configuration, security or implementing features. The problem is that the answers often contain so-called hallucinations: AI suggests non-existent options, wrong paths, made-up commands. From the outside it looks professional, but in practice it leads to serious problems – from incorrect system configurations, through incompatibility with the required environment, to recommendations that simply don't work. Users can waste time, introduce wrong settings or even expose systems to additional risk – just because AI gave something that sounded convincing.

The "slopsquatting" phenomenon (that is, AI generating references to non-existent libraries) is a real attack vector. Fraudsters can register such libraries in popular repositories (e.g. PyPI, NPM) and take control of applications that import them. Worse still, non-technical people may not even notice that they're using a fake package.

I'm also seeing a rise in the number of made-up tips. On forums and discussion groups, users publish AI-generated "solutions" – often wrong or even dangerous. Others copy them, without verifying, and this way the mistakes spread like rumours.

The problem isn't AI, but who's asking

It's not that AI is bad. It's that AI won't replace knowledge and experience. A poorly formulated prompt, a lack of understanding of the topic, blind faith in the result – these are the main causes of mistakes. Language models have no awareness of context, don't know our realities, don't understand legal regulations. They have no intuition, won't understand business exceptions or the subtle nuances that are everyday matters for a human specialist.

Meanwhile, many people assume that since the answer looks convincing – it's correct. That's very deceptive. In a study by Snyk, as many as 80% of developers considered AI-generated code safer than their own. That's not just naive – it's downright dangerous, because it leads to giving up any verification at all. Meanwhile, every fragment of code – regardless of its source – should go through tests, reviews and a security analysis.

AI can help – but not everyone and not in every way

A well-prepared prompt, combined with expert knowledge, can bring great results. AI works well as a supporting tool – but only when it's used by someone who knows the topic. A person who understands what they're doing can ask the right question, verify the result and understand the consequences of an implementation. In the hands of a beginner it's like giving the steering wheel to a child. Will it drive? Maybe. But safely? Definitely not.

This doesn't mean that beginners can't use AI. It means they should do it carefully – as educational support, and not a direct source of ready-made solutions. It's a tool that has potential, but its effectiveness depends on the user's competence. For example: an experienced developer using AI to generate a REST API code skeleton can significantly speed up their work, and then improve and secure the code in line with good practices. In such a scenario AI becomes real support, and not a source of risk.

It's also worth talking about responsibility. Who is responsible for mistakes generated by AI? The developer? The company? The model's creator? This question is increasingly being asked by regulators and lawyers – and for now there's no single answer to it.

The conclusions are clear: AI is a powerful tool, but it requires awareness, responsibility and humility. The point isn't to give up using it – but to know when and how to use it. It's not magic or an all-knowing expert, just a tool – and like every tool, in the wrong hands it can do more harm than good. If we're going to use AI, let's do it with our heads. And if we don't know what we're doing – let's stop before we click "generate". critical thinking and technical skills. Otherwise it becomes a source of shoddiness, botch jobs and serious security threats. And although it can speed up work, without quality control it does more harm than good.

In times when anyone can generate code with one click, asking the question becomes even more important: who checked this code? Who verified it? Who will be liable when customer data leaks online? And who will tell the user: "this doesn't work well – it just looks like it works"?