How to Secure an LLM Application: Prompt Injection, Data Leakage and Access Control

A secure LLM application is not defined by the model alone. It depends on whether untrusted content is contained, sensitive data is minimized before inference, and every retrieval or action is authorized outside the model. For companies deploying generative AI in Poland and the EU, that is the production test that matters. A polished assistant with weak boundaries should slow procurement, not accelerate it.

Buyers have become more demanding, and that is healthy. A vendor can promise EU hosting, encryption in transit, and enterprise readiness, then still fail the review that counts: where authorization happens, what enters the prompt, what gets retained in logs, and whether hostile retrieved content can steer the system. Those are not edge cases. They decide whether the deployment survives legal review, internal audit, and day-two operations.

The recurring mistake is familiar. Teams assess the model provider carefully, then under-secure the application wrapped around it. They review API terms, region options, and security certificates, but connect the model to internal documents, tickets, CRM records, or workflow tools with broad permissions and vague retention defaults. The risky part is usually the orchestration layer: the component deciding what context reaches the model, which tools can be called, and what traces are stored afterward.

The decision frame is narrower than many teams want it to be. The real question is not whether the model is safe in the abstract. It is whether the application can enforce trust boundaries under real operating conditions. If it cannot, reduce scope instead of adding more prompt wording and calling that control.

Why LLM security fails at the application boundary

Large language models do not understand trust the way enterprise systems need them to. They generate output from the context they receive. If system instructions, user text, retrieved documents, hidden notes, and tool descriptions are blended into one prompt, the model has no native mechanism for treating one source as authoritative and another as hostile. That is why OWASP's Top 10 for LLM Applications treats prompt injection as an application security problem, not a prompt-writing problem.

This matters commercially because weak deployments often look fine in a demo. Clean inputs make the model appear obedient. Production traffic is not clean. Customer messages contain pasted content, attachments, malformed markup, copied emails, and instructions that were never meant to be trusted. Internal data is messier still: free-text notes, policy exceptions, and comments written for humans rather than machines. If the application does not separate and govern those inputs before inference, the model becomes an efficient way to spread mistakes.

For companies operating under GDPR, this is not just technical hygiene. Article 5 principles such as data minimization and purpose limitation push architecture toward narrower context contracts. If a support assistant only needs order status, approved policy text, and the latest case summary, sending the full customer history, internal comments, and attachments into the model is difficult to defend. The legal issue follows from the design issue.

EU hosting is often oversold. It helps, but it is not a serious proxy for safety. A system can process data in the EU and still fail review because retrieval ignores source permissions, prompts are copied into observability tools, retention defaults are vague, or support access is poorly controlled. Buyers who stop at residency are screening for the easiest answer, not the most important one.

Prompt injection is a system design problem

Prompt injection is usually described as malicious instructions manipulating the model. Accurate, but incomplete. The deeper failure is that the application allowed untrusted content to influence a privileged reasoning path. A stronger system prompt may reduce some obvious failures. It does not fix a design where hostile text can shape retrieval, output, or tool use.

A support assistant for service agents makes the point quickly. The assistant can read ticket content, search an internal knowledge base, summarize prior cases, and prepare a refund request for review. A customer message includes hidden instructions in pasted HTML from an email thread, and the support platform preserves that markup while the AI layer strips almost none of it before retrieval. If that content is blended with trusted instructions, the model may treat it as operational guidance rather than untrusted input.

That is enough.

The result is rarely cinematic compromise. It is usually more ordinary and more dangerous: the assistant cites another customer's case, exposes internal notes, or proposes an action that should never have been available. That is enough for a personal-data incident, an avoidable dispute, or a failed internal control review.

Mature defenses start with source separation. System instructions, user input, retrieved content, and tool definitions should remain distinct in the orchestration layer even if they are later assembled into one model request. The application needs to preserve source identity because policy decisions depend on it. A customer-uploaded PDF should not be treated like an approved internal policy document. A ticket body should not be allowed to redefine tool behavior.

The next control is trust labeling. Retrieved content should be tagged by source type, trust level, and sensitivity before prompt assembly. If the use case depends on user-supplied documents, a safer pattern is often to extract structured facts first and pass those facts forward, rather than letting raw text compete with trusted instructions inside the same reasoning context.

Then comes action gating. The model may recommend an action, but the application must decide whether that action is allowed. Sensitive operations should sit behind server-side policy checks, fixed schemas, and explicit approval thresholds. A model can request create_refund_request(order_id, reason_code). It should not improvise arbitrary API calls or free-form database operations.

A buyer can test prompt injection maturity quickly. Ask for a live demonstration in the real application flow, not a slide about guardrails. Insert a known malicious string into a document, ticket, or note that the assistant is allowed to retrieve. Then watch what happens. If the answer relies mainly on a stronger system prompt, the control set is weak. A credible answer shows source tagging, retrieval filtering, and policy enforcement outside the model.

There is also a product decision hiding inside the security discussion. Many teams want a broad assistant because it looks more strategic. In enterprise settings, that ambition is often misplaced. A task-specific assistant with limited retrieval and no autonomous high-impact actions is usually easier to approve, easier to monitor, and harder to break. Some buyers will disagree because the broader vision sounds more valuable. In practice, the narrower system is often the one that reaches production.

Data leakage usually starts with overbroad context

Many buyers still begin with one question: does the model provider train on prompts? That matters, but it is not the most common enterprise failure mode. The more frequent problem is simpler. The application sends too much data into the model because nobody defined the smallest useful context for the task.

The EDPB guidance on data protection by design and by default matters here because it pushes teams toward technical measures that reduce unnecessary processing at the architecture level. For LLM systems, that means deciding what the model actually needs before integrations are built. A claims triage assistant may need claim type, policy status, and an approved summary. It probably does not need raw attachments, full correspondence history, or every internal note ever written on the case.

Context sprawl usually enters through convenience. The team exposes a broad API to the orchestration layer because that is faster than designing a narrower contract. Then prompts tell the model to ignore sensitive fields. That is not a control. It is evidence that the wrong data was sent in the first place.

This matters even more in operational environments where free text carries dense personal data. Customer service, HR, insurance, healthcare-adjacent workflows, and legal operations often contain identifiers, exceptions, informal comments, and copied correspondence. In the Polish market, language adds another layer. Local-language workflows include inflected names, address variants, and unstructured notes that are harder to sanitize with simplistic redaction rules built for English-only demos. A vendor that cannot explain how it handles retention, redaction, and retrieval discipline for Polish-language data is not ready for serious deployment.

Leakage risk extends beyond inference. It can appear in retrieval indexes, observability tools, support consoles, evaluation datasets, and debugging traces. The ENISA work on AI cybersecurity challenges is useful because it treats AI systems as composite environments rather than isolated models. That is the right lens for enterprise buyers. If prompts and outputs are tightly controlled but traces are copied into vendor dashboards with long retention, the system still leaks.

A workable baseline for leakage prevention is not complicated, but it does require discipline:

• Context minimization: pass structured fields or approved summaries instead of raw records whenever possible.
• Redaction before inference: remove identifiers, account numbers, contact details, and free-text notes unless they are necessary for the task.
• Permission-aware retrieval: enforce source-system entitlements before any chunk reaches the model.
• Logging discipline: separate operational telemetry from content storage and avoid full prompt logging by default.
• Retention limits: define how long prompts, outputs, traces, and evaluation samples are kept, and who can access them.

Retention is where many otherwise competent deployments fail review. Teams secure the API path and forget that observability products, vendor dashboards, and support tooling may retain prompts or outputs longer than the business intended. If a vendor cannot state default retention periods, redaction options, deletion mechanisms, and support-access conditions clearly, treat that as a buying warning rather than a documentation gap.

The same logic applies to vector stores and caches. Embeddings are not outside the governance perimeter just because they are not human-readable in their original form. If the source text contains personal or confidential information, the derived retrieval layer remains inside the same risk envelope and should be governed accordingly.

Access control must cover retrieval and actions

Traditional application security already knows how to authenticate users. LLM systems complicate the picture because the user is no longer opening one record at a time. They are asking a system to synthesize across documents, notes, and tools. If access control stops at the front-end login, the deployment is under-secured.

A workable model has three layers: application access, retrieval authorization, and action authorization. Application access answers who can use the assistant at all. Retrieval authorization answers what the assistant may fetch for that specific user from source systems. Action authorization answers what the assistant may do, or request to do, on the user's behalf. All three need to align. If one layer is broad, the whole system becomes broad.

The common failure pattern is flattening permissions during integration. A broad service account is used for retrieval. The vector index mixes documents from different sensitivity levels. Tool connectors are configured once for convenience and left overprivileged. The interface may look polished, but the back end is effectively bypassing the controls the source systems already had.

The strongest retrieval pattern is to enforce permissions before retrieval, not after generation. If a user could not open a document directly in the source system, the assistant should not retrieve it indirectly. That sounds obvious, yet many naive retrieval setups embed everything first and try to filter later. Once mixed content is already in the candidate set, mistakes become harder to reason about and harder to explain during audit.

There are several workable implementation patterns. Teams can carry source-system ACLs or entitlement tags into the retrieval index and filter before similarity ranking is finalized. They can partition indexes by tenant, business unit, or sensitivity class where the use case allows it. They can also place a retrieval broker in front of the index so user identity and policy attributes are checked before any chunk is returned to the orchestration layer. The exact pattern varies. The decision rule does not: authorization must happen before context reaches the model.

Action authorization needs even tighter control because the model can turn a suggestion into an operational event. A safer pattern is to expose only narrow server-side functions with fixed schemas. High-impact actions should be classified in advance. Reading approved reference content is low risk. Drafting internal notes or summaries is moderate risk. Sending external communications, updating customer data, approving payments, or altering regulated records is high risk and should never execute solely because the model proposed it.

For those actions, require explicit user confirmation with a clear preview, a second approver, or a human queue outside the model path. If the business insists on autonomous execution for consequential actions, the burden of proof should be much higher. In many EU deployments, that is the point where the project should be narrowed, redesigned, or paused. Hidden authority inside a probabilistic system is a poor fit for regulated or customer-facing operations.

What buyers in Poland and the EU should verify before rollout

Security and procurement reviews become more useful when the buyer asks a small number of hard questions instead of working through a long generic checklist. The goal is to determine whether the deployment has a real control boundary or only reassuring language.

Contractually, buyers should verify whether the vendor will sign a DPA that clearly defines processor responsibilities, subprocessors, retention, deletion, and support access. They should also verify where prompts, outputs, and traces are processed, under what exceptions support personnel can access them, and whether unnecessary retention can be disabled. If the deployment processes personal data at scale or touches employee, customer, or regulated records, DPIA questions should be raised early rather than after procurement has already advanced.

The EU AI Act should be handled carefully rather than used as generic fear marketing. Not every LLM application falls into a high-risk category. Still, the Act raises expectations around transparency, risk management, and oversight, especially when AI is embedded in workflows affecting employment, access to services, or regulated decisions. The practical implication is straightforward: document the use case, define human oversight, and avoid giving the model hidden authority over consequential actions.

One buying signal matters more than polished sales language: can the vendor explain failure conditions concretely? Mature teams can show where authorization happens, where logs live, how retention is configured, what happens when a malicious document is retrieved, and which actions are blocked by policy. Immature teams talk about guardrails in broad terms and move quickly to commercials.

The strongest production stance for most companies is selective ambition. Keep the model inside a narrow role. Minimize context. Authorize retrieval before inference. Gate actions outside the model. Retain less content. Demand more evidence.

Secure LLM application design is not about making the model obedient. It is about making the surrounding system governable. In this market, that is what separates a pilot that looks impressive from a deployment that survives procurement, legal review, and real operational use.