LOKALIT

We tested 13 AI models on public office tasks. What works locally

· 8 min read · LOKALIT

AI can run on hardware inside a public office, without sending data to the cloud and without paying for every query. Before we propose anything to a client, we check it ourselves. In November 2026 we tested 13 open-weight models on tasks taken from the everyday work of a municipal office and a cultural centre, and then ready-made assistants built on the best of them. All data in the test was fictitious. The tasks were in Polish, as they would be for our clients.

How we tested

Everything ran locally, on a laptop with a 16 GB graphics card and 64 GB of memory.

Stage 1, models. Each of the 13 models received the same 8 tasks in Polish, each repeated twice:

  1. a reply to an application in line with the regulations,
  2. a summary of meeting minutes with a list of decisions, including a document of about 30 pages,
  3. a Facebook post, an Instagram caption with a character limit and screen text of up to 12 words,
  4. moving summer day camp sign-ups from emails into a table,
  5. anonymising a letter: names, addresses, phone numbers, PESEL national ID numbers, emails, bank account numbers,
  6. questions about the regulations, including one the document does not answer,
  7. rewriting an official notice in plain language,
  8. correcting spelling and punctuation errors.

On top of that there were two agent tests: using tools (booking a room and sending an email, with deliberate traps) and an agent browsing a municipality website. We scored each answer on a scale from 1 to 5, flagging fabrications and data leaks separately1.

Stage 2, assistants. On the best models we built ready-made assistants, each with its own recipe and automatic output checks. We tested them in the browser, the way an employee would use them, together with sign-up automation, booking with approval, and graphics and animation2.

Model results

Average score across 8 tasks (scale 1 to 5)
Gemma 4 26B: 4.9Gemma 4 26B4.9Qwen3.8 27B: 4.9Qwen3.8 27B4.9Gemma 4 31B: 4.9Gemma 4 31B4.9Gemma 4 12B: 4.6Gemma 4 12B4.6gpt-oss 20B: 4.3gpt-oss 20B4.3Bielik 11B: 4.0Bielik 11B4.0PLLuM 12B: 3.6PLLuM 12B3.6

Selected models out of 13 tested. Laptop with a 16 GB graphics card, fictitious data.

Source: LOKALIT test, November 2026

Data table
ModelScore
Gemma 4 26B4.9
Qwen3.8 27B4.9
Gemma 4 31B4.9
Gemma 4 12B4.6
gpt-oss 20B4.3
Bielik 11B4.0
PLLuM 12B3.6

Gemma 4 26B came out on top: an average of 4.9, no fabrications, no data leaks and both agent tests passed. On a 16 GB card a ready draft letter appeared in about 4 to 5 seconds. Among the smaller models the best was Gemma 4 12B (4.6), which needs about 9 GB of memory1.

20 of 20personal data items masked by the best models in the anonymisation test
0fabrications by good models when the regulations contained no answer
4–5 sto a ready draft letter (Gemma 4 26B, 16 GB graphics card)

Source: LOKALIT test, November 2026

A Polish model does not mean a better one. Bielik 11B wrote excellent Polish, but did not keep to character limits and performed poorly as an agent. PLLuM 12B, without safeguards, let through 16 of 20 personal data items in the anonymisation task. Gemma 4 wrote Polish just as well, while keeping to the format better and protecting data. We deploy Polish models as an option when a client wants them1.

All the models we recommend are licensed under Apache 2.0, which allows commercial deployment, including in public institutions1.

Assistants in practice

8 of 8correct answers from the regulations, each in 1 to 3 seconds, with a reference to the document
6 of 6sign-ups from 5 emails entered correctly into the table, in 9 seconds
0actions performed without staff approval in the room booking test

Source: LOKALIT test, stage 2, 17 November 2026

Employees use the assistants in the browser, under their own account, while the automations run on a server in the building.

Why configuration matters. Without preparation, the same models could make up an address, let data through or return an empty answer. With default settings the document assistant returned blank replies or got stuck in a loop; after our configuration it answered 8 of 8 questions correctly. The value of a deployment is not the model itself but the recipes, the institution's templates, the approval step and properly configured tools2.

Posters and graphics

A local model can also prepare an illustration for a poster, a Facebook graphic or an animated slide for the screen in the lobby. The test showed a clear limit, though: image models mangle Polish text. Instead of "Bożonarodzeniowy" (Christmas) they wrote, for example, "Bożonarodzieniówy". That is why the model only creates the illustration and the text is added by the institution's template. A poster prepared this way had flawless Polish2.

TaskTime on a 16 GB card
Poster illustration or Facebook graphicapprox. 40 s
Animated screen slide, 5 secondsapprox. 3.3 min
Removing the background from a photoapprox. 3 s

When chat and image generation run on the same graphics card at the same time, both slow down roughly fourfold. With several people at once, image jobs are best queued or given a separate card2. All the image models we use have licences that allow commercial use.

Which hardware, which model

SetupHardwareFor whomModel
StartPC or Mac, 16 to 32 GB of memory1 to 3 workstations, a small cultural centre, a libraryGemma 4 12B, with 32 GB Gemma 4 26B
Server S/Mserver with one 24 GB graphics cardoffice, cultural centre, small municipalityGemma 4 26B
Server M+/L48 GB of graphics memory or 128 GB of unified memorylarger office, many users at once, graphics alongside chatGemma 4 31B or Qwen3.8 27B

We measured model quality. Speed on a Mac and on servers with a 24 GB or larger card is for now our estimate, which we will confirm in further tests. As a rough guide, a Mac with 36 GB of memory is enough for 1 to 3 people with Gemma 4 26B, while an NVIDIA graphics card remains faster for graphics and animation. If an office lacks suitable hardware, we help choose it to fit its needs and budget.

Honest limits

We say plainly what AI should not do on its own, because our test showed it12:

  1. Anything that leaves the building is approved by a person. Without approval, 4 of 11 models made up the recipient's email address, and one announced that it had sent the message. With approval, no action was carried out without the employee's consent.
  2. Letters only with the institution's template. Without a template, 6 of 13 models added a made-up address or phone number for the office. With a template, none of the four models checked did.
  3. Anonymisation is initial masking, a filter and human review, never automatic publication, for example in a public information bulletin.
  4. Character limits are counted by the system, not the model, because some models do not keep to them.
  5. Polish text on graphics is added by the template, not the model.
  6. We do not restore archival photos of people. In the test, upscaling an old photo distorted the faces.
  7. A browser agent works only on a list of allowed sites. In the test, one model tried to visit an external online shop without being asked.

AI will not replace an employee and does not give one hundred percent security. Deployed well, though, it takes repetitive work off people's hands, and the data stays in the building.

Why data in public offices needs special protection is covered in Data security in public offices.

Sources

  1. LOKALIT test, stage 1, 14 and 15 November 2026: 13 models, 8 tasks in Polish, 2 agent tests, fictitious data.
  2. LOKALIT test, stage 2, 17 November 2026: 6 assistants with automatic checks on 5 models, a browser interface with a document base, sign-up automation, an agent with approval, graphics and animation, fictitious data. The full technical report is available on request.

Read also