Inside Version 3 of the VisualText NLP Engine: Compiled, Cloud-Built, on npm — and AI-Assisted

Article, Blog, Export, Top

For most of its life, the NLP++ engine has been a powerful tool for writing rule-based analyzers — explainable, glass-box, and deterministic natural-language processing you can actually read, debug, and reproduce exactly. It has run on Windows, Linux, and macOS for years. What Version 3 changes is how you build, deploy, and install it — and, now, how you start. Compiling your analyzers to native code is a one-click job rather than a C++ expedition; those binaries can be built in the cloud; and — for the first time — you can pull the engine into a Node.js project with a single npm install. The VS Code extension goes one step further: a built-in help system and ready-to-paste prompts let Claude scaffold a working analyzer from scratch — and even gather real text off the web to test and harden it — so you no longer have to be fluent in NLP++ to begin. And what comes back is still plain, glass-box rules you own.

Here’s what shipped, and why it matters.

Compiling analyzers — now a one-click job, not a C++ expedition

Here’s a subtlety worth being precise about: compiling NLP++ analyzers and knowledge bases to native C++ was never impossible. The capability has existed for a long time. But it was hidden, fragile, and realistically reachable only by someone with serious C++ build expertise — the right toolchain, the right flags, the right link order. For everyone else, it might as well not have existed.

Version 3 takes that buried, expert-only capability and turns it into a one-click process any user can run — no C++ knowledge required. You write rules, functions, dictionaries, and knowledge bases; the engine code-generates C++ from your NLP++ rule files and knowledge bases and builds them into native C++ shared libraries on whatever platform you are on whethe it be Linux, the Mac, or Windows. The payoff is faster execution and a deployable artifact, without ever opening a compiler.

Making that one click trustworthy took real engineering under the hood: dozens of type-correct overloads for NLP++ builtins in the generated code, provenance comments (/* nlp-source */) so a line of generated C++ can be traced back to the NLP++ rule that produced it, and a series of correctness fixes — flushing knowledge-base writes that were silently coming up empty, guarding against a null-pointer crash, and fixing a subtle C++ evaluation-order bug by hoisting function arguments into temporaries.

Compile in the cloud — no toolchain required

Not everyone wants a C++ build environment on their machine. So Version 3 introduced a cloud-compile service (nlp-compile-service): a lightweight dispatcher that hands your analyzer to GitHub-hosted runners, compiles it for Windows, Linux, and macOS, and hands back native libraries.

The genuinely hard problem here was cross-platform native linking — getting ICU and the engine’s own static libraries to link cleanly on Linux and macOS (the kind of --whole-archive / --start-group / -force_load incantations that decide whether you get a working binary or an undefined-symbol error). With that solved, “compile my analyzer” became a button in the VisualText VS Code extension (now v3.1.24), complete with progress, an elapsed-time counter, and automatic staging of the result into your project.

New in v3: the engine on npm

The headline packaging change in Version 3 is a brand-new Node.js package. NLP++ has had a native Python package for a while (pip install NLPPlus), but the JavaScript world had nothing first-class. Now it does:

  • Node.js (new): npm install nlpplus (npm-package-nlpengine, v1.0.5) — a self-contained native addon that bundles its runtime dependencies. Built from scratch this cycle.
  • Python (enhanced): pip install NLPPlus (py-package-nlpengine, v2.0.9) — the existing package, now with the new compile() and cloud_compile() APIs, context-manager support, and Python 3.13 wheels.

Both embed the engine directly — no subprocess shell-out, no manual binary wrangling — just import and analyze.

Smarter analyzers and lighter dictionaries

The language runtime got upgrades too. Large lexicons now lazy-load one word at a time instead of reading an entire dictionary into memory up front — a big win for startup time and footprint. New loadkbb and loaddict functions, an _xVAR("attribute") match-list special, and support for digits in underscore-prefixed token names round out the NLP++ improvements.

On the content side, the shared package-analyzers collection (email, telephone, links, address) was re-architected from line-based to zone-based processing, so it works on real-world HTML and Markdown pages rather than tidy single lines — and the telephone analyzer gained international support.

Built with an AI collaborator

One more thread runs through all of this: much of Version 3 was developed in collaboration with Claude (Anthropic’s Claude Code). You can see it in the commit history — the systematic, traceable changes that hardened compiled mode, ironed out the platform-specific build problems compiled mode exposed, stood up the cloud-compile backend, scaffolded the new Node package, and wired up automatic cross-repo release propagation.

There’s a deeper point here other than convenience. NLP++ is glass-box AI — rule-based, inspectable, explainable, and deterministic: identical input yields identical output, every time, with rules a human can audit. That’s exactly what statistical and probabilistic models can’t guarantee, and it’s why deterministic NLP is the right fit for critical-path systems — the places where “usually correct” isn’t good enough and you need to be able to prove why a decision was made.

So the symmetry is striking: a large language model (Claude) is helping build a deterministic NLP system — one that can go where statistical systems like LLMs can’t. The LLM is a development partner, not the runtime. It helps create the tools that itself cannot perform reliably; the tools themselves stay transparent and reproducible.

Help is now built in — and Claude can write your first analyzer

For years the hardest part of NLP++ was the blank editor. Developers who didn’t already know the language found rule-based analyzer creation intimidating, and many reached for an LLM instead — accepting an opaque black box in exchange for not having to start from nothing. Version 3 removes that barrier from both directions: it puts the documentation an arm’s length away, and it hands Claude everything it needs to write real NLP++ with you, while the code it produces stays plain and inspectable.

The first half is a built-in Help view. The VS Code extension now carries quick-start guides, compilation instructions, regression-testing documentation, and the full NLP++ reference in the sidebar, so the answer to “how do I run this, compile this, test this” is in the editor rather than somewhere on the web.

The second half is a small library of ready-to-paste Claude prompts. You open one and it auto-fills the machine-specific paths — the engine executable, the bundled example analyzers, and the templates — so it’s ready to paste with nothing to look up. The pairing works because NLP++ is a logical, rule-based language: its passes, rules, wildcards, and knowledge-base operations are structured and pattern-like, so once Claude has studied the example and template analyzers it reproduces the conventions faithfully. And the prompts reach beyond a first build to the whole workflow of an analyzer — gathering and cleaning a development corpus off the web, adding grammar passes, writing dictionaries and knowledge bases, generating edge-case test inputs, and debugging why a rule does or doesn’t fire. The guidance is simple: start from the closest-fitting prompt and edit it rather than writing from a blank page.

From scratch: chemical formulas

A complete, self-contained worked example that carries an analyzer from nothing to working output. The prompt first has Claude assemble a corpus — short excerpts from Wikipedia chemistry articles (Water, Sodium chloride, Photosynthesis, Combustion, Carbon dioxide) that are dense with inline formulas — transcribing Unicode subscripts to plain ASCII (H₂O to H2O) and deliberately seeding near-miss distractors like Roman numerals and catalog codes. It then builds ChemFormulas from the Knowledge Base template: an analyzer that finds formulas embedded in prose, splits each into element symbols and atom counts, validates every symbol against a generated element.dict, and emits JSON through JsonKB. Reach for it to watch an analyzer built end to end, or as a concrete model for your own from-scratch build.

NLP++ local install — build an analyzer

The generic starting point for any brand-new analyzer. It hands Claude the installed engine, example, and template paths, then sets the conventions that keep it on the rails: study a few example and template analyzers first to learn the analyzer.seq sequence and pass regions; copy the Knowledge Base template and use it as intended; accumulate results into a knowledge base and serialize them with SaveKB and JsonKB rather than hand-rolling JSON; and run with the -WORK switch pointed at the engine directory. Two blanks at the bottom are yours to fill — a description of the corpus and of the extraction task. Use it whenever you’re starting a new analyzer and supplying your own domain.

Harden an existing analyzer

For an analyzer you already have and want to make more robust. The prompt asks Claude to generate additional, varied text files — with realistic variety and the edge cases you name — under the analyzer’s input directory, run them through the engine, and report back any inputs where the extraction looks wrong. That gives you a targeted list of failures to tighten rules against before you re-bless your regression goldens. You fill in the kind of text and which variations to emphasize. Use it to grow test coverage and flush out precision and recall gaps on an analyzer that already works.

Create dictionaries and knowledge bases

Also for an existing analyzer: it builds NLP++ dictionaries (.dict, the flat “word attr=val” format) and knowledge bases (.kbb, the indented concept hierarchy), learning the exact layout — including how a .dict is paired with a .kbb of the same stem — from the shared language and misc libraries (currencies, country codes, timezones, and the like), and places the results under the analyzer’s kb/user directory. You describe what the dictionary or KB should contain and how its entries should be organized. Use it when your grammar needs a gazetteer, a lexicon, or a structured knowledge source.

Add missing words to the English dictionary

A precision-minded prompt for extending the shared full English dictionary from the words an analyzer failed to recognize. Claude locates missing_words.txt in the analyzer’s output log, de-duplicates and lowercases it, and for each genuine word writes a properly featured entry — part of speech, a root lemma on inflected forms, verb form and tense readings, noun number — into both en-full.dict and en-full.kbb, keeping them alphabetized with identical headword sets and matching line endings. Obvious noise — typos, fragments, single letters, proper nouns — is diverted to a missing_words_skipped.txt for your review rather than allowed to pollute the dictionary, and if en-full-feat.dict (the featured source of truth) is present, the same entry is added there so it survives a future regeneration. It finishes by re-running the analyzer to confirm the words are now found. Use it when an analyzer leans on the English dictionary and keeps tripping over unknown words.

In every case, what Claude hands back is plain, deterministic, glass-box NLP++ — passes, rules, and dictionaries you can read, diff, version, and re-run to the same result. Claude gathers the texts, mimics the patterns, and explains the passes, but it is a development partner, not the runtime. The engine that ships your extraction is still the compiled, reproducible NLP++ you can see all the way down.

Try it

Version 3 is the release where NLP++ became a compilable, cloud-buildable, pip-and-npm-installable engine — without giving up the thing that makes it special: you can always read the rules.