ADCS on a Luna HSM: the KSP service-account gotcha that stops your CA

Your issuing CA has run flawlessly for fourteen months. A routine patch window reboots it, and Active Directory Certificate Services refuses to start. The event log says the keyset does not exist. The HSM is healthy, the partition is intact, the key is demonstrably still there — and yet the one Windows service that needs it is blind. Nobody deleted anything. What broke is something almost nobody documents in the build guide: who the Luna KSP was registered for.

This chapter of the Luna Field Guide explains how CNG key storage provider registration actually works on Windows, why registering the Luna KSP under your own admin session produces a CA that works right up until its first cold restart in the wrong conditions, and how to diagnose and design out the single most common ADCS-on-Luna failure we see discussed in vendor knowledge bases and community threads. If you’re new to the platform, start with our Luna HSM design and operational guide for the architecture context.

How CNG provider registration actually works

Windows CNG (Cryptography API: Next Generation) doesn’t talk to a Luna HSM directly. It talks to a key storage provider — the SafeNet Key Storage Provider that ships with the Luna HSM Client — which translates CNG calls into PKCS#11 operations against a partition. Installing the client puts the KSP binaries on disk, but that alone does nothing. The provider has to be configured, and here is the property that drives everything else in this article: Luna KSP slot registration is per Windows account, not per machine.

When you run the registration utilities — KspConfig.exe (graphical) or kspcmd.exe (command line), both living in the KSP folder of the client install — you register two things. First, the security library: the path to cryptoki.dll, so the KSP knows how to reach the HSM. Second, one or more HSM slots, and each slot registration is made for a specific user and domain. The utility literally presents “Register For User” and “Domain” selections. The registration stores the partition and its Crypto Officer challenge in the KSP’s own configuration (kept in the registry under the SafeNet KSP hive), keyed against that account.

The consequence: when a process asks CNG for a key through the SafeNet KSP, the provider looks up the slot registration for the account that process runs as. If there’s no registration for that account, the provider has no partition, no credentials, and — from that process’s point of view — no keys. Not an access-denied error against a visible key. No keys at all.

Why “it worked when I built it” proves nothing

Now put yourself in the build session. You’re logged in as a domain admin. You run KspConfig.exe, register the library, register the slot — for yourself, because that’s the account the dropdown offers first. You verify:

certutil -csplist

The SafeNet Key Storage Provider appears and passes. You install the CA role, select the SafeNet KSP as the cryptographic provider, and key generation succeeds — because the wizard is doing its cryptographic work in a context where your registration is visible. Every check you can conveniently run passes, because every check you can conveniently run executes as you.

But the Certificate Services service (certsvc) does not run as you. It runs as Local System — NT AUTHORITY\SYSTEM. For the CA service to open its private key, the slot must be registered a second time, for the SYSTEM account in the NT AUTHORITY domain. Thales’s own integration guidance for AD CS is explicit that both registrations are required. Miss the second one and you have built a CA whose key is visible to your logon session and invisible to the service that owns it.

Why doesn’t this always explode immediately? Sometimes it does — the service fails on first start and gets fixed during the build. The dangerous cases are the ones where the gap hides: the SYSTEM registration was done on the primary node but forgotten on the standby; a Luna client upgrade or reinstall during a patch cycle reset the KSP configuration; the partition’s Crypto Officer challenge was rotated and the stale cached credential only matters at the next cold key-open. In all three, the running CA keeps its established session and open key handles, and nothing surfaces until the service has to start from scratch — which is precisely what a reboot or a failover makes it do.

The failure pattern: months of runtime, one patch reboot

Here is the composite scenario, assembled from vendor KB threads, community forum posts and recurring field patterns in regulated environments — not from any specific engagement.

An agency runs a two-node issuing CA arrangement against a Luna Network HSM: an active CA and a cold-standby build intended for recovery. The standby was built by a contractor months ago; they registered the KSP, saw certutil -csplist pass in their own session, confirmed the partition was reachable, and signed the runbook. Fourteen months later a monthly patch window reboots the active node — or, in the uglier variant, forces a swing to the standby. certsvc fails to start. The Application event log records Event ID 100: Active Directory Certificate Services did not start: Could not load or verify the current CA certificate … Keyset does not exist 0x80090016 (NTE_BAD_KEYSET).

The incident call goes exactly where you’d expect. “Keyset does not exist” reads like the CA key is gone, so the first hour is spent confirming the HSM didn’t lose it. It didn’t — the partition contents are intact, and an admin who logs on interactively can even see and verify the key, which deepens the confusion: the key exists for one human and not for the machine. Someone raises restoring from backup; someone else asks whether the root needs to re-sign anything. (It doesn’t — and if your root is an offline Luna-backed CA, resist the urge to power it up mid-incident; that ceremony has its own discipline, covered in our chapter on proving the restore of an offline root CA on a Luna HSM.) The actual fix, once someone thinks to ask which account the KSP was registered for, is ten minutes of registration and a service start.

The pattern is so common because every element of it is individually reasonable. The build verification was done honestly — in the wrong security context. The CA ran long enough that the build session was forgotten. And the error message points at the key, not at the per-account lookup that failed to find it.

Diagnosing it: event signatures and certutil

Three signatures, in rough order of frequency:

• Event ID 100, error 0x80090016 (NTE_BAD_KEYSET), “Keyset does not exist”. The KSP loaded, but found no slot registration (or no key) for the calling account. This is the classic missing-SYSTEM-registration signature.
• Event ID 100, error 0x8009001d (NTE_PROVIDER_DLL_FAIL), “Provider DLL failed to initialize correctly”. The KSP itself couldn’t come up in the service context — library not registered, client not installed correctly, or HSM connectivity dead at service start.
• Intermittent start failures after reboot only. The service raced NTLS: certsvc started before the Luna client could reach the appliance. Same event ID, but it succeeds on manual restart — a dependency-ordering problem, not a registration one.

The single most useful diagnostic habit is to run your checks in the service’s context, not yours. certutil -csplist in your admin window tells you about your account. Run it as SYSTEM (Sysinternals PsExec is the usual tool) and you’re finally testing what certsvc will see:

:: What YOUR account sees
certutil -csplist

:: What the CA service sees (run as SYSTEM)
psexec -s -i cmd.exe
certutil -csplist
certutil -verifykeys

If the SafeNet Key Storage Provider passes in your session and fails (or the CA key verification fails) in the SYSTEM session, you have your answer without touching the HSM. You can also confirm which provider the CA is bound to under HKLM\SYSTEM\CurrentControlSet\Services\CertSvc\Configuration\<CA name>\CSP — if that says SafeNet and the SYSTEM-context checks fail, the diagnosis is complete.

The correct registration model

Conceptually, a correct ADCS build registers the Luna KSP twice per slot: once for the administrative account performing the installation (so the CA setup wizard and your maintenance sessions can operate), and once for NT AUTHORITY\SYSTEM (so certsvc can). Don’t be thrown if the utility’s registered-slots view appears to show only one entry afterwards — Thales’s documentation notes both registrations exist even when the display collapses them.

Three operational corollaries fall out of the per-account model:

• Every node gets both registrations. Standbys, DR builds, rebuilt servers. KSP registration is local configuration; it does not replicate, restore with the CA database, or survive a client reinstall.
• Credential rotation touches the KSP. The registration caches the partition’s Crypto Officer challenge. Rotate that challenge and every registration holding the old value is now a delayed-action outage, armed until the next service restart.
• Custom service accounts change the target. If a service runs as a named domain account rather than Local System — common for non-ADCS CNG consumers — the slot must be registered for that account. The rule is always the same: register for the account the process runs as.

The full step-by-step, including the kspcmd command-line equivalents and the pre-production reboot test, is in the how-to companion to this chapter: how to register the Luna KSP for the ADCS CA service account.

Two trust layers on one CA server

It’s worth being precise about what KSP registration is not. Your CA server holds two entirely separate trust relationships with the Luna appliance, and they fail independently:

• The NTLS client identity — machine-level, certificate-based, binding the server (by hostname or IP) to the appliance. If this breaks, nothing on the box can reach the partition, in any account context.
• The KSP slot registration — per-Windows-account, credential-based, binding one account to one partition. If this is missing for an account, only that account is blind; others may be fine.

This is why “the admin can see the key but the service can’t” is such a strong diagnostic: it proves NTLS is healthy and isolates the fault to the account layer. Conversely, if no context can reach the HSM after a network change or DR event, you’re in NTLS territory — and that failure mode has its own chapter: NTLS vs STC, and why your HSM connection dies after a DR failover. In an IRAP-assessed environment you’ll want both layers named in the system security plan, because an assessor asking “what happens when this CA restarts unattended?” is really asking about both.

Where this leaves you

Your CA now survives its own reboots: the key lives on the HSM and the service account can actually see it. The next fragility is the HSM itself — a single Luna appliance behind your issuing CA is a single point of failure for every certificate issuance in the estate, and the fix (an HA group that replicates the CA’s key material across members) has sharp edges of its own around what replicates and what doesn’t. That’s the next chapter: the issuing CA on a Luna HA group. The full series lives at The Luna Field Guide.