By most checklists, the environment was “working”, except it wasn’t.

This article documents what happened next. No new directions, no future training steps. Just the troubleshooting journey required to turn a technically valid Workbench project into something I could actually experiment in.

Starting From a “Working” Container That Wasn’t

After creating the Workbench project:

• The custom base container validated and pulled successfully ^✅

• The GPU was visible ✅

• CUDA was available inside the container ✅

And yet:

• No JupyterLab

• No VS Code

• No obvious interactive entry point

• The Packages UI was completely empty

At the infrastructure level, everything looked correct. At the usability level, the project was effectively dead on arrival.

The First Wrong Assumption: Packages and Applications

The obvious next step was to add JupyterLab through the Workbench UI.

I clicked Add JupyterLab.

The UI accepted the click.

Nothing happened.

No error. No feedback. No application created.

That’s when I realized:

In AI Workbench, “Packages” and “Applications” are not the same thing.

Applications require two things:

• The runtime dependencies must already exist in the container

• Metadata must be present to tell Workbench how to manage them

Without both, the UI quietly does nothing. Great.

Why JupyterLab Could Not Be Added

So why couldn’t it be added, well, it’s pretty simple. The custom GHCR-based base image did not include JupyterLab. AI Workbench does not install applications into custom containers.

If the binary does not exist, the Add JupyterLab action fails silently.

At this point, I had a choice:

• Install JupyterLab manually inside the running container (not the best choice)

• Or fix the base image properly (went with this one)

Ad-hoc installs would work temporarily, but they would not be reproducible and they would fight the way Workbench is designed to operate. It’s better to have a correct base image to build on.

Fix the Base Image, Not the Project

A pattern was already emerging through my testing. The base image had required wrapping to satisfy Workbench metadata in Part 1. Tooling needed the same treatment. Rather than layering fixes inside a single project, I decided to rebuild the GHCR base image, saves time in the long wrong and makes this reproducible (fewer moving parts) and reusable (scales for future projects too).

How Does AI Workbench Actually Discover Capabilities?

NVIDIA’s documentation confirmed something important: AI Workbench does not infer or scan container contents. It discovers capabilities entirely through labels.

For package management, two labels are required:

• com.nvidia.workbench.package-manager.apt.binary

• com.nvidia.workbench.package-manager.pip.binary

This explained everything:

• Why the Packages UI was empty

• Why dependency management was unavailable

Workbench had no declared way to manage packages!

Verifying Package Manager Paths

We can’t go guessing the paths, not good enough. I verified the actual binary locations inside the NGC PyTorch base image:

• apt at /usr/bin/apt

• pip at /usr/local/bin/pip

The nuance here matters:

If the label path does not match the real binary exactly, Workbench fails silently.

Baking JupyterLab Into the Base Image

With the model clear, JupyterLab belonged in the base image.

During the build:

• Installed JupyterLab

• Installed ipykernel

No runtime installs. No state drift. JupyterLab now exists as a first-class capability of the environment.

Rebuilding the Wrapper Image Correctly

The updated Dockerfile now included:

• Required Workbench OS labels

• CUDA metadata

• Package manager labels

• JupyterLab installation

The image was rebuilt using --no-cache to avoid stale layers.

Tagging, Pushing, and Relearning the Same Lesson

A new pinned tag was used. The image was pushed to GHCR. Labels were verified on the pulled image, not just the local build. Once again, the lesson held:

AI Workbench validates remote metadata, and latest should never be used for base environments.

Repointing the Workbench Project

The Workbench project was updated to reference the new pinned image tag.

This time:

• OS metadata validated

• Package managers were recognized

• The Packages UI appeared

• JupyterLab functioned correctly

The project finally functioned like a true development environment, similar to the default one included with NVIDIA Workbench.

Final Sanity Check

A dedicated GPU validation script was run.

Confirmed:

• CUDA availability

• GPU detected

• BF16 support

• Tensor operations executed on GPU successfully

This was the first point where the environment wasn’t just “working”. It was actually usable.

Core Lessons From This Phase

• AI Workbench is metadata-driven, not convenience-driven

• Custom containers must explicitly declare capabilities

• Silent failures usually indicate missing metadata, not runtime errors

• Fixing the base image once is cheaper than fixing every project

• latest is actively harmful in Workbench workflows

Closing the Loop

This article intentionally closes the loop on environment readiness.

From here on, the platform is stable. Training, fine-tuning, and experimentation can finally begin.

Stay tuned for Part 3 where that work begins actively.

This article documents the steps, decisions, and some of the failures I hit while getting NVIDIA AI Workbench running cleanly on DGX Spark with a Blackwell-capable PyTorch base image.

By the end of this article, you will have:

• AI Workbench running on DGX Spark (This is pretty straightforward)

• A CUDA-enabled PyTorch environment that supports sm_121 (This is where I had some issues)

• A clean, reproducible base image

• A foundation suitable for fine-tuning modern models

What I Was Trying to Accomplish

The requirements were non-negotiable:

• Use NVIDIA AI Workbench as the development control plane

• Run on DGX Spark (GB10, Blackwell, sm_121)

• Fine-tune modern models (starting with Phi-4)

• Use CUDA-enabled PyTorch, not CPU-only fallbacks

• Avoid silent GPU architecture incompatibilities

Failure #1: Generic Python Base Images

Starting from a standard Python base image fails exactly how you would expect:

• PyTorch installs as +cpu

• torch.cuda.is_available() returns False

AI Workbench does not magically make PyTorch CUDA-aware. The base image determines everything.

Failure #2: Built-in Workbench PyTorch Images

The next logical step was to use NVIDIA-provided PyTorch base images directly from AI Workbench.

Symptoms:

• CUDA appears available

• GPU is visible

• Warnings about unsupported architecture: sm_121

Root cause:

• These images were compiled for sm_80, sm_86, and sm_90

• Blackwell (sm_120 / sm_121) support was not present

At this point the issue was not Docker or Workbench; it was the PyTorch build target.

Choosing a PyTorch Base That Supports Blackwell

Once the constraint was clear, the solution was straightforward.

NVIDIA NGC PyTorch Container

The working base image:

• nvcr.io/nvidia/pytorch:25.12-py3

Why this image:

• Blackwell-capable (sm_120, compatible with sm_121)

• CUDA 13.x user-space

• NVIDIA-validated for DGX-class systems

From a GPU and framework standpoint, this is the correct foundation.

Unfortunately, AI Workbench adds another constraint, this led to some trial and error.

Why NGC Images Fail in AI Workbench by Default

NGC images are valid Docker images. They are not valid AI Workbench base environments.

AI Workbench validates image metadata before pulling layers. If required labels are missing, the image is rejected immediately with errors such as:

• invalid base environment (invalid OS)

• no OSDistro set

• no OSDistroRelease set

You must explicitly provide the metadata Workbench expects.

The Fix: A Minimal Wrapper Image

This is the cleanest solution and the one that scales. No recompiles. No rebuilding PyTorch. Just metadata.

Inspect the Base OS

Inside the NGC container:

• OS: linux

• Distro: ubuntu

• Release: 24.04

Verified via /etc/os-release.

Wrapper Dockerfile with Workbench Metadata

The wrapper image does one thing only; it adds the labels required by AI Workbench.

FROM nvcr.io/nvidia/pytorch:25.12-py3

LABEL com.nvidia.workbench.schema-version="v2" \
      com.nvidia.workbench.name="NGC PyTorch 25.12 (Workbench)" \
      com.nvidia.workbench.description="Wrapper for nvcr.io/nvidia/pytorch:25.12-py3 with Workbench metadata" \
      com.nvidia.workbench.image-version="25.12.1" \
      com.nvidia.workbench.cuda-version="13.0" \
      com.nvidia.workbench.os="linux" \
      com.nvidia.workbench.os-distro="ubuntu" \
      com.nvidia.workbench.os-distro-release="24.04" \
      com.nvidia.workbench.programming-languages="python3"

No software changes are introduced.

Building and Verifying the Wrapper Image

docker build --no-cache -t nvwb-pytorch-25.12:latest .

Verify that the labels are present:

docker inspect nvwb-pytorch-25.12:latest \
  --format '{{range $k,$v := .Config.Labels}}{{println $k "=" $v}}{{end}}' \
  | grep 'com.nvidia.workbench'

If these labels are missing or incorrect, AI Workbench will reject the image.

Publishing the Image to GitHub Container Registry (GHCR)

Why GHCR

• AI Workbench needs a container URL that can be pulled, and I chose GHCR for this purpose. You can choose any option you prefer, as long as the image can be pulled from a valid URL.

• Local images are ignored

• GHCR works reliably for personal and public images

Authentication Requirements

I used a classic GitHub Personal Access Token. It might be better to use a fine-grained token, but since this is a local development setup, I didn't think it was necessary to determine the exact permissions needed and kept it basic.

• read:packages

• write:packages

echo "$GH_TOKEN" | docker login ghcr.io -u <username> --password-stdin

Docker Credential Helper Pitfall

My Docker config initially contained:

"credHelpers": {
  "ghcr.io": "workbench"
}

This silently overrides standard Docker authentication and causes:

• Failed pushes

• wb-svc errors

• AI Workbench unable to read image metadata

Fix:

• Remove the credHelpers entry for ghcr.io

• Allow Docker to use auths normally

This is subtle and easy to miss.

Tagging and Pushing (Pinned Tags Only)

Avoid latest.

docker tag nvwb-pytorch-25.12:latest ghcr.io/brianbaldock/nvwb-pytorch-25.12:25.12.1
docker push ghcr.io/brianbaldock/nvwb-pytorch-25.12:25.12.1

Why latest Causes Workbench Validation Failures

AI Workbench validates metadata before pulling layers.

With latest:

• Multiple digests

• Registry-side caching

• Inconsistent metadata resolution

Pinned tags remove ambiguity and resolve validation failures immediately.

Creating the AI Workbench Project

In AI Workbench:

• New Project → Custom Container

• Container URL:

ghcr.io/<username>/nvwb-pytorch-25.12:25.12.1

Result:

• Base environment validated

• Project created successfully

• GPU visible inside the container

Final Verification

Inside the container:

import torch
print(torch.__version__)
print(torch.cuda.is_available())
print(torch.cuda.get_device_name(0))

Expected output:

• CUDA-enabled PyTorch

• NVIDIA GB10

• No architecture warnings

Key Takeaways

• AI Workbench is metadata-driven, not just Docker-driven

• NGC containers require wrapper labels

• Blackwell requires the correct PyTorch build target

• GHCR authentication is not Git authentication

• Docker credHelpers can silently override auth

• latest is unsafe for Workbench base images

• Wrapper images are the cleanest long-term approach

What’s Next

This article establishes a clean, reproducible base.

Next articles in this series will build on it:

• Fine-tuning Phi-4 on DGX Spark

• LoRA vs QLoRA on Blackwell

• Serving models with vLLM

• Multi-container AI Workbench workflows

Python environments have burned me more times than I want to admit. Different projects, different versions, weird PATH collisions. The usual mess. PyEnv cleaned that up for me on MacOS and Linux, so when I found the Windows version, I was all in.

PyEnv for MacOS and Linux: https://github.com/pyenv/pyenv
PyEnv for Windows: https://github.com/pyenv-win/pyenv-win

My workflow lives in VS Code, and I want things to just work. If I open a repo that needs a specific Python version, I want that version active immediately. No guessing. No scrolling through interpreter lists. So I wired up a small PowerShell trick to keep everything in sync.

Here is the flow.

I open a repo. I know it needs Python 3.10. I run:

pyenv local 3.10

PyEnv drops a .python version file into the folder. The only job left is making sure my shell respects it.

This line in my PowerShell profile does exactly that. It puts the PyEnv shims at the front of PATH:

$env:Path = "$($env:USERPROFILE)\.pyenv\pyenv-win\shims;$($env:USERPROFILE)\.pyenv\pyenv-win\bin;" + $env:Path

I keep this line in two places so everything behaves the same way:

%userprofile%\Documents\PowerShell\Microsoft.PowerShell_profile.ps1
%userprofile%\Documents\PowerShell\Microsoft.VSCode_profile.ps1

Why put it in the profile instead of editing the global Windows PATH.

For me, it is safer and more predictable. The change applies only to the current shell session, and PyEnv’s shims always load first. Nothing else on the system silently overrides them.

In the end, it is one small line that keeps my Python setup clean and my VS Code projects consistent on Windows. No more surprises.

Like this post if you find it useful!

If you’ve ever tried to embed a Microsoft Copilot Studio agent inside Salesforce, you’ve probably learned the hard way that it’s not as simple as dropping in an iframe. Between Salesforce Locker restrictions, Entra ID redirect rules, and Copilot Studio’s federated auth model, it takes some serious tinkering to get everything to line up.

That’s why I built LightningCopilot. It’s a Lightning Web Component that brings a Copilot Studio agent to life inside Salesforce with full Entra ID SSO, MSAL-based auth, and clean token flow from start to finish. The chat experience runs through a BotFramework WebChat based lightweight chat experience with adaptive cards, all without breaking Locker.

This post walks through the whole thing so you can wire it up yourself across Entra ID, Copilot Studio, and Salesforce.

The repository is published here: github.com/brianbaldock/LightningCopilot

What you’ll build

• A Salesforce Lightning Web Component that securely hosts a Copilot Studio Agent

• Seamless Entra ID SSO with MSAL.js

• Properly scoped Salesforce Static Resources, Custom Labels and CSP Trusted URLs

Architecture Overview

Everything starts in the Lightning Web Component. It loads the required libraries from Salesforce Static Resources, signs in users through Entra ID using MSAL, and passes authentication to the Copilot Studio agent.

At runtime, the component connects to Microsoft’s Power Platform APIs and Bot Framework endpoints to enable real-time conversations.

Using Static Resources was intentional. In Salesforce, scripts loaded directly from CDNs can break under Locker restrictions or CSP rules. Packaging them as Static Resources, you get to control versioning, avoid runtime issues, and keep the entire bundle self-contained, no external script calls needed. I chose this method for simplicity but feel free to test with script loading from CDN if you prefer.

Custom Labels follow the same logic. Instead of hardcoding client IDs, region URLs, or agent endpoints in your JavaScript, labels let you manage those values from Salesforce Setup. It’s cleaner, secure, and easier to update between environments without touching the code.

Let’s get started!

Prerequisites (I’m assuming you have this already)

• A Salesforce org (and know how to create a utility bar component)

• A published Copilot Studio Agent

• Access to Microsoft Entra ID (Application Administrator is sufficient RBAC on the app registration itself)

• Access to Azure Cloud Shell to enable the Copilot SPN.

• Notepad - for copy pasting information from this guide as well as the different platforms outlined below.

Prepare the Agent

Chances are you have a Copilot Studio Agent already. If not, create one.

Assuming you already have a published agent, open it in https://copilotstudio.microsoft.com. Depending on your Power Platform environment type, some authentication or configuration options might not appear. I’d recommend checking that these settings are available; if they’re missing, you’ll likely need your IT team to spin up a new Power Platform environment for Copilot Studio, ideally using a Dev or Sandbox environment type. See the screenshot below:

1. Once able, select “Authenticate manually” and click “Save”.

   Note: Much of this form is filled out by default and you need only note down the values.

2. Add the “Redirect URL” to your notes.

3. Select “Microsoft Entra ID V2 with federated credentials” from the drop down.

4. Add the “Federated credential issuer” to your notes.

5. Add the “Federated credential value” to your notes.

6. Add the “Client ID” to your notes.

7. Note*:* The “Scopes” will likely only contain “profile” and “openapi”, this is okay for now. We’ll come back to this later.

Prepare the Entra ID App Registration

• Access the Entra Admin Center.

• Click “App Registrations”.

• Select your agent from the list. You should see your agent name with the (Microsoft Copilot Studio) suffix.

• Note down your Application (Client) ID, Directory (Tenant) ID.

1) Authentication settings

• Click “Authentication”, and click “Add platform”

• Select “Web”

  • Enter the following value: https://token.botframework.com/.auth/web/redirect

• Click “Add platform” again, this time selecting “Single-page application” (SPA)

• Add the following SPAs:

  • https://api.powerplatform.com/CopilotStudio.Copilots.Invoke

  • https://<YOUR-ORG>.<develop,sandbox, etc>.lightning.force.com

  • https://<YOUR-ORG>.<develop,sandbox, etc>.lightning.force.com/ ← note the trailing slash

  • https://<YOUR-ORG>.<develop,sandbox, etc>.lightning.force.com/lightning/page/home

• Select “ID tokens (used for implicit and hybrid flows)”

• Should look something like this when you’re done:

2) Certificates & Secrets

1. Click “Federated credentials”, there are likely already 2 credentials here.

2. Click “Add credential”.

1. From the “Federated credential scenario” dropdown select “Other issuer”.

2. The “Issuer” in this context is the following with your tenant ID: https://login.microsoftonline.com/<TENANT ID>/v2.0

   Leave the “Type” as is: “Explicit subject identifier”.

3. In “Value” drop in the “/eid1” generated when you created the manual authentication settings in your agent. This field should be populated in your agent’s “Federated credential value” field.

4. Name this credential something clear and easily identifiable like, “FED_ID_AGENTNAME”.

5. Provide a description, example: “Federated credential for <AGENTNAME> for Salesforce SSO flow“

6. Click “Add” to save the settings

3) API Permissions

For this section we need to create the Power Platform API service principal for the tenant. The easiest and quickest way to do this is via the Azure Cloud Shell. Open a new tab in your browser and access the Cloud Shell at the following link: https://shell.azure.com

Later we’ll be adding the CopilotStudio.Copilots.Invoke delegated permission in Entra ID, it shows up as part of the Power Platform API. That’s expected, Copilot Studio is built on top of the Power Platform service layer. The documentation under Power Platform Admin API covers this same backend service.

az ad sp create --id 8578e004-a5c6-46e7-913e-12f58912df43

With the SPN enabled we can now go back to Entra ID and add additional API Permissions. Open up your Agent’s App Registration again and click on the “API Permissions” pane.

1. Under API Permissions select “Add Permission”.

1. Select "APIs my organization uses”.

2. Search for “Power Platform API” and select it.

1. Search for “CopilotStudio.Copilots.Invoke”.

2. Check the “CopilotStudio.Copilots.Invoke” option.

3. Click “Add permissions”.

1. Click “Grant admin consent for <Your Org Name>”.

4) Expose an API

1. Select “Expose and API”.

2. Click “Add" next to “Application ID URI”

1. Click “Save” (the generated URI is sufficient for our purposes)

1. Next, click “Add a Scope”.

2. For "Scope name” enter: "Chat.Invoke”.

3. Who can consent?: “Admins and users”.

4. Admin consent display name: “Copilot Invocation”.

5. Admin consent description: “Invoke Copilot Agent”.

6. User consent display name: “Copilot Invocation”.

7. User consent description: “Invoke Copilot Agent”.

8. State: “Enabled”.

9. Click “Add scope”.

When you’re done it should look something like this:

Update the Scopes in your Copilot Agent

• Now copy the full scope URI from under “Scopes” see the Clipboard button next to the api:// address.

• Go back to Copilot Studio and select you Agent, click Settings, Security, Authentication and paste the full api://<GUID>/Chat.Invoke at the end of the “Scopes” text field, leave a space after the existing scopes there.

• 

Building the LWC

1. Clone the GitHub repository here: https://github.com/brianbaldock/LightningCopilot

2. At the root of the project, type the following commands in order

# Install node modules:
npm install

I’ve added a build-static-resources.mjs which pulls the latest components required for Adaptive Cards, MSAL, and Copilot Studio Client from various modules and SDKs. The script is here: scripts/build-static-resources.mjs

You should see a result like this.

Rebranding

This is a good time to rename your agent project because chances are you’re not calling it LightningCopilot (though it is a pretty cool name huh 😉)

To publish under a different brand:

• Rename this folder: lightningCopilot/main/default/lwc/lightningCopilotAuth

• Rename classes/export: Update exported class LightningCopilotAuth to new name.

• Event names: Replace: lightningcopilotauthsignin, lightningcopilotauthsignout, lightningcopilotautherror.

• HTML labels: Change “Sign in to Lightning Copilot” to the new branding text.

• CSS namespace: Root class .lightning-copilot-shell → new scoped class.

• Logging prefix: Update [LightningCopilotAuth] occurrences.

• sfdx-project.json: Adjust package directory path if source folder renamed.

• README: Replace occurrences of LightningCopilot with new brand. Keep naming internally consistent to avoid broken imports.

Pushing the project to SalesForce

At the root of the project folder do the run the following commands.

sf org login web --alias MyOrg --instance-url https://login.salesforce.com
sf config set target-org MyOrg --global
sf project deploy start

Creating the Static Resources

Static Resources in Salesforce are essentially packaged files (JavaScript, CSS, images, or libraries) that you upload once and reference securely inside Lightning Web Components or Aura apps. Instead of linking to external CDNs or hardcoding paths, Static Resources let you bundle everything you need (like msalBrowser, copilotStudioClient, and adaptiveCards) directly into your org. This keeps your component self-contained, avoids Locker and CSP violations, and gives you full control over versioning and deployment between environments.

Let’s create the static resources:

1. In SalesForce select the “Setup” gear.

2. Search for “Static Resources” and enter the Static Resource screen.

3. Create three new Static Resources with the following details:

When you’re done, it should look something like this:

Creating the TrustedURLs

Trusted URLs in Salesforce define which external domains your org is allowed to load resources or make network calls to. Because of Salesforce’s strict Content Security Policy (CSP), anything not explicitly trusted will be blocked, including scripts, iframes, or API requests from your Lightning Web Component. By adding these Microsoft endpoints as Trusted URLs, you’re telling Salesforce that communication with Copilot Studio, Power Platform, and Entra ID is safe and intentional. This step ensures that token exchanges, Adaptive Card rendering, and chat functionality all work correctly within Locker’s sandboxed environment.

1. In SalesForce select the “Setup” gear.

2. Search for “CSP” and select “Trusted URLs”

3. Click “New Trusted URL”

4. All of the Trusted URLs listed below follow the same recipe, the only difference is the URL and the API Name.

   • Trusted URL Recipe:

Here is the list of URLs:

When you’re done, it should looks something like this:

Creating the Custom Labels

Custom Labels in Salesforce let you store configurable values, like URLs, client IDs, and scopes, that your Lightning Web Component can reference at runtime. Instead of hardcoding these details into your code, labels make it easier to manage environments (Dev, Test, Prod) without redeploying the component. It’s also more secure and keeps sensitive identifiers out of your source files. Think of them like environment variables.

1. In SalesForce select the “Setup” gear.

2. Search for “Custom Labels” and select “Custom Labels”

3. Click “New Custom Label”

4. You’ll need custom labels for all of the values listed below. This activity will require you to lookup values in Entra ID, Copilot Studio, and do a bit of discovery using Azure App Insights.

List of Custom Labels:

Great list right? Where do we find all the things? Let’s start with the easy ones:

• MSAL_ClientId - Your Copilot Agent’s App Registration Application (Client) ID - Find this in Entra ID.

• MSAL_RedirectUri - Your SalesForce page where the component will be running.

• MSAL_Scopes - These are what we defined (with some additions) to the App Registration earlier in this article. List these out in the custom label with spaces separating each one. - Find this in Entra ID.

• MSAL_TenantId - Your Entra ID Tenant ID - Find this in Entra ID.

• COPILOT_AgentURL

  1. Go to https://copilotstudio.microsoft.com.

  2. Open your agent.

  3. Click Channels, select Demo Website

  4. Copy the URL listed in “Share the URL” to your browsers address bar:

  5. When the page opens notice how the URL changed. Copy this new URL to your notes.

  6. Update the URL the following way:

     • Remove the highlighted parts in the example below:

       • https://copilotstudio.microsoft.com/environments/<ENVIRONMENTGUID>/bots/<NAME>/canvas?version=2&enableFileAttachment=true

     • Replace with the following (ensure to remove any trailing or doubled slashes)

       • /webchat?__version__=2

     • It should looks like this now:

       • https://copilotstudio.microsoft.com/environments/<ENVIRONMENTGUID>/bots/<NAME>/webchat?version=2

• COPILOT_EmbedURL

  1. This one is really tricky to find but luckily you don’t have to do the network tracing, CSP monitoring, trials and errors that I went through to lock this one in. Just take the Environment GUID from the COPILOT_AgentURL above and drop it into this preconstructed link:

     • https://api.bap.microsoft.com/providers/Microsoft.BusinessAppPlatform/environments/<ENVIRONMENTGUID>/copilotAgents?api-version=2023-10-01-preview

When you’re finished, it should look something like this:

Troubleshooting

Agent keeps asking to sign in

• Verify every Salesforce SPA redirect URI exists in Entra ID. Include the trailing slash and /lightning/page/home.

• Confirm https://token.botframework.com/.auth/web/redirect is listed under Web.

• Check the federated credential issuer and value. Issuer must be https://login.microsoftonline.com/<tenant-id>/v2.0. Value should match the /eid1/... from Copilot Studio.

• If MSAL silent auth fails inside Salesforce, set cacheLocation: "localStorage" and storeAuthStateInCookie: true in your MSAL config.

Power Platform API not found in Entra ID

• Create the service principal first (see the section earlier in the article)

• Reopen API permissions and search for Power Platform API. Add CopilotStudio.Copilots.Invoke and grant admin consent.

Adaptive Cards show [object Object]

• Use the non-minified Adaptive Cards build (this should be the case if you followed the article so far) This is mainly noted for posterity.

• Confirm the Static Resource name matches exactly what the LWC imports expect.

MSAL redirect loop or stale cache

• Clear browser storage for your Salesforce domain.

• Verify the SPA redirect URIs exactly match the domain you are testing on.

• In sandboxes, double-check you are not mixing lightning.force.com and my.salesforce.com origins.

Known gotchas by environment

• Locker: avoid dynamic eval or libraries that assume window globals.

• Static Resources: names are case sensitive. Change a name and your LWC fails to load.

• Multiple Salesforce domains: Entra SPAs must exist for each domain you use to test.

• Region drift: moving the agent to another Power Platform environment without updating labels breaks calls silently.

Security notes

• Minimum role to create the Power Platform API SPN is Application Administrator. Cloud App Admin, Privileged Role Admin, or Global Admin also work.

• Keep client IDs, tenant IDs, and environment GUIDs in Custom Labels. Do not hardcode in JS.

• Pin your Static Resource versions with your build script to avoid unplanned upgrades.

Quick validation checklist

• Copilot Studio agent is published in the expected environment.

• Entra ID app has Web and SPA redirects configured.

• Power Platform API with CopilotStudio.Copilots.Invoke is granted and consented.

• Chat.Invoke scope exposed and referenced in MSAL_Scopes custom label.

• Static Resources uploaded and named exactly: msalBrowser, adaptiveCard, copilotStudioClient.

• Trusted URLs added for login.microsoftonline.com, api.bap.microsoft.com, copilotstudio.microsoft.com, directline.botframework.com, your environment-specific HTTPS and WSS endpoints.

• Custom Labels populated and referenced by the LWC.

Wrap up

You should now have a Copilot Studio agent running inside Salesforce with Entra ID SSO, MSAL handling the token flow, and a lightweight Bot Framework chat that behaves under Locker. The setup is opinionated for reliability in Salesforce: Static Resources for scripts, Custom Labels for environment config, and a short list of Trusted URLs to satisfy CSP. From here you can extend the component with richer telemetry, swap the hosted chat for a custom Direct Line client, or light it up inside other Salesforce workspaces.

Repo is here if you want to clone or open a PR: github.com/brianbaldock/LightningCopilot

The research, minus the lab coat

An IBM team studied how people use large language models to write content that is supposed to be tied to real documents. Picture a simple setup; folks are asked to answer questions using a source doc, and an AI offers “helpful” suggestions along the way. Sometimes the AI’s answer is faithful to the source, sometimes it quietly makes stuff up. They also tried a few speed bumps to slow people down and make them think first:

• Answer first; write your own response before seeing the AI.

• Read first; skim the source doc before you start.

• Highlight; show which parts of the AI’s answer actually line up with the source.

What shook out matches what most of us have seen in the wild:

• When the AI invents facts, quality drops. No surprise; if the suggestion is off, the end result gets worse.

• People still lean on bad AI, often by “topping up” their own answer with AI text. I’ve done this, you’ve done this: you write something correct, then paste in a smart-sounding line that quietly disagrees with you. Now you have a confident, wrong paragraph.

• Those speed bumps help mindset, but they are not magic. They nudge people to think, they do not erase bad inputs.

• Three simple checks matter most: faithfulness to the source, factual accuracy, and completeness. If your response fails any one of those, it should not ship.

Bottom line; when the model makes things up, humans can make it worse by trusting or blending the output. The Ars Technica piece is the public version of that lab result.

The public face‑plant

That education report called for “ethical AI”, yet the bibliography had ghosts; over a dozen citations that do not exist. Eighteen months of work, still shipped with phantom sources. That is the “paste the smart sentence and keep moving” problem, just at report scale. Nobody stopped to ask the only question that matters: “where did this claim come from, and can I open it?”

Why we fall for it

A few human things get in the way:

• Anchoring, and saving brain cycles. If you see the AI answer first, it frames your thinking. Even if you write first, you may paste in a clever line to save time.

• Plausibility beats provenance. Smooth, on-topic text feels accurate at a glance. Without a hard source check, fluent hallucinations slide through. And, “I used less AI on this one” is not the same as, “I verified the facts.”

A practical playbook so “ethical AI” stops citing ghosts

Given that report, everything around it, and the research, it’s time for a playbook. Here are some ideas:

1. No source, no ship. If a claim has no link or citation you can actually open and verify, it does not publish. Use three dials to review: faithfulness, accuracy, completeness. Fail one, it fails all.

2. Answer first, compare second, justify third. Capture a human draft before any AI suggestion, then show the AI response side-by-side with citation checks. Add one required sentence: “what changed after seeing AI, and why.” Not every org can build this workflow today, but even a lightweight version in Word or Copilot helps (maybe an Agent?).

3. Automate the boring stuff.

   • Resolve every reference; flag dead links and junky journals.

   • Run a basic overlap check between claims and sources; even a highlight-style view helps reviewers see what is actually grounded.

4. Block on grounding when confidence is low. If the system cannot point to real evidence, kick it to a human. Do not allow publish.

5. Catch the “append the AI” bug. Lint for contradictions between the human draft and pasted AI text. If they disagree, stop and reconcile.

6. Person-in-the-Loop for real. Make escalation a main path, not a side door. If a source cannot be verified, a human decides: fix it or cut it. Log that decision.

7. Spot-check after publish. Sample a few items weekly against the same rubric. Celebrate “caught in review” saves, not just volume shipped.

Bring it home

The IBM study shows how easily hallucinations and overreliance can drag quality down in AI-assisted writing. The “ethical AI” report shows the reputational blast radius when that slips into production. If we want AI to help us move faster without burning trust, we need hard gates for provenance, simple checks humans can actually use, and workflows that make the right thing the easy thing. Otherwise we will keep shipping polished documents that cite ghosts 👻.

References

• Ashktorab, Desmond, Pan, Johnson, Brachman, Dugan, Danilevsky, Geyer. Emerging Reliance Behaviors in Human‑AI Content Grounded Data Generation: The Role of Cognitive Forcing Functions and Hallucinations. CHIWORK ’25. Key findings summarized above; see Figure 5, Table 1, Table 4, and Table 5 for the results I reference. https://arxiv.org/pdf/2409.08937v2

• Edwards, Ars Technica. Education report calling for ethical AI use contains over 15 fake sources. Summary and context for the Newfoundland and Labrador report. https://arstechnica.com/ai/2025/09/education-report-calling-for-ethical-ai-use-contains-over-15-fake-sources/

Whether you’re setting up Internet Access, enabling Private Access, or exploring the benefits of Token Protection, we’ve got you covered.

FastTrack partnered with Product Marketing, GTM, WWL Studios, and Microsoft Learn to build a library of short deployment videos that show you how to get Entra features running quickly.

Each video is hosted on the M365Accelerator site, a hub for FastTrack resources, playbooks, and tools to help IT pro’s move from planning to deployment with confidence. Bookmark it as your go-to resource for accelerating Microsoft 365 adoption.

With these videos, you can go from learning to deploying in minutes.

All of these and more are available on the M365Accelerator website. Start exploring today and deploy with confidence.

Why I built a mini-PowerShell Module

The official documentation explains that you can have more than one access policy and the most restrictive one always wins. To clarify:

• Direct user or group assignments take precedence over org-wide.

• If a user lands in multiple policies, Disabled beats Enabled.

Given that there is no other way to create or manage these policies than the cmdlets called out in the documentation I wanted to create some light automation that handles this in an idempotent way (create or update if needed, no duplicates, safe re-runs) and silent by default (user -verbose to see the noise).

The module

Repo is here: Set-CopilotForEngage

The core cmdlet is Set-EngageFeatureAccess. It does the heavy lifting:

• Ensures the Exchange Online module is installed/updated.

• Connects to Exchange Online if needed.

• Resolves the Viva Engage feature IDs.

• Creates or updates policies for Copilot and/or AI Summarization.

• Handles scope (-Everyone, -GroupIds, -UserIds).

• Supports user controls (-UserOptInByDefault).

Recommended approach

Keep it simple:

• Create an org-wide disabled policy for each feature (baseline).

• Create targeted enable policies scoped to security groups.

• Don’t bother with “disable groups” unless you want carve-outs from a permissive baseline.

That way:

• Default is deny (no one gets access unless explicitly added).

• Membership in an Enable group grants access.

• Most restrictive still applies, but you’re not juggling extra disable layers.

Quick start

Requires: PowerShell 5.1+, ExchangeOnlineManagement 3.9.0+ (the module helper can auto‑install/update with switches) see step 2a

# 1) Dot source the mini module
. .\Set-CopilotForEngage.ps1

# 2a) Baseline: Use this to disable both features org-wide and install/update the to the latest version of Exchange Online Managment PowerShell Module
Set-EngageFeatureAccess -Mode Disable -Copilot -AISummarization -Everyone -PolicyNamePrefix "All" -AutoInstallEXO -AutoUpdateEXO -Confirm:$false -Verbose

# 2b) Baseline: Disable both features org‑wide 
Set-EngageFeatureAccess -Mode Disable -Copilot -AISummarization `
  -Everyone -PolicyNamePrefix "All" -Confirm:$false -Verbose

# 2) Enable Copilot for one or more groups
Set-EngageFeatureAccess -Mode Enable -Copilot `
  -GroupIds "GROUP GUID HERE" `
  -PolicyNamePrefix "Enable" -Confirm:$false -Verbose

# 3) Enable AI Summarization for one or more groups
Set-EngageFeatureAccess -Mode Enable -AISummarization `
  -GroupIds "GROUP GUID HERE" `
  -PolicyNamePrefix "Enable" -Confirm:$false -Verbose

# 4) Verify policy layout
Get-VivaModuleFeaturePolicy -ModuleId VivaEngage

What “good” looks like

The view below shows two org‑wide block policies (one per feature) and two group‑targeted enable policies. With this layout, anyone not in an enable group stays blocked; group members are enabled.

So a short blog article for September where I created a tight little module which solves a real admin pain point, and makes policy management repeatable and predictable.

Code’s up on GitHub if you want to try it: Set-CopilotForEngage

What Secure Score actually is

Secure Score pulls together recommended security improvements across Microsoft 365 into one place with a point-based system. Each recommendation tells you what to do, why it matters, and how many points you’ll earn by completing it. Think of it as a prioritized sanitation checklist for identity, devices, apps, and data.

The number of points isn’t the goal. The real value is in the visible breadcrumb trail that proves you’re making continuous improvements. It’s your “cleaning the line.”

Check it out here (you will need the right role to view it): Microsoft Secure Score

Turn Secure Score into a living backlog

Treat every Secure Score recommendation like a task in your security backlog.

• Pull items weekly into your work queue.

• Filter them by category: Identity, Devices, Apps, Data

• Label them as Quick Clean, Hot Clean, Deep Clean:

  • Quick Clean: low-risk configuration fixes; like wiping down a station on the line during a lull.

  • Hot Clean: high impact controls that attackers target first; like sanitizing the cutting boards, washing the hoods.

  • Deep Clean: structural changes that require planning; like pulling out appliances to clean behind them.

• Use your existing work management tool. If you are Microsoft-heavy, a Planner or Azure Boards view works well so Product, Infra, and SecOps can see the same queue.

Prioritize like a pro

Do not chase easy points first. Use a simple triage that fits enterprise reality.

• Impact: How much real risk it cuts given how attackers operate today. Use exposure management and define critical devices to map attack paths. (Microsoft Security Exposure Management (MSEM) Attack Paths)

• Effort: People hours, complexity, approvals.

• Dependencies: Does something else need to happen first?

• Blast radius: Who could be affected if it goes wrong.

• Regulatory tie-in: Does it map to controls that auditors will ask about.

Score each on a 1 to 3 scale. Hot Clean items with high impact and low to medium effort go to the top.

Examples that usually pay off early:

• Require phishing-resistant MFA for admins; enforce conditional access for high risk sign-ins.

• Disable legacy auth where feasible.

• Safe Links and Safe Attachments policies.

• Attack surface reduction rules in audit, then enforce.

• Device compliance policies with a minimum OS and encryption.

Assign owners; set a cadence; show progress

This is where teams succeed or stall.

• Owner per control: Security writes the “why”, the platform owner runs the change, and the helpdesk knows the support path.

• Weekly working sessions: 30 to 45 minutes. Review last week’s moves, pull two to five new items, unblock dependencies.

• Monthly showcase: One slide that tells a story. Current score, what changed, what risk you removed, and what is next.

Tie it to culture, if we use Microsoft Cultural Attributes as an example (Microsoft cultural attributes): Growth Mindset means you’re iterating; Customer Obsession means you protect users while uplifting controls; One Microsoft means SecOps, Identity, Endpoint and Applications ship together.

Automate the easy wins

Use policy and configuration at scale so hygiene stays done.

• Where possible use baseline policies. Conditional access templates in Entra ID. Security Baselines in Intune.

• Configuration as code for repeatability where possible; even simple scripts tracked in a repo beat one-off portal clicks.

• Alerting for drift. If a control falls out of compliance, it reopens in your backlog automatically.

• Document exceptions with an owner and an expiry date. Exceptions without an end date become permanent debt.

Report upwards with context, not just a number

Executives want to know if the business is safer this month than last.

• Show trend: last quarter, last month, today.

• Translate actions to attack friction. Use wording like: “Blocking legacy auth removed password spray success paths for X% of accounts.”

• Map to frameworks and programs: What is Zero Trust? | Zero Trust Assessment & Workshop

• Keep a small “narrative log” of notable changes and why you chose them.

Common pitfalls

• Point chasing: Hitting a target score without reducing meaningful risk.

• Big-bang changes: Shipping ten identity policies at once and flooding the helpdesk.

• No rollback plan: Always know how to revert a control safely.

• Unowned exceptions: If everyone owns it, no one owns it.

• Stale backlog: If recommendations sit untouched for 90 days, re-evaluate or explicitly accept the risk (at this point you already have).

A simple Microsoft Secure Score 30-60-90 starter plan

Days 1-30 “Set the system”

• Stand up one visible backlog for Secure Score in Planner or Azure Boards; tag items as Quick Clean, Hot Clean, or Deep Clean.

• Define your triage rubric: Impact, Effort, Dependencies, Blast radius, Regulatory tie-in; simple scoring.

• Lock the cadence: a weekly 30 or 45 minute working session with clear entry and exit criteria; clear entry and exit criteria; one owner per item with a rollback checklist.

• Baseline metrics and rules: score by pillar, average days to close, exception policy with owner and expiry.

Days 31-60 “Run the loop, make it boring”

• Work the cadence; move a steady batch from proposed to validated to done.

• Apply the rubric before pulling anything; no point chasing, document user impact and the helpdesk path.

• Add drift checks and auto-reopen on deviation; track rollbacks and lessons learned.

• Publish a monthly showcase with trend, risks reduced, and next moves; close or explicitly accept items that sit for 90 days.

Days 61-90 “Scale it and age it”

• Break Deep Clean items into small, testable steps; add a simple RACI so accountability is clear.

• Map changes to Zero Trust pillars and Secure Future Initiative outcomes; include this view in the showcase.

• Formalize exception reviews; auto-notify owners 2 weeks before expiry, renew or remove with justification.

• Ship v1.0 of the playbook: intake, triage, change, validate, rollback, report; set quarterly objectives for the backlog.

Closing

In the kitchen, you never stand still because there is always something to clean. In security, there is always something to improve. Secure Score is the list on the cooler door. Work it daily, celebrate small wins, and keep moving!

This blog is for security architects and identity engineers trying to build scalable agentic systems that don’t compromise your entire environment. If your agents need to talk to APIs, legacy databases, and each other.

Why agent identity is a different beast

Agentic workloads don’t follow the old patterns. They:

• Spin up and disappear, or maybe not.

• Call services in parallel.

• Act on behalf of users (or other agents).

• Break assumptions about static infrastructure.

We can’t just treat them like apps or users. We need first-class identity that’s dynamic, scoped, and trackable. Microsoft Entra Agent ID helps, but architecture matters more than tooling.

The core protocols (and what they expect from identity)

All of them assume identity is solved, our job is to make that true.

The Agent Identity Architecture Recipe

Here’s a simple truth: you can’t bolt on identity after you’ve built an agentic system. It needs to be part of the foundation. I threw together a recipe you can use to secure agents in real-world environments.

1. Give every agent an identity

   • Agents shouldn’t be invisible, they should show up in your Entra ID tenant like any other principal.

   • Entra Agent ID or app registrations, no shared accounts. No mystery agents.

2. Use modern auth only

   • Use OAuth2 client credentials flow or managed identity for agents calling APIs.

   • Prefer federated identity for agents outside Azure (GitHub, K8s, etc.).

   • Support on-behalf of flows when an agent acts on behalf of a user.

3. Wrap legacy systems with identity-aware proxies

• Put an API layer or proxy in front of legacy tools.

• Let the agent call the API with a token; the API handles legacy auth using credentials from a vault.

4. Vaut every secret

   • Use Azure Key Vault (or HashiCorp Vault) to store DB passwords, tokens, and certs.

   • Grant agents minimal read access scoped to what they need.

   • Rotate secrets on a schedule. Better: make them ephemeral where possible.

5. Enforce Zero Trust

   • Assign scoped roles and use Conditional Access for workload identities.

   • Leverage Agent Cards or scoped JWTs for delegation (especially with A2A).

   • Never assume an agent is “safe” just because it’s internal. Validate everything.

6. Monitor and decomission

   • Use access reviews and audit logs to track usage.

   • Deprovision stale agents. Revoke access. Rotate keys.

   • Feed everything into Sentinel or your SIEM so you can catch weird behavior early.

Quick-scan checklist

• ◻️ Every agent has a unique identity in Entra ID (Agent ID / service principal)

• ◻️ No raw secrets in code or memory—everything is pulled from Key Vault

• ◻️ Service-to-service auth uses OAuth 2.0, mTLS, or federated identity

• ◻️ Scoped permissions only—never more than what’s needed

• ◻️ Legacy systems are abstracted behind modern identity-aware proxies

• ◻️ Conditional Access & role policies apply to agents like any other principal

• ◻️ Secrets are rotated regularly (or eliminated via managed identity)

• ◻️ Full audit trail from user → agent → resource

• ◻️ Monitoring for abnormal behaviour or usage patterns

• ◻️ Lifecycle automation—agents are created, monitored, and decommissioned cleanly

Example: End-to-end Identity Flow from user to SQL via MCP

Let’s try and make this real. Using the previous recipe, let’s outline what it would look like for a user to trigger an action via an agent, which then queries SQL Server via an MCP server. The MCP server could just be used as a wrapper for another API in this case.

Here’s a matrix view of the different flows:

Wrap up

Agentic AI isn’t the future, it’s already live in your environment. I guarantee someone in your org is trying to build an agent right now. The question is: do you know what it’s doing?

The good news is you don’t need to start over. You just need to start being deliberate.

Treat your agents like non-human teammates. Secure them with the same Zero Trust mindset, enforce guardrails like you would for any user, and apply governance from day one. If you’re already using Microsoft Entra ID, you’re not far off. You’ve got the foundation, now it’s just about wiring it up the right way.

Start small. Register your agents. Vault your secrets. Lock down what they can do.

Because when that next agent spins up and hits your data layer, you want to know exactly who kicked it off, what it accessed, and why.

References

• Microsoft Tech Community – Announcing Microsoft Entra Agent ID

• Model Context Protocol – Anthropic MCP Overview

• Agent Card & A2A Protocol – Agent-to-Agent

• Azure Identity Platform Docs – Workload identity federation

• Azure Key Vault – Best practices for secrets management

• Britive Blog – Agentic AI Is Redefining Identity Security in the Cloud

• Techolution Report – How Legacy Systems Sabotage Agentic AI

• How Strata Identity and Microsoft Entra ID solve identity challenges in mergers and acquisitions

• The Identity Problem at AI Scale: Why Agentic AI Demands More From OAuth

I’ve spent over a decade helping customers harden environments, deploy secure solutions, and chase down alerts. But it wasn’t until I started trying to break into systems myself that I really began to understand how attackers think, and honestly, it gave me a headache.

I’m not trying to be some elite super hacker. At best, I’m a proud script kiddie still learning the ropes. The real pros out there see the world sideways; they think in crooked lines and find gaps I wouldn’t have considered. It’s like that classic Abraham Wald story: the places without bullet holes are what really bring the plane down. That analogy fits security a little too well.

I’ve been using my homelab as a safe playground to simulate attacks and sharpen my thinking. There’s something humbling about exploiting your own environment and realizing how many assumptions you’ve made as a defender. I’m also a huge fan of the Hack The Box platform. Shoutout to those folks for keeping it real and challenging.

So, while I’m still early in my offensive journey, I’ve already picked up a few lessons that completely changed how I think about defense. Here are some of the biggest ones so far.

Lesson 1: That moment I got my first reverse shell

The first time I landed a reverse shell honestly blew my mind. Before that moment, I'd spent my entire career locking down systems, hardening environments, and chasing security alerts. Flipping roles and becoming the attacker (even ethically in a controlled setting) completely changed my perspective.

I still remember it clearly. I set up my listener, launched a PowerShell payload on the target, and almost instantly saw a new prompt appear. Just like that, I was inside the Windows machine.

Seeing the command prompt switch and reveal the target’s hostname was eye-opening. It felt like discovering a hidden world beneath a familiar surface, similar to the first time you realize there's an entire community dedicated to geocaching right under your nose. I'd spent years strengthening barriers without fully appreciating the creativity attackers use to bypass them.

Two big insights came from that experience:

1. Attackers always have the initiative. Defenders usually react, plugging gaps as threats appear. Tools like Microsoft Security Exposure Management help shift defenders to a more proactive stance, but attackers still control the tempo. Recognizing this fundamentally changed my approach to security.

2. Simple techniques remain highly effective. Before getting hands-on with ethical hacking, I thought attackers mainly relied on sophisticated exploits that grab headlines. While advanced threats exist, basic attacks (like phishing emails that deliver reverse shell payload) are incredibly common. Solid foundational security measures are as crucial as ever in 2025.

Breaking into my first system, even ethically, taught me more in a few hours than months of defensive and sysadmin work ever did. To defend effectively, you have to deeply understand how attackers think, operate, and exploit vulnerabilities.

Lesson 2: Overlooking the obvious

Early on in my offensive journey, one moment stands out clearly. I was deep into a Hack The Box challenge, convinced there had to be some complex exploit hidden beneath the surface. Hours later, I realized I'd completely overlooked checking basic file permissions.

When I finally looped back, the vulnerability was painfully obvious—misconfigured permissions on a sensitive directory handed me immediate access. Oops.

Trust your process, cover the basics thoroughly, and don’t assume complexity. Skipping the obvious steps only costs you time.

Lesson 3: Thinking sideways (Attackers don’t play by your rules)

One of my favorite "aha" moments came from a box that initially seemed impossible. I was convinced the vulnerability had to be web-based because everything pointed in that direction. After exhausting every possible web exploit, I took a break, frustrated. It turned out the entry point wasn't web at all; it was an outdated, seemingly harmless print service that I'd dismissed early on.

That pivot taught me something critical: attackers don’t care how things “should” work. They exploit how they “actually” work. If defenders only look at what's typical or documented, we leave blind spots wide open.

So, forget your assumptions. Security is about expecting the unexpected. To be effective, defenders must learn to look at their environments with fresh eyes (and think sideways) just like the attackers do.

Bonus: What I changed in my own environment

After spending time trying to break into my own systems, I quickly saw where my blind spots were, and immediately got to work plugging them. Here are some practical changes I've made to strengthen my own security posture:

• Firmware Updates: Let's be honest, keeping firmware up to date isn't fun or flashy, but outdated firmware can be a goldmine for attackers. Now, I make it a routine to regularly check and update firmware on everything from routers and switches to IP cameras and printers.

• Isolating IoT Devices: IoT devices are notoriously sketchy when it comes to security. My smart home gear is now isolated on its own VLAN, keeping these potentially vulnerable devices separated from my main network.

• VLAN Segmentation: Beyond IoT, I’ve segmented my network into different VLANs, creating clear boundaries between sensitive data, regular browsing, and guest access. It might feel a little paranoid at first, but compartmentalizing access drastically reduces lateral movement opportunities for attackers.

• Zero Trust Mindset: Probably the biggest mindset shift has been embracing Zero Trust principles. Instead of assuming internal traffic is safe, I now operate as if every device and user is potentially compromised. It takes discipline, but it's worth it.

Taking these steps not only improved my security but gave me greater peace of mind. Nothing sharpens your defensive instincts quite like actively testing your own limits.

Final Thoughts: Why every defender should break in (safely)

If you've spent your career on the defensive side like I did, switching perspectives can feel intimidating, but it's essential. Understanding attackers requires thinking like one. And there's no substitute for hands-on experience.

By safely and ethically exploring offensive security techniques, you will:

• Gain practical insights that theory alone can't teach.

• Recognize weak spots in your environment that you might have overlooked.

• Develop empathy and respect for the creativity and persistence attackers bring to their craft.

Even basic, controlled exercises (like those available on platforms such as Hack The Box or TryHackMe) can profoundly change your approach. You don’t need to become a world-class pen tester but spending some time on the other side can sharpen your defensive instincts and make your security efforts more effective.

Every defender benefits from knowing how to break in safely. So give it a try, your environment (and your skills) will thank you.

• Where basic push-notification MFA falls short.

• Why improvements like number matching help, but don’t fully solve the problem.

• Which methods are actually “phishing-resistant”.

• How Microsoft Entra MFA ties into device ‘trust’, and why “hybrid join” doesn’t automatically mean “secure device”.

The problem with push MFA

The Problem with Push MFA

Push-based MFA is that nice, convenient method where you get a notification on your phone: “Was this you signing in?” You tap “Approve” or “Deny,” and you’re in. It uses services like Firebase (Android) and APNS (iOS) to pop up that request in your authenticator app.

People tend to like this method because it’s fast and easy. No fuss.

Attackers also tend to like this method because they’ll just keep spamming you at all hours of the night until you tap “Approve” and they’re in. Or, they’ll do a little social engineering, “Hey, I’m so-and-so from IT, can you accept the prompt you’re getting please?” Let’s face it, when you’re in a hurry, you might just tap Approve by mistake or pure annoyance.

Real-World Breach Examples

• Cisco (2022): Attackers hammered an employee’s phone with MFA prompts and impersonated IT on a phone call. Eventually, the user caved and hit Approve. Boom! Network compromised. [Reference: https://blog.talosintelligence.com/recent-cyber-attack/]

• Uber (2022): Same deal. An attacker texted the user, spammed them with prompts, eventually wearing them down. [Reference: https://www.wired.com/story/uber-hack-mfa-phishing/]

Push MFA is convenient, but way too reliant on human judgment. If someone can just pester or trick you into tapping Approve, they win.

Number matching - A partial fix

A crafty phisher can call up the user, do a bit of social engineering and say something like, “Hey, please approve the following request, the confirmation code is 56,” or something along those lines. Basically, number matching reduces accidental approvals but still can’t stop a determined social engineer.

Phishing-Resistant MFA: FIDO2, Passkeys, and Certificates

A truly phishing-resistant MFA method doesn’t rely on the user handing over a code or tapping a random approval prompt. Instead, it uses cryptographic tricks bound to the real site or service, so a fake site can’t fool it. Let’s check out the big kids in the neighborhood:

• FIDO2 Security Keys: USB or NFC “keys” that do public-key cryptography with each site. They’re awesome because they will not authenticate you with the wrong domain. So even if a user stumbles onto a phishing page, the key sees it’s not the real site and refuses to sign.

  • SMB Reality Check: Physical keys cost money and require some admin overhead. But many businesses find them worth it, especially for high-risk accounts. Employees can also use built-in authenticators on devices (like Windows Hello for Business) to skip buying physical keys.

• Passkeys: These wrap all the goodness of FIDO2 but sync across devices, so it’s more consumer-friendly. Microsoft, Google, Apple, and Amazon are jumping on the bandwagon. Adoption is climbing fast—people are tired of dealing with passwords. There is a bit of friction in setup, but once it’s running, people notice the convenience: no password, just a quick biometric or PIN.

• Certificate-Based Authentication: The user or device has a certificate that never leaves the system. The server checks it via a cryptographic challenge. This is phishing-resistant because you can’t phish someone into reading off a certificate. It’s all behind the scenes—no one-time codes to intercept. The downside? PKI can be complicated. Usually larger enterprises or government organizations go this route, but it’s an option for smaller businesses who need an ironclad approach (and can handle the overhead).

But wait, you need more than just MFA

Security in depth is the best way to approach authentication. A layered approach will help you secure your environment hence why phishing-resistant MFA is so important but so is the compliant device tag. Let’s explore a scenario:

• An attacker using a proxy phishes a user and provides a link to what appears to the user as the M365 login page.

• The user performs the login and gets the MFA prompt, if they are using push MFA, TOTP or number matching, they get the same prompt on the login page and enter the details.

  • Meanwhile the attacker has grabbed a copy of the token.

• The user is redirected to the actual M365 login page or passed through to M365.

• Now the attacker takes the token and replays it on their device allowing them to login as the user.

How do we stop this? Simple: add the Require Compliant Device checkbox to the Conditional Access policy, or at least the Require Hybrid Joined Device checkbox. This doesn’t allow any access to your tenant resources if the device is not explicitly enrolled with your tenant. Maybe the attacker can capture the user’s token, but they can’t do anything with it unless they own one of your devices, which means they’re already in your network, which brings on a whole other set of problems you’d hopefully detect.

• Hybrid Azure AD Joined: Means a Windows device is both on-prem domain-joined and registered with Azure AD. Great for single sign-on and identity, but doesn’t guarantee the device is secure—it’s more of a static “corporate device” check.

• Compliant Device (MDM): Means the device is enrolled in Intune (or another MDM) and meets your security rules (encryption, OS updates, no jailbreak, etc.). Intune continuously verifies compliance via a certificate-based attestation.

Key Point: Hybrid join alone won’t catch a laptop that’s loaded with malware or missing patches. Compliance via MDM is the stronger signal because it’s an active, policy-based check.

Recommendation: For sensitive access, don’t rely solely on hybrid join. Combine it with “require compliant device” so you know the machine is actually up to policy.

Recommendations

1. Don’t over-rely on Push MFA

   It’s convenient but social engineering and phishing can break it. Number matching which is enabled by default for all Microsoft Authenticator users will reduce accidental approvals but realize that it’s not foolproof.

2. Use phishing-resistant MFA where it matters most

   Admins, executives, and anyone with access to critical data should have FIDO2 or passkeys. Even if you roll it out gradually, it’s worth the peace of mind.

3. Consider going passwordless

   Tools like Microsoft Authenticator phone sign-in or Windows Hello for Business are leaps ahead of memorized passwords. You’ll cut out password-based phishes entirely. Just remember that phone sign-in still relies on user caution; FIDO2 is more bulletproof.

4. Embrace “Compliant Device”

   Require that devices are enrolled and meet your security policies. That way, even if an attacker steals someone’s creds, they can’t just log in from any random device.

5. Learn from the breaches

   Set up user education to prevent “push fatigue” approvals, and get rid of weaker MFA factors (SMS, voice calls) as soon as you can.

6. Keep watching the future

   Passkeys are on a fast upward trend, and user acceptance is growing. It might feel daunting to jump on new tech right away, but these days it’s often less painful than you’d think. The payoff: better security and fewer password headaches.

Phishing resistant MFA isn’t just for Fortune 500s or government agencies anymore. If you’re running a business, these modern authentication methods could be the difference between a small incident and a massive breach. I hope this rundown helps you navigate the next steps in your MFA journey. Have questions or want to share your own experience? Let me know in the comments.

If you’re an IT admin or business owner looking to streamline your Microsoft 365 onboarding and configuration, you need to see this!

This tool is a game-changer for setting up secure and efficient environments in no time. Whether you’re new to M365 or a seasoned pro, the setup expert AI is an epic resource to streamline your expertise.

Check out the video to see how it works and let me know what you think!

It’s a simple concept: an infinite pinup board for text-only messages. No replies, no moderation, no tracking; just pure, unfiltered expression, the way I remember the internet of the late 90s early 00s. Drop a note anywhere on the board, and it stays. Every user starts in a random location, and if you want to post, the app will nudge you to the nearest open space. That’s it. No noise, no engagement metrics, just a raw stream of whatever people feel like saying.

Why? Because every platform these days tries to shape what we see. LeaveA.Gripe doesn’t. There are no hidden rules, no boosted posts, no content policing. It’s the wild west of anonymous text, where people can say whatever they want, and it just exists.

It’s live now at LeaveA.Gripe. Check it out, leave your mark, and let the gripes flow!

What’s Changed?

Since 2023, Microsoft has dramatically reduced the number of URLs needed for allow-listing, consolidating Defender for Endpoint’s cloud endpoints into a much smaller set. Instead of dealing with a long list of domains, most organizations now only need to allow *.endpoint.security.microsoft.com and a few others. Microsoft also introduced static IP ranges and Azure service tags, making firewall configurations much more manageable.

For organizations with disconnected networks, these changes mean fewer headaches when setting up proxy rules. But even with these improvements, you still need a path to the cloud—and that’s where proxies remain essential.

Why Proxies Still Matter

Many organizations don’t allow direct internet access from endpoints, especially in high-security environments. A proxy allows MDE to connect to Microsoft’s cloud while maintaining network control. This isn’t a security compromise; it’s a smart way to ensure MDE can leverage AI-driven protection and real-time threat intelligence without opening the floodgates.

To make it work, you need to:

• Use a system-wide proxy configuration (WinHTTP) so Defender can always communicate, even when no user is logged in.

• Allow required Microsoft endpoints without SSL inspection; Defender uses certificate pinning, and inspecting traffic will break its connection.

• Ensure outbound connections don’t require user authentication, since Defender’s telemetry is sent by the system, not a logged-in user.

With a properly configured proxy, you get full cloud protection without sacrificing security.

Airgapped Environments Need a Different Approach

If your environment is fully airgapped (no internet at all), then cloud-based protection just isn’t an option. Defender for Endpoint isn’t designed for fully offline use, and while you can keep Defender Antivirus running with offline signature updates, you lose out on AI-driven threat detection, EDR, and cloud analytics.

For true airgap scenarios, your focus should be on offline update mechanisms (WSUS, Configuration Manager) and strict network segmentation to prevent lateral movement. But if there’s even a tiny opportunity to establish controlled, intermittent connectivity; say, syncing telemetry weekly; it’s worth doing.

Let’s Talk About Trust

One of the biggest pushbacks I still hear is trust. Some organizations hesitate to open a proxy for Defender’s cloud security, despite already trusting Microsoft with their emails (Exchange Online), files (SharePoint and OneDrive), and collaboration (Teams). If your business-critical data already lives in Microsoft’s cloud, why would you suddenly draw the line at security telemetry?

Defender for Endpoint sends security signals, not sensitive business data. It’s about identifying threats, improving detection, and keeping your environment safe. If your security model is still based on “we don’t trust cloud security,” it might be time to rethink that stance.

Best Practices for 2025

If you’re operating in a disconnected or hybrid network, here’s what you should be doing:

• Use the new streamlined allow-list instead of managing dozens of URLs.

• Disable SSL inspection for Defender traffic; it’ll break functionality.

• Use a dedicated proxy configuration so Defender always has cloud access.

• Regularly check connectivity using the client analyzer tool.

• Educate security teams; this isn’t about opening everything, it’s about controlled access to a trusted security cloud.

References:

Announcing a streamlined device connectivity experience for Microsoft Defender for Endpoint

Disconnected environments, proxies and Microsoft Defender for Endpoint

Defender for Endpoint and disconnected environments. Cloud-centric networking decisions

Here's the reality: just because something requires an internet connection doesn't mean it's exposed to the "open internet." When you set up a tightly controlled, encrypted tunnel between your endpoint and a specific service, you're not throwing data into the wild. You're essentially creating a “VPN”, whether you call it that or not.

With HTTPS/TLS, your data is encrypted, directed to a single, defined endpoint, and protected from interception; just like a VPN tunnel. The difference? One is application-specific, and the other typically routes all traffic. This distinction is at the heart of why the "Defender needs the internet" argument is flawed. It uses an HTTPS/TLS connection to send telemetry and signals to the Defender service. But before we dive too deep, let's talk about why encryption became the backbone of modern security in the first place.

The Internet "Goes Dark"

Before 2013, encryption was primarily used for sensitive data like banking transactions and login credentials, while most internet traffic remained unencrypted. The Snowden leaks in 2013 exposed extensive global surveillance and the exploitation of unencrypted data, triggering a shift to an "encrypt everything" mentality.

In 2016, Let's Encrypt revolutionized the scene by offering free, automated SSL/TLS certificates, making HTTPS the norm overnight. Today, encryption is standard for almost all online activities; but it also poses challenges for law enforcement, as robust encryption can hinder traditional surveillance methods and complicate data access during investigations.

The Cultural Shift

The industry transitioned from "only encrypt sensitive data" to "encrypt everything by default." Today, whether you're checking email, using a SaaS app, or running a cloud-based AV/EDR like Defender for Endpoint, encryption is the norm. This means that when an endpoint communicates with the cloud, it's through a controlled, encrypted tunnel; not just general web surfing.

Key Takeaways from the Encryption Revolution:

• Pre-2013: Encryption was limited to specific use cases.

• Post-Snowden (2013+): "Encrypt everything" became the standard (it was a slow process).

• Google’s HTTPS Push (2014-2018): Encouraged HTTPS adoption by ranking HTTPS sites higher in search results and marking HTTP sites as insecure in Chrome.

• 2016 and beyond: Let's Encrypt removed cost barriers, making HTTPS ubiquitous.

• Regulatory Changes (e.g., GDPR, PCI DSS v3.2): Strengthened encryption requirements for compliance.

The Basics of Secure Tunnels

Now, let's break down what a VPN does versus what HTTPS/TLS does.

At their core, both VPNs and HTTPS/TLS tunnel encrypted traffic to a specific destination. The main difference is that a VPN typically works at the network level, whereas HTTPS secures individual services.

How HTTPS/TLS ‘Mimics’ a VPN

Pinning and Allowlisting

• Many security teams allowlist only the endpoints that services like Defender for Endpoint require.

• If a device can only communicate with Microsoft's secure Defender services, labeling it as "internet-exposed" is misleading.

Equivalent Security

• A properly pinned HTTPS connection to a cloud service offers the same level of encryption as a VPN, but at the application level rather than the entire network.

Addressing the OT concerns

Let’s talk about “air-gapped”

• A true air-gapped system has zero external connectivity. These are rare and not foolproof.

• In reality, most environments have controlled exceptions, such as telemetry, updates, or security tools.

Challenges with Air-Gapped Networks

• Human Factors: Air-gapped systems may require data transfer via physical media like USB drives. This necessity introduces a vector for malware.

• Insider Threats: Employees or contractors with legitimate access can inadvertently or maliciously introduce malware into the system.

• Advanced Persistent Threats (APTs): Sophisticated adversaries develop methods to breach air-gapped systems, such as exploiting electromagnetic emissions or using compromised hardware.

• Maintenance and Updates: Keeping air-gapped systems updated is challenging, often leading to outdated software that is vulnerable to exploits.

While air-gapped networks add a layer of security, they should not be the sole defense mechanism. Implementing strict access controls, regular security audits, and comprehensive monitoring is essential. Isolation can lead to complacency. Just look at the damage Stuxnet caused.

HTTPS/TLS as a "Micro-VPN"

By pinning HTTPS/TLS traffic to a specific set of endpoints, you create a tunnel just as restricted as a VPN.

Benefits of a Strict Allowlist

If a network only allows outbound HTTPS traffic to a handful of approved domains, then that traffic is as "air-gapped" as possible while still allowing required security functions.

Final thoughts

"Connecting to the internet" does not equate to exposing your network. HTTPS/TLS is a secure tunnel, just like a VPN; just scoped differently.

Sources

'Five Eyes' security alliance calls for access to encrypted material

Canada's National Cyber Security Strategy

Canada’s national police force admits use of spyware to hack phones

Microsoft Digital Defense Report 2024

An Unprecedented Look at Stuxnet, the World's First Digital Weapon

What are the Biggest Challenges to Federal Cybersecurity? (High Risk Update)

High-Risk Series:Federal Government Needs to Urgently Pursue Critical Actions to Address Major Cybersecurity Challenges

The Air Gap Is Dead. It’s Time for Industrial Organisations to Embrace the Cloud

Securing OT Systems: The Limits of the Air Gap Approach

SCADA Security: Is the Air Gap Debate Over?

Scientist-developed malware prototype covertly jumps air gaps using inaudible sound

I’ve always been fascinated by AI and self-hosted solutions, so with my home lab setup, I figured - why not experiment with AI and containers?

Since I already had the hardware, building a local inference server seemed like a natural next step.

I started researching different options and, as a longtime Hugging Face lurker, decided to try out some of their tools. That’s when I came across vLLM, a fast, OpenAI-compatible model server, and ChatUI, a clean, customizable frontend. It looked like a straightforward setup.

Yeah, it wasn’t.

Between networking issues, container misconfigurations, and a handful of other headaches, getting everything running was far more involved than I expected. But after plenty of troubleshooting, rebuilding, and debugging, I got it working.

This article walks through that process - what I learned, the challenges I ran into, and the resources that helped along the way. I’ve also included the final working configuration for anyone looking to set up a local AI inference server using Docker, NVIDIA GPUs, and open-source tools.

The Goal

I wanted to deploy a self-hosted AI chatbot with the following components:

• vLLM serving as the AI model (replacing cloud APIs like Hugging Face or OpenAI)

• ChatUI providing the frontend (so I wouldn’t have to build a UI from scratch)

• MongoDB storing chat history for persistence

• NGINX handling reverse proxying and TLS (SSL) termination

• Docker managing deployment for a consistent and reproducible setup

• GPU acceleration ensuring snappy responses

The Final Architecture

Here’s what I ended building/using

Getting the prerequisites ready

With the physical setup complete, the next step was choosing an operating system. I wanted something lightweight with minimal overhead since I’d be managing everything remotely over SSH using Visual Studio Code. A streamlined OS also helps keep configuration and maintenance simple.

I went with Debian, a solid and reliable choice.

Setting Up the Environment

Once Debian was installed, I ran the following commands to update the system, install essential packages, and set up Docker.

# Baseline updates and setup
sudo apt update
sudo apt upgrade -y
sudo apt install ca-certificates curl gcc git git-lfs wget

# Docker specific prerequisite setup
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Installing NVIDIA Drivers & CUDA

For GPU acceleration, I needed CUDA and the latest NVIDIA drivers. The best way to ensure compatibility is by downloading the appropriate CUDA version directly from NVIDIA’s site. The site dynamically generates the correct install commands based on your OS version - always check for updates before running these commands.

As of writing, CUDA 12.8 was the latest version. Here’s how I installed it:

# Installing Nvidia Drivers and Cuda
wget https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-8
sudo apt-get install -y nvidia-open

Setting Up the NVIDIA Container Toolkit

To enable GPU access inside Docker containers, I installed the NVIDIA Container Toolkit following NVIDIA’s documentation.

# Installing the Nvidia Container Toolkit
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker

Verifying Everything Works

After installation, I rebooted the system and verified that the GPU and container toolkit were working properly.

# Reboot (start fresh)
sudo reboot

# Now check prereqs
nvidia-smi # This should output driver information

# Test that the container toolkit is working properly with Docker and that containers can access the GPU
sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

The nvidia-smi output should display your GPU details. Running the same command inside a Docker container should produce identical output. If anything looks off, retrace the setup steps to catch any missed configurations.

Setting Up the Working Directory

This is where I stored all Docker Compose files, models, and configuration files for the various containers. Most directories will be generated later, but a few need to be created manually.

cd ~
mkdir working
mkdir working/chat-ui
mkdir working/nginx

Full Setup Commands (For Reference)

If you want to set everything up in one go, here’s a consolidated version.

# Baseline updates and setup
sudo apt update
sudo apt upgrade -y
sudo apt install ca-certificates curl gcc git git-lfs wget

# Docker specific prerequisite setup
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

# Installing Nvidia Drivers and Cuda
wget https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-8
sudo apt-get install -y nvidia-open

# Installing the Nvidia Container Toolkit
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker

# Reboot host
sudo reboot

# Now check prereqs
nvidia-smi # This should output driver information

# Test that the container toolkit is working properly with Docker and that containers can access the GPU
sudo docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

# Prep the working folder(s)
cd ~
mkdir working
mkdir working/chat-ui
mkdir working/nginx

Why vLLM?

After setting up the environment, the next step was choosing an inference engine to serve the AI model. With so many options available - like Hugging Face’s Text Generation Inference (TGI) - I wanted something fast, compatible with OpenAI’s API, and optimized for high-throughput inference.

I came across an excellent blog post by Malte Büttner at inovex.de. While the article focused on TGI rather than vLLM, it helped frame my approach to deploying this solution efficiently.

Why vLLM Over Other Solutions?

Inference engines evolve rapidly, but I chose vLLM for its balance of speed, efficiency, and support for modern models. Here’s a quick comparison:

vLLM is OpenAI-compatible API made it easy to integrate without modifying other components like ChatUI. It also supports efficient batching and memory handling, which is crucial when running inference on consumer-grade GPUs.

Choosing the Right Model

Selecting the right model wasn’t just about grabbing the biggest, most powerful LLM available. It had to fit within hardware constraints while still performing well in chat-based interactions.

The RTX 3080 (10GB VRAM) is a solid GPU, but large models like LLaMA 3 8B or Mistral 7B would have pushed its memory limits, making them impractical for real-time chat. I needed a model that was:

• Compact enough to run efficiently on my GPU

• Strong in conversation

• Fully compatible with vLLM for seamless inference

Why Phi-3?

Phi-3 stood out because it delivers strong chat performance despite its smaller size. At just 3.8B parameters, it competes with (and even outperforms) some 7B models while maintaining efficiency.

Key Factors in My Decision

1. ✅ Performance vs. Hardware Constraints

   • The RTX 3080 has limited VRAM (10GB), so full-precision LLMs were out.

   • Phi-3 runs efficiently without excessive slowdowns.

   • Larger models like Mistral 7B would have been too memory-intensive for real-time use.

2. ✅ Optimized for Chat

   • Some models excel at code generation, but fall short in conversation.

   • Phi-3 is specifically fine-tuned for instruction-following, making it ideal for a chatbot.

   • Benchmarks show it outperforms some larger models in reasoning tasks.

3. ✅ Seamless vLLM Integration

   • Since vLLM supports OpenAI’s API, I needed a model that played well with that framework.

   • Phi-3 works out of the box with vLLM, eliminating compatibility headaches.

Alternative Models I Considered

Why Phi-3 Was the Best Fit

Phi-3 struck the right balance between size, performance, and efficiency, making it the best option for local inference on a 3080.

• ✅ Fast

• ✅ Lightweight

• ✅ Strong chat performance

• ✅ Easily deployable with vLLM

Starting with one solid model made sense, but ChatUI supports multiple models, so I can expand later.

Preparing Phi-3 for vLLM

Since this is a self-hosted setup, the model must be stored locally. There are several ways to download models, but I opted for git to keep things simple.

Downloading Phi-3

cd ~/working
git lfs install
git clone https://huggingface.co/microsoft/Phi-3-mini-4k-instruct

After running this, the folder structure should look like this:

/home/username
 |-- working
 │    |-- nginx
 │    |-- Phi-3-mini-4k-instruct
 │    |-- chat-ui

Breaking Down the Container Configuration

Each container in this setup plays a specific role, and getting them to work together required deliberate configuration choices. Below, I’ll walk through:

• Design Decisions – Why I configured it this way and the trade-offs involved.

• Configuration Details – Key settings and how they impact functionality.

• Troubleshooting Steps – How I validated that each container was running correctly.

Understanding the Configuration

This is not a plug-and-play setup. You can’t just copy and paste the full docker-compose.yml and expect it to work out of the box. Certain containers require additional configuration files and environment variables that I’ll cover in their respective sections.

If you’re following along, make sure to review the highlighted sections, those require extra setup.

Container Overview

In the following sections, I’ll break down each container’s configuration, highlight potential pitfalls, and explain the reasoning behind key decisions.

vLLM: The Model Inference Server

At the core of this self-hosted AI chatbot is vLLM, a highly optimized inference framework designed for efficient text generation. My goal was to maximize performance on an RTX 3080 while ensuring compatibility with ChatUI, which integrates easily using an OpenAI-compatible API.

I previously covered why I chose vLLM over TGI, but to reiterate, vLLM offers out-of-the-box compatibility with OpenAI’s API, making it a drop-in replacement for services like OpenAI or Hugging Face APIs. This meant seamless integration with other components in my setup without needing to rewrite API interactions.

vLLM Configuration

Here’s the docker-compose.yml configuration for vLLM:

text-generation:
    image: vllm/vllm-openai:latest
    ports:
      - "8080:8000"
    volumes:
      - ./Phi-3-mini-4k-instruct:/data/model/Phi-3-mini-4k-instruct 
    command:
      - "--model"
      - "/data/model/Phi-3-mini-4k-instruct"
      - "--dtype"
      - "bfloat16" 
      - "--tensor-parallel-size"
      - "1"
      - "--gpu-memory-utilization"
      - "0.9"
      - "--max-model-len"
      - "3264"
      - "--max-num-batched-tokens"
      - "3264"
      - "--trust-remote-code"
    environment:
      NVIDIA_VISIBLE_DEVICES: all
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    runtime: nvidia
    container_name: text-generation
    restart: always

vLLM - Key configuration choices

Troubleshooting vLLM

vLLM wasn’t plug-and-play, and I hit a few roadblocks along the way. Here’s how I validated that it was working correctly and debugged common issues.

1️⃣ Ensuring vLLM is running properly

After deployment, I tested whether vLLM was active by running:

curl -X GET http://localhost:8080/v1/models

✅ Expected response (formatted for readability)

{
    "object": "list",
    "data": [
        {
            "id": "/data/model/Phi-3-mini-4k-instruct",
            "object": "model",
            "created": 1738295351,
            "owned_by": "vllm",
            "root": "/data/model/Phi-3-mini-4k-instruct",
            "parent": null,
            "max_model_len": 3264,
            "permission": [
                {
                    "id": "modelperm-18926d767c1346399251c729e0cd251a",
                    "object": "model_permission",
                    "created": 1738295351,
                    "allow_create_engine": false,
                    "allow_sampling": true,
                    "allow_logprobs": true,
                    "allow_search_indices": false,
                    "allow_view": true,
                    "allow_fine_tuning": false,
                    "organization": "*",
                    "group": null,
                    "is_blocking": false
                }
            ]
        }
    ]
}

If this request fails, check the logs for errors:

sudo docker logs text-generation

2️⃣ Common issues you may encounter

Model Path Mismatch

Ensure that the volume mounts are correct. The host directory ./Phi-3-mini-4k-instruct should be properly mapped to /data/model/Phi-3-mini-4k-instruct in the container.

GPU Visibility Issues

If vLLM can’t access the GPU, check nvidia-smi:

sudo docker exec -it text-generation nvidia-smi

If the GPU isn’t detected, the issue is likely related to missing drivers, an incorrect runtime setting in Docker, or the NVIDIA Container Toolkit not being configured correctly.

3️⃣ Validating Inference Works via API

To ensure text generation was functioning properly, I ran a test request directly from the Docker host:

curl -X POST http://localhost:8080/v1/chat/completions -H "Authorization: Bearer sk-fake-key" -H "Content-Type: application/json" -d '{
    "model": "/data/model/Phi-3-mini-4k-instruct",
    "messages": [{"role": "user", "content": "Hello, what is AI?"}]
  }'

✅ Expected response (formatted for readability)

{
    "id": "chatcmpl-9375af51aa3f479db7c3053e33eb753d",
    "object": "chat.completion",
    "created": 1738296405,
    "model": "/data/model/Phi-3-mini-4k-instruct",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": " Artificial Intelligence (AI) .... blah blah blah",
                "tool_calls": []
            },
            "logprobs": null,
            "finish_reason": "stop",
            "stop_reason": 32007
        }
    ],
    "usage": {
        "prompt_tokens": 10,
        "total_tokens": 179,
        "completion_tokens": 169,
        "prompt_tokens_details": null
    },
    "prompt_logprobs": null
}

Debugging networking issues

One of the trickiest parts of setting up vLLM was getting the container networking right. Since containers handle networking behind the scenes, misconfigurations can cause connection failures between ChatUI and vLLM.

4️⃣ Testing Connectivity from Another Container

I confirmed that ChatUI could talk to vLLM by running a cURL test inside the ChatUI container:

sudo docker exec -it chatui bash
curl -X POST http://text-generation:8000/v1/chat/completions \
-H "Authorization: Bearer sk-fake-key" \
-H "Content-Type: application/json" \
-d '{
    "model": "/data/model/Phi-3-mini-4k-instruct",
    "messages": [{"role": "user", "content": "Hello, what is AI?"}]
}'

If this request fails, the issue is likely container networking. I resolved it by explicitly defining a dedicated Docker network for this deployment in my final configuration.

Thoughts on the vLLM deployment

Deploying vLLM with Phi-3 Mini was an exercise in balancing performance and efficiency. The RTX 3080 handles it well, but it’s clear that more powerful models (LLaMA 3, Mistral, etc.) would need more VRAM.

Would I recommend vLLM for local inference?

Absolutely. If you need solid performance and compatibility with OpenAI-style APIs, vLLM is a great choice.

Next Steps for My Setup

• Deploy multiple models for on-the-fly model switching in ChatUI.

• Optimize memory usage for longer prompts.

• Enhance functionality with web search and retrieval-augmented generation (RAG).

Final Thoughts - vLLM

Setting up vLLM wasn’t as easy as I expected, but once everything was dialed in, it worked flawlessly. The biggest hurdles were container networking and GPU resource management, but debugging logs and running manual API tests made troubleshooting straightforward.

If you’re looking to self-host an AI chatbot, vLLM is a powerful, flexible option - and one that scales well for local deployments.

ChatUI: The Frontend Interface

With vLLM handling inference, the next step was setting up ChatUI - a web-based chat interface that makes interacting with the model easy. ChatUI provides an OpenAI-style frontend, so I didn’t need to build my own UI from scratch.

Why ChatUI?

I briefly mentioned ChatUI earlier, but let’s break down why I chose it:

• OpenAI-compatible – Works seamlessly with vLLM without major modifications.

• Lightweight and simple – Minimal dependencies, making deployment straightforward.

• Multi-model support – Allows loading multiple models and switching between them in the UI.

• Customizable API endpoints – Ensures all requests are routed to my self-hosted AI instead of external APIs.

ChatUI Docker Configuration

Here’s how I set up ChatUI using Docker Compose, connecting it to MongoDB for chat history and vLLM for inference:

  chat-ui:
    image: ghcr.io/huggingface/chat-ui:latest # Chat UI container
    ports:
      - "3000:3000" # Expose Chat UI on port 3000
    environment:
      MONGODB_URL: mongodb://mongo-chatui:27017 # MongoDB URL for frontend
    volumes:
      - ./chat-ui/.env.local:/app/.env.local
    container_name: chatui
    restart: always

This configuration ensures ChatUI:

• ✅ Connects to MongoDB for storing conversations.

• ✅ Runs on port 3000 for easy web access.

• ✅ Reads settings from .env.local, making it easy to tweak configurations.

Creating the ChatUI Configuration File

The .env.local file controls ChatUI’s settings, including which AI models are available and where to send requests.

1️⃣ Create the Configuration File

cd ~/working/chat-ui
nano .env.local

2️⃣ Add Configuration Details

### Models ###
MODELS=`[
    {
      "name": "/data/model/Phi-3-mini-4k-instruct",
      "description": "Microsoft's Phi-3 Model",
      "promptExamples": [
        {
          "title": "Write an email from bullet list",
          "prompt": "As a restaurant owner, write a professional email to the supplier to get these products every week: \n\n- Wine (x10)\n- Eggs (x24)\n- Bread (x12)"
        }, {
          "title": "Code a snake game",
          "prompt": "Code a basic snake game in python, give explanations for each step."
        }, {
          "title": "Assist in a task",
          "prompt": "How do I make a delicious grilled cheese sandwich?"
        }
      ],
      "endpoints": [
      {
        "type": "openai",
        "baseURL": "http://text-generation:8000/v1"
      }
      ]
    }
]`

### MongoDB ###
MONGODB_URL=mongodb://mongo-chatui:27017
USE_HF_API=false
USE_OPENAI_API=false

## Removed models, useful for migrating conversations
# { name: string, displayName?: string, id?: string, transferTo?: string }`
OLD_MODELS=`[]`

### Task model ###
# name of the model used for tasks such as summarizing title, creating query, etc.
# if not set, the first model in MODELS will be used
TASK_MODEL="/data/model/Phi-3-mini-4k-instruct"

### Authentication ###

# if it's defined, only these emails will be allowed to use login
ALLOWED_USER_EMAILS=`[]` 
# If it's defined, users with emails matching these domains will also be allowed to use login
ALLOWED_USER_DOMAINS=`[]`
# valid alternative redirect URLs for OAuth, used for HuggingChat apps
ALTERNATIVE_REDIRECT_URLS=`[]` 
### Cookies
# name of the cookie used to store the session
COOKIE_NAME=hf-chat

## Websearch configuration
PLAYWRIGHT_ADBLOCKER=true
WEBSEARCH_ALLOWLIST=`[]` # if it's defined, allow websites from only this list.
WEBSEARCH_BLOCKLIST=`[]` # if it's defined, block websites from this list.
WEBSEARCH_JAVASCRIPT=true # CPU usage reduces by 60% on average by disabling javascript. Enable to improve website compatibility
WEBSEARCH_TIMEOUT=3500 # in milliseconds, determines how long to wait to load a page before timing out
ENABLE_LOCAL_FETCH=false # set to true to allow fetches on the local network. /!\ Only enable this if you have the proper firewall rules to prevent SSRF attacks and understand the implications.

## Public app configuration ##
PUBLIC_APP_GUEST_MESSAGE= #a message to the guest user. If not set, no message will be shown. Only used if you have authentication enabled.
PUBLIC_APP_NAME="Bri-Chat" # name used as title throughout the app
PUBLIC_APP_ASSETS=chatui # used to find logos & favicons in static/$PUBLIC_APP_ASSETS
PUBLIC_ANNOUNCEMENT_BANNERS=`[
    {
    "title": "Welcome to ChatUI",
    "linkTitle": "Check out Brian's blog",
    "linkHref": "https://blog.brianbaldock.net"
  }
]`
PUBLIC_SMOOTH_UPDATES=false # set to true to enable smoothing of messages client-side, can be CPU intensive

### Feature Flags ###
LLM_SUMMARIZATION=false # generate conversation titles with LLMs
ENABLE_ASSISTANTS=false #set to true to enable assistants feature
ENABLE_ASSISTANTS_RAG=false # /!\ This will let users specify arbitrary URLs that the server will then request. Make sure you have the proper firewall rules in place. 
REQUIRE_FEATURED_ASSISTANTS=false # require featured assistants to show in the list
COMMUNITY_TOOLS=false # set to true to enable community tools
EXPOSE_API=true # make the /api routes available
ALLOW_IFRAME=true # Allow the app to be embedded in an iframe

### Tools ###
# Check out public config in `chart/env/prod.yaml` for more details
TOOLS=`[]` 

USAGE_LIMITS=`{}`

### HuggingFace specific ###
# Let user authenticate with their HF token in the /api routes. This is only useful if you have OAuth configured with huggingface.
USE_HF_TOKEN_IN_API=false

### Metrics ###
METRICS_ENABLED=false
METRICS_PORT=5565
LOG_LEVEL=info

# Remove or leave blank any unused API keys
OPENAI_API_KEY=

The file configures:

• ✅ Model selection (Phi-3 Mini)

• ✅ MongoDB connection for chat history

• ✅ Frontend branding (custom name, banners, etc.)

• ✅ API exposure

Troubleshooting ChatUI

Like vLLM, ChatUI wasn’t plug-and-play. Here’s how I debugged the common issues.

1️⃣ Making sure ChatUI talks to vLLM

One of the biggest problems I faced was ChatUI trying to connect to OpenAI instead of my local vLLM server. To check where requests were going, I used browser dev tools:

1. Open ChatUI in a browser (http://<docker host IP>:3000).

2. Press F12 (or right-click → Inspect) to open Developer Tools.

3. Go to the Network tab and send a test message.

4. Look for requests to /v1/chat/completions.

✅ Expected request URL

http://text-generation:8000/v1/chat/completions

❌ If the request goes to api.openai.com, then:

• The .env.local file is incorrect.

• The baseURL value might be missing or incorrectly set.

💡 Fix: Update .env.local and restart ChatUI:

sudo docker restart chatui

2️⃣ Running a Direct API Test from Inside the ChatUI Container

If the frontend isn’t returning responses, I tested the API connection manually inside the ChatUI container:

sudo docker exec -it chatui bash

# Inside the ChatUI container now
curl -X POST http://text-generation:8000/v1/chat/completions -H "Authorization: Bearer sk-fake-key" -H "Content-Type: application/json" -d '{
    "model": "/data/model/Phi-3-mini-4k-instruct",
    "messages": [{"role": "user", "content": "Hello, what is AI?"}]
  }'

✅ Expected response (formatted for readability):

{
    "id": "chatcmpl-9375af51aa3f479db7c3053e33eb753d",
    "object": "chat.completion",
    "created": 1738296405,
    "model": "/data/model/Phi-3-mini-4k-instruct",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": " Artificial Intelligence (AI) .... blah blah blah",
                "tool_calls": []
            },
            "logprobs": null,
            "finish_reason": "stop",
            "stop_reason": 32007
        }
    ],
    "usage": {
        "prompt_tokens": 10,
        "total_tokens": 179,
        "completion_tokens": 169,
        "prompt_tokens_details": null
    },
    "prompt_logprobs": null
}

❌ If this request fails, there’s likely a networking issue between ChatUI and vLLM.

3️⃣ Debugging Networking Issues

One of the most frustrating parts of the setup was container networking.

To check whether ChatUI could reach vLLM, I ran a simple connectivity test inside the ChatUI container:

sudo docker exec -it chatui curl -I http://text-generation:8000/v1/models

✅ Expected output:

HTTP/1.1 200 OK

❌ If the request fails:

• The docker network isn’t configured properly.

• vLLM isn’t running (docker ps to check).

💡 Fix: Explicitly define a Docker network in docker-compose.yml:

networks:
  chat-network:
    name: chat-network
    driver: bridge

Then restart everything:

sudo docker-compose down && sudo docker-compose up -d

Final Thoughts on ChatUI

Once ChatUI was configured properly, it worked exactly as expected:

• ✅ Low-latency responses even on an RTX 3080

• ✅ Fully private – no external API calls

• ✅ Scalable – can support multiple models later

Next Steps:

• Enable authentication for secure access

• Deploy more models and allow switching in the UI

• Improve caching & performance

With ChatUI in place, I now had a functional, self-hosted AI assistant running entirely on my own hardware.

MongoDB: Storing Chat History and Context

With vLLM handling inference and ChatUI providing the frontend, the next piece was storing chat history. MongoDB made the most sense here since ChatUI requires it for storing past conversations.

This setup ensures:

• ✅ Users can continue conversations even after closing ChatUI.

• ✅ The system maintains chat history for context-aware responses.

• ✅ Queries remain fast and efficient without unnecessary complexity.

Why MongoDB?

MongoDB wasn’t a choice I actively made, it was required by ChatUI. That said, it fits well for this use case:

• Native ChatUI Support – ChatUI is built to use MongoDB, making setup seamless.

• Flexible Schema – Since chat logs don’t have a fixed structure, MongoDB handles this well.

• Lightweight – Minimal resource overhead, making it ideal for containerized environments.

• Persistent Storage – Conversations aren’t lost between restarts, improving usability.

MongoDB Docker Configuration

Here’s the Docker Compose configuration for MongoDB, ensuring it integrates smoothly with ChatUI:

  # MongoDB for storing history/context
  mongo-chatui:
    image: mongo:latest # MongoDB container
    ports:
      - "27017:27017" # Expose MongoDB on the default port
    volumes:
      - ./mongo-data:/data/db # Persist MongoDB data
    container_name: mongo-chatui
    restart: always

Key Configuration Choices

Troubleshooting MongoDB

This container worked as expected right out of the box, so I didn’t run into major issues. However, here are a few quick verification steps if something goes wrong:

1️⃣ Check if the container is running

sudo docker ps | grep mongo-chatui

✅ Expected Output:

The container should be running with a stable uptime.

2️⃣ Verify MongoDB is accepting connections

mongo --host localhost --port 27017 --eval "db.stats()"

✅ Expected Output:

A JSON response with database statistics, confirming the DB is accessible.

Final Thoughts on MongoDB

With MongoDB properly configured, ChatUI now maintains session history, but there are areas to improve:

• ✅ Implement indexing – To speed up query times.

• ✅ Optimize storage – Automate log cleanup over time.

• ✅ Monitor performance – As usage scales, keeping track of resource consumption will be important.

This setup provides a simple, persistent chat history solution, laying the groundwork for future optimizations.

NGINX: Reverse Proxy for Secure Access

Once vLLM, ChatUI, and MongoDB were up and running, I needed a way to make the deployment secure and accessible via a domain. While I haven’t exposed this setup to the internet, I’m using a public domain for SSL certificates. This is where NGINX comes in, acting as a reverse proxy, handling SSL termination, routing, and ensuring everything is accessible from a single domain.

Why use NGINX?

• Reverse Proxy: Routes traffic from a single domain to multiple internal services.

• SSL Termination: Handles HTTPS encryption via Certbot and Let’s Encrypt.

• Performance Boost: Enables caching and optimizes request handling for faster responses.

• Security: Hides internal containers from direct exposure, securing ports like 8080, 3000, and 27017.

NGINX Docker Configuration

Here’s the Docker Compose configuration for NGINX:

# NGINX Reverse Proxy
  nginx:
    image: nginx:latest
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - ./nginx/certs:/etc/letsencrypt:ro
    container_name: nginx
    depends_on:
      - chat-ui
      - text-generation
    restart: always

Key Configuration Choices

The NGINX Configuration (nginx.conf)

Navigate to the NGINX folder to create the configuration file:

cd ~/working/nginx
nano nginx.conf

Paste the following configuration:

events {
    worker_connections 1024;
}

http {
    server_tokens off;
    charset utf-8;

    # General HTTP server for nginx status or redirects
    server {
        listen 80 default_server;
        listen [::]:80 default_server;

        location /nginx_status {
            stub_status on;
            allow 127.0.0.1; # Allow only localhost to access this
            deny all;
        }
    }

    # Frontend (Chat UI)
    server {
        listen 443 ssl http2;
        listen [::]:443 ssl http2;
        server_name chat.brianbaldock.net;

        ssl_certificate /etc/letsencrypt/live/<YOURDOMAINNAME>/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/<YOURDOMAINNAME>/privkey.pem;

        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_prefer_server_ciphers on;
        ssl_ciphers HIGH:!aNULL:!MD5;

        client_max_body_size 15G;

        location / {
            proxy_pass http://chat-ui:3000;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            # WebSocket support (optional, depending on your frontend)
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }
    }

    # Backend (Text Generation API)
    server {
        listen 443 ssl http2;
        listen [::]:443 ssl http2;
        server_name api.chat.brianbaldock.net;

        ssl_certificate /etc/letsencrypt/live/<YOURDOMAINNAME>/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/<YOURDOMAINNAME>/privkey.pem;

        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_prefer_server_ciphers on;
        ssl_ciphers HIGH:!aNULL:!MD5;

        client_max_body_size 15G;

        location / {
            proxy_pass http://text-generation:8000/v1/;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }

    # HTTP Redirect (Redirect HTTP to HTTPS)
    server {
        listen 80;
        listen [::]:80;
        server_name <YOURDOMAINNAME> api.<YOURDOMAINNAME>;

        return 301 https://$host$request_uri;
    }
}

What this configuration does

• Redirects HTTP to HTTPS (forces encrypted connections).

• Routes <YOURDOMAINNAME> to ChatUI.

• Proxies API calls to vLLM (/v1/ endpoint).

• Handles SSL Certificates via Certbot.

• Supports HTTP/2 for better performance.

⚠️ What You Need to Modify

Troubleshooting NGINX

1️⃣ Check if NGINX is running

sudo docker ps | grep nginx

If NGINX isn’t running or keeps restarting:

sudo docker logs nginx

2️⃣ Test ChatUI access via NGINX

From the Docker host, verify that ChatUI is accessible through NGINX:

curl -I https://<YOURDOMAIN>

✅ Expected output

HTTP/2 200 
server: nginx
date: ""
content-type: text/html; charset=utf-8

Final Thoughts on the NGINX deployment

With NGINX properly configured

• ChatUI is now accessible via HTTPS.

• All traffic is proxied through a single, controlled entry point.

• TLS encryption keeps your data secure while improving performance with HTTP/2.

This setup adds a solid security layer while keeping everything manageable and scalable.

Certbot: Automating SSL Certificates for Secure Connections

With NGINX acting as the reverse proxy, the next step was securing all connections with TLS. Instead of manually managing certificates, I used Certbot, an automated tool for obtaining Let’s Encrypt SSL certificates and handling renewals.

While many DNS registrars offer APIs to automate Let’s Encrypt certificate creation, the process often requires some tweaking. Fortunately, GitHub is packed with scripts, plugins, and Docker container builds for different registrars.

In my case, I customized a Certbot container to support Porkbun (who are awesome, by the way, https://porkbun.com). It was a straightforward configuration but required some trial and error to get right.

Why Use DNS-Based Validation?

I chose DNS-based validation over the traditional HTTP method because:

• No Need to Expose Port 80: I didn’t want to open HTTP ports on my Docker host just for certificate validation.

• API Integration: Let’s Encrypt can communicate directly with my registrar via API, making the process seamless.

If you’re using a different registrar, you’ll need to adjust the process accordingly.

Certbot configuration and customizations

1️⃣ Pull the Latest Certbot Image

# Pull down the default image for Certbot
sudo docker pull certbot/certbot:latest

2️⃣ Set Up the Working Directory

mkdir ~/working/certbot-porkbun
mkdir ~/.secrets

The .secrets folder keeps sensitive API credentials out of your bash history.

3️⃣ Create the Custom Dockerfile

Navigate to the certbot-porkbun directory and create a Dockerfile:

cd ~/working/certbot-porkbun
nano Dockerfile

FROM certbot/certbot:latest

# Install pip if not already installed
RUN apk add --no-cache python3 py3-pip

# Install the certbot_dns_porkbun plugin
RUN pip3 install certbot-dns-porkbun

4️⃣ Add Your API Credentials

nano ~/.secrets/porkbun.ini

Insert your API key and secret:

dns_porkbun_key=<keyid>
dns_porkbun_secret=<secretid>

Ensure the file has secure permissions:

chmod 600 ~/.secrets/porkbun.ini

5️⃣ Build the Custom Certbot Image

docker build -t certbot-porkbun .

6️⃣ Request your SSL Certificate

Run the following command to spin up the new container image and generate your certificate:

sudo docker run --rm -it \
    -v ~/working/nginx/certs:/etc/letsencrypt \
    -v ~/.secrets/porkbun.ini:/etc/letsencrypt/porkbun.ini \
    certbot-porkbun certonly \
    --non-interactive \
    --agree-tos \
    --email your@email.com \
    --preferred-challenges dns \
    --authenticator dns-porkbun \
    --dns-porkbun-credentials /etc/letsencrypt/porkbun.ini \
    --dns-porkbun-propagation-seconds 600 \
    -d <YOURDOMAINNAME> \
    -d api.<YOURDOMAINNAME>

7️⃣ Automate Certificate Renewal with Docker Compose

Add this to your docker-compose.yml file:

  # Certbot for SSL Certificates
  certbot:
    image: certbot-porkbun
    volumes:
      - ./nginx/certs:/etc/letsencrypt
    entrypoint: /bin/sh -c "trap exit TERM; while :; do sleep 6h & wait $${!}; certbot renew; done"
    container_name: certbot-porkbun
    restart: unless-stopped

• The container checks for certificate renewals every 6 hours.

• The certbot renew command ensures certificates stay up-to-date without manual intervention.

Thoughts on the Certbot Deployment

With the Certbot container properly configured:

• SSL/TLS certificates are issued and renewed automatically.

• ChatUI and vLLM are securely accessible via HTTPS.

• All data in transit is encrypted for end-to-end security.

This setup makes SSL management effortless, don’t have to think about manual renewals.

Full docker-compose.yml: Bringing Everything Together

After configuring vLLM, ChatUI, MongoDB, NGINX, and Certbot, it’s time to bring everything together into a unified docker-compose.yml file. This file defines the entire deployment, ensuring all services work seamlessly.

If you’ve followed the prerequisites and configuration instructions for NGINX, ChatUI, and Certbot, you should be able to copy this file into your working directory, run one command, and spin up your own chatbot. (Notice the networks section at the end, this is important for container communication.)

Final docker-compose.yml configuration

services:
  # vLLM Backend
  text-generation:
    image: vllm/vllm-openai:latest
    ports:
      - "8080:8000"  # vLLM API Server
    volumes:
      - ./Phi-3-mini-4k-instruct:/data/model/Phi-3-mini-4k-instruct 
    command:
      - "--model"
      - "/data/model/Phi-3-mini-4k-instruct"
      - "--dtype"
      - "bfloat16" 
      - "--tensor-parallel-size"
      - "1"
      - "--gpu-memory-utilization"
      - "0.9"
      - "--max-model-len"
      - "3264"
      - "--max-num-batched-tokens"
      - "3264"
      - "--trust-remote-code"
    environment:
      NVIDIA_VISIBLE_DEVICES: all
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    runtime: nvidia
    container_name: text-generation
    restart: always

  # The frontend
  chat-ui:
    image: ghcr.io/huggingface/chat-ui:latest # Chat UI container
    ports:
      - "3000:3000" # Expose Chat UI on port 3000
    environment:
      MONGODB_URL: mongodb://mongo-chatui:27017 # MongoDB URL for frontend
    volumes:
      - ./chat-ui/.env.local:/app/.env.local
    container_name: chatui
    restart: always

  # MongoDB for storing history/context
  mongo-chatui:
    image: mongo:latest # MongoDB container
    ports:
      - "27017:27017" # Expose MongoDB on the default port
    volumes:
      - ./mongo-data:/data/db # Persist MongoDB data
    container_name: mongo-chatui
    restart: always

  # NGINX Reverse Proxy
  nginx:
    image: nginx:latest
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - ./nginx/certs:/etc/letsencrypt:ro
    container_name: nginx
    depends_on:
      - chat-ui
      - text-generation
    restart: always

  # Certbot for SSL Certificates
  certbot:
    image: certbot-porkbun
    volumes:
      - ./nginx/certs:/etc/letsencrypt
    entrypoint: /bin/sh -c "trap exit TERM; while :; do sleep 6h & wait $${!}; certbot renew; done"
    container_name: certbot-porkbun
    restart: unless-stopped

networks:
  chat-network:
    name: chat-network
    driver: bridge

How to deploy everything

Once the docker-compose.yml file is ready, deployment is as simple as:

sudo docker-compose up -d

✅ Verifying Deployment

To check if everything’s running smoothly:

sudo docker ps

• All containers should show a similar uptime.

• If a container is restarting, check its logs:

sudo docker logs <container_name>

You should now be able to access ChatUI and start chatting with your model!

Example: I think Phi-3 is happy to be featured in this blog post.

Lessons learned

1. Use a dedicated docker network

   • Avoids conflicts between containers.

   • Prevents duplicate networks causing routing issues.

2. Double-check environment variables

   • Ensure the right endpoints are set.

3. Verify connectivity between containers

   • Running cURL inside the containers helped troubleshoot API connection issues.

4. Monitor logs for debugging

   • Don’t be afraid to get your hands dirty and check out the container logs.

Final Thoughts

This self-hosted AI assistant should now be fully operational with

• ✅ A local LLM inference server

• ✅ A secure web interface

• ✅ Full HTTPS support

• ✅ Persistent chat history

The deployment is scalable, secure, and future-ready.

Wrapping It Up: The Journey, The Lessons, and What’s Next!

What started as a simple curiosity project quickly turned into a deep dive into self-hosted AI, containerized deployments, and debugging Docker networking nightmares. Along the way, I learned a lot. Some of it the hard way.

At the end of the day, this setup works. It’s fast, secure, and completely under my control. No API costs, no privacy concerns, just a local AI assistant running on my own hardware.

Lessons That Stuck With Me

• Nothing is truly plug-and-play – Even with solid documentation, real-world deployments always require tweaks, troubleshooting, and some extensive searching.

• Container networking can be a pain – Debugging why one container can’t talk to another inside Docker’s virtual network was probably the biggest headache. Using a dedicated network and running cURL tests from within containers made all the difference.

• Self-hosting an LLM is totally doable – Even on a consumer-grade GPU like the RTX 3080, inference speeds are surprisingly snappy with Phi-3 and vLLM.

• Certbot with DNS validation is the way to go – No need to expose port 80, no manual renewal, just hands-off, automated SSL certs.

• Future-proofing matters – I designed this setup to be scalable, allowing for multiple models, additional tools, and even more powerful hardware down the line.

What’s Next?

This was only the beginning. Some of the things I’d love to explore next include:

• Multi-Model Support – Deploying multiple LLMs and being able to switch between them on the fly in ChatUI.

• Integrating RAG (Retrieval-Augmented Generation) – Adding a local document index so the model can search and reference private documents.

• Expanding the Infrastructure – Maybe a multi-GPU setup or upgrading to a 4090 to push inference speeds even further.

• Web-Connected AI – Implementing a secure web search tool to allow the model to fetch real-time information.

Self-hosting AI is a game-changer. The more control we have over our own tools, the better.

Want to Try This Yourself?

Everything in this article is reproducible, and I’d love to hear how it works for you! If you have any questions, improvements, or want to share your own builds, let’s connect. Drop a comment, or hit me up on LinkedIn!

Here’s to building cool things, breaking them, fixing them again, and learning along the way!

The grand setup

On July 31st, Contoso’s users were initially synced to Fabrikam via Microsoft’s Multi-Tenant Organization (MTO) and Cross-Tenant Sync (CTS). Everything was humming along nicely; users appeared where they should.

The unexpected twist

The influx of over 10,000 users from Contoso into Fabrikam caused Fabikam’s org wide “All Company” team to hit its membership limit (All Company Teams are created by default if your tenant has less than 5000 users). Remember, org wide teams max out at a total of 10,000 users (as of 2024-10) [Reference].

Fabrikam hadn’t anticipated the sheer volume of users syncing over, and the automatic group membership expansion threw a wrench into their communications and permissions setup. When an org wide team expands beyond 10,000 users it is automatically converted to a public group. Teams are front ends for the Microsoft 365 group.

The quick fix - Or so they thought

Realizing the issue, the Fabrikam swiftly acted. On August 4th, 4 days following the initial sync, they applied a dynamic rule to remove the now converted org wide groups members that are from Contoso. Crisis averted, or so it seemed.

Dynamic Rule Example:
user.userPrincipalName -notContains "_contoso.com#EXT#@fabrikam.com" and user.UserType -eq "Member"

The hidden ripple effect

This is where things got interesting. Between the initial sync on July 31st and the cleanup on August 4th, Contoso had disabled some users (as they are a large organization, turnover is high - so not a rare scenario). When an external tenant disables (soft deletes) a user in their own tenant, the corresponding synced user in the partner tenant (in this case Fabrikam) gets put into a soft deleted state. After August 4th, some of these users were re-enabled on Contoso’s side (again, large org) Remember, before the group was cleaned up, these users were members of the org wide team/now converted to public group, and soft deleted users will not display as members of a team. So, due to the restore process in cross-tenant sync, the “removed” users were effectively resurrected in Fabrikam and, unexpected, but logically, re-enabled as members of the “All Company” group/team, even though Fabrikam had previously cleaned up the membership.

This process meant that every time a user was disabled and then re-enabled on Contoso’s side, they could potentially pop-up into Fabrikam’s “All Company” team/group, adding to communication compliance concerns. After some investigative work we pinpointed a robust solution and it’s surprisingly easy. Just permanently delete any soft-deleted users in the partner tenant. In this example, the fix was having Fabrikam delete all soft-deleted users in their tenant that contain _contoso.com#EXT#@fabrikam.com in their UPN. This ensures that if a deleted user in Contoso’s tenant gets re-enabled, the cross-tenant sync process will recreate the user with a new ObjectID ignoring any previous group memberships. This prevents the user from being “restored” to the “All Company” group/team.

Hopefully you do not run into a similar situation in your organization(s), but, if you do, and you’re seeing similar oddities, double-check your soft-deleted users and do a cleanup. Complex systems can sometimes have unintended consequences.

TLDR

1. Train your employees on cybersecurity basics

2. Enforce strong password policies and go passwordless when able

3. Do not ignore software updates, rip that band-aid off as soon as possible

4. Test your backups, have backups if you don't already, but test them.

5. Don't forget about Mobile Devices, they need protection too

6. Beef up your network security, segment-segment-segment

7. Monitor security alerts, make sure you stay on top of them

8. Upgrade your software, bit the bullet and get it done, maintaining the old is going to hurt you in the long run

9. Lock your doors and windows, that server running in the closet that people put their winter boots on, not a good idea, especially if it's running your crown jewels

10. Have plans for any incidents and make sure your employees and teams know what to do when something bad happens

If you're worried about the cost, ask yourself this: would you rather invest in these preventive measures now or face the potentially crippling expenses of a cyber breach? Imagine losing everything and being unable to restore production because your backups failed, and no one knows how to respond. The initial cost of proper security measures pales in comparison to the financial and reputational damage of a full-scale breach.

Common Cybersecurity Mistakes and Solutions

1) Lack of Employee Training

• Problem: Sure, we can throw tech solutions at problems all day, but let's be real—teaching humans is the hard part. Cybercriminals know this and target people as the primary entry point for attacks. Social engineering tactics like phishing, smishing, and vishing are getting more sophisticated, making them the top human risks.

• Solution: You need a solid security awareness program that's more than just a checkbox exercise for compliance. This means regular, engaging training sessions that go beyond the basics and keep employees up to date on the latest threats. The goal is to create a security-conscious culture where everyone knows how to spot and avoid these scams. It's not just about a yearly meeting; it’s about continuous learning and behavior change.

• Check out this article I wrote that talks about options for end user cybersecurity training: Cybersecurity Training Programs: Empower Your Employee

2) Weak Password Policies

• Problem: Weak or reused passwords are still a huge problem for many businesses. Even with all the tech advancements, a lot of organizations stick with outdated password habits, making them easy targets. The Microsoft Digital Defense Report 2023 (MDDR) points out that password-related breaches are still a major issue, with cybercriminals taking advantage of simple or compromised passwords to break in. The SANS Security Awareness Report 2024 also highlights those human errors, like poor password practices, are a big risk factor​.

• Solution: To up your security game, start by implementing Multi-Factor Authentication. It's a game-changer that adds an extra layer of protection and can stop 99% of attacks. Next, use a password manager like Bitwarden or 1Password to generate and securely store strong, unique passwords, helping you avoid the bad habit of reusing passwords. Get an Enterprise version if you have a team so that you can collaborate and share access. Finally, make sure your team knows why strong passwords matter (refer to point 1) and enforce policies requiring complex passwords and regular updates​. These steps are crucial for preventing unauthorized access and strengthening your overall security.

3) Ignoring Software Updates

• Problem: Skipping software updates is like leaving the front door unlocked. It’s an open invitation for cybercriminals to exploit known vulnerabilities. When you don't update your systems, you're missing out on critical security patches that protect against threats. Cyber attackers are constantly evolving, and outdated software is a prime target for their exploits. The Microsoft Digital Defense Report 2023 emphasizes that unpatched vulnerabilities are one of the most common ways systems get compromised​. Failing to update can lead to serious consequences, from data breaches to ransomware attacks, costing your business time, money, and reputation.

• Solution: Keep your systems and software up to date, plain and simple. This means enabling automatic updates wherever possible and regularly checking for updates on systems that require manual intervention. Make sure you're leveraging advanced security features like Hypervisor-Protected Code Integrity (HVCI), which helps protect against malware, and Smart App Control, which blocks unauthorized applications. These features are part of the latest Windows updates and offer an extra layer of security to prevent unauthorized access and potential threats​​. Staying current with updates isn't just about getting the latest features—it's a crucial step in defending your business against cyber threats.

4) Insufficient and Untested Data Backup

• Problem: Not having a solid backup plan is like playing with fire. If your data isn't properly backed up, you're just one cyberattack away from losing everything. The Microsoft Digital Defense Report 2023 (MDDR) highlights the rise of double extortion tactics, where attackers not only encrypt your data but also threaten to leak it unless you pay up​. It's not just cyberattacks you need to worry about, hardware failures, accidental deletions, and natural disasters can also wipe out your critical information. The Windows Resiliency Best Practices article emphasizes that without a robust backup and recovery strategy, you're leaving yourself wide open to these risks​.

• Solution: To avoid disaster, you need a comprehensive backup strategy that covers all your bases. Start by scheduling regular backups of all critical data, and don't just rely on one type of storage. Use a mix of on-site and off-site solutions, like cloud storage, to protect against physical and cyber threats. Make sure all your backups are encrypted, both when they're stored and when they're being transferred. It's crucial to regularly test your backups and recovery processes so you're not caught off guard in an emergency. Make sure your backup strategy is integrated into your overall business continuity and disaster recovery plan, so everyone knows what to do when things go south. By taking these steps, you'll be better equipped to bounce back quickly from any data loss incident and keep your business running smoothly.

5) Overlooking Mobile Device Security

• Problem: In today's work environment, mobile devices are everywhere, making work more convenient but also introducing significant security risks. Many businesses drop the ball on securing these devices, leaving them wide open to malware, data breaches, and unauthorized access. According to the Microsoft Digital Defense Report 2023 (MDDR), unmanaged devices are a major target for ransomware and other attacks​​. With employees often using personal devices for work tasks, the risk of data leakage skyrockets, especially if these devices aren't secured properly. SANS Security Awareness Report 2024 also highlights the importance of having comprehensive security policies that cover all devices accessing corporate data, not just the typical desktops and laptops​.

• Solution: To lock down mobile devices, a multi-layered security approach is essential. Start by deploying an MDM solution like Microsoft Intune or VMware Workspace One, which allows for enforcing security policies, managing app permissions, and controlling access to sensitive company data. Make sure all devices are encrypted to protect data at rest and in transit; this is crucial if a device gets lost or stolen. Keep mobile operating systems and apps up to date, as outdated software can become a backdoor for cyberattacks, see point 3. And don't forget to implement strong authentication methods like MFA to prevent unauthorized access.

6) Inadequate Network Security

• Problem: Network security often gets overlooked, especially in smaller businesses that might lack the resources or expertise to manage a complex setup. This oversight can leave your business wide open to attacks like malware, unauthorized access, and data breaches. The Microsoft Digital Defense Report 2023 (MDDR) points out that poor network configurations and weak security measures are common ways attackers get in​. Many companies don't segment their networks properly or set up advanced firewalls, which creates security gaps. The SANS Security Awareness Report 2024 also emphasizes the importance of continuous monitoring and a quick response to detect and deal with threats in real-time​.

• Solution: To boost your network security, start by adopting a Zero Trust model. This approach assumes that threats could be lurking both inside and outside your network, so it requires constant verification of user identities and device security. Next, make sure to implement network segmentation. By dividing your network into different segments, you can contain breaches and limit their impact. Advanced firewalls with/and Intrusion Detection Systems or Intrusion Prevention Systems are also key; they help monitor and block malicious traffic. Finally, regularly audit your network configurations and keep them updated to close any security holes.

7) Not Monitoring Security Alerts

• Problem: A major pitfall for many businesses is neglecting to properly monitor and respond to security alerts. It's not uncommon for alerts to get ignored, delayed, or lost in the noise, especially if the organization lacks a dedicated security team or relies on basic, limited tools. The Microsoft Digital Defense Report 2023 (MDDR) emphasizes that quick detection and response are crucial for minimizing damage from security incidents​. Yet, many SMBs either don't have the resources or the expertise to handle this effectively, consider finding a third party to assist with resourcing requirements. There are many excellent partners that provide SOC services. The SANS Security Awareness Report 2024 points out that many breaches could have been prevented or at least mitigated if alerts had been handled appropriately and promptly. Ignoring these alerts can lead to extended dwell times, giving attackers more opportunity to cause significant damage.

• Solution: To stay on top of security alerts, businesses need to set up a comprehensive Security Information and Event Management (SIEM) system like Microsoft Sentinel or Splunk. This tool can aggregate and analyze data from multiple sources, giving you a full picture of what's happening in your environment. It's essential to have a dedicated team or at least a designated person responsible for monitoring and responding to these alerts, ensuring that potential threats don't slip through the cracks. Regularly fine-tuning your alert settings is also important to minimize false positives, so your team can focus on real issues. Lastly, integrating automated response mechanisms can help speed up threat containment and mitigation, reducing the time attackers have to cause harm. By implementing these strategies, you can significantly improve your incident response capabilities and better protect your business from potential breaches. Hire a SOC provider to assist where resources are limited.

8) Using Unsupported or Outdated Software

• Problem: Relying on unsupported or outdated software is like leaving your backdoor unlocked for cybercriminals. Many businesses stick with old systems because they think they "get the job done," but this can be a risky move. Outdated software often lacks essential security updates, making it an easy target for attackers. Without support, software doesn't receive patches for new vulnerabilities, leaving your systems wide open for exploitation. Keeping your software up to date is one of the simplest and most effective ways to boost your security posture. Ignoring this not only increases the risk of data breaches but also puts you at odds with compliance requirements, especially as governments ramp up regulations. It's better to get ahead of the curve now and avoid the potential fallout later.

• Solution: To mitigate these risks, businesses must prioritize regular updates and timely upgrades. Start by ensuring that all systems and applications are running on supported versions that receive regular security updates. Implement a software asset management system to keep track of all software licenses and their support status. This system can help in identifying which software needs to be updated or replaced. Also, consider transitioning to cloud-based solutions where possible, as they often provide continuous updates and security enhancements. Finally, train employees on the importance of using updated software and the risks associated with unsupported systems. By adopting these practices, businesses can significantly reduce the risk of cyber-attacks, ensuring their systems are secure and compliant with industry standards.

9) Inadequate Physical Security

• Problem: In the digital age, it's easy to focus solely on cyber threats and forget about physical security, but the two are closely intertwined. Inadequate physical security measures can lead to unauthorized access to critical infrastructure, resulting in data breaches and other security incidents. The Microsoft Digital Defense Report 2023 (MDDR) highlights that physical breaches often go hand-in-hand with cyber-attacks, as physical access can enable malicious actors to tamper with hardware, install malware, or steal sensitive information. The SANS Security Awareness Report 2024 notes that businesses often overlook physical security, especially in smaller offices or remote locations, making them easy targets for insider threats or external attackers​. Without proper physical security measures, companies are at risk of significant data loss and operational disruption.

• Solution: To address these risks, businesses should implement a comprehensive physical security strategy. Start by securing all entry points with measures like keycard access, biometric scanners, or even traditional locks and alarms. It's essential to monitor these access points with surveillance cameras and motion detectors, providing a real-time view of who is entering and exiting the premises. Additionally, ensure that sensitive areas, such as server rooms, are restricted to authorized personnel only, with strict access controls in place. Regular security audits and drills should be conducted to identify and address potential vulnerabilities, keeping everyone prepared for emergency situations. By integrating these physical security measures with your overall cybersecurity strategy, you can create a more secure and resilient environment that protects both digital and physical assets.

10) Neglecting Incident Response Planning

• Problem: Many businesses overlook the importance of having a well-defined incident response plan, thinking they'll deal with issues as they arise. This lack of preparation can lead to chaos and confusion during a cyber incident, resulting in delayed responses and greater damage. Without a solid incident response strategy, organizations struggle to contain and mitigate the effects of a security breach​. Quick and coordinated action is crucial to minimizing the impact of incidents. Having a pre-defined plan helps ensure that all team members know their roles and responsibilities during a crisis​. The SANS Security Awareness Report 2024 points out that businesses that conduct regular incident response drills are significantly better prepared to handle real-world attacks​. The absence of a well-practiced plan can result in increased recovery time, higher costs, and a greater likelihood of data loss or regulatory penalties.

• Solution: To effectively manage security incidents, businesses must develop and maintain a comprehensive incident response plan. Start by clearly defining the roles and responsibilities of all team members, ensuring that everyone knows their part in the event of an incident. Regularly conduct incident response drills to practice these roles and improve the team's ability to respond swiftly and efficiently. It’s also crucial to establish communication protocols, both internally and externally, to keep all stakeholders informed during an incident. Additionally, document and review all incidents to learn from each event and refine your response plan. By implementing these measures, businesses can ensure a coordinated and efficient response to security incidents, minimizing damage and facilitating a quicker recovery.

References

• Windows resiliency: Best practices and the path forward - Microsoft Community Hub

• Securely design your applications and protect your sensitive data with VBS enclaves - Microsoft Community Hub

• New Windows 11 features strengthen security to address evolving cyberthreat landscape | Microsoft Security Blog

• Windows Security best practices for integrating and managing security tools | Microsoft Security Blog

• Microsoft Digital Defense Report 2023 (MDDR)

• 2024 Security Awareness Report | SANS Institute

• The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win: Kim, Gene, Behr, Kevin, Spafford, George: 9780988262591: Books - Amazon.ca *** While not directly referenced, many of the concepts discussed align with those covered in the book, which also happens to be a fantastic read.

• Bonus: You’re Doing It Wrong! Common Security Anti Patterns | RSA Conference *** Not referenced but a really great session that I highly recommend!

KnowBe4

A comprehensive platform that offers security awareness training and simulated phishing. They’ve got a ton of interactive training modules, videos, and games to educate your team about various cybersecurity threats. More info

Proofpoint Security Awareness Training

Proofpoint Security Awareness Training provides targeted training modules based on user behavior and the current threat landscape. It’s a solid option if you're looking for phishing simulations and other interactive content to engage your team. More info

Cofense PhishMe

Cofense PhishMe focuses specifically on phishing awareness and response training. They offer real-world phishing simulations and educational content to help users spot and avoid phishing attacks. More info

SANS Security Awareness Training

SANS is well-known for its high-quality training courses. They cover a wide range of topics, including phishing, social engineering, and data protection. Check out the SANS Security Awareness Report for more insights on building a strong security culture. More info

Microsoft Defender for Office 365 P2 (included in E5)

MDO P2 includes a great attack simulation training platform that you can run regular campaigns and provide training directly. It’s a fantastic way to keep your employees on their toes and educate them about the latest threats. You can simulate real-world attacks and give your team practical experience in recognizing and responding to these threats. It's all about making sure your people are prepared and know exactly what to do when faced with potential security issues. More info

Terranova Security

Terranova Security provides a full suite of security awareness training, including phishing simulations, interactive modules, and compliance training. They’ve got a good variety of content to keep things fresh and engaging. More info

Cybersecurity & Infrastructure Security Agency (CISA)

CISA offers a bunch of free resources and training materials to help businesses build a security-conscious culture. Their materials are especially useful if you're on a tight budget but may require some work to integrate into any campaigns you're running. More info

Internal Workshops and Seminars

Don't underestimate the power of in-house training sessions. Regular workshops, possibly with guest speakers from the cybersecurity industry, can keep your team up to date on the latest threats and best practices.

Get started

There are plenty of options available, as you can see, but the most important thing is to start implementing a security awareness program now rather than waiting until it's too late. Being proactive is key; addressing security risks before they become issues is always better than trying to fix problems after they've occurred. And let's face it, the human problem is the toughest one to deal with. People are often the weakest link in cybersecurity, so educating your team is crucial to staying ahead of potential threats.