How to connect ChatGPT to a company knowledge base safely

Choose the connection by job and control boundary

The phrase 'connect ChatGPT to our knowledge base' can describe four different projects. One is enabling a product feature. Another is approving a connected app. A third is publishing a custom MCP app inside a managed workspace. The fourth is building an application that uses OpenAI models through the API and owns its own retrieval stack.

These routes should not be evaluated only by setup speed. They differ in who authenticates the user, where source permissions are enforced, which administrator controls are available, whether write actions exist, what is logged, and who owns the application lifecycle.

Route one: start with ChatGPT company knowledge for sourced answers

OpenAI's company knowledge documentation says the feature is available to ChatGPT Business and Enterprise/Edu users with at least one eligible enabled app. It searches and fetches from connected sources, returns citations, and respects the permissions the user already has in those apps.

This is the lowest-customization route for a company already using a supported workspace. Test it before commissioning a new retrieval stack. Use real questions that require evidence from more than one connected system, and open the citations rather than grading only the prose.

There are current product limits to include in the decision. Company knowledge must be selected in the conversation flow described by OpenAI, availability depends on the plan and enabled apps, and write actions are not available when an app is called through company-knowledge mode. The feature is designed around information retrieval.

Route two: govern each app as a separate capability

OpenAI's current app administration guidance separates who can use an app from what the app can do and when ChatGPT asks before an action. Depending on the workspace and app, administrators can use app access, roles, action controls, and app permissions.

That separation should survive procurement. Do not approve a provider because the search action is useful and assume every future action is equally safe. Inventory the enabled reads and writes, the OAuth scopes they require, the roles that need them, and the behavior when the provider adds an action later.

Use one test account per important role. Confirm the connected app returns only material that role can open directly in the source system. Then remove a permission in the source and verify the result changes. A successful connection is not a permission test.

Route three: use a custom MCP app for approved private tools

Model Context Protocol, or MCP, gives a model a structured way to discover and call tools exposed by a server. A custom app can present company-specific search, fetch, and action capabilities inside ChatGPT without making the underlying system public.

OpenAI's developer mode and MCP guidance says administrators or authorized developers can test and publish custom apps, manage access, and control actions where supported. It also warns that custom apps are the organization's responsibility to vet and that untrusted MCP servers increase security exposure, including prompt injection.

Expose narrow tools named for real jobs. `find_current_client_agreement` is easier to govern than `search_everything`. `prepare_extension_request` is easier to review than `update_record`. Keep the server's own authorization checks; do not rely on the model to omit a tool it should not use.

For writes, separate preparation from execution. Return a structured preview containing the target, proposed change, source evidence, and required approver. Let application or workspace controls decide whether the final tool is available and when a person must confirm it.

Route four: build an API/RAG application when the company needs to own the operating layer

A separate application is justified when the business needs a custom interface, a retrieval design spanning databases and document stores, customer or tenant isolation, its own model-routing choices, a dedicated evaluation layer, or a workflow that cannot be expressed safely through existing apps.

Microsoft's RAG guidance describes retrieval-time document access control and recommends role-based production identity instead of relying on development API keys. Its secure multitenant architecture places tenant filtering in the data and retrieval design.

The model API is one component. The application must authenticate the user, resolve the permitted evidence, record citations, control tool availability, validate proposed changes, collect approval, and retain enough audit data to reconstruct an incident or disputed result.

OpenAI states that data from eligible business products and the API is not used to train models by default. That does not remove the need to review retention, regional processing, connected-service terms, and the company's own logging before sensitive data is sent.

Permission inheritance needs an end-to-end test

Write the path for one answer from the employee account to the source record. Which identity does ChatGPT or the custom application present to the app? Does the app query on behalf of the user or through a shared service identity? Where are client, group, document, or row filters applied? What identifier appears in the audit log?

A shared service credential can be appropriate inside a controlled application, but only when the application reliably applies the user's narrower permissions before retrieval. A broad key pasted into a prompt, browser extension, or client application is not an access model.

Test a user who should see the record, a user who should see only a redacted subset, a user from another client or department, and a recently revoked user. Inspect citations and tool inputs as well as visible answer text.

Assume a retrieved document may try to control the model

OWASP describes indirect prompt injection as malicious instructions embedded in content the model later processes, including documents, email, and web pages. A knowledge connection increases the amount of untrusted text that can reach the model.

Add a test document containing an instruction that conflicts with the application's rules. The safe result is that the text is treated as evidence to quote or ignore, not as authority to reveal another source or call a tool. Repeat the test with a permitted read-only tool and with an unavailable write tool.

No single prompt or filter proves safety. Reduce the blast radius with source trust levels, retrieval boundaries, limited tools, structured inputs, output validation, approval, and monitoring. The system should be able to show which retrieved passage influenced a disputed answer or proposed action.

A pre-launch test set for all four routes

Use the same business questions across the routes being considered. That makes product convenience visible without confusing it with answer quality or control.

• A known answer with a direct, current source
• A question whose evidence is split across two connected systems
• A restricted record requested by an unauthorized role
• Two approved sources that disagree
• A missing fact that should produce a limitation, not a guess
• A retrieved document containing a hostile instruction
• A proposed write with no approver available
• A destination-system failure after approval

Start with the route you can govern today

For many teams, the correct first result is a configured ChatGPT workspace that answers from approved apps with citations. That is faster and easier to reverse than a custom platform. The test may also reveal that the source content and permissions need work before any AI connection can be trusted.

Move to a custom MCP or API application when the evidence shows a real gap: unsupported systems, tenant-aware retrieval, custom authority rules, controlled workflow completion, or assurance requirements the built-in route cannot satisfy. The custom work should buy a specific operating capability, not merely reproduce a chat screen under a new logo.

If the system will write to business software, define the decision owner and acceptance test before enabling the credential. Connection is the beginning of the control problem, not the end of the implementation.