LLM

Connect a language model to ask about findings and products in a chat.

Updated

Connect a language model, and an LLM chat opens from the top bar of every page. Ask it how to fix a finding, what a product’s riskiest findings have in common, or anything else; you can attach findings and products to the question.

LLM integrations need a paid license. See License.

An external LLM provider receives what you send to the chat, including the findings you attach, and findings can contain secrets. When that matters, use a model that runs on your own servers, such as Ollama.

Connect a model

Open the settings, then LLM Settings under Integrations, and create an integration.

  1. Choose your LLM provider: Claude, Open AI, Gemini, Ollama, or Custom for any other HTTP API.
  2. Authentication: the API key from the provider’s console, or for Ollama the URL of your server.
  3. Payload Config and Response Config are filled in for the listed providers. For a custom API, enter the Request payload (JSON), the Prompt path in request payload and the Path to answer in the response.
  4. Settings:
    • Use as default LLM in chat: the model a new chat starts with.
    • Use as default LLM for summary generation: the model that summarizes the earlier messages of a chat, when the chat sends a summary instead of the whole history.
    • Allow Manual Context: users can attach findings and products to their questions.
    • Allow Read Data Tools: the model can look up data in Portal itself, for example to find the findings of a product. Click View Read Tools to see what it can read.
    • Request Timeout (seconds).

Run test request checks the integration before you save it.

Use the chat

Open the chat from the top bar. Each chat remembers its own history; Add chat starts another.

  • Ask next to a finding’s name opens the chat with the finding attached.
  • In the lists of findings and products, the menu of each row adds it to the chat’s context.
  • Context mode sets how much of the conversation the model receives with each question: No context, Last messages, a Summary of the earlier messages, or the Full history.
  • Each chat can use another model, chosen in its settings.

Check the answers before you act on them. Tool calls under an answer show what the model read in Portal.