API: Using sendRequest and getting cut off response (including in chat-sample)

[worksofliam]

My Copilot responses are being cut of when using the API to implement my chat. It doesn't seem to happen all the time.

Code

async function streamModelResponse(
  messages: vscode.LanguageModelChatMessage[],
  stream: vscode.ChatResponseStream,
  token: vscode.CancellationToken
) {
  const chosenProvider = AiConfig.getProvider();
  const chosenModel = AiConfig.getModel();

  if (chosenProvider === `none`) {
    stream.markdown(
      `No AI provider selected. Please select an AI provider and model.`
    );
    stream.button({
      command: `vscode-db2i.ai.changeModel`,
      title: `Select AI Provider and Model`,
    });
    return;
  }

  showModelProviderIfNeeded(stream, chosenProvider, chosenModel);
  stream.progress(`Provider: ${chosenProvider} Model: ${chosenModel}`);

  return copilotRequest(chosenModel, messages, {}, token, stream);
}

async function copilotRequest(
  model: string,
  messages: LanguageModelChatMessage[],
  options: LanguageModelChatRequestOptions,
  token: CancellationToken,
  stream: vscode.ChatResponseStream
): Promise<void> {
  const models = await vscode.lm.selectChatModels({ family: model });
  if (models.length > 0) {
    const [first] = models;
    const response = await first.sendRequest(messages, options, token);

    try {
      for await (const fragment of response.text) {
        stream.markdown(fragment);
      }
    } catch (e) {
      console.error(e);
      stream.markdown("An error occurred while processing the request.");
    }
  }
}

More details with screenshots

When using standard copilot, it does not cut off. Could it be the case that my input/context is too big?

Further progress using a different model but same issue.

Took out all my User messages / context and still getting the same issue.

With logs:

2024-07-01 14:49:23.623 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 855 ms
2024-07-01 14:49:26.248 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-07-01 14:49:26.249 [info] [streamChoices] request done: requestId: [2495000d-a584-4652-80f2-a4b1af36754c] model deployment ID: []

[worksofliam]

I have a colleague who is checked out with the same code and the issue is not happening for them.

[worksofliam]

More requests that finish but seemingly cut off.

Screenshot

2024-07-01 15:51:47.308 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 1089 ms
2024-07-01 15:52:02.460 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-07-01 15:52:02.461 [info] [streamChoices] request done: requestId: [2f06e0be-77a4-40b3-a539-f794b69e916e] model deployment ID: []
2024-07-01 15:53:31.062 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 1203 ms
2024-07-01 15:53:34.183 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-07-01 15:53:34.184 [info] [streamChoices] request done: requestId: [9e77cd62-0045-4818-9f29-26f48f0aa608] model deployment ID: []

[worksofliam]

I have recreate this issue with the vscode-extension-samples -> chat-sample: https://github.com/microsoft/vscode-extension-samples/tree/main/chat-sample

2024-07-01 16:00:55.412 [info] [chat fetch] request.response: [https://api.githubcopilot.com/chat/completions], took 329 ms
2024-07-01 16:00:55.929 [info] [streamMessages] message 0 returned. finish reason: [stop]
2024-07-01 16:00:55.929 [info] [streamChoices] request done: requestId: [94db1a75-8a4f-4dba-814c-6fddb30bc239] model deployment ID: [dep-3]

Copilot Diags

Copilot

• Version: 1.208.962
• Build: nightly
• Editor: vscode/1.91.0-insider

Environment

• http_proxy: n/a
• https_proxy: n/a
• no_proxy: n/a
• SSL_CERT_FILE: n/a
• SSL_CERT_DIR: n/a
• OPENSSL_CONF: n/a

Feature Flags

• Send Restricted Telemetry: enabled
• Chat: enabled

Node setup

• Number of root certificates: 141
• Operating system: Darwin
• Operating system version: 23.5.0
• Operating system architecture: arm64
• NODE_OPTIONS: n/a
• NODE_EXTRA_CA_CERTS: n/a
• NODE_TLS_REJECT_UNAUTHORIZED: n/a
• tls default min version: TLSv1.2
• tls default max version: TLSv1.3

Network Configuration

• Proxy host: n/a
• Proxy port: n/a
• Kerberos SPN: n/a
• Reject unauthorized: disabled
• Fetcher: HelixFetcher

Reachability

• github.com: HTTP 200
• api.github.com: HTTP 200
• copilot-proxy.githubusercontent.com: HTTP 200
• api.githubcopilot.com: HTTP 200
• default.exp-tas.com: HTTP 200

VS Code Configuration

• HTTP proxy:
• HTTP proxy authentication: n/a
• Proxy Strict SSL: true
• Extension HTTP proxy support: override

Extensions

• Is win-ca installed?: false
• Is mac-ca installed?: false

Authentication

• GitHub username: worksofliam

Screenshot showing normal Copilot works fine

[roblourens]

So you're seeing the issue with the chat-sample, and no modifications to it? A couple days later, is this still happening? Could have been a transient service issue.

It looks like you found the "Copilot: Collect Diagnostics" command, thanks. Could you also run "Developer: GitHub Copilot Chat Diagnostics"?

I'm not sure what else to look at, any ideas @jrieken?

[jrieken]

What version of VS Code and chat do you use? Around a week ago there was a bug in this area which since has been fixed