非你网

AI 对话项目，会用到流式返回。一般客户端不会直接调用 AI 对话，而是调用自己的服务，自己的服务再调用 AI. 服务端可以通过 websocket 的方式来达到这个效果，也可以通过 SSE 的方式来达到这个效果。而 AI 也可能会提供这两种方式来提供服务。这样一来，项目就可能有四种组合方式来进行 AI 对话。如下：

客户端	服务端	AI
APP、浏览器	websocket	websocket
APP、浏览器	websocket	SSE
APP、浏览器	SSE	websocket
APP、浏览器	SSE	SSE

https://developer.mozilla.org/zh-CN/docs/Web/API/Server-sent_events/Using_server-sent_events

找 AI

websocket 阵营

https://www.xfyun.cn/doc/spark/Web.html#_1-%E6%8E%A5%E5%8F%A3%E8%AF%B4%E6%98%8E

SSE 阵营

https://agents.baidu.com/docs/develop/out-deployment/conversation/

尝试实现四种组合

从最简单的开始。服务端通过 SSE 的方式, AI 也是 SSE 的方式。

先封装一个 curl 工具包,创建 internal/pkg/curl/curl.go,内容如下。

package curl

import (
    "bytes"
    "context"
    "fmt"
    "io"
    "net/http"

    "github.com/gogf/gf/v2/frame/g"
    "github.com/gogf/gf/v2/net/gclient"
)

type HttpClient struct {
    client *gclient.Client
}

func NewHttpClient() *HttpClient {
    return &HttpClient{
        client: g.Client(),
    }
}

func (h *HttpClient) Get(ctx context.Context, url string, headers map[string]string) (string, error) {
    res, err := h.client.Get(ctx, url, headers)
    if err != nil {
        return "", err
    }
    defer res.Close()

    return res.ReadAllString(), nil
}

func (h *HttpClient) Post(ctx context.Context, url string, data interface{}, headers map[string]string) (string, error) {
    for k, v := range headers {
        h.client.SetHeader(k, v)
    }

    res, err := h.client.Post(ctx, url, data, headers)
    if err != nil {
        return "", err
    }
    defer res.Close()
    return res.ReadAllString(), nil
}

func (h *HttpClient) PostStream(ctx context.Context, url string, data interface{}, headers map[string]string) (io.ReadCloser, error) {
    // 将 data 转换为 io.Reader
    var body io.Reader
    switch v := data.(type) {
    case string:
        body = bytes.NewReader([]byte(v))
    case []byte:
        body = bytes.NewReader(v)
    default:
        return nil, fmt.Errorf("unsupported data type: %T", v)
    }

    // 创建 HTTP 请求
    req, err := http.NewRequestWithContext(ctx, "POST", url, body)
    if err != nil {
        return nil, err
    }

    // 设置请求头
    for k, v := range headers {
        req.Header.Set(k, v)
    }

    // 发起请求
    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return nil, err
    }

    // 检查响应状态码
    if resp.StatusCode != http.StatusOK {
        return nil, fmt.Errorf("request failed with status code: %d", resp.StatusCode)
    }

    return resp.Body, nil
}

现在来完善 chat 接口。

AI 聊天，会用到流式返回的接口。用 goframe 框架尝试尝试。

开始

初始化项目

gf init stream.demo -u

cd stream.demo

gf run main.go

默认路由信息是这样的。

build: main.go
go build -o .\main.exe  main.go
.\main.exe
build running pid: 44364
2025-07-11T13:44:19.160+08:00 [INFO] pid[44364]: http server started listening on [:8000]
2025-07-11T13:44:19.160+08:00 [INFO] {ac79854f201c51188f16b244dba07b30} swagger ui is serving at address: http://127.0.0.1:8000/swagger/
2025-07-11T13:44:19.161+08:00 [INFO] {ac79854f201c51188f16b244dba07b30} openapi specification is serving at address: http://127.0.0.1:8000/api.json

  ADDRESS | METHOD |   ROUTE    |                           HANDLER                           |           MIDDLEWARE
----------|--------|------------|-------------------------------------------------------------|----------------------------------
  :8000   | ALL    | /api.json  | github.com/gogf/gf/v2/net/ghttp.(*Server).openapiSpec       |
----------|--------|------------|-------------------------------------------------------------|----------------------------------
  :8000   | GET    | /hello     | stream.demo/internal/controller/hello.(*ControllerV1).Hello | ghttp.MiddlewareHandlerResponse
----------|--------|------------|-------------------------------------------------------------|----------------------------------
  :8000   | ALL    | /swagger/* | github.com/gogf/gf/v2/net/ghttp.(*Server).swaggerUI         | HOOK_BEFORE_SERVE
----------|--------|------------|-------------------------------------------------------------|----------------------------------

提交版本

git init -b main
git add .
git commit -m 'initialize'

定义接口，创建控制器，定义路由

新建 api/chat/v1/chat.go 文件，定义以下请求和返回

package v1

import "github.com/gogf/gf/v2/frame/g"

type ChatReq struct {
    g.Meta  `path:"/chat" method:"post" tags:"聊天"  summary:"聊天请求"`
    Uid     uint64 `v:"required#用户ID不能为空" json:"uid" dc:"用户ID"`
    Message string `v:"required#消息内容不能为空" json:"message" dc:"消息内容"`
}

type ChatRes struct {
    IsEnd bool `json:"is_end" dc:"是否结束"`
}

执行 gf gen ctrl 生成控制器。修改 internal/cmd/cmd.go,增加路由配置。

import (
    ……
    "stream.demo/internal/controller/hello"
)

……
s.Group("/api", func(group *ghttp.RouterGroup) {
    group.Bind(
        chat.NewV1(),
    )
})

再运行 gf run main.go,把项目跑起来看看。发现 chat 路由已经好了。

build: main.go
go build -o .\main.exe  main.go
.\main.exe
build running pid: 44148
2025-07-11T14:13:29.676+08:00 [INFO] pid[44148]: http server started listening on [:8000]
2025-07-11T14:13:29.676+08:00 [INFO] {5065f8ccb71d5118823d646fdde40224} swagger ui is serving at address: http://127.0.0.1:8000/swagger/
2025-07-11T14:13:29.677+08:00 [INFO] {5065f8ccb71d5118823d646fdde40224} openapi specification is serving at address: http://127.0.0.1:8000/api.json

  ADDRESS | METHOD |   ROUTE    |                           HANDLER                           |           MIDDLEWARE
----------|--------|------------|-------------------------------------------------------------|----------------------------------
  :8000   | ALL    | /api.json  | github.com/gogf/gf/v2/net/ghttp.(*Server).openapiSpec       |
----------|--------|------------|-------------------------------------------------------------|----------------------------------
  :8000   | POST   | /api/chat      | stream.demo/internal/controller/chat.(*ControllerV1).Chat   | ghttp.MiddlewareHandlerResponse
----------|--------|------------|-------------------------------------------------------------|----------------------------------
  :8000   | GET    | /hello     | stream.demo/internal/controller/hello.(*ControllerV1).Hello | ghttp.MiddlewareHandlerResponse
----------|--------|------------|-------------------------------------------------------------|----------------------------------
  :8000   | ALL    | /swagger/* | github.com/gogf/gf/v2/net/ghttp.(*Server).swaggerUI         | HOOK_BEFORE_SERVE
----------|--------|------------|-------------------------------------------------------------|----------------------------------

开始流式 demo

编辑 internal/controller/chat/chat_v1_chat.go

package chat

import (
    "context"
    "fmt"
    "time"

    "github.com/gogf/gf/v2/net/ghttp"

    v1 "stream.demo/api/chat/v1"
)

func (c *ControllerV1) Chat(ctx context.Context, req *v1.ChatReq) (res *v1.ChatRes, err error) {
    r := ghttp.RequestFromCtx(ctx)
    r.Response.Header().Set("Content-Type", "text/event-stream")
    r.Response.Header().Set("Cache-Control", "no-cache")
    r.Response.Header().Set("Connection", "keep-alive")
    r.Response.Header().Set("Content-Encoding", "identity")

    for i := range 10 {
        data := fmt.Sprintf(`{"uid":%d, "message":"%s","index":%d}`, req.Uid, req.Message, i)
        r.Response.Writefln("data: %s\n\n", data)
        r.Response.Flush()
        time.Sleep(time.Millisecond * 1000)
    }

    fmt.Println("发送结束")
    res = &v1.ChatRes{}
    res.IsEnd = true
    return
}

重新运行项目。

测试

可以使用 curl 命令，可以使用 apifox 这种工具，也可以使用浏览器。 这里使用浏览器加 js 的方式来搞。

使用 chrome 打开 http://127.0.0.1:8000 ，然后打开控制台。输入以下测试代码，回车。

fetch("http://127.0.0.1:8000/api/chat", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    message: "舞低杨柳楼心月，歌尽桃花扇底风",
    uid: 1,
  }),
})
  .then(readStream)
  .catch(console.error);

async function readStream(res) {
  const reader = res.body.getReader();
  const decoder = new TextDecoder("utf-8");
  let buffer = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });

    let lines = buffer.split("\n\n");
    buffer = lines.pop();

    for (const chunk of lines) {
      const dataLine = chunk.trim();
      if (dataLine.startsWith("data: ")) {
        const data = dataLine.slice(6);
        console.log(getNowTime(), "消息:", data);
      }
    }
  }
}

function getNowTime() {
  const date = new Date();
  return date.toLocaleTimeString("zh-CN", {
    hour12: false,
    timeZone: "Asia/Shanghai",
  });
}

截图如下：

提交版本控制。

git add .
git commit -m '增加 chat 接口，并测试流式响应'

nginx 处理

如果走 nginx 代理，需要处理关闭相应缓冲

location /api/ {
    proxy_pass http://你的后端服务地址;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_buffering off;
    proxy_cache off;
    proxy_request_buffering off;
    chunked_transfer_encoding on;
}