在公司CodeX/Claude code 無法使用本地自建LLM服務伺服器問題排查順序

資料來源:https://chatgpt.com/share/6a69681e-fc94-83ee-a7c7-1218ca37c306

https://chatgpt.com/share/6a696e23-00f8-83e8-bf08-1b5dcb299226

公司本地LLM系統架構

OpenAI Codex / VSCode...
        │
        ▼
     CCSWITCH
        │
        ▼
   LiteLLM Proxy
        │
        ▼
      vLLM
        │
        ▼
    LLM模型

AI提問: 我現在透過CCSWITCH 連結公司的liteLLM 後面是 VLLM 當我的CODEX 沒反應 我要如何進行除錯排查 看看哪裡有問題

排查順序步驟:

01.檢查VLLM是否能正常反應[WINDOWS CMD]

curl http://192.168.1.30:8000/v1/models -H "Content-Type: application/json" -H "Authorization: Bearer MySecretKey_123456"

上述命令正常回復備份(比對用)

C:\Users\jashv>curl http://192.168.1.30:8000/v1/models  -H "Content-Type: application/json" -H "Authorization: Bearer MySecretKey_123456"
{"object":"list","data":[{"id":"Qwen3.6-35B-FP4","object":"model","created":1785292770,"owned_by":"vllm","root":"/model","parent":null,"max_model_len":262144,"permission":[{"id":"modelperm-aeed806f834f4731","object":"model_permission","created":1785292770,"allow_create_engine":false,"allow_sampling":true,"allow_logprobs":true,"allow_search_indices":false,"allow_view":true,"allow_fine_tuning":false,"organization":"*","group":null,"is_blocking":false}]}]}
C:\Users\jashv>

02.VLLM基本對話測試[WINDOWS CMD]

curl http://192.168.1.30:8000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer MySecretKey_123456" -d "{\"model\": \"Qwen3.6-35B-FP4\", \"messages\": [{\"role\": \"user\", \"content\": \"HELLO\"}], \"temperature\": 0.7}"

上述命令正常回復備份(比對用)

C:\Users\jashv>curl http://192.168.1.30:8000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer MySecretKey_123456" -d "{\"model\": \"Qwen3.6-35B-FP4\", \"messages\": [{\"role\": \"user\", \"content\": \"HELLO\"}], \"temperature\": 0.7}"
{"id":"chatcmpl-8146e1e463644aa4","object":"chat.completion","created":1785293142,"model":"Qwen3.6-35B-FP4","choices":[{"index":0,"message":{"role":"assistant","content":"\n\nHello! 👋 How can I help you today? Feel free to ask me anything!","refusal":null,"annotations":null,"audio":null,"function_call":null,"reasoning":"Here's a thinking process:\n\n1.  **Analyze User Input:**\n   - User said: \"HELLO\"\n   - It's a simple greeting in all caps.\n   - No specific question or request is attached.\n\n2.  **Identify Intent:**\n   - The user is initiating a conversation.\n   - They likely expect a friendly, welcoming response.\n   - The tone is casual but enthusiastic (due to all caps).\n\n3.  **Determine Response Strategy:**\n   - Acknowledge the greeting warmly.\n   - Match the tone appropriately (friendly, helpful, not overly formal).\n   - Invite them to ask a question or share what they need help with.\n   - Keep it concise.\n\n4.  **Draft Response (Mental):**\n   Hello! 👋 How can I help you today? Feel free to ask me anything!\n\n5.  **Refine Response:**\n   - Check tone: Friendly, open, helpful.\n   - Check length: Short and direct.\n   - Check appropriateness: Matches user's simple greeting, invites next step.\n   - No extra fluff.\n\n   Final version matches the draft.\n\n6.  **Output Generation:** (Proceeds to output)✅\n"},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null,"routed_experts":null}],"service_tier":null,"system_fingerprint":"vllm-0.25.1-4203034a","usage":{"prompt_tokens":12,"total_tokens":299,"completion_tokens":287,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"prompt_text":null,"kv_transfer_params":null,"metrics":null}
C:\Users\jashv>

如果VLLM沒有回應 -> 

開啟終端機並執行下列命令

cd /data
sudo ./vllm_crash_log_shell.sh 

然後系統重開(反正模型載入要很久 接重開最簡單)

PS.

# 重啟 vLLM
docker compose restart vllm-qwen

# 看啟動 Log
docker compose logs -f vllm-qwen

============

03.檢查LiteLLM是否能正常反應[WINDOWS CMD]

curl http://192.168.1.30:4000/v1/models  -H "Content-Type: application/json" -H "Authorization: Bearer sk-master-key-1234567890"

上述命令正常回復備份(比對用)

C:\Users\jashv>curl http://192.168.1.30:4000/v1/models  -H "Content-Type: application/json" -H "Authorization: Bearer sk-master-key-1234567890"
{"data":[{"id":"Qwen3.6-35B-FP4","object":"model","created":1677610602,"owned_by":"openai"}],"object":"list"}
C:\Users\jashv>

04.LiteLLM基本對話測試[WINDOWS CMD]

curl http://192.168.1.30:4000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer sk-master-key-1234567890" -d "{\"model\": \"Qwen3.6-35B-FP4\", \"messages\": [{\"role\": \"user\", \"content\": \"HELLO\"}], \"temperature\": 0.7}"

上述命令正常回復備份(比對用)

C:\Users\jashv>curl http://192.168.1.30:4000/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer sk-master-key-1234567890" -d "{\"model\": \"Qwen3.6-35B-FP4\", \"messages\": [{\"role\": \"user\", \"content\": \"HELLO\"}], \"temperature\": 0.7}"
{"id":"chatcmpl-bbdecf7ced1bfd65","created":1785293636,"model":"Qwen3.6-35B-FP4","object":"chat.completion","system_fingerprint":"vllm-0.25.1-4203034a","choices":[{"finish_reason":"stop","index":0,"message":{"content":"\n\nHello! How can I assist you today?","role":"assistant","tool_calls":null,"function_call":null,"refusal":null,"annotations":null,"reasoning":"Here's a thinking process:\n\n1.  **Analyze User Input:**\n   - User said: \"HELLO\"\n   - It's a simple greeting in all caps.\n   - No specific question or request is attached.\n\n2.  **Identify Intent:**\n   - The user is initiating a conversation.\n   - Standard response should be friendly, acknowledge the greeting, and invite further interaction.\n\n3.  **Determine Response Strategy:**\n   - Match the tone (friendly, professional).\n   - Acknowledge the greeting.\n   - Ask how I can help.\n   - Keep it concise.\n\n4.  **Draft Response (Mental):**\n   Hello! How can I assist you today?\n\n5.  **Refine Response:**\n   - Check for appropriateness: Yes, it's polite, open-ended, and ready for follow-up.\n   - No need to overcomplicate a simple greeting.\n\n6.  **Final Output Generation:** (Matches the refined draft)\n   \"Hello! How can I assist you today?\"✅\n"}}],"usage":{"completion_tokens":240,"prompt_tokens":12,"total_tokens":252,"completion_tokens_details":null,"prompt_tokens_details":null},"service_tier":null,"prompt_logprobs":null,"prompt_token_ids":null,"prompt_text":null,"kv_transfer_params":null,"metrics":null}
C:\Users\jashv>

如果LiteLLM沒有回應 -> 那就重啟對應DOCKER[下面是Linux shell]

# 重啟 LiteLLM
docker compose restart litellm-proxy
# 查看 LiteLLM Log
docker compose logs -f litellm-proxy

# 查看所有容器狀態
docker compose ps