CodingDebuggingIntermediate30 minSaves 30 minutes

Diagnosing TLS Handshake Failures Between Internal Services

Platform engineers can quickly isolate and resolve common mTLS handshake issues, minimizing downtime for critical internal service communications.

Platform engineers facing TLS handshake failures between internal services can use this playbook to diagnose issues. It provides hypotheses, command-line checks for certificate chains, SNI, and cipher suites, common fixes, and verification steps for rapid resolution.

READY-TO-USE PROMPT

Copy Prompt

prompt.txt
As a seasoned network diagnostics expert specializing in secure communications, your role is to guide a platform engineer through a common yet complex issue: TLS handshake failures between internal services.

### Context
A platform engineer is encountering a persistent TLS handshake failure between two internal microservices: `{{failing_service_name}}` and `{{peer_service_name}}`. The error logs indicate a generic TLS handshake failure, without specific root causes. The engineer suspects issues related to certificate chain validation, Server Name Indication (SNI) mismatches, or cipher suite incompatibilities, common culprits in mTLS environments.

### Task
Generate a comprehensive debugging playbook. This playbook should systematically address the most common hypotheses for TLS handshake failures in an internal service context. For each hypothesis, provide specific diagnostic checks, including command-line tools and expected outputs, along with likely fixes and verification steps.

### Constraints
*   **Focus Areas**: The playbook must specifically cover certificate chain validation, SNI issues, and cipher suite mismatches. Do not deviate into other networking issues unless directly related to these TLS aspects.
*   **Practical Checks**: Each diagnostic check must include actionable command-line examples, assuming a Linux environment (e.g., `openssl`, `curl`, `ss`, `netstat`, `tcpdump`).
*   **Output Structure**: The playbook must follow the exact structure: `Symptom` → `Hypothesis List` → `Checks (with commands)` → `Likely Fixes` → `Verification`.
*   **Clarity**: Ensure the language is precise, diagnostic, and avoids ambiguity.
*   **No Generic Advice**: Provide concrete steps and commands, not high-level suggestions.

### Output Format
The output must be a markdown-formatted debug playbook with the following sections for each major hypothesis:

1.  **Symptom**: Brief description of the observed failure.

2.  **Hypothesis List**:
    *   Hypothesis 1: `[Specific issue, e.g., Mismatched Certificate Chain]`
    *   Hypothesis 2: `[Specific issue, e.g., SNI Mismatch]`
    *   Hypothesis 3: `[Specific issue, e.g., Incompatible Cipher Suites]`
    *   ...

3.  **Checks (with commands)**:
    *   **Check for Hypothesis 1**: `[Description of check]`
        *   `[Command-line example]`
        *   `[Expected output / interpretation]`
    *   **Check for Hypothesis 2**: `[Description of check]`
        *   `[Command-line example]`
        *   `[Expected output / interpretation]`
    *   ...

4.  **Likely Fixes**:
    *   **Fix for Hypothesis 1**: `[Actionable solution]`
    *   **Fix for Hypothesis 2**: `[Actionable solution]`
    *   ...

5.  **Verification**:
    *   `[Steps to confirm resolution, e.g., re-running service, checking logs]`

Estimated results

DifficultyIntermediate
Setup time30 min
Time saved30 minutes
Best modelsChatGPT, Gemini, Claude
Best audienceTechnology, Cloud Computing

Editor's note

Why this prompt matters

Debugging TLS handshake failures in an internal microservices environment can be a time-consuming process. When services fail to communicate securely, it often points to issues like misconfigured certificates, incorrect Server Name Indication, or incompatible cipher suites. These errors can be particularly opaque, with generic logs offering little specific guidance.

This workflow is designed for platform engineers who need to quickly diagnose and resolve mTLS communication breakdowns. It provides a structured approach to tackle the most common culprits, moving beyond guesswork to targeted diagnostic steps. When you're facing a critical service interruption and suspect a TLS-related problem between your internal applications, this playbook offers a systematic path to identify the root cause and implement a fix, reducing the impact of downtime. It's particularly useful when generic error messages obscure the underlying certificate or protocol negotiation issues.

Anatomy

Prompt engineering breakdown

Role

seasoned network diagnostics expert specializing in secure communications

Context

A platform engineer is encountering a persistent TLS handshake failure between two internal microservices: {{failing_service_name}} and {{peer_service_name}}. The error logs indicate a generic TLS handshake failure, without specific root causes. The engineer suspects issues related to certificate chain validation, Server Name Indication (SNI) mismatches, or cipher suite incompatibilities, common culprits in mTLS environments.

Goal

Generate a comprehensive debugging playbook. This playbook should systematically address the most common hypotheses for TLS handshake failures in an internal service context. For each hypothesis, provide specific diagnostic checks, including command-line tools and expected outputs, along with likely fixes and verification steps.

Constraints

Focus Areas: The playbook must specifically cover certificate chain validation, SNI issues, and cipher suite mismatches. Do not deviate into other networking issues unless directly related to these TLS aspects. Practical Checks: Each diagnostic check must include actionable command-line examples, assuming a Linux environment (e.g., openssl, curl, ss, netstat, tcpdump). Output Structure: The playbook must follow the exact structure: Symptom Hypothesis List Checks (with commands) Likely Fixes Verification. Clarity: Ensure the language is precise, diagnostic, and avoids ambiguity. No Generic Advice: Provide concrete steps and commands, not high-level suggestions.

Output format

markdown-formatted debug playbook with the following sections for each major hypothesis: Symptom, Hypothesis List, Checks (with commands), Likely Fixes, Verification.

Why this structure works

Role priming establishes the model's authority as an expert, ensuring diagnostic and precise guidance. Explicit constraints on focus areas and output structure direct the model to produce a highly relevant and actionable debugging playbook. The structured output ensures that the engineer receives a consistent, easy-to-follow guide for troubleshooting.

Pick your version

Prompt variations

BeginnerWorks with any model

For engineers new to TLS debugging or requiring a simpler, step-by-step guide with fundamental checks.

prompt.txt
Act as a helpful guide for platform engineers troubleshooting secure communication issues. Your goal is to walk a beginner through diagnosing common TLS handshake failures between internal services.

### Context
You're seeing a TLS handshake error when `{{failing_service_name}}` tries to talk to `{{peer_service_name}}`. The error messages aren't very clear, but often these problems come from issues with certificates, how services identify themselves (SNI), or incompatible security settings (cipher suites).

### Task
Create a simple, step-by-step debugging guide. For each common problem area, give clear checks to run, what commands to use (like `openssl` or `curl`), what results to look for, and how to fix it.

### Focus Areas
Stick to checking:
*   Certificate chain validation (making sure certificates are trusted)
*   SNI (Server Name Indication) issues (service identity)
*   Cipher suite mismatches (agreeing on encryption methods)

### Guide Structure
Organize your guide like this for each issue:
*   **Symptom**: What you see.
*   **Hypothesis**: What might be wrong.
*   **Checks (with commands)**: How to find out, with specific commands and what to expect.
*   **Likely Fixes**: How to solve it.
*   **Verification**: How to confirm it's fixed.
ProfessionalBest with claude

When a detailed, command-line focused playbook is needed for specific, common TLS issues by an experienced engineer.

prompt.txt
As a seasoned network diagnostics expert specializing in secure communications, your role is to guide a platform engineer through a common yet complex issue: TLS handshake failures between internal services.

### Context
A platform engineer is encountering a persistent TLS handshake failure between two internal microservices: `{{failing_service_name}}` and `{{peer_service_name}}`. The error logs indicate a generic TLS handshake failure, without specific root causes. The engineer suspects issues related to certificate chain validation, Server Name Indication (SNI) mismatches, or cipher suite incompatibilities, common culprits in mTLS environments.

### Task
Generate a comprehensive debugging playbook. This playbook should systematically address the most common hypotheses for TLS handshake failures in an internal service context. For each hypothesis, provide specific diagnostic checks, including command-line tools and expected outputs, along with likely fixes and verification steps.

### Constraints
*   **Focus Areas**: The playbook must specifically cover certificate chain validation, SNI issues, and cipher suite mismatches. Do not deviate into other networking issues unless directly related to these TLS aspects.
*   **Practical Checks**: Each diagnostic check must include actionable command-line examples, assuming a Linux environment (e.g., `openssl`, `curl`, `ss`, `netstat`, `tcpdump`).
*   **Output Structure**: The playbook must follow the exact structure: `Symptom` 
 `Hypothesis List` 
 `Checks (with commands)` 
 `Likely Fixes` 
 `Verification`.
*   **Clarity**: Ensure the language is precise, diagnostic, and avoids ambiguity.
*   **No Generic Advice**: Provide concrete steps and commands, not high-level suggestions.

### Output Format
The output must be a markdown-formatted debug playbook with the following sections for each major hypothesis:

1.  **Symptom**: Brief description of the observed failure.

2.  **Hypothesis List**:
    *   Hypothesis 1: `[Specific issue, e.g., Mismatched Certificate Chain]`
    *   Hypothesis 2: `[Specific issue, e.g., SNI Mismatch]`
    *   Hypothesis 3: `[Specific issue, e.g., Incompatible Cipher Suites]`
    *   ...

3.  **Checks (with commands)**:
    *   **Check for Hypothesis 1**: `[Description of check]`
        *   `[Command-line example]`
        *   `[Expected output / interpretation]`
    *   **Check for Hypothesis 2**: `[Description of check]`
        *   `[Command-line example]`
        *   `[Expected output / interpretation]`
    *   ...

4.  **Likely Fixes**:
    *   **Fix for Hypothesis 1**: `[Actionable solution]`
    *   **Fix for Hypothesis 2**: `[Actionable solution]`
    *   ...

5.  **Verification**:
    *   `[Steps to confirm resolution, e.g., re-running service, checking logs]`
Short VersionWorks with any model

For quick reference or when needing a summary of common TLS debugging steps without extensive detail.

prompt.txt
Generate a focused debugging playbook for a platform engineer experiencing TLS handshake failures between `{{failing_service}}` and `{{peer_service}}`. As a network expert, create a concise guide covering certificate chain validation, SNI issues, and cipher suite mismatches. For each, provide a specific diagnostic check with a command-line example (e.g., `openssl`, `curl`), expected output, a likely fix, and a verification step. Structure the output as: Symptom, Hypothesis List, Checks (with commands), Likely Fixes, and Verification, ensuring all advice is practical and command-line driven for Linux environments.
EnterpriseBest with chatgpt

In regulated environments where compliance, auditing, and stakeholder communication are critical alongside technical resolution.

prompt.txt
As a Principal Security Architect specializing in mTLS and incident response, guide a platform engineer and relevant stakeholders through a critical TLS handshake failure between `{{failing_service_name}}` and `{{peer_service_name}}`. This failure impacts critical internal service communication and requires a structured, auditable approach.

### Context
A platform engineer has reported a persistent TLS handshake failure, with generic logs. Initial suspicions point to certificate chain, SNI, or cipher suite issues, which are common in our mTLS framework. The resolution process must account for security best practices, compliance requirements, and potential audit trails.

### Task
Develop a comprehensive debugging and incident response playbook. This playbook must systematically address the identified TLS failure hypotheses, providing technical diagnostic steps alongside considerations for documentation, stakeholder communication, and compliance.

### Constraints
*   **Technical Scope**: Strictly adhere to certificate chain, SNI, and cipher suite issues.
*   **Auditability**: All diagnostic checks and fixes must be documented in a way that supports post-incident review.
*   **Communication**: Include prompts for informing security, compliance, and service owners.
*   **Risk Assessment**: Briefly note potential impact of prolonged failure.
*   **Practical Checks**: Include command-line examples (Linux: `openssl`, `curl`, `tcpdump`).
*   **Output Structure**: Follow: `Symptom` 
 `Hypothesis List` 
 `Checks (with commands) & Audit Notes` 
 `Likely Fixes & Compliance Steps` 
 `Verification & Incident Closure`.

### Output Format
A markdown-formatted debug and incident response playbook, structured as specified, emphasizing clear, actionable steps and enterprise-level considerations for each hypothesis.

What you'll get

Expected output

Symptom: Persistent TLS handshake failure observed in order-service logs when attempting to connect to payment-gateway. Logs show generic "TLS handshake failed" or "remote host closed connection" messages, without specific certificate or protocol errors.

Hypothesis List:

  • Hypothesis 1: Certificate Chain Validation Failure
  • Hypothesis 2: Server Name Indication (SNI) Mismatch
  • Hypothesis 3: Incompatible Cipher Suites

Checks (with commands):

Check for Hypothesis 1: Verify payment-gateway's certificate chain and order-service's trust store. * Inspect payment-gateway's presented certificate chain: openssl s_client -connect payment-gateway.internal.svc.cluster.local:443 -showcerts -servername payment-gateway.internal.svc.cluster.local < /dev/null * *Expected output / interpretation*: The output should show the full certificate chain, including the root CA. Look for Verify return code: 0 (ok) and ensure all certificates are valid and not expired. Any error codes here indicate an issue with the certificate itself or the chain. * Check order-service's trust store for the issuing CA: sudo keytool -list -keystore /etc/ssl/certs/java/cacerts (for Java applications) or ls /etc/ssl/certs/ (for system-wide CAs) * *Expected output / interpretation*: Confirm the root CA or intermediate CA that signed payment-gateway's certificate is present in order-service's trust store.

Check for Hypothesis 2: Confirm SNI is being sent correctly by order-service and accepted by payment-gateway. * Simulate order-service's connection with explicit SNI: openssl s_client -connect payment-gateway.internal.svc.cluster.local:443 -servername payment-gateway.internal.svc.cluster.local * *Expected output / interpretation*: A successful connection and handshake details. If this works but order-service fails, it suggests order-service might not be sending the correct SNI. If it fails, the payment-gateway might not be configured for the SNI. * Capture traffic on payment-gateway to inspect SNI: sudo tcpdump -i any -s 0 -A 'port 443 and tcp[((tcp[12:1]&0xf0)>>2):4]=0x16030100' | grep 'Server Name' * *Expected output / interpretation*: Look for the Server Name field in the captured TLS Client Hello packets. It should match payment-gateway.internal.svc.cluster.local.

Check for Hypothesis 3: Identify common cipher suites between order-service and payment-gateway. * List payment-gateway's supported cipher suites: nmap --script ssl-enum-ciphers -p 443 payment-gateway.internal.svc.cluster.local * *Expected output / interpretation*: A list of cipher suites supported by the server. Note preferred ones. * Inspect order-service's configured cipher suites (application-dependent, example for Java): grep -r "jdk.tls.client.cipherSuites" /path/to/java/config * *Expected output / interpretation*: Configuration files should list the cipher suites order-service is configured to offer. Compare this list with payment-gateway's supported ciphers to find common ground.

Likely Fixes:

Fix for Hypothesis 1: * Missing Root/Intermediate CA: Import the necessary CA certificate into order-service's trust store. * Expired Certificate: Renew payment-gateway's certificate. * Incorrect Chain Order: Ensure payment-gateway is serving the full chain in the correct order (leaf, intermediate, root).

Fix for Hypothesis 2: * `order-service` not sending SNI: Configure order-service to send the correct SNI (payment-gateway.internal.svc.cluster.local). This is often a client-side library configuration. * `payment-gateway` misconfigured for SNI: Adjust payment-gateway's server configuration to correctly handle the expected SNI, ensuring the corresponding certificate is served.

Fix for Hypothesis 3: * No Common Cipher Suite: Configure order-service or payment-gateway to support at least one common, secure cipher suite. Prioritize modern, strong ciphers.

Verification: * Restart order-service and payment-gateway (if configuration changes require it). * Monitor order-service logs for successful connection messages to payment-gateway. * Execute a test transaction or API call from order-service to payment-gateway to confirm end-to-end functionality. * Review payment-gateway access logs for successful TLS handshakes from order-service.

Under the hood

Why this prompt works

This prompt is effective because it employs several targeted prompt engineering techniques to structure a complex diagnostic task. Firstly, role priming establishes the model as a "seasoned network diagnostics expert specializing in secure communications." This instructs the model to adopt a knowledgeable, authoritative tone and to focus its reasoning on deep technical understanding of TLS.

Secondly, explicit constraints are crucial. The prompt specifies the exact focus areas (certificate chain, SNI, cipher suites) and forbids deviations. It also dictates a precise output structure: SymptomHypothesis ListChecks (with commands)Likely FixesVerification. This structured output ensures the generated content is directly usable as a playbook, preventing verbose explanations or irrelevant information. The requirement for "practical checks" with "actionable command-line examples" and "expected output / interpretation" forces the model to generate concrete, executable advice rather than abstract recommendations. Without these constraints, a one-liner prompt would likely produce a high-level list of potential issues without the necessary depth, actionable commands, or structured flow required for effective troubleshooting in a production environment. The detailed structure and explicit requirements guide the model to produce a comprehensive, ready-to-use diagnostic tool.

Model fit

Best AI models for this prompt

Claude

Claude models excel at maintaining conversational context and following complex, multi-step instructions like the playbook structure requested here. Their strong reasoning capabilities help in generating coherent diagnostic steps and commands that align with the given hypotheses. Claude's ability to handle extensive input and output ensures a thorough and detailed playbook. See the full Claude hub for deeper guidance.

ChatGPT

ChatGPT models are effective for this task due to their broad knowledge base across networking protocols and command-line tools. They can quickly generate a range of diagnostic commands and interpret potential error messages, which is critical for debugging. While generally proficient, ensuring precise command syntax might require minor validation. See the full ChatGPT hub for deeper guidance.

Gemini

Gemini models perform well in tasks requiring structured output and technical accuracy. Their strong code generation and understanding capabilities mean they can produce accurate command-line examples and interpret the implications of various network configurations. Gemini tends to be precise, which is valuable for diagnostic playbooks where exact steps are critical. See the full Gemini hub for deeper guidance.

When to use

  • When service logs show generic TLS handshake failed or SSL_ERROR_SYSCALL messages.
  • For issues where two internal services fail to establish mTLS, despite basic network connectivity.
  • When deploying new services or rotating certificates leads to connection rejections.
  • To diagnose intermittent connection drops on encrypted channels between microservices.
  • If services are migrating between environments with different Certificate Authorities (CAs).

When not to use

  • For general network connectivity issues like DNS resolution failures or firewall blocks, not directly TLS related.
  • When error messages explicitly indicate a protocol mismatch, such as an HTTP client connecting to an HTTPS port.
  • For debugging TLS issues with external, public-facing services where client configuration is beyond direct control.
  • If the services in question are not configured to use TLS or mTLS, or rely solely on self-signed certificates in non-production.

Get more from it

Pro tips

  • 1

    Always start by examining service logs on both the client and server; specific error codes often provide immediate direction.

  • 2

    Use `openssl s_client -debug -msg` to observe the TLS handshake in detail, revealing certificate exchange and SNI problems.

  • 3

    Verify system clock synchronization across all services; clock skew can invalidate certificates due to validity period issues.

  • 4

    Isolate the problem by testing connections from a known-good client using `curl --resolve` to simulate SNI conditions accurately.

  • 5

    When using `tcpdump`, filter by port and dissect the captured TLS traffic with Wireshark to analyze cipher suite negotiation frames.

  • 6

    Ensure all intermediate CA certificates are correctly chained and trusted by both the client and server's trust stores to avoid validation errors.

Don't ship this

Common mistakes

  • Neglecting to update the trust store on the client service after server certificate rotation or CA changes.

    Fix — Ensure the client's CA bundle or trust store includes the new issuing CA for proper server certificate validation.

  • Misconfiguring Server Name Indication (SNI), causing the server to present an incorrect default certificate.

    Fix — Explicitly set the `ServerName` in client requests or verify the `Host` header matches a certificate's SAN entry.

  • Assuming default cipher suites are universally compatible across different service versions or language runtimes.

    Fix — Manually specify a common, mutually supported cipher suite list on both the client and server configurations.

  • Overlooking hostname mismatches between the certificate's Common Name (CN)/SAN and the actual service endpoint.

    Fix — Verify the client connects to a hostname that is precisely listed within the server certificate's CN or Subject Alternative Names.

  • Not restarting services after updating certificate files, key files, or trust store configurations.

    Fix — Confirm services are properly reloaded or fully restarted to ensure they pick up the latest security configurations.

  • Using outdated or insecure TLS protocol versions that are no longer supported by peer services or security policies.

    Fix — Configure both services to negotiate a mutually agreed-upon, modern TLS protocol version, such as TLS 1.2 or 1.3.

People also ask

Frequently asked questions

Q.Can this playbook help with TLS issues involving external, internet-facing services?

This playbook primarily targets internal mTLS environments. While the openssl commands are universally applicable for TLS diagnostics, external B2B TLS often involves different network topologies, client control, and specific CDN/proxy configurations not directly covered here.

Q.How much prior TLS knowledge is assumed to effectively use this debugging guide?

It assumes a foundational understanding of networking, common openssl syntax, and core TLS concepts like Certificate Authorities (CAs), X.509 certificates, and mutual TLS. Familiarity with how your specific services handle TLS configuration is also beneficial for faster diagnosis.

Q.What if the service logs do not provide specific enough error messages for diagnosis?

If logs are vague, increase the logging verbosity for both services to DEBUG or TRACE levels. Supplement this with tcpdump to capture raw TLS handshake packets. Analyze these captures in Wireshark for detailed protocol-level insights into the failure.

Q.Will this playbook help identify basic network connectivity problems that aren't TLS-related?

No, this playbook is strictly scoped to TLS handshake failures. Before diving into these steps, ensure basic network connectivity using tools like ping, telnet, or netcat to verify the target service port is reachable and listening.

Q.Is this debugging approach specific to a particular cloud provider or operating system?

The playbook leverages standard Linux command-line tools and universal TLS concepts, making it cloud-agnostic. The diagnostic commands apply directly to any Linux-based environment, regardless of the underlying cloud infrastructure or specific distribution.

Q.How long should I expect a typical debugging session to last when using this playbook?

A straightforward issue with clear log messages might be resolved in 30-60 minutes. However, complex scenarios involving intermittent failures, distributed systems, or difficult-to-reproduce bugs can extend debugging efforts to several hours of iterative diagnosis and verification.

Version 1.0Last reviewed July 20, 2026
Reviewed by PromptInFlow Editorial Team