3. Troubleshooting the network from the terminal: ping, dig, curl, and ss

Description

By the end of this lesson you will be able to diagnose, layer by layer, why a network connection fails — does the name not resolve?, is there no reachability?, does the port not respond?, is the response wrong? — and you will be able to pick the right tool for each layer: dig for resolution, ping and traceroute for reachability (with their real limits), ss, netstat, and lsof -i for local ports, and curl -I, -v, and -w to inspect the full HTTP response.

This is exactly what an engineer does when an alert goes off at three in the morning and the message says "the payments endpoint is not responding." That sentence says nothing about the cause. It could be that nobody can resolve the domain, that the datacenter is unreachable, that the process crashed and the port is no longer listening, or that the service responds but with a 500 error. Each cause points to a different team — networking, infrastructure, the application itself — and whoever cannot rule out layers ends up escalating to the wrong person or, worse, restarting services at random until something "works" without knowing why.

Connection to the module: the previous lesson gave you the vocabulary — IP, ports, DNS, what it means for a port to "listen." This lesson gives you the tools to watch that vocabulary in action and decide, with evidence, which layer the problem is in.

The method: rule out layer by layer, do not try commands at random

An electrician who cannot find why a lamp will not turn on does not start swapping wires at random. They follow the circuit from the panel: does power reach the panel?, does it reach the switch?, does it reach the lamp box?, does the bulb itself work? At every point they measure with a tester and only move forward once they confirm that part of the circuit is fine. If they measure power at the lamp box but the bulb will not turn on, they already know the problem is the bulb, not the wiring — they save themselves from opening walls that had nothing to do with it.

Diagnosing a network connection is the same exercise, with four measurement points instead of an electrical circuit:

1. Does the name resolve? The domain you typed — does it translate into an IP address? This is pure DNS, without touching the network toward the destination yet.
2. Is there reachability? With that IP in hand, can your machine reach that network? This is host-level reachability, regardless of what service runs there yet.
3. Does the port respond? You reached the host, but is anything listening on the specific port you care about? A host can be perfectly alive with the port you are after closed.
4. Is the response correct? The port accepts the connection, but does the service that responds do what it is supposed to, with the expected status code and body?

Each layer has its own tool, and — this is the part learned through practice, not theory — each layer can fail in a way different from what a surface-level glance suggests. A ping that gets no response does not prove the host is down. A curl -I that returns a 301 does not prove the site is broken. The method keeps you from drawing the wrong conclusion by stopping at the first symptom.

Worked example

Suppose you need to open a connection against internal-api.example.com, port 8443, and the request just hangs. (We use example.com and addresses from the 203.0.113.0/24 block because they are addresses reserved for documentation by RFC 5737 — they genuinely resolve, but never point to a real service, so you can copy these commands without worrying about hitting something that belongs to someone else.)

Step 1 — does the name resolve?

dig internal-api.example.com +short

What to expect:

203.0.113.24

A single line with the IP. If instead it returns nothing, the problem is already solved: it is DNS, and it is not even worth trying to connect yet.

Step 2 — is there reachability?

ping -c 3 internal-api.example.com

What to expect:

PING internal-api.example.com (203.0.113.24): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2

--- internal-api.example.com ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

This is where most people stop and conclude "it is down." It is the most common mistake in this lesson — we will see below why that is a rushed conclusion. Let us move to the next measurement point before drawing conclusions.

Step 3 — does the port respond?

curl -v --connect-timeout 5 https://internal-api.example.com:8443/health

What to expect:

*   Trying 203.0.113.24:8443...
* Connected to internal-api.example.com (203.0.113.24) port 8443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN: server accepted h2
> GET /health HTTP/2
> Host: internal-api.example.com:8443
>
< HTTP/2 200
< content-type: application/json
<
{"status":"ok"}

The key line is * Connected to internal-api.example.com (203.0.113.24) port 8443. The port does respond, the TLS negotiation completes, and the server answers with a 200. The correct diagnosis is not "the host is down": it is that this host has ICMP (what ping uses) blocked, but port 8443 (what your actual application uses) is perfectly alive.

Step 4 — is the response correct?

You already saw it respond with 200 and a valid body in the previous step. If instead the request takes a long time, the next command tells you exactly which stage the time is going into:

curl -o /dev/null -s -w "dns:%{time_namelookup}s connect:%{time_connect}s ttfb:%{time_starttransfer}s total:%{time_total}s\n" https://internal-api.example.com:8443/health

What to expect:

dns:0.045s connect:0.089s ttfb:0.312s total:0.315s

Low time_namelookup and time_connect, but a high time_starttransfer (time until the first byte of the response) points to the server processing slowly — not the network. If instead time_connect were the large number, the problem would be in the network path, not the service.

With those four measurements you now have evidence, not suspicion: DNS resolves, the port responds, TLS negotiates, the service answers 200 in 315 milliseconds. The failed ping was never the real symptom.

Going deeper: dig and nslookup — looking more closely at resolution

dig +short gives you the quick answer, but dig without +short shows you everything behind that IP:

dig internal-api.example.com

In the full output, the section that matters is ANSWER SECTION:

;; ANSWER SECTION:
internal-api.example.com. 300 IN A 203.0.113.24

That row tells you four things in order: the name queried, the TTL (300 seconds — how long resolvers can cache this answer before asking again), the class (IN, internet), and the record type (A, an IPv4 address). If you expected a CNAME and instead see a direct A, or if the TTL looks suspiciously high after someone "already changed the DNS," there is the explanation.

nslookup does basically the same thing and is still common, especially on Windows, but its output is less practical to read in scripts and it no longer receives active development — if you have dig available, use it first.

Going deeper: ping and its real limits

ping measures exactly one thing: whether the host responds to ICMP Echo packets. That is useful, but it is a different measurement from "does the service I care about work?", and the difference matters in production for a concrete reason: it is a widespread practice among firewalls, load balancers, and cloud security groups to deliberately block ICMP as a security measure, without that affecting the application's actual TCP traffic. A server can have ICMP closed on purpose and, at the same time, serve HTTP traffic without any problem.

That is why the order of the method matters: ping is useful as a quick first signal when it DOES respond (it confirms reachability unambiguously), but a ping that fails is not evidence of anything on its own — it only tells you that you need to move to the next layer with a tool that speaks the protocol you actually care about (curl, nc, or your application's real client).

Going deeper: curl -I, -v, and -w — headers, negotiation, and timing

Three different modes of curl, for three different questions:

curl -I makes a HEAD request and shows only the response headers, with no body. It is good for a quick check of status and content type without downloading anything:

curl -I https://internal-api.example.com:8443/health

HTTP/1.1 200 OK
content-type: application/json
content-length: 16

curl -v (verbose) shows the whole process: resolution, TCP connection, TLS negotiation, and the request and response headers with the prefixes > (what curl sends) and < (what the server responds). It is the tool to reach for when -I is not enough because you need to see exactly which step the connection breaks at — TCP, TLS, or the application.

curl -w (write-out) lets you ask for specific metrics in whatever format you want, using variables like %{time_namelookup}, %{time_connect}, %{time_appconnect} (end of the TLS handshake), %{time_starttransfer}, and %{time_total}. It is the difference between "the request took 2 seconds" and "the request took 2 seconds because DNS took 1.8 of those 2" — two completely different diagnoses that point to completely different fixes.

A detail that trips people up often: if a site redirects (301/302), curl -I without -L shows you that code and stops there. That is not a site error — it is a perfectly valid HTTP response saying "the resource moved." The location: header tells you where. Adding -L makes curl follow the redirect automatically.

Going deeper: ss, netstat, and lsof -i — who is listening on your own machine

These three tools answer the question from the local side: which process has which port open on this machine?

ss -tulpn (Linux) is the fastest and the one most modern distributions use, because it comes from the iproute2 package that replaced the older net-tools tools:

sudo ss -tulpn

Netid State  Recv-Q Send-Q Local Address:Port   Peer Address:Port  Process
tcp   LISTEN 0      128    0.0.0.0:8443         0.0.0.0:*          users:(("api-server",pid=4821,fd=6))
tcp   LISTEN 0      128    127.0.0.1:5432       0.0.0.0:*          users:(("postgres",pid=1190,fd=7))

Every letter in the flag filters something: -t TCP only, -u UDP only, -l only sockets in LISTEN state (the ones accepting new connections, not already-established ones), -p shows the process that owns the socket, -n keeps ss from trying to resolve ports to service names, so it responds faster. The Local Address:Port column is the one you already know from the previous lesson: 0.0.0.0:8443 listens on every interface, 127.0.0.1:5432 only accepts connections from the machine itself — postgres, for instance, should almost never listen on 0.0.0.0.

netstat -tulpn does the same thing on Linux and still shows up in older documentation, but it is in maintenance mode compared to ss. On macOS, netstat does not accept -p to show the owning process, so there the equivalent tool is lsof -i:

sudo lsof -i :8443

COMMAND    PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
api-serv  4821 mikeni    6u  IPv4  0x1a2      0t0  TCP *:8443 (LISTEN)

lsof -i lists open files that are network sockets (-i, for "internet"); filtering with :8443 limits the output to that exact port. It works the same on Linux if you prefer its syntax over ss. You almost always need it with sudo: without privileges, the kernel will not let you see which process owns sockets that are not yours.

Going deeper: traceroute — locating where the path breaks

Once you already know there is a real reachability problem (not just ICMP blocked at the final destination), traceroute shows you every network hop between your machine and the destination, with the round-trip time for each one:

traceroute internal-api.example.com

traceroute to internal-api.example.com (203.0.113.24), 64 hops max
 1  192.168.1.1        2.104 ms  1.876 ms  1.654 ms
 2  10.20.0.1          8.221 ms  7.998 ms  8.045 ms
 3  * * *
 4  * * *
 5  203.0.113.1       24.331 ms 23.876 ms 24.001 ms
 6  203.0.113.24      25.012 ms 24.876 ms 24.998 ms

The * * * at hops 3 and 4 do not, by themselves, mean the path breaks there: they only mean that particular router did not respond to the probe packet, something common when a router is configured to not respond, or to silently drop that kind of traffic. The proof that the path is still alive is at hops 5 and 6, which do respond — the packet went through hops 3 and 4 just fine, those routers simply did not announce that they did. The real sign of a break is when the * * * continue all the way to the end with no later hop ever responding.

A platform detail worth knowing: on Linux and macOS, traceroute sends UDP packets to high ports by default; on Windows, tracert sends ICMP. If a corporate firewall allows one protocol and blocks the other, the same network path can look broken on one operating system and healthy on another. The -I option forces traceroute to use ICMP just like ping, useful when you want to compare apples to apples against a Windows tracert.

Honesty: what you cannot diagnose from your own machine

Everything above gives you evidence about the network as far as your machine can observe. There are things none of these tools will show you:

• What happens inside the server. If the port responds 200 but with incorrect data, the cause is in that application's logs or its database — not the network. Your terminal sees the response, not the reason for the response.
• Firewall or WAF rules that only block your IP or your specific traffic pattern. A coworker on a different network can succeed with the exact same command that fails for you, and the difference can be a rule you will never see from your side.
• Whether "the other side" already knows something is wrong. Sometimes the correct, faster diagnosis is to message the team that owns the service with the evidence you already gathered (dig resolved, the port does not respond from anywhere) instead of continuing to try new commands.

Knowing how to say "I cannot see this from here anymore, I need to ask the other side" with evidence in hand — not as an excuse — is part of the method, not a failure of it.

Common mistakes

1. Concluding the server is "down" because ping does not respond. What happens: as you saw in step 2 of the example, many production hosts block ICMP on purpose without that affecting the application's real traffic. Why it happens: ping feels like "the proof of life" because it is the best-known command, and we confuse "does not respond to ICMP" with "does not respond to anything." How to spot it: if you suspect this is the case, test the real port with curl -v or nc before escalating. How to fix it: never declare a service down based only on ping — use the real protocol (HTTP, the database, whatever it is) to confirm.

2. Reading a 301/302 from curl -I as if it were an error. What happens: the student sees a code that is not 200 and assumes the site is broken. Why it happens: they lack the habit of reading the status code as information, not as a binary "works / does not work" verdict — a 3xx is a perfectly valid HTTP response that says "the resource is somewhere else." How to spot it: check the location: header in the same output; if it points to a valid URL, the server is working fine. How to fix it: add -L to follow the redirect, or treat the 3xx as the real destination of your diagnosis, not as a failure.

3. Running ss -tulpn without privileges and concluding nothing is listening on that port. What happens: without sudo, the process column (users:(...)) can show up empty for sockets that are not yours, even though the socket is indeed in LISTEN. Why it happens: the kernel hides the socket's owner from users without enough privilege, but still shows the row with the port. How to spot it: if the port shows up in LISTEN but the process column is empty or says -, that is a permissions clue, not an absence-of-process clue. How to fix it: repeat the command with sudo ss -tulpn (or sudo lsof -i :port) before concluding the port is free.

Exercises

Exercise 1. A coworker tells you: "I tried ping-ing payments.internal and got no response, so the payments service must be down." Write, in order, the sequence of commands (with their flags) you would run to confirm or rule out that conclusion before escalating the alert.

dig payments.internal +short
curl -v --connect-timeout 5 https://payments.internal:PORT/health-path
curl -I https://payments.internal:PORT/health-path

Exercise 2. You have this traceroute output:

 1  192.168.1.1     1.2 ms  1.1 ms  1.0 ms
 2  10.0.0.1        4.5 ms  4.3 ms  4.1 ms
 3  * * *
 4  * * *
 5  * * *
 6  * * *

and the traceroute stops there (it hit the maximum 64 hops with no hop after 2 ever responding, and the final connection never completes either). Is this result evidence of a real break in the path, or just of a router that does not respond to probes? Justify your answer.

Exercise 3. You ran this command and got this output:

curl -o /dev/null -s -w "dns:%{time_namelookup}s connect:%{time_connect}s ttfb:%{time_starttransfer}s total:%{time_total}s\n" https://api.example.com/reports

dns:0.041s connect:0.098s ttfb:3.812s total:3.815s

Which layer is the bottleneck in, and which team (network or application) would you take this data to?

Exercise 4. On your own machine, bring up a simple server (for example python3 -m http.server 8000) and, without stopping it, use ss -tulpn (Linux) or sudo lsof -i :8000 (macOS/Linux) to confirm it is listening. Note which address it shows bound to (Local Address) and explain what it would mean if, instead of 127.0.0.1:8000, you saw 0.0.0.0:8000.

Summary and next step

This lesson's method does not change: resolution, reachability, port, response — in that order, with a different tool for each layer, and without confusing "this tool did not respond" with "the service is down." ping measures ICMP, not your application. dig measures DNS, not the network toward the destination. ss/lsof -i look at your own machine, not the other side's. And there is always a point where the evidence stops being on your side and it is time to ask the other end, with that evidence in hand.

Before moving on you should be able to: explain, without looking at any service's code, which layer a connection broke at with just two or three commands; explain why a failed ping is not enough as a diagnosis on its own; read a curl -w output and say whether the problem is network or application; and find which process is listening on a given port on your own machine.

All of this measured whether a network channel exists and responds. The next lesson takes the natural next step: opening an encrypted, authenticated channel over that same network path with SSH, so that instead of only diagnosing a remote machine, you can work inside it.

Resources

• curl manual — full reference of flags, including every -w format variable.
• dig — BIND 9 Administrator Reference Manual — official dig syntax and its output sections.
• ss(8) — Linux manual page — every ss -tulpn flag documented by the Linux man-pages project.
• netstat(8) — Linux manual page — for comparing against ss and understanding why it is considered in maintenance mode.
• traceroute(8) — Debian manpages — default protocol and the -I option to force ICMP.
• RFC 5737 — IPv4 Address Blocks Reserved for Documentation — why 203.0.113.0/24 is safe to use in examples.