Module 5: Gitops With Argocd

6. Hands-on: syncing `andes-cargo-status-api` from Git

Description

This is the lesson where the three pieces — Gitea (lesson 3), ArgoCD (lesson 4), and the Application lesson 5 dissected — connect for the first time. You're going to apply a single kubectl apply, and you're going to watch ArgoCD take control of the nine manifests you've applied yourself, by hand, one by one, across four different modules until now. You're also going to run into a real problem, inherited from an honest decision in Module 4: metrics-server was never reinstalled after that module recreated the cluster, and that leaves ArgoCD reporting the whole Application as Degraded — not because of any error in this module, but because of a preexisting cluster condition ArgoCD correctly detects.

Connection to the module

This lesson is the whole module's first real convergence — the moment lesson 2's thermostat analogy stops being a promise and turns into a kubectl get application you can run yourself. Lesson 8 (final project) builds directly on this state: the same Application, already synced, receiving a real change.


Step 1 — The bootstrap problem: who applies the first Application

Before applying anything, it's worth resolving a question lesson 5 left open on purpose: if ArgoCD syncs everything Git declares, who syncs the Application itself, the first time? The answer is a standard GitOps pattern, not an improvisation of this lab: a cluster's first Application always gets applied by hand, once — it's the equivalent of turning the thermostat on for the first time, an act that, by definition, can't depend on the thermostat itself. After that first kubectl apply, the Application stays managed by Git like any other resource (in fact, you already saw this in lesson 3: application.yaml lives inside the same andes-cargo-k8s repository, alongside the other nine manifests) — future changes to that same file are going to flow through Git, with no additional kubectl apply.

              THE GitOps BOOTSTRAP, ONCE

  kubectl apply -f application.yaml   ← the only manual step
              │                          in this entire module
              ▼
     Application created on the cluster
              │
              ▼
     From here on, EVERY future change to application.yaml
     (or to any of the other nine manifests) flows
     through git push — never again a manual kubectl apply

Step 2 — Apply the Application

kubectl apply -f application.yaml

What to expect (literal, executed):

application.argoproj.io/andes-cargo-status-api created
kubectl get application -n argocd

What to expect (literal, executed — right after creating it, still unevaluated):

NAME                     SYNC STATUS   HEALTH STATUS
andes-cargo-status-api

Both columns are empty for an instant — ArgoCD just registered the object, but the first comparison cycle (lesson 2's loop) hasn't run even once yet.


Step 3 — The first result: Synced, but Degraded

After a few seconds, check again:

kubectl get application andes-cargo-status-api -n argocd -o jsonpath='{.status.sync.status} {.status.health.status}{"\n"}'

What to expect (literal, executed):

Synced Degraded

Synced is the good half: ArgoCD already applied the nine manifests — including confirming they already existed on the cluster since Modules 1-4, so in practice nothing changed, it just "took ownership" of resources that already matched Git. Degraded, on the other hand, deserves investigation:

argocd app get andes-cargo-status-api

What to expect (literal, executed — the commit hash 820515f is your variable value, whichever one corresponds to your own git push from lesson 3):

Name:               argocd/andes-cargo-status-api
Project:            default
Server:             https://kubernetes.default.svc
Namespace:          andes-cargo
URL:                https://localhost:8080/applications/andes-cargo-status-api
Source:
- Repo:             http://gitea-http.gitea.svc.cluster.local:3000/andes-cargo/andes-cargo-k8s.git
  Target:           main
  Path:             .
SyncWindow:         Sync Allowed
Sync Policy:        Automated (Prune)
Sync Status:        Synced to main (820515f)
Health Status:      Degraded

GROUP              KIND                     NAMESPACE    NAME                            STATUS   HEALTH    HOOK  MESSAGE
                   Namespace                andes-cargo  andes-cargo                     Running  Synced          namespace/andes-cargo configured
networking.k8s.io  NetworkPolicy            andes-cargo  default-deny-ingress            Synced                   networkpolicy.networking.k8s.io/default-deny-ingress configured
networking.k8s.io  NetworkPolicy            andes-cargo  allow-from-ingress-nginx        Synced                   networkpolicy.networking.k8s.io/allow-from-ingress-nginx configured
                   Secret                   andes-cargo  andes-cargo-status-api-secrets  Synced                   secret/andes-cargo-status-api-secrets configured
                   ConfigMap                andes-cargo  andes-cargo-status-api-config   Synced                   configmap/andes-cargo-status-api-config configured
                   Service                  andes-cargo  status-api-service              Synced   Healthy         service/status-api-service configured
apps               Deployment               andes-cargo  andes-cargo-status-api          Synced   Healthy         deployment.apps/andes-cargo-status-api configured
autoscaling        HorizontalPodAutoscaler  andes-cargo  andes-cargo-status-api-hpa      Synced   Degraded        horizontalpodautoscaler.autoscaling/andes-cargo-status-api-hpa configured
networking.k8s.io  Ingress                  andes-cargo  status-api-ingress              Synced   Healthy         ingress.networking.k8s.io/status-api-ingress configured
argoproj.io        Application              argocd       andes-cargo-status-api          Synced                   application.argoproj.io/andes-cargo-status-api configured
                   Namespace                             andes-cargo                     Synced

There's the exact source: HorizontalPodAutoscaler andes-cargo-status-api-hpa is the only resource with HEALTH: Degraded. Everything else — Deployment, Service, Ingress — is Healthy. Application as a whole inherits the worst health state of any of its resources, so one single degraded HPA is enough for the whole Application to get reported as Degraded, even though andes-cargo-status-api is serving traffic perfectly.


Step 4 — The diagnosis: the same cause you already saw in Module 4

kubectl describe hpa andes-cargo-status-api-hpa -n andes-cargo

What to expect (literal, executed — the relevant fragment):

Conditions:
  Type           Status  Reason                   Message
  ----           ------  ------                   -------
  AbleToScale    True    SucceededGetScale        the HPA controller was able to get the target's current scale
  ScalingActive  False   FailedGetResourceMetric  the HPA was unable to compute the replica count: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)

This is the same finding Module 4's project already documented, with an explicit note: "HorizontalPodAutoscaler shows cpu: <unknown>/50% because this cluster got recreated in lesson 4 and metrics-server (Module 3, lesson 7) hasn't been reinstalled yet". ArgoCD didn't invent any new problem — it's simply the first tool in this lab that translates that preexisting condition into a health state visible at the whole-application level, instead of leaving it hidden inside a kubectl describe hpa no one runs unless they already suspect something.

Fix it by installing metrics-server, exactly as Module 3 already explained:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
kubectl patch deployment metrics-server -n kube-system --type='json' \
  -p='[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--kubelet-insecure-tls"}]'
kubectl rollout status deployment/metrics-server -n kube-system --timeout=120s

What to expect (literal, executed — the --kubelet-insecure-tls is the same gotcha Module 3, lesson 7 already documented, because of kind's self-signed certificates):

serviceaccount/metrics-server created
clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created
clusterrole.rbac.authorization.k8s.io/system:metrics-server created
rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created
clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
service/metrics-server created
deployment.apps/metrics-server created
apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created
deployment.apps/metrics-server patched
Waiting for deployment "metrics-server" rollout to finish: 1 old replicas are pending termination...
deployment "metrics-server" successfully rolled out

Wait about twenty seconds — metrics-server needs a couple of collection cycles before it reports real values — and confirm:

kubectl get hpa -n andes-cargo
kubectl get application andes-cargo-status-api -n argocd -o jsonpath='{.status.sync.status} {.status.health.status}{"\n"}'

What to expect (literal, executed):

NAME                         REFERENCE                           TARGETS       MINPODS   MAXPODS   REPLICAS   AGE
andes-cargo-status-api-hpa   Deployment/andes-cargo-status-api   cpu: 1%/50%   2         6         3          76m

Synced Healthy

cpu: 1%/50%, a real number instead of <unknown> — and the whole Application goes from Degraded to Healthy, with no one touching any Git manifest. ArgoCD reevaluates each resource's health on every comparison cycle, completely independent of whether it synced anything new or not.


Step 5 — Confirm the complete state, with both tools

argocd app list

What to expect (literal, executed):

NAME                           CLUSTER                         NAMESPACE    PROJECT  STATUS  HEALTH   SYNCPOLICY  CONDITIONS  REPO                                                                            PATH  TARGET
argocd/andes-cargo-status-api  https://kubernetes.default.svc  andes-cargo  default  Synced  Healthy  Auto-Prune  <none>      http://gitea-http.gitea.svc.cluster.local:3000/andes-cargo/andes-cargo-k8s.git  .     main
kubectl get all -n andes-cargo

What to expect (literal, executed — Pod names with a hash suffix and AGE are your variable values; the rest, including the three-replica pattern, is literal at this point of the module):

NAME                                          READY   STATUS    RESTARTS   AGE
pod/andes-cargo-status-api-548966dd97-bpqjq   1/1     Running   0          76m
pod/andes-cargo-status-api-548966dd97-ls2bc   1/1     Running   0          76m
pod/andes-cargo-status-api-548966dd97-z7bpd   1/1     Running   0          41s

NAME                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/status-api-service   ClusterIP   10.96.239.125   <none>        80/TCP    79m

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/andes-cargo-status-api   3/3     3            3           79m

NAME                                                DESIRED   CURRENT   READY   AGE
replicaset.apps/andes-cargo-status-api-548966dd97   3         3         3       79m

NAME                                                             REFERENCE                           TARGETS       MINPODS   MAXPODS   REPLICAS   AGE
horizontalpodautoscaler.autoscaling/andes-cargo-status-api-hpa   Deployment/andes-cargo-status-api   cpu: 1%/50%   2         6         3          79m

argocd app list (ArgoCD's view) and kubectl get all (Kubernetes' native view) describe exactly the same state, from two different angles — ArgoCD never replaces the Kubernetes API, it just adds a layer of continuous comparison on top of it. This is the first time, in this entire guide, that the andes-cargo namespace's state reflects a Git repository's content, not the history of commands you ran yourself.


Common mistakes

Getting alarmed at Degraded without reading which specific resource causes it (a flow mistake, the most likely one in this lesson). What happens: someone sees Health Status: Degraded in the overall output and assumes something broke badly, without checking the resource table below it. How to spot it: if your first reaction was to look for which manifest is misconfigured, instead of checking argocd app get's HEALTH column. How to fix it: the resource table always points to exactly which object causes the degraded state — in this lesson, a single HorizontalPodAutoscaler, with a cause already known since Module 4. Always check that table before assuming a new problem.

Applying application.yaml a second time thinking "it didn't sync" (impatience, the same pattern from lesson 2's polling-latency point). What happens: someone, seeing SYNC STATUS still empty right after Step 2, runs kubectl apply -f application.yaml again. How to spot it: if you didn't wait at least a few seconds before repeating the command. How to fix it: a freshly created Application's first comparison cycle takes a moment to run — repeating kubectl apply doesn't speed anything up (the object already exists, Kubernetes just confirms nothing changed) and can create confusion about whether the Application got created once or twice. Wait for Step 3 before investigating.

Forgetting metrics-server is lab infrastructure, not part of the andes-cargo-k8s repository (scope, matters for the rest of this guide). What happens: someone looks for metrics-server inside andes-cargo-k8s and doesn't find it, or wonders why ArgoCD didn't install it on its own. How to spot it: if you expected to see a metrics-server manifest in the Gitea repository. How to fix it: metrics-server lives in kube-system, it's infrastructure of the cluster itself (like ingress-nginx or local-path-storage), not an Andes Cargo business resource — this module's Application only manages what lives in andes-cargo, explicitly defined by destination.namespace in lesson 5.


Exercises

Exercise 1 — Explain why Degraded doesn't mean "ArgoCD found an error in your YAML." In two or three sentences, explain to a colleague the difference between Sync Status: Synced (which you did confirm in this lesson) and Health Status: Degraded (which you also confirmed) — why can both be true at the same time?

See solution

A reasonable explanation: "Sync Status answers 'does the cluster have exactly what Git declares?' — and the answer was yes, ArgoCD applied the nine manifests with no syntax or permission error at all. Health Status answers a different question: 'are the resources that already exist working well?' — and there the answer was no, because the HorizontalPodAutoscaler couldn't read CPU metrics, due to a cluster condition (missing metrics-server) that has nothing to do with whether the YAML was well-formed. An Application can be perfectly synced with Git and, at the same time, degraded in health, because they're two different questions."

Exercise 2 — Diagnose without argocd app get. If you only had kubectl available (without the argocd CLI), which command would you use to reach Step 4's same diagnosis — that the HorizontalPodAutoscaler is the cause of the Degraded state?

See solution

kubectl get hpa -n andes-cargo would show cpu: <unknown>/50% instead of a real percentage — the same signal you already saw in Module 4 — and kubectl describe hpa andes-cargo-status-api-hpa -n andes-cargo would show the ScalingActive: False condition with the FailedGetResourceMetric message, exactly the same text Step 4 captured. argocd app get doesn't discover anything kubectl couldn't show on its own — what it adds is the consolidated view, which gathers the nine resources' state in one place, without you having to check each one separately.

Exercise 3 — Predict what would happen if you deleted andes-cargo-status-api-hpa by hand right now. With selfHeal: true active (lesson 5) and the Application already synced (this lesson), if you ran kubectl delete hpa andes-cargo-status-api-hpa -n andes-cargo, what would you expect to see seconds later?

See solution

The HorizontalPodAutoscaler would reappear on its own, recreated by ArgoCD — the same mechanism lesson 5 demonstrated with kubectl scale, applied here to a full deletion instead of a field change. selfHeal: true doesn't distinguish between "someone changed a value" and "someone deleted the whole resource": in both cases, the cluster's real state stopped matching what hpa.yaml declares in Git, and ArgoCD fixes the difference on its next comparison cycle, with no one running kubectl apply or git push.


Summary and next step

This lesson applied this entire module's one manual step — kubectl apply -f application.yaml, the necessary bootstrap no GitOps setup can avoid on its own — and confirmed, with literal evidence, the first real convergence: nine manifests, managed from that moment on by ArgoCD instead of by your command history. You also found, diagnosed, and fixed a real problem (missing metrics-server, inherited from an honest decision in Module 4), which left the Application Degraded until you resolved it — the first time, in this guide, that a GitOps tool translates a cluster condition into a health state visible at the whole-application level.

Before moving on you should be able to: explain why a cluster's first Application always gets applied by hand; distinguish Sync Status from Health Status with your own example; and diagnose, with argocd app get or with just kubectl, which specific resource causes a Degraded state.

Next lesson: deployment strategies. There you learn, with technical precision, how Kubernetes replaces Pods when the Deployment changes — RollingUpdate, already configured in deployment.yaml since Module 2 — versus what a cluster would need to add for blue/green or canary, before the module's final project.

Resources

  1. Argo CD — Automated Sync Policy — the automated sync mechanism this lesson runs for the first time.
  2. Argo CD — Health — official documentation on how ArgoCD calculates health status for each resource type, including HorizontalPodAutoscaler.
  3. Kubernetes — Horizontal Pod Autoscaling — reference for the mechanism behind Step 4's diagnosis.
  4. kubernetes-and-eks-in-production-guide (NIEVA), Module 4, lesson 8 — the original note about metrics-server and the cluster recreation this lesson picks back up with new evidence.