스카이워킹 배경 및 개요

분산 시스템과 마이크로서비스 아키텍처에서 요청의 흐름을 추적하는 것은 필수적입니다. 이 과정은 APM(Application Performance Management)이라고 불리며, 트래픽이 프론트엔드에서 시작하여 데이터베이스까지 도달하는 모든 단계를 기록하고 가시화합니다. 이를 통해 의존성 관계나 성능 분석 등 다양한 정보를 제공하며, 문제 해결에 매우 유용합니다.

사용한 환경

• Kubernetes v1.22.5 클러스터

• master: 192.168.10.100

• node01: 192.168.10.101

• node02: 192.168.10.102

• Elasticsearch v7.12.0

• SkyWalking 8.4.0

구성 요소:

• skywalking-oap-server: 백엔드 서비스
• skywalking-ui: 프론트엔드 UI
• skywalking-es-init: Elasticsearch 데이터 초기화
• elasticsearch: SkyWalking 메트릭 데이터 저장

NFS 설정을 통한 Kubernetes 스토리지 준비

네임스페이스 생성

kubectl create ns efk

NFS 서버 구성

모든 노드에서 다음 명령어 실행:

yum -y install nfs-utils rpcbind
mkdir -p /nfsdata
chmod 777 -R /nfsdata
echo "/nfsdata 192.168.10.0/24(rw,no_root_squash,sync)" > /etc/exports
systemctl start rpcbind && systemctl enable rpcbind
systemctl start nfs-server && systemctl enable nfs-server
exportfs -rv
showmount -e 192.168.10.100

StorageClass 설정

rbac.yaml 파일 작성:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io

provisioner-deployment.yaml 작성:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: quay.io/external_storage/nfs-client-provisioner:latest
          env:
            - name: PROVISIONER_NAME
              value: wuchang-nfs-storage
            - name: NFS_SERVER
              value: "192.168.10.100"
            - name: NFS_PATH
              value: "/nfsdata"

storageclass.yaml 작성:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: wuchang-nfs-storage
allowVolumeExpansion: true
parameters:
  archiveOnDelete: "false"

Elasticsearch 설치

es-pvc.yaml 작성:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: es-pvc
  namespace: default
spec:
  storageClassName: "managed-nfs-storage"
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 10Gi

es-svc.yaml 작성:

apiVersion: v1
kind: Service
metadata:
  name: elasticsearch-single
  namespace: default
spec:
  ports:
    - port: 9200
      protocol: TCP
      targetPort: 9200
  selector:
    k8s-app: elasticsearch-single

elasticsearch-single.yaml 작성:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: elasticsearch-single
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: elasticsearch-single
  template:
    metadata:
      labels:
        k8s-app: elasticsearch-single
    spec:
      containers:
        - image: elasticsearch:7.12.0
          name: elasticsearch-single
          resources:
            limits:
              cpu: 2
              memory: 3Gi
            requests:
              cpu: 0.5
              memory: 500Mi
          env:
            - name: discovery.type
              value: "single-node"
            - name: ES_JAVA_OPTS
              value: "-Xms512m -Xmx2g"
          ports:
            - containerPort: 9200
              name: db
              protocol: TCP
          volumeMounts:
            - name: elasticsearch-data
              mountPath: /usr/share/elasticsearch/data
      volumes:
        - name: elasticsearch-data
          persistentVolumeClaim:
            claimName: es-pvc

SkyWalking 설치

Helm 설치

curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash

SkyWalking Charts 준비

git clone https://github.com/apache/skywalking-kubernetes.git
cd skywalking-kubernetes/chart
helm repo add elastic https://helm.elastic.co
helm dep up skywalking

Namespace 생성

kubectl create namespace skywalking

Values 수정

외부 Elasticsearch 사용 설정:

oap:
  image:
    tag: 8.4.0-es7
  storageType: elasticsearch7

ui:
  image:
    tag: 8.4.0
  service:
    type: NodePort
    externalPort: 80
    internalPort: 8080
    nodePort: 30008

elasticsearch:
  enabled: false
  config:
    host: elasticsearch-single.default
    port:
      http: 9200

Helm으로 배포

helm install skywalking ./skywalking -n skywalking -f values.yaml

Pod 상태 확인

kubectl get pod -n skywalking -w

접속 URL 확인

NODE_PORT=$(kubectl get --namespace skywalking -o jsonpath="{.spec.ports[0].nodePort}" services skywalking-ui)
NODE_IP=$(kubectl get nodes --namespace skywalking -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT