K3S@Home | XuLog

type

status

date

slug

summary

VM Cluster

在PVE机器上创建三台虚拟机。如果用CT，k3s启动会有一点问题，新手为了避免不必要的麻烦，还是安装VM，首先需要的就是VM Template，不然每次都重新安装系统，也太麻烦了。

Cloud-Init is the de facto multi-distribution package that handles early initialization of a virtual machine instance. Using Cloud-Init, configuration of network devices and ssh keys on the hypervisor side is possible. When the VM starts for the first time, the Cloud-Init software inside the VM will apply those settings.

https://pve.proxmox.com/wiki/Cloud-Init_Support

文中需要的Cloud Image可以从系统官网下载，比如

Ubuntu Cloud Images

Canonical and Amazon have collaborated on the launch of Amazon's Elastic Container Service for Kubernetes (EKS) to make Ubuntu worker nodes available. Amazon EKS is a fully managed service that makes it easy for you to use Kubernetes on AWS without having to be an expert in managing Kubernetes clusters.

https://cloud-images.ubuntu.com

k3s

安装官网就是一句话

但是最好不要照着广告走，还是得照着文档走。虽然k3s可以用其他runtime，但为了避免麻烦，还是安装&使用docker。

安装Docker

Install Docker Engine on Ubuntu

Estimated reading time: 12 minutes To get started with Docker Engine on Ubuntu, make sure you meet the prerequisites, then install Docker. To install Docker Engine, you need the 64-bit version of one of these Ubuntu versions: Ubuntu Hirsute 21.04 Ubuntu Groovy 20.10 Ubuntu Focal 20.04 (LTS) Ubuntu Bionic 18.04 (LTS) Docker Engine is supported on x86_64 (or amd64), armhf, and arm64 architectures.

https://docs.docker.com/engine/install/ubuntu/

安装k3s

接下来就是安装k3s

K3s Server Configuration Reference

In this section, you'll learn how to configure the K3s server. Throughout the K3s documentation, you will see some options that can be passed in as both command flags and environment variables. For help with passing in options, refer to How to Use Flags and Environment Variables.

https://rancher.com/docs/k3s/latest/en/installation/install-options/server-config/

K3s Agent Configuration Reference

In this section, you'll learn how to configure the K3s agent. Throughout the K3s documentation, you will see some options that can be passed in as both command flags and environment variables. For help with passing in options, refer to How to Use Flags and Environment Variables.

https://rancher.com/docs/k3s/latest/en/installation/install-options/agent-config/

这个时候，服务端的kubectl 就可以正常使用了，文档后面是安装dashboard，那个dashboard对新手来说，没用。先不装。

客户端登陆

在自己电脑使用kubectl 显然是比每次都要ssh到服务器上更方便，在自己电脑上装好kubectl ，然后把k3s server上的配置文件下载到~/.kube/config

本机的kubectl就可以正常使用了

第一个应用

就安装httpbin吧，创建一个资源yaml，一般这个时候，会用helm，但是新手啥都不会，先从k8s yaml开始吧。

我们先创建一个namespace，目的只是为了保证演示和效果一致。

部署这个deployment

可以看到，正常运行了，但是我们暂时没有办法访问。因为它并没有暴露出来，我们现在创建一个service来暴露它。

部署这个service

显然，通过IP和端口来访问，显然不是我们想要的，我们来给他分配一个域名。

再次测试

配置Let's Encrypt

现在已经能够通过http来访问k8s上的服务了，但是如果没有https，在很多场景下并不方便，比如搭建docker私服，没有https需要额外配置。给k8s ingress加上证书，需要添加一个cer-manager

Kubectl apply

Installing with regular manifests Prerequisites kubectl version >= v1.19.0-rc.1 (otherwise, you will have issues updating the CRDs. see v0.16 upgrade notes) A Kubernetes or OpenShift cluster running a supported version cert-manager not already installed on the cluster Prerequisites specific to your cloud provider Steps All resources (the CustomResourceDefinitions and the cert-manager, cainjector and webhook components) are included in a single YAML manifest file: Install all cert-manager components: $ kubectl apply -f https://github.

https://cert-manager.io/docs/installation/kubectl/

安装完成之后，会多一个cert-manager的namespace，可以不用理会，现在要做的就是配置好cert manager

我们需要创建一个ClusterIssuer，相较于Issuer，适用范围更广。

我们得先配置好cf的相关信息

Cloudflare

To use Cloudflare, you may use one of two types of tokens. API Tokens allow application-scoped keys bound to specific zones and permissions, while API Keys are globally-scoped keys that carry the same permissions as your account. API Tokens are recommended for higher security, since they have more restrictive permissions and are more easily revocable.

https://cert-manager.io/docs/configuration/acme/dns01/cloudflare/

接下来，我们就要使用这个issuer，修改上面的httpbin的ingress.yaml

再次部署测试

https就正常工作了！

搭建nexus repository manager

这里以nexus repository manager作为例子是有下面几个原因。

这个需要持久化

可能需要多端口

需要https

Volume

Volumes

On-disk files in a container are ephemeral, which presents some problems for non-trivial applications when running in containers. One problem is the loss of files when a container crashes. The kubelet restarts the container but with a clean state. A second problem occurs when sharing files between containers running together in a Pod.

https://kubernetes.io/docs/concepts/storage/volumes/

由于我还有一台nas，加上服务对io要求应该不高，所以我选择了nfs挂载，如果要使用nfs挂载，就需要cluster每台机器都装了nfs相关的组件

多端口

上面提到的多端口，docker 对于 registry 是不带path的，但是oss是带path的，所以需要在oss的repository上开启独立的端口来供docker使用，而且由于oss的free版本是不支持push到grouped repository的，所以我可能需要两个额外的端口，一个用于self host，放一些制成品，一个用户proxy，解决后面CI编译的问题。