Kubernetes 集成 LDAP 认证

Kubernetes 集成 LDAP 认证

基础知识

每一个到 API SERVER 的请求都会经过三个阶段（除了读请求，读请求只会经过前两个阶段）：authentication, authorisation 和 admission control

Authentication 检查请求用户是否合法Authorisation 校验用户对请求的资源是否有权限Admission control 对请求的资源进行校验，比如请求内容是在a命名空间创建资源，但是a命名空间并不存在，此时就会拒绝该请求

在 authentication  这个阶段包括多个认证插件，它们是这么工作的：

进来的请求会依次进入每个插件，每个插件会尝试进行认证：

如果一个插件完成了认证，则请求会直接进入授权验证阶段如果一个插件认证不通过，则请求会进入下一个插件进行验证如果所有插件都认证不通过，则请求会被拒绝并抛出 401 的状态码

以下是 Kubernetes v1.18 版本中所有的认证插件：

Static Token File X.509 Client Certificate OpenID Connect Token  Authenticating ProxyWebhook TokenService Account TokenBootstrap Token

在 v1.22 版本又新增了一种插件：client-go credential plugins

其中 Service Account Token 和 Bootstrap Token 是系统调用的插件，前者用于对 SA 的认证，后者主要用于集群创建过程中的一些认证

概要

本文主要讲解通过 Webhook Token 认证的方式实现集成 LDAP，整个流程如下：

Kubernetes 会通过 Webhook Token 认证插件向我们定义的webhook发送一条包含 HTTP bearer token 的请求

请求格式及内容大概如下：

{  "apiVersion": "authentication.k8s.io/v1beta1",  "kind": "TokenReview",  "spec": {    # Opaque bearer token sent to the API server    "token": "014fbff9a07c...",       # Optional list of the audience identifiers for the server the token was presented to.    # Audience-aware token authenticators (for example, OIDC token authenticators)     # should verify the token was intended for at least one of the audiences in this list,    # and return the intersection of this list and the valid audiences for the token in the response status.    # This ensures the token is valid to authenticate to the server it was presented to.    # If no audiences are provided, the token should be validated to authenticate to the Kubernetes API server.    "audiences": ["https://myserver.example.***", "https://myserver.internal.example.***"]  }}

更详细内容：https://github.com/kubernetes/api/blob/master/authentication/v1beta1/types.go#L36

为了简单，我们定义token的格式如下：

username:password  # 分别表示ldap中存储的用户名和密码

如果此时ldap中有一个用户是poorops，密码是poorops，则该用户请求的流程如下：

Webhook Token 认证插件收到请求后，将请求中的token提取出来，即 poorops:poorops ，然后封装到上面提到的固定格式中Webhook Token 认证插件将封装好的请求通过HTTP发送给我们定义的webhook我们定义的webhook收到请求后提取里面的token，并解析出用户名和密码我们定义的webhook拿着解析出来的用户名和密码去LDAP进行认证我们定义的webhook将认证结果使用同样的格式以response的方式返回给 Webhook Token 认证插件Webhook Token 认证插件拿到响应后，会以将里面 Status.Authenticated 字段的值来判断接受还是拒绝该请求

响应的格式及内容大概如下：

{  "apiVersion": "authentication.k8s.io/v1beta1",  "kind": "TokenReview",  "status": {    "authenticated": true,    "user": {      # Required      "username": "janedoe@example.com",      # Optional      "uid": "42",      # Optional group memberships      "groups": ["developers", "qa"],      # Optional additional information provided by the authenticator.      # This should not contain confidential data, as it can be recorded in logs      # or API objects, and is made available to admission webhooks.      "extra": {        "extrafield1": [          "extravalue1",          "extravalue2"        ]      }    },    # Optional list audience-aware token authenticators can return,    # containing the audiences from the `spec.audiences` list for which the provided token was valid.    # If this is omitted, the token is considered to be valid to authenticate to the Kubernetes API server.    "audiences": ["https://myserver.example.***"]  }}

正式开始

前提是已经部署好了ldap服务及kubernetes集群

使用go写webhook的好处是，可以直接拿kubernetes源码中定义的TokenReview结构体使用，只需要安装两个包

go get github.com/go-ldap/ldap/v3go get k8s.io/api/authentication/v1beta1

kubernetes 默认使用 authentication.k8s.io/v1beta1 版本的TokenReview格式，如果想用v1版本的，则需要 go get k8s.io/api/authentication/v1，并且 API server 启动参数必须添加：--authentication-token-webhook-version=v1

直接上webhook代码

注意点：必须使用 http.ListenAndServeTLS 来启动HTTPS服务，否则无效

package mainimport (  "encoding/json"  "flag"  "fmt"  "github.com/go-ldap/ldap/v3"  "github.com/pkg/errors"  "io"  "k8s.io/api/authentication/v1beta1"  "log"  "net/http"  "strings")var (  ldapURL   string  baseDN    string  ldapUser  string  ldapPass  string  ldapGroup string  addr      string  certFile  string  keyFile   string)func init() {  flag.StringVar(&ldapURL, "ldap-url", "ldap://192.168.111.102:389", "ldap url, format: ldap://addr:port")  flag.StringVar(&baseDN, "base-dn", "dc=poorops,dc=com", "base dn, format: dc=example,dc=com")  flag.StringVar(&ldapUser, "ldap-user", "admin", "ldap user")  flag.StringVar(&ldapPass, "ldap-pass", "poorops", "ldap password")  flag.StringVar(&ldapGroup, "ldap-group", "ou", "the name of the ldap group attribute")  flag.StringVar(&addr, "addr", ":443", "addr to listen, format: host:port")  flag.StringVar(&certFile, "cert-file", "", "cert-file")  flag.StringVar(&keyFile, "key-file", "", "key-file")  flag.Parse()}func main() {  http.HandleFunc("/", handler)  log.Printf("start to listen %s\n", addr)  log.Fatal(http.ListenAndServeTLS(addr, certFile, keyFile, nil))}func handler(w http.ResponseWriter, r *http.Request) {  // Read body of POST request  b, err := io.ReadAll(r.Body)  defer func() {    if e := r.Body.Close(); e != nil {      log.Printf("error to close body: %s\n", e.Error())    }  }()  if err != nil {    w.WriteHeader(http.StatusInternalServerError)    log.Printf("error to read body: %s", err.Error())    return  }  log.Printf("Receiving: %s\n", string(b))  // Unmarshal JSON from POST request to TokenReview object  var tr v1beta1.TokenReview  err = json.Unmarshal(b, &tr)  if err != nil {    w.WriteHeader(http.StatusInternalServerError)    log.Printf("error to umarshl json to TokenReview object: %s", err.Error())    return  }  // Extract username and password from the token in the TokenReview object  s := strings.SplitN(tr.Spec.Token, ":", 2)  if len(s) != 2 {    w.WriteHeader(http.StatusBadRequest)    log.Printf("invalid ldap token: %s, valid ldap token format is name:password", tr.Spec.Token)    return  }  username, password := s[0], s[1]  // Make LDAP Search request with extracted username and password  userInfo, err := ldapSearch(username, password)  if err != nil {    w.WriteHeader(http.StatusInternalServerError)    log.Printf("failed LDAP Search request: %s", err.Error())    return  }  // Set status of TokenReview object  if userInfo == nil {    tr.Status.Authenticated = false  } else {    tr.Status.Authenticated = true    tr.Status.User = *userInfo  }  // Marshal the TokenReview to JSON and send it back  b, err = json.Marshal(tr)  if err != nil {    w.WriteHeader(http.StatusInternalServerError)    log.Printf("error to marshl TokenReview object to json: %s", err.Error())    return  }  _, _ = w.Write(b)  log.Printf("Returning: %s\n", string(b))}func ldapSearch(username, password string) (*v1beta1.UserInfo, error) {  // Connect to LDAP directory  l, err := ldap.DialURL(ldapURL)  if err != nil {    return nil, err  }  defer l.Close()  // Authenticate as LDAP admin user  err = l.Bind(fmt.Sprintf("cn=%s,%s", ldapUser, baseDN), ldapPass)  if err != nil {    return nil, err  }  // Execute LDAP Search request  searchRequest := ldap.NewSearchRequest(    baseDN,    ldap.ScopeWholeSubtree, ldap.NeverDerefAliases, 0, 0, false,    fmt.Sprintf("(&(objectClass=organizationalPerson)(uid=%s))", ldap.EscapeFilter(username)), // Filter    //[]string{"dn"},    nil,    nil,  )  result, err := l.Search(searchRequest)  if err != nil {    return nil, err  }  if len(result.Entries) != 1 {    return nil, errors.New("User does not exist or too many entries returned")  } else {    userdn := result.Entries[0].DN    err = l.Bind(userdn, password)    if err != nil {      return nil, errors.Errorf("User %s authenticate failed", username)    } else {      extra := make(map[string]v1beta1.ExtraValue)      for _, item := range result.Entries[0].Attributes {        if item.Name != ldapGroup {          extra[item.Name] = result.Entries[0].GetAttributeValues(item.Name)        }      }      return &v1beta1.UserInfo{        Username: username,        UID:      username,        Groups:   sanitize(result.Entries[0].GetAttributeValues(ldapGroup)),        Extra:    extra,      }, nil    }  }}func sanitize(a []string) []string {  var res []string  for _, item := range a {    res = append(res, strings.ToLower(item))  }  return res}

main:  启动HTTPS服务handler: 处理HTTPS 请求ldapSearch: 查询LDAP

验证

1、启动webhook服务，使用如下命令生成伪证书

$ openssl req -x509 -newkey rsa:2048 -nodes -subj "/CN=localhost" -keyout key.pem -out cert.pem

启动：

$ ./ldap-for-k8s -cert-file=cert/cert.pem -key-file=cert/key.pem2021/11/18 15:06:53 start to listen :443

2、编写 Webhook Token 认证插件使用的配置文件 webhook-config.yaml

apiVersion: v1kind: Configclusters:  - name: authn    cluster:      server: https://192.168.111.102:443  # webhook服务地址      insecure-skip-tls-verify: trueusers:  - name: kube-apiservercurrent-context: authncontexts:- context:    cluster: authn    user: kube-apiserver  name: authn

主要是 server 和 insecure-skip-tls-verify 字段，其他字段没有实际意义，主要是为了满足要求，格式必须和  kubeconfig 一样，使用有效证书认证的例子如下：

# Kubernetes API versionapiVersion: v1# kind of the API objectkind: Config# clusters refers to the remote service.clusters:  - name: name-of-remote-authn-service    cluster:      certificate-authority: /path/to/ca.pem         # CA for verifying the remote service.      server: https://authn.example.***/authenticate # URL of remote service to query. 'https' recommended for production.# users refers to the API server's webhook configuration.users:  - name: name-of-api-server    user:      client-certificate: /path/to/cert.pem # cert for the webhook plugin to use      client-key: /path/to/key.pem          # key matching the cert# kubeconfig files require a context. Provide one for the API server.current-context: webhookcontexts:- context:    cluster: name-of-remote-authn-service    user: name-of-api-server  name: webhook

3、配置api-server启用Webhook Token认证插件，编辑 /etc/kubernetes/manifests/kube-apiserver.yaml

# 在相应位置添加如下内容  - command:  ...    - --authentication-token-webhook-config-file=/etc/webhook-config.yaml        volumeMounts:    ...    - mountPath: /etc/webhook-config.yaml      name: webhook-config      readOnly: true  # 这里为了简单，把配置文件直接放在master机器上，如果是多master的情况建议创建一个ConfigMap挂载  volumes:   ...  - hostPath:      path: /root/webhook-config.yaml    name: webhook-config

总共支持三个参数：--authentication-token-webhook-config-file 指定配置文件--authentication-token-webhook-cache-ttl 指定缓存结果时长，默认是2分钟--authentication-token-webhook-version 指定版本，默认是v1beta1，可以指定v1

保存之后，api-server pod 会自动重启

4、添加用户凭证

此时ldap中存储有poorops用户，密码为poorops

$ kubectl config set-credentials poorops --token poorops:pooropsUser "poorops" set.

5、使用该用户

$ kubectl get nodes --user pooropsError from server (Forbidden): nodes is forbidden: User "poorops" cannot list resource "nodes" in API group "" at the cluster scope

看到这个报错不要慌，其实请求已经通过了第一阶段，该报错是在 authorisation 阶段返回的，即表示poorops用户没有相关资源的权限

6、 给用户添加资源权限

$ kubectl create clusterrole poorops --resource nodes --verb list$ kubectl create clusterrolebinding poorops --clusterrole poorops --user poorops

7、此时再看

$ kubectl get nodes --user pooropsNAME     STATUS   ROLES                  AGE    VERSIONmaster   Ready    control-plane,master   7d3h   v1.20.0node     Ready7d3h   v1.20.0# 当然其他资源还是没有权限的$ kubectl get secrets --user poorops     Error from server (Forbidden): secrets is forbidden: User "poorops" cannot list resource "secrets" in API group "" in the namespace "default"

8、 把poorops的密码改成别的再试下

$ kubectl get nodes --user poorops                               error: You must be logged in to the server (Unauthorized)

Bingo!

其他

如果安装了kubernetes原生的Dashboard验证其他更简单， 只需要在登录页填写 username:password 即可

参考

https://itnext.io/implementing-ldap-authentication-for-kubernetes-732178ec2155