Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Ask a Question
Welcome To Ask or Share your Answers For Others

Categories

routing - how to terminate ssl at ingress-gateway in istio?

0 votes
378 views
in Technique[技术] by (71.8m points)

routing - how to terminate ssl at ingress-gateway in istio?

I am trying to experiment ssl connection in istio ingress gateway.

From here istio ssl gateway without termination, i assume that istio ingress gateway by default should terminate ssl.

I have installed istio with demo profile, via istioctl. I have also installed my service svc1.

Apart from these, below are what my resources are with routng logic:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: vs-gateway
  namespace: myns
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-certs/tls.key
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: vs
  namespace: myns
spec:
  hosts:
  - "*"
  gateways:
  - vs-gateway
  http:
  - match:
    - uri:
        prefix: "/svc1/"
    rewrite:
      uri: "/"

    route:
    - destination:
        host: svc1
        port:
          number: 80

I found the Gateway url via this. For experiment i had also enabled http on ingress gateway. That way curl http://172.17.0.2:<http_node_port> works with 200 response. Later i removed http from ingress gateway and kept only https (as https is my primary goal to be recieved by ingress gateway)

And then trying curl with https on gateway url. I get 503.

$ curl -ivk https://172.17.0.2:<https_node_port>/svc1/user
*   Trying 172.17.0.2...
* TCP_NODELAY set
* Connected to 172.17.0.2 (172.17.0.2) port 30278 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=CA; O=Abc; CN=example.com
*  start date: Dec 31 08:22:32 2019 GMT
*  expire date: Jan 30 08:22:32 2020 GMT
*  issuer: C=US; ST=CA; O=Abc; CN=example.com
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* Using Stream ID: 1 (easy handle 0x55c961626580)
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET /svc1/user HTTP/2
> Host: 172.17.0.2:30278
> User-Agent: curl/7.58.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
< HTTP/2 503 
HTTP/2 503 
< content-length: 95
content-length: 95
< content-type: text/plain
content-type: text/plain
< date: Thu, 02 Jan 2020 08:13:49 GMT
date: Thu, 02 Jan 2020 08:13:49 GMT
< server: istio-envoy
server: istio-envoy

< 
* Connection #0 to host 172.17.0.2 left intact
upstream connect error or disconnect/reset before headers. reset reason: connection termination

I also enabled sidecar istio proxy debuggin on svc pod sidecar. I got below logs

[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [external/envoy/source/extensions/filters/listener/original_dst/original_dst.cc:18] original_dst: New connection accepted
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [external/envoy/source/extensions/filters/listener/tls_inspector/tls_inspector.cc:72] tls inspector: new connection accepted
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:30] Called tcp filter: Filter
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:40] Called tcp filter: initializeReadFilterCallbacks
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [external/envoy/source/common/tcp_proxy/tcp_proxy.cc:204] [C88] new tcp proxy session
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:133] [C88] Called tcp filter onNewConnection: remote 10.244.0.5:34148, local 10.244.0.16:3000
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][filter] [external/envoy/source/common/tcp_proxy/tcp_proxy.cc:347] [C88] Creating connection to cluster inbound|80|serviceport|svc1.myns.svc.cluster.local
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:83] creating a new connection
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:364] [C89] connecting
[Envoy (Epoch 0)] [2020-01-02 06:53:19.392][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:711] [C89] connecting to 127.0.0.1:3000
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:720] [C89] connection in progress
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:109] queueing request due to no available connections
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][conn_handler] [external/envoy/source/server/connection_handler_impl.cc:333] [C88] new connection
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:559] [C89] connected
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:285] [C89] assigning connection
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][filter] [external/envoy/source/common/tcp_proxy/tcp_proxy.cc:541] TCP:onUpstreamEvent(), requestedServerName: 
[Envoy (Epoch 0)] [2020-01-02 06:53:19.393][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:143] Called tcp filter completeCheck: OK
[Envoy (Epoch 0)] [2020-01-02 06:53:19.395][28][debug][filter] [src/istio/control/client_context_base.cc:139] Report attributes: attributes {
  key: "connection.event"
  value {
    string_value: "open"
  }
}
attributes {
  key: "connection.id"
  value {
    string_value: "38a9b348-1730-4e0b-9664-fbbaeedd9215-88"
  }
[Envoy (Epoch 0)] [2020-01-02 06:53:19.395][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:100] [C88] Called tcp filter onRead bytes: 664
[Envoy (Epoch 0)] [2020-01-02 06:53:19.396][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:123] [C88] Called tcp filter onWrite bytes: 28
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:123] [C88] Called tcp filter onWrite bytes: 0
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:100] [C88] Called tcp filter onRead bytes: 34
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:527] [C88] remote close
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:193] [C88] closing socket: 0
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:174] [C88] Called tcp filter onEvent: 0 upstream 127.0.0.1:3000
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][filter] [src/istio/control/client_context_base.cc:139] Report attributes: attributes {
  key: "connection.duration"
  value {
    duration_value {
      nanos: 6151000
    }
  }
}
attributes {
  key: "connection.event"
  value {
    string_value: "close"
  }
}
at
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:104] [C89] closing data_to_write=34 type=0
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][conn_handler] [external/envoy/source/server/connection_handler_impl.cc:88] [C88] adding to cleanup list
[Envoy (Epoch 0)] [2020-01-02 06:53:19.398][28][debug][filter] [src/envoy/tcp/mixer/filter.cc:35] Called tcp filter : ~Filter
[Envoy (Epoch 0)] [2020-01-02 06:53:19.399][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:589] [C89] write flush complete
[Envoy (Epoch 0)] [2020-01-02 06:53:19.399][28][debug][connection] [external/envoy/source/common/network/connection_impl.cc:193] [C89] closing socket: 1
[Envoy (Epoch 0)] [2020-01-02 06:53:19.399][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:124] [C89] client disconnected
[Envoy (Epoch 0)] [2020-01-02 06:53:19.399][28][debug][pool] [external/envoy/source/common/tcp/conn_pool.cc:238] [C89] connection destroyed

With the logs, it seems like ingress gateway is forwarding ssl request to svc. (please correct here if i am wrong)

So in the end, can anyone please help me out getting ssl terminated at ingress gateway, and forward plain http request to svc?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

Based on this github issue and this istio documentation

Named service ports: Service ports must be named. The port name key/value pairs must have the following syntax: name: [-]. See Protocol Selection for more details.

Based on mockserver service 

ports:
    - name: serviceport

I would recommend to change it to http/https like mentioned there and confirmed by community member who had same issue on github.

Manual protocol selection

Protocols can be specified manually by naming the Service port name: [-]. The following protocols are supported:

  • grpc
  • grpc
  • web
  • http
  • http2
  • https
  • mongo
  • mysql*
  • redis*
  • tcp
  • tls
  • udp

*These protocols are disabled by default to avoid accidentally enabling experimental features. To enable them, configure the corresponding Pilot environment variables.


与恶龙缠斗过久,自身亦成为恶龙；凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

Just Browsing Browsing

2.1m questions

2.1m answers

60 comments

57.0k users

Most popular tags

Powered by Question2Answer
Donate
Theme by Q2A Market&&OStack.cn
...