Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HAproxy Reverse Proxy Setup #390

Open
awptechnologies opened this issue Mar 22, 2024 · 3 comments
Open

HAproxy Reverse Proxy Setup #390

awptechnologies opened this issue Mar 22, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@awptechnologies
Copy link

Reverse proxy is supported from what i can tell in the research i have done. i am having troubles setting up a relay server that is accessible from outside my network through the reverse proxy.

@awptechnologies awptechnologies added the bug Something isn't working label Mar 22, 2024
@eltorio
Copy link

eltorio commented Mar 26, 2024

@awptechnologies
I tried to host hbbs/hbbr and a minimalist admin server in a Kubernetes cluster.
It uses Haproxy 2.7 as the ingress proxy and a modified Haproxy-ingress-controler 1.10 as the Haproxy controler.
First issue is that Haproxy cannot handle udp -except for quic- and it probably never supports it.
Second issue hbbs will see the IP address of Haproxy and shows a log like this one:

2024-03-26T07:41:22.860694378Z [2024-03-26 07:41:22.860521 +00:00] DEBUG [src/rendezvous_server.rs:1097] Tcp connection from [::ffff:172.28.5.218]:58786, ws: false
2024-03-26T07:41:22.861155579Z [2024-03-26 07:41:22.861075 +00:00] DEBUG [src/rendezvous_server.rs:1137] Tcp connection from [::ffff:172.28.5.218]:58786 closed

but this is the generated haproxy config:

frontend https
  mode http
  bind 0.0.0.0:443 name v4 crt /etc/haproxy-ingress/certs/frontend ssl alpn h2,http/1.1
  bind [::]:443 name v6 crt /etc/haproxy-ingress/certs/frontend ssl alpn h2,http/1.1
  http-request set-var(txn.base) base
  http-request set-var(txn.path) path
  http-request set-var(txn.host) req.hdr(Host),field(1,:),lower
  http-request set-var(txn.host_match) var(txn.host),map(/etc/haproxy-ingress/maps/host.map)
  http-request set-var(txn.host_match) var(txn.host),regsub(^[^.]*,,),map(/etc/haproxy-ingress/maps/host.map,'') if !{ var(txn.host_match) -m found }
  http-request set-var(txn.path_match) var(txn.host_match),concat(,txn.path,),map(/etc/haproxy-ingress/maps/path-exact.map)
  http-request set-var(txn.path_match) var(txn.host_match),concat(,txn.path,),map_beg(/etc/haproxy-ingress/maps/path-prefix.map) if !{ var(txn.path_match) -m found }
  http-request auth realm Protected-Content if { var(txn.path_match) -m dom 706d41ac94352467187ae04f79c5413b } !{ http_auth_group(briac-code-server-hcf-coder-ingress-coder-briac-cf) authenticated-users }
  http-request auth realm Protected-Content if { var(txn.path_match) -m dom 1a8a05768edae0436e25b1e49cc824c2 } !{ http_auth_group(longhorn-system-longhorn-ui) authenticated-users }
  http-request auth realm Protected-Content if { var(txn.path_match) -m dom 67785cfd7a61cf6cf21ad6ac536622cd } !{ http_auth_group(monitoring-prometheus-kube-prometheus-prometheus) authenticated-users }
  http-request set-header X-Forwarded-Proto https
  use_backend %[var(txn.path_match),field(1,.)]
  default_backend _default-local-service_http

frontend sctgdesk-hbbr-hbbs-gateway-port-21115
  mode tcp
  bind 0.0.0.0:21115 name v4
  bind [::]:21115 name v6
  option tcplog
  default_backend sctgdesk_port-21115

frontend sctgdesk-hbbr-hbbs-gateway-port-21116
  mode tcp
  bind 0.0.0.0:21116 name v4
  bind [::]:21116 name v6
  option tcplog
  default_backend sctgdesk_port-21116

frontend sctgdesk-hbbr-hbbs-gateway-port-21117
  mode tcp
  bind 0.0.0.0:21117 name v4
  bind [::]:21117 name v6
  option tcplog
  default_backend sctgdesk_port-21117

frontend sctgdesk-hbbr-hbbs-gateway-port-21118
  mode tcp
  bind 0.0.0.0:21118 name v4
  bind [::]:21118 name v6
  option tcplog
  default_backend sctgdesk_port-21118

frontend sctgdesk-hbbr-hbbs-gateway-port-21119
  mode tcp
  bind 0.0.0.0:21119 name v4
  bind [::]:21119 name v6
  option tcplog
  default_backend sctgdesk_port-21119

backend sctgdesk_port-21116
  mode tcp
  default-server check
  server SRV_1 172.28.2.86:21116 enabled

backend sctgdesk_port-21117
  mode tcp
  default-server check
  server SRV_1 172.28.2.86:21117 enabled

backend sctgdesk_port-21118
  mode tcp
  default-server check
  server SRV_1 172.28.2.86:21118 enabled

backend sctgdesk_port-21119
  mode tcp
  default-server check
  server SRV_1 172.28.2.86:21119 enabled

backend sctgdesk_sctgdesk-api-server-service_port-21114
  mode http
  balance roundrobin
  option forwardfor
  no option abortonclose
  default-server check
  server SRV_1 172.28.5.91:21114 enabled

I also added a iptables rule for forwarding udp:

iptables -t nat -A PREROUTING -i enp0s6 -p udp --dport 21116 -j DNAT --to-destination 172.31.255.221:21116

But yet it doesn't work…
I studied the possibility to use Haproxy v2 protocol. It needs a small modification of the code. I made some tests at https://github.com/sctg-development/proxyv2-test/blob/main/src/main.rs#L47-L89

In fact it will be better to study the possibility to use the Websockets. I use the RustDesk web it uses websockets on ports 21118/21119. I just modified the code for using secure websockets on 21120/21121 . It is easy to proxy websockets and to proxy secure websockets to websockets.
hbbs/hbbr use tokio_tungstenite for getting the inner stream via websocket, we'll need to add that support on RustDesk

@eltorio
Copy link

eltorio commented Mar 26, 2024

I made a fork publically available with my test https://github.com/sctg-development/sctgdesk-server I don't have yet enough time to test but docker image are built at https://hub.docker.com/search?q=eltorio%2Frustdesk-server and the built package are at https://github.com/sctg-development/sctgdesk-server/releases
But as I wrote before Haproxy cannot handle UDP, you need something else.

@eltorio
Copy link

eltorio commented Mar 28, 2024

Another idea may use QUIC which can be enabled in HAProxy, I saw that the hbb_common code contains a quic feature. Currently it does not compile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants