At Flexiant, we had a need for a general websocket to tcp proxy. A cute apache module to do extensible web-sockets programming has been developed by self.disconnect, and is available on github here. I’ve hacked around with this to produce a generic, apache licensed, websocket proxy. A websocket connection is made over HTTP or HTTPS to apache, this calls the websocket module, which in turn calls my module, which makes an outbound tcp connection.

Some features:

  • It supports base64 and text format sockets. So it will thus process (for instance) VNC which works using base64 encoding.
  • Configuration is supplied through normal apache directory based configuration (see details below).
  • It’s extensible (currently by changing the code, rather than calling additional modules)

The configuration section looks like this, in this case set up to proxy VNC using a novnc client:

Note that the parameter you specify for Location must not be a file or directory that exists with your docroot, or you will just get 404 or similar. So, in the above example you must not have a file or directory at ${DOCROOT}/vncproxy.

I discovered that on a default apache install, you may also want to change the RequestReadTimeout, i.e.

I’m going to try and persuade self.disconnect to stick this in the examples directory for apache-websocket, but in the mean time the source is available here. Put it in the examples directory of apache-websocket, then compile, install and enable it with it with:
sudo apxs2 -c -i -a -I.. mod_websocket_tcp_proxy.c

[Update: see new blog post, git repo link here]

77 responses to “Using apache-websocket to proxy tcp connections”

  1. Leonardo Moreno Avatar
    Leonardo Moreno

    Alex,

    First let me thankyou for your time and answers.

    It looks like I’m still lost in the darkness here, because apparently your code is not going to work for me as what I really need is to proxy my ws connections between the browser and my java web application.

    Having said that, and with a giant please and a even bigger shame, can you guide me to search for the right documentation to be able to do that? I’ve lost the entire week trying to figure this out and, like you just saw, I’m not even close. Any help would be really appreciated.

    Leonardo

  2. Alex Bligh Avatar
    Alex Bligh

    Leonardo:

    NGINX does it
    http://www.letseehere.com/reverse-proxy-web-sockets

    HAProxy does it:
    http://stackoverflow.com/questions/2419346/can-nginx-be-used-as-a-reverse-proxy-for-a-backend-websocket-server

    I thought I remembered someone saying mod_proxy could be got to work where there was no CONNECT used, but I can’t now find it.

    Alex

  3. Leonardo Moreno Avatar
    Leonardo Moreno

    Alex, thanks a lot.

    I have an installation with apache and right now it can’t be changed, so I’m going to keep doing my research.

  4. Hi,

    I am very new to Apache modules and need your help in finding an Apache module which forwards WebSocket requests to Tomcat instead of handling the requests itself.

    Scenario:
    I have a Java web application deployed on Tomcat 7 which uses the Tomcat websocket implementation. Tomcat sits behind Apache 2.2.3 server where the static content is served by Apache itself and requests for dynamic content(JSP/Servlet) are forwarded to Tomcat. I have linked both the servers using the AJP connector but the AJP connector does not seem to support WebSocket protocol as yet.

    It would be very helpful if anyone could point me to such a module or just guide me through the process of creating one.

    Thanks in advance for your help.

    1. If you want to have your Java program just listening for tcp, you need my module. I unwrap the tcp stream by taking off the websocket layer.

      If you want your Java program to speak websockets, and want a proxy for that, this is hard in the general case. mod_proxy might be persuaded to do it but do not expect it to be problem free. NGINX does it. See my replies to Leonardo above.

  5. Thanks for the help.

    I think we (Leonardo and I) are sailing in the same boat (in the darkness).
    But as you suggested, configuring mod_proxy might help, I need to research more on this and hopefully I might find something helpful. I will be tracking this post for my future reference and look forward to any help from the comments as well.

    Thanks a lot, again.

  6. Finally found a module to proxy websocket using Apache. Here is the link: https://github.com/kawasima/mod_proxy_websocket.This module proxies websocket requests to Tomcat.

    1. Thanks for that useful link. For others reading this, what Nishant linked to is a proxy that simply forwards the websockets connection unchanged (rather than unwrapping the data into a TCP session without the Websockets encapsulation as my module does).

  7. Sebastien Cramatte Avatar
    Sebastien Cramatte

    Does anyone have tested the module that Nishant talk about ? Anyone can post some apache config example.

    I would like to have my PHP websocket server listening on 127.0.0.1 and got these proxy to been able to access it through port 443 and 80

  8. Hi
    Unfortunately my connection to server is suddenly closed.
    In apache error log file I got these errors:
    Thu Nov 29 17:26:14 2012] [debug] mod_websocket_tcp_proxy.c(187): tcp_proxy_run apr_socket_recv failed len=0 rv=70007, The timeout specified has expired
    [Thu Nov 29 17:26:14 2012] [debug] mod_websocket_tcp_proxy.c(216): tcp_proxy_run stop
    [Thu Nov 29 17:26:14 2012] [debug] mod_websocket_tcp_proxy.c(421): tcp_proxy_on_disconnect
    [Thu Nov 29 17:26:14 2012] [info] [client 192.168.1.9] (9)Bad file descriptor: core_output_filter: writing data to the network
    Do you have any idea on this error??

    Thanks in advance

    1. That error indicates that the websocket module did not receive any data from the server for the amount of time specified in the timeout parameter, and thus closed the socket. I’d examine why your server has not sent any data.

      (Also, I’d ensure you are using the version from github as the timeout code did have some bugs in).

  9. Hi Alex
    Thanks a lot for your quick reply.
    According to your clarify about that error,We changed server code so that server sends acknowledge on client’s data. So connection was not closed 🙂

    Thanks again

  10. Nishant, could you successfully use mod_proxy_websocket to forward requests to tomcat? Can you please share the example configuration you used in Apache. I tried to use that, though looks like mod_proxy_websocket strips Connection and Upgrade headers in the response and websocket handshake fails.

    Anyone else able to make that work?

    1. Alex Bligh Avatar
      Alex Bligh

      If you are running tomcat to serve websockets, this module won’t work for you. This module provides a websocket proxy of a tcp connection, i.e. the backend connects to is expected to talk raw tcp, and the front end is that same tcp stream wrapped in websockets.

      There is a little more documentation on github.

  11. Alex, I tried to use your tcp proxy module too, though i was HTTP 403 the browser for web socket requests.

    In the logs i also saw a log entry from mod_websocket_vnc_proxy.c(1241): “tcp_proxy_on_connect bad protocol”. Any idea what’s happening?

  12. Rahul Jain Avatar
    Rahul Jain

    Thanks Alex for responding. I’ll keep looking at mod_proxy_websocket then.

  13. Hi Alex,
    I’m compiling the mod_websocket module and then your mod_websocket_vnc_proxy.
    I’ve configured the httpd on Centos like this:

    But when I permor a request on apache:80/rpc the httpd server seems try to look inside the htdocs if there is a folder named “rpc”.
    Where I’ve missing something?
    Thanks for the support!

    1. As per the instructions, you need to put a block around the stuff with SetHandler and WebSocket handler, if you expect it to respond to location /rpc. The example used a /vncproxy as the location.

      Make sure you do NOT have a directory called /rpc in your docroot.

  14. Thanks a billion!!! seems very promising

  15. Hi Alex thanks for the wonderful apache module.
    I have one question in vnc proxy code.
    /* see the tutorial about the reason why we have to specify options again */
    apr_socket_opt_set(tpd->tcpsocket, APR_SO_NONBLOCK, 1);
    apr_socket_opt_set(tpd->tcpsocket, APR_SO_KEEPALIVE, 1);
    apr_socket_timeout_set(tpd->tcpsocket, 0);

    Can you plz point to the tutorial explaining above , sorry i did not find it ??
    Thanks in advance

Leave a Reply to ToniCancel reply

Discover more from Alex Bligh's blog

Subscribe now to keep reading and get access to the full archive.

Continue reading