Architectural Overview
Kaazing Gateway 8.12 (Battlestar) Release
February 2009
There is an ever-growing need to provide bi-directional (full-duplex), real-time communication between the browser and the Web server, similar to that which enables desktop clients to interact with back-end systems. For example, the demand for real-time services on the Web is increasing at a torrid pace, requiring Web applications to display information such as live stock feeds or facilitate participation in games, bidding, chat, and so on. However, HTTP only supports half-duplex communication (requests and responses can flow only in one direction at a time), making it difficult to deliver real-time content.
Techniques such as Comet and Reverse Ajax were developed to allow server-initiated message delivery, a process in which information is pushed from the server to the client. However, these techniques have many limitations such as standardization, performance, and scalability. To address these issues, the HTML 5 specification defines Web sockets and server-sent events. The combination of these standards makes full-duplex, direct TCP communication a reality for the next generation of browsers.
Fortunately, Kaazing Gateway makes full-duplex communication a reality today, even for the most antiquated browsers. Kaazing Gateway delivers a WebSocket server, built for real-time business, which supports bi-directional browser communication as proposed by the HTML 5 specification. Kaazing Gateway integrates with a wide array of business services.
This document describes how Kaazing Gateway works and contains the following sections:
History of the Real-Time Web
Polling, a technique in which the client (browser) sends HTTP requests at regular intervals and immediately receives a response, represents the initial attempt to deliver real-time information on the Web. However, the downside of polling is that it generates a lot of unnecessary requests, which can impact both client and server performance. For example, if a request is sent by the browser every second and the server only sends messages at a 17-second interval, then there are 16 unnecessary requests taking up server resources and network bandwidth. Obviously, there is a simple solution to this problem if the interval of message delivery is known. The solution is to synchronize the interval at which a request is made by the client with that at which a message is ready for delivery on the server. However, real-time data is often not that predictable, making unnecessary requests inevitable.
Current attempts to solve the problem now largely center around server-side push technology, the most notable of which is Comet or Reverse Ajax. Comet introduces a way to delay the completion of an HTTP response to deliver messages to the client, a technique often called a hanging-GET or pending-POST. Comet-based push is generally implemented in JavaScript and uses connection strategies such as long-polling or streaming.
Long-Polling and Streaming
In the case of long-polling, also known as asynchronous-polling, the browser sends a request to the server and the server keeps the request open for a set period. If a notification is received within that period, a response containing the message is sent to the client. If a notification is not received within the set time period, the server sends a response to terminate the open request. In a case where a large number of messages must be sent, long-polling does not provide any substantial performance improvements over traditional polling. This lack of performance can be attributed to the number of HTTP headers, present in both long-polling and polling, which often accounts for more than half of the network traffic. Additionally, if secure connections (connections using SSL) are used in combination with long-polling, the setup and tear down of each connection is costly, unless the client establishes an HTTP 1.1 persistent connection.
When streaming is used, the browser sends a complete request, but the server sends and maintains an open response that is continuously updated. A request is sent and kept open indefinitely (or for a set period of time) and the response is updated whenever a message is ready to be sent, but the server never signals to complete the response, thus keeping the connection open to deliver future messages. One benefit of streaming is reduced network traffic, which is the result of sending packets that only contain data rather than packets that contain both data and HTTP headers. The downside of streaming is that it is still encapsulated in HTTP, so intervening HTTP proxies may choose to buffer the response, increasing the latency of the message delivery.
Traditional Java EE Architecture with Streaming Support
By way of example, let's look at a fairly common scenario in which the requirement is to stream data from a back-end data feed (for example, a feed that contains stock information) to a client browser. Using traditional application design, the browser connects to an application server and application-specific bridge code in the application layer handles the translation between the APIs and the browser. This can be implemented as a custom Servlet, which internally uses a JMS client. The custom Servlet interprets custom JavaScript application protocols and maps these to standard APIs. Comet or reverse Ajax techniques might be used to push data to the clients. The following figure shows a traditional Java EE architecture used to extend various protocols (including JMS) to the browser.
Since the Java EE Servlet uses HTTP to talk to the browser, a full-duplex communication all the way between the server and the browser is never achieved; The Java EE Container has, in this case, become the throughput bottleneck. There are additional issues with this implementation. Many protocols, including those used by Java EE, are based on TCP and therefore have specific message-delivery guarantees. It is now up to the application developer to ensure that the same guarantees are enforced all the way to the browser and back.
Additionally, there may be a need to manually implement support for long-polling or streaming to push data to Web clients. Furthermore, developers must design an application protocol between browser and server, implementing their own JavaScript and Web Servlet listener that understands a custom protocol, and map the JavaScript calls into standard API calls. Finally, the JMS Client may not originally be designed to scale to a large number of clients, but may be forced to, causing competition for CPU resources on the Web application host between dealing with the many concurrent synchronous requests and processing the Web application logic.
HTML 5
The HTML 5 specification and, more specifically, its Communication section defines Web sockets and server-sent events, which makes full-duplex, direct TCP communication a reality for browsers. This section of the HTML 5 specification is drawing a lot of attention. The first public draft of the HTML 5 specification was published by the W3C (World Wide Web Consortium) in January 2008, and browser vendors are already targeting certain features in the specification.
The idea of putting a full-duplex channel into the specification is not new; the TCPConnection API and protocol were initially drafted over two years ago. With recent development, in which Kaazing Corporation played a contributing role, the HTML 5 Communication section has been solidified and is now nearly complete with few, or no, additions planned. This makes it easy for browser vendors to implement native support for the Communication section of the specification, despite the full HTML 5 proposal remaining in draft form.
Three significant areas in the HTML 5 specification that impact cross-site, real-time Web development are:
Cross-Document Messaging
To prevent cross-site scripting attacks, browsers enforce a same-origin policy, which prevents documents with different origins from affecting each other. Unfortunately, this security feature also blocks non-malicious communication between pages on different origins. Cross-Document Messaging provides a system that allows documents to securely communicate across different origins.
The postMessage function, as defined in HTML 5, allows you to communicate between documents served by different origins. You call postMessage on the target document’s window, passing the message and the expected origin of the destination, as shown in the following example:
document.getElementsByTagName('iframe')[0].contentWindow.postMessage
('KZNG', 'http://stock.server.com');
In this example, the message KZNG is sent to the document of the first iframe element, expecting the target origin to be http://stock.server.com. The receiving iframe, served by http://stock.server.com, has to add an event listener to handle the incoming events, as shown in the following example:
function onmessage(event) {
if (event.origin == 'http://www.server.com') {
if (event.data == 'KZNG') {
event.source.postMessage('KZNG:1.01', event.origin);
}
}
}
window.addEventListener('message', onmessage, false);
Server-Sent Events
Server-sent events standardizes and formalizes how a continuous stream of data can be sent from a server to a browser. Server-sent events is designed to enhance native, cross-browser streaming. Server-sent events introduces eventsource, a new HTML DOM element that connects to a server URL to receive an event stream as shown in the following example:
<eventsource src="http://stocks.kaazing.com"
onmessage="alert(event.data)">
A connection to the server stocks.kaazing.com is established and when this server sends an event stream, a message event is dispatched on the eventsource element. For example, if the server sends the following event (where \n represents a newline character):
event: message\n
data: {'KZNG': 1.01}\n
\n
Then the eventsource element in the browser dispatches a message event with a data attribute that contains the string {'KZNG': 1.01}. In this case, the stock price for KZNG is received by the client and displayed on the page.
Additionally, if the server includes the id header for an event, then the client adds a Last-Event-ID header when it reconnects, so that the event stream can resume without repeating or missing any events when the HTTP response completes, thus guaranteeing message delivery. The HTTP response can complete normally or else abruptly, if the underlying TCP connection is broken due to some network error. Furthermore, the server can control how long the client must wait before reconnecting by specifying an optional retry header as part of an event in the event stream.
Web Sockets
The HTML 5 Web sockets specification introduces the WebSocket interface, which defines a full-duplex communications channel that operates over a single socket. WebSocket:
- Traverses firewalls and routers seamlessly
- Allows authorized cross-domain communication
- Integrates with cookie-based authentication
- Integrates
with existing HTTP load balancers
A WebSocket connection is established by upgrading from the HTTP protocol to the WebSocket protocol during the initial handshake between the client and the server. Once established, WebSocket data frames can be sent back and forth between the client and the server in full-duplex mode.
Although the WebSocket protocol is ready to support a diverse set of clients, WebSocket cannot deliver raw binary data to JavaScript, because JavaScript does not support a byte type. Therefore, binary data is ignored if the client is Javascript.
WebSocket detects the presence of a proxy server and automatically sets up a tunnel to pass through the proxy. The tunnel is established by issuing an HTTP CONNECT statement to the proxy server, which requests for the proxy server to open a TCP/IP connection to a specific host and port. Once the tunnel is set up, communication can flow unimpeded through the proxy. Since HTTP/S works in a similar fashion, support for SSL is inherent. Server-sent events and Web sockets are not natively supported by modern browsers. Kaazing Gateway to the rescue!
Kaazing Gateway
Kaazing Gateway is a cutting-edge, WebSocket server that enables full-duplex communication from the browser to any TCP-based back-end service (for example, JMS, JMX, IMAP, Jabber, and so on). Kaazing Gateway eliminates the need for complex server- and client-side architectures to bridge various protocols to the browser over HTTP.
This simplified architecture allows developers to code directly against back-end services, using JavaScript, which eliminates the need for complex server- and client-side architectures to bridge various protocols to the browser over HTTP.
Developers simply configure Kaazing Gateway to communicate directly with the back-end APIs (for example, Stomp or XMPP), and develop their client application, allowing them to maintain focus on what matters most: building applications, instead of reinventing the wheel.
Kaazing Gateway Components
Kaazing Gateway consists of the following components:
- Kaazing Gateway Server—A WebSocket server that enables direct TCP connectivity between browsers and back-end services
- Kaazing Gateway Client Libraries—A set of protocol-specific client libraries available for JavaScript
Kaazing Gateway Architecture
To fully appreciate the benefits of the Kaazing Gateway architecture, it is useful to compare it to current real-time Web application architectures. Kaazing Gateway can dramatically simplify and increase the performance of a similar solution.
HTML 5 Server-Sent Events Architecture
Kaazing Gateway supports broadcast notification of data sent from a backend service to all connected clients. The backend data source typically sends data to the Kaazing Gateway over UDP (User Datagram Protocol) and browsers can then receive new notifications using server-sent events. The following figure shows a news feed that sends custom UDP messages to the Kaazing Gateway Server, which in turn fans the packet data out to all connected browser clients using server-sent events.
HTML 5 WebSocket Architecture
Kaazing Gateway is configured to interpret emulated or native TCP calls and pass them directly to any TCP-based back-end service (for example, a JMS broker, a database, an IMAP server, or an instant messaging server). An end-to-end TCP connection is established, as shown in the following Figure, over which messages are sent with minimal overhead.
As illustrated, the browser connects directly to the Kaazing Gateway. The Kaazing Gateway Client Libraries, loaded in the client browser, direct requests immediately to the Kaazing Gateway Server. The Kaazing Gateway Server supports the WebSocket protocol and can also emulate the WebSocket protocol using two HTTP connections—one for upstream communication and one for downstream communication. The downstream connection is implemented using the server-sent events wire protocol, so that native browser server-sent events and WebSocket implementations will work immediately when they become available.
Enterprise HTML 5 WebSocket Architecture
Setting up Kaazing Gateway in an enterprise topology is not much different from the simple setup shown earlier. Kaazing Gateway TCP connections seamlessly traverse firewalls and proxy servers between the different enterprise tiers, ensuring that message delivery guarantees are maintained. Additional hardware load-balancing routers and additional Kaazing Gateway Servers can be configured for high availability and failover.
Kaazing Gateway Features
The following are the key features of Kaazing Gateway:
- Reliability—Message delivery from back-end systems and services is guaranteed all the way from the server to the browser and vice versa in the same way that the HTML 5 Specification guarantees TCP message delivery natively.
- Standards-Compliance—The current implementation of Kaazing Gateway is based on the HTML 5 standard. Once browsers support full-duplex connectivity, per the HTML 5 specification, there is no need to change any server or client code; applications will automatically take advantage of the native implementations of HTML 5's server-sent events and Web sockets with improved performance.
- Binary and Text-Based Protocol Support—While the Web Sockets specification only supports text-based protocols, Kaazing Gateway also supports binary protocols, enabling raw TCP communication between client and server.
- Scalability—Kaazing Gateway is stateless and therefore scales quite easily to enormous amounts of concurrent users. Kaazing Gateway runs separately from application logic hosted in an application server, which mitigates competition for resources.
- Performance—HTTP is a request and response-driven protocol, whereas TCP is centered around the reliable delivery of message data. Header information is sent with each HTTP request and response, which is an unnecessary overhead, and virtually eliminated by the Kaazing Gateway, which favors TCP traffic.
- Client-Side APIs for JavaScript—Kaazing Gateway provides a set of protocol-specific client libraries implemented in JavaScript.
- Low Latency – There is no longer a need for the browser to contact server systems through custom Servlets or complex application server logic; the browser can now send binary data packages and receive an immediate response.
- Cross-Browser Support—Kaazing Gateway supports all major browsers (Firefox version 1.5 and higher, Internet Explorer version 5.5 and higher, Safari version 3.0 and higher, Opera version 9.5 and higher, and Google Chrome version 0.2 and higher).
- Resilient Messaging—Network proxy servers and firewalls are no longer a problem. In case of a broken request, the Kaazing Gateway automatically reconnects, guaranteeing message delivery. Since all connectivity, including downstream server-sent events requests, is client-initiated, Kaazing Gateway can seamlessly traverse firewalls and proxy servers. Additionally, Kaazing's emulation layer automatically honors the browser's proxy settings, eliminating any potential problems with connections that must pass through a Web proxy server.
- Security—Kaazing Gateway supports wire encryption using HTTPS, W3C Cross-Origin Resource Sharing, and HTTP-based authentication and authorization. Furthermore, Kaazing Gateway integrates with JAAS, supporting pluggable authentication modules. See also: Kaazing Security Overview.
- Password Keyring—Kaazing Gateway Client Libraries provide storage to remember the credentials for the backend systems to expedite application startup.
- Management—Kaazing Gateway exposes a set of MBeans that allow administrators to manage and monitor the internal state of the Kaazing Gateway.
- Storage—Kaazing Gateway Client Libraries provides browser session storage and local storage as defined in the Storage section of the HTML 5 specification.
- Proxy Streaming--Proxy servers typically buffer unencrypted HTTP responses introducing unpredictable latency during HTTP response streaming. Kaazing Gateway automatically redirects to use encrypted HTTP traffic to enable HTTP response streaming for server-sent events and WebSocket connections when an HTTP proxy is detected. If the Kaazing Gateway Server has not been configured to support HTTP encryption then long-polling is used as a last resort to guarantee functional correctness.
- Broadcast Notification--Kaazing Gateway supports broadcast notification of data sent from a backend service to all connected clients. The backend data source typically sends data to the Kaazing Gateway over UDP (User Datagram Protocol) and browsers can then receive new notifications using server-sent events.
Understanding the Kaazing Gateway Components
The following is a detailed description of the various Kaazing Gateway components:
Kaazing Gateway Server
Kaazing Gateway Server is a WebSocket server that enables full-duplex TCP connectivity between browsers and back-end services. In case native support for WebSocket is not available in the browser, the socket connectivity between browsers and the Kaazing Gateway Server is emulated using two HTTP connections. The Kaazing Gateway Server proxies communication to the remote server over raw TCP.
Kaazing Gateway Server is based on SEDA (Staged Event-Driven Architecture), and it leverages Java New I/O (NIO) for enhanced Java networking functionality. For example, instead of assigning a single thread per request, the Kaazing Gateway Server shares threads for optimal performance.
Kaazing Gateway Server can serve static HTML content. It can also be used in conjunction with a Web application server. However, if a Web application server is used to serve HTML resources, then the Kaazing Gateway Server must reside on a subdomain of the application server.
Kaazing Gateway Client Libraries
Kaazing Gateway Client Libraries are protocol-specific JavaScript client libraries. Currently, the following client libraries are available:
- PostMessage Client Library
- XMLHttpRequest Client Library
- ServerSentEvents Client Library
- WebSocket Client Library
- ByteSocket Client Library
- StompClient Client Library
- XmppClient Client Library
PostMessage Client Library
The PostMessage client library, available in JavaScript, enables cross-document messaging, such as between a main document and an embedded iframe. If the browser does not support postMessage natively, the library automatically falls back to emulated mode, a feature that allows any browser to enjoy the benefits of cross-document messaging.
The emulated mode of cross-document messaging is implemented using fragment identifier messaging, which uses the fragment part of the URL to transfer data between documents. The sender and the receiving documents have an iframe delegate responsible for receiving incoming messages. Document location fragments have a maximum size of approximately 4 KB, so large messages must be split into multiple fragments.
The document location fragment is write-only, therefore each write must be acknowledged to confirm the receipt before performing the next write operation. Integrity of the communication channel is maintained by having a separate shared secret for each direction of communication.
XMLHttpRequest Client Library
The XMLHttpRequest client library, available in JavaScript, enables cross-origin XMLHttpRequest, leveraging the W3C Cross-Origin Resource Sharing specification. If the browser does not support cross-origin XMLHttpRequest natively, the library automatically falls back to emulated mode.
The emulated mode of cross-origin XMLHttpRequest creates an iframe targeted at the origin of the cross-site request to retrieve the trusted JavaScript emulation implementation. PostMessage is used to communicate with that iframe, which then makes a same-origin XMLHttpRequest request to the originally requested location. The emulation JavaScript sets the effective request origin to the origin of the postMessage event.
According to the specification, a preflight request will be made under specific circumstances to determine if the server allows the browser to make the actual request. For browsers that do not support XMLHttpRequest response streaming, such as Internet Explorer, a text-based iframe is used to stream the response
ServerSentEvents Client Library
The ServerSentEvents JavaScript client library allows clients to connect to any standards-compliant server-sent events stream. If the browser does not support server-sent events natively, the library automatically falls back to emulated mode, a feature that allows any browser to benefit from the server-sent events standard.
In JavaScript, server-sent events emulation is implemented using cross-origin XMLHttpRequest response streaming, which keeps an HTTP response open while listening for additional messages. The server-sent events specification dictates that a connection must be reestablished using a retry delay. By default, this delay is between two and three seconds, but it can be set in the server configuration. While reconnecting, the server inspects the Last-Event-ID HTTP header to ensure the event stream is resumed without repeating or skipping any events.
Because emulation is implemented using a full HTTP response, each message remains in memory even after it is delivered. Therefore, messages must be flushed occasionally, which is accomplished by completing the HTTP response. This requirement to flush-and-reconnect is common to all of today's HTTP streaming solutions that rely on JavaScript—such JavaScript implementations are often referred to as Comet solutions. The Kaazing Gateway Server reestablishes the connection immediately after the messages are flushed in an effort to minimize the degradation of latency, and regardless of the retry delay used. If a network error occurs during the flush-and-reconnect, the regular retry delay is honored before an attempt is made to reconnect again.
WebSocket Client Library
The WebSocket JavaScript client library can be used to open a WebSocket connection to communicate directly to a back-end service using text-based protocols (Jabber, IMAP, and so on). A connection is established by specifying a target server URL. In case the browser does not natively support WebSocket, the library automatically falls back to emulated mode, a feature that allows any browser to benefit from the WebSocket standard.
In the case of client-side emulation, the browser first makes a request to the target server to authorize and create a new WebSocket session. After that, bi-directional WebSocket communication is emulated using Server-sent events for the downstream messages and cross-domain HTTP requests for the upstream messages.
The server-sent events message syntax is also used for upstream message delivery in the emulation. WebSocket's reliable message delivery is thereby guaranteed, because event ids are used and the events are buffered until an acknowledgment message has been received.
ByteSocket Client Library
The ByteSocket JavaScript client library, can be used to open a WebSocket connection that communicates directly to a back-end service using binary protocols, such as AMQP. A connection is established by specifying the URL of a target server.
The ByteSocket implementation is layered on top of WebSocket, and transports bytes as strings using Base64 encoding. The ByteSocket client library enables raw TCP communication between client and server.
StompClient Client library
Implemented using the ByteSocket Client Library, the StompClient client library allows developers to send and receive Stomp (Streaming Text Orientated Messaging Protocol) messages to and from a Stomp-compliant server such as Apache ActiveMQ or RabbitMQ with the rabbitmq-stomp adapter. The StompClient client library is available in JavaScript.
The StompClient client library allows developers to take advantage of the many JMS (Java Message Service) features, like queuing, broadcasting, and publish and subscribe (pub/sub) from JavaScript in the same way that these JMS APIs can be used in Java, using the Stomp-compliant server's JMS client library. This makes JavaScript a first-class citizen in the JMS messaging bus.
Message receipt acknowledgment and other JMS features allow for persistence and reliability and Kaazing Gateway extends that guarantee all the way to the browser, because the implementation is layered on top of the ByteSocket, WebSocket, and ServerSentEvents Client Libraries.
Note: The StompClient client library is layered on top of ByteSocket. Although Stomp is a text-based protocol, the body of a Stomp message can contain bytes, provided a Content-Length header is specified. Therefore, WebSocket, which is text-based, cannot be used to implement a fully-compatible Stomp client, because it would break on non-text body content. Additionally, each Stomp message ends with a null byte, which can cause browsers to ignore the content of the string that follows that null byte.
XmppClient Client library
Layered on top of WebSocket, the XmppClient client library allows developers to send and receive messages to and from an XMPP server. The XmppClient library is available in JavaScript. The XmppClient client library allows JavaScript developers to take advantage of the many XMPP features. To support group chat, the XmppClient client library exposes the XmppRoom JavaScript API.
Because the implementation is layered on top of the WebSocket and ServerSentEvents client libraries, Kaazing Gateway provides accurate buddy lists by extending presence awareness all the way to the browser.
Summary
Until now, achieving bi-directional browser communication, which is vital for real-time Web applications, has been an elusive goal only partially achieved with an assortment of hacks. Techniques such as Comet and Reverse Ajax were developed to allow server-initiated message delivery, but these techniques have many limitations.
Recent updates to the HTML 5 specification, and specifically the definitions of Web sockets and server-sent events have changed the landscape. Developers can now take advantage of a full-duplex communication channel that operates over a single socket, as soon as browser vendors implement native support for this standard.
Kaazing Gateway is an HTML 5-compliant WebSocket server that makes full-duplex binary and text communication a reality today, for all major browsers. It enables browsers to open a socket connection to any TCP-based back-end service (for example, JMS, JMX, IMAP, Jabber, and so on). Therefore, it is now possible to simplify the convoluted Java EE architectures of the past and build applications that directly communicate with back-end services using native protocols over HTTP.
Thus, with the help of Kaazing Gateway and Web sockets, the browser now becomes a first-class citizen of network communications; a benefit that has long been enjoyed only by desktop applications. As a result, Kaazing Gateway brings the promise of migrating the desktop to the Web.
If no native support is available in the browser, Kaazing Gateway falls back to the client-side emulation of the standard automatically. Once browsers support full-duplex connectivity, per the HTML 5 specification, there is no need to change any server or client code; applications will automatically take advantage of the native implementations of HTML 5 with optimal performance.
Therefore, Kaazing Gateway delivers the future of Web application development today, and ensures that your applications are future-proof for the Web of tomorrow. Kaazing Gateway is built for real-time business.