If you are reading this, I’m fairly confident you might have heard, over-heard or unintentionally stumbled on the term gRPC.
With gRPC you can define your service once in a .proto file and implement clients and servers in any of gRPC’s supported languages, which in turn can be run in environments ranging from servers inside a large data center to your own tablet - all the complexity of communication between different languages and environments is handled for you by gRPC.
You also get all the advantages of working with protocol buffers, including efficient serialization, a simple IDL, and easy interface updating.
gRPC-Web lets you access gRPC services built in this manner from browsers using an idiomatic API.
Learn more: https://grpc.io/docs/what-is-grpc/introduction/
The topic is vast and there's a lot to cover. This blog post however is targeted at unravelling the mysteries of grpc-Web.
What is grpc-Web? You may wonder.
In other words, grpc-Web enables your client app/browser app communicating over HTTP 1.1 to successfully communicate to the gRPC services with gRPC working on top of HTTP 2 protocol.
But, how is this possible? Where and how does the protocol shift take place?
The answer lies in the proxy used: in this case Envoy.
gRPC-Web is supported by a filter that allows a gRPC-Web client to send requests to Envoy over HTTP/1.1 and get proxied to a gRPC server.
(More support present in Envoy for gRPC: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_protocols/grpc )
So with those concepts arranged rightly in our head, let's now deploy a gRPC Web application and see the actual packets! Follow along.
Step 1: Deploy grpc-Web services
Follow the steps here: https://grpc.io/docs/platforms/web/quickstart/
>git clone https://github.com/grpc/grpc-web >cd grpc-web >docker-compose pull prereqs node-server envoy commonjs-client >docker-compose up -d node-server envoy commonjs-client
The last command spins up three docker containers: one for envoy, one for the grpc node server and one for common utilities.
Now visit: http://localhost:8081/echotest.html
You'll see the example test app.
Step 2: Send a message "Hello" and intercept the request/response in Burp Suite
First call is to OPTIONS:
The Access-Control-Request-Headers request header is used by browsers when issuing a preflight request to let the server know which HTTP headers the client might send when the actual request is made (such as with setRequestHeader()). The complementary server-side header of Access-Control-Allow-Headers will answer this browser-side header.
Note the values for these headers and you can spot some grpc specific headers:
The second call is a POST request to GRPC endpoint
Note that it's HTTP 1.1 , with interesting grpc headers:
The text "AAAAAAcKBWhlbGxv" is Base64 encoded "Hello"
Now, look at the reponse:
The response value:
is Base64 decoded as:
Step 3: What does the traffic look like in Envoy?
Get a shell on the envoy node & run tcpdump.
> tcpdump -vvv -s 0 'tcp port 8080 and (((ip[2:2] - ((ip&0xf)<<2)) - ((tcp&0xf0)>>2)) != 0)'
Note the incoming & outgoing request from the proxy:
Note that these are similar values to what we saw in the Burp Suite.
However, one hop is missing, between the first & second log in the image above, the call is converted to HTTP2 and sent over the grpc node server.
How do I know that? Follow the next step.
Step 4: What does the traffic look like in gRPC node?
This is tricky. It might not be intuitive at first but tcpdump won't show you a lot here. HTTP 2 is all about binary, how do you make sense of things you'd see here, because you'd likely see gibberish like this:
Here's a tip, capture the pcap and extract it from the docker container & view it in Wireshark!
> tcpdump -s 0 port 9090 -i eth0 -w grpctraffic.pcap
Then, extract this file to your system to view it in Wireshark.
> docker cp ead29df0b605:/github/grpc-web/net/grpc/gateway/examples/echo/node-server/grpctraffic.pcap .
Now open it in Wireshark! However, note that you'd still not see a lot of details, you'd need to configure your wireshark to understand grpc and http 2.
Follow this to do that: https://grpc.io/blog/wireshark/
Once done, you'd be able to see the traffic in a much more meaningful sense:
Note, this is GRPC over HTTP2.
There are 24 headers in the incoming request to grpc server:
Header: :authority: localhost:8080
Header: :path: /grpc.gateway.testing.EchoService/Echo
Header: :method: POST
Header: :scheme: http
Header: sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="96", "Google Chrome";v="96"
Header: sec-ch-ua-mobile: ?0
Header: custom-header-1: value1
Header: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36
Header: content-type: application/grpc
Header: accept: application/grpc-web-text
Header: x-grpc-web: 1
Header: sec-ch-ua-platform: "macOS"
Header: origin: http://localhost:8081
Header: sec-fetch-site: same-site
Header: sec-fetch-mode: cors
Header: sec-fetch-dest: empty
Header: referer: http://localhost:8081/
Header: accept-encoding: gzip, deflate, br
Header: accept-language: en-GB,en-US;q=0.9,en;q=0.8
Header: <unknown>: 9a23e6ee-45f8-42ee-ab18-91d925d9525a
Header: te: trailers
Header: grpc-accept-encoding: identity
Note some of these are populated with values from the HTTP 1.1 request, however some of them are totally HTTP 2 exclusive.
Here's the first look into protobuf message:
Note that it's built up of parts like field number, wiretype, value length, value, as described in the protobuf spec: https://developers.google.com/protocol-buffers/docs/encoding#structure
So, that was the sneak peek into end to end request flow of grpc-Web app. Hope you found it insightful. Should you have any further queries feel free to reach out to me & I'd be happy to engage.