If you are reading this, I’m fairly confident you might have heard, over-heard or unintentionally stumbled on the term gRPC.
With gRPC you can define your service once in a .proto file and implement clients and servers in any of gRPC’s supported languages, which in turn can be run in environments ranging from servers inside a large data center to your own tablet - all the complexity of communication between different languages and environments is handled for you by gRPC.
You also get all the advantages of working with protocol buffers, including efficient serialization, a simple IDL, and easy interface updating.
gRPC-Web lets you access gRPC services built in this manner from browsers using an idiomatic API.
Learn more: https://grpc.io/docs/what-is-grpc/introduction/
The topic is vast and there's a lot to cover. This blog post however is targeted at unravelling the mysteries of grpc-Web.
What is grpc-Web? You may wonder.
In other words, grpc-Web enables your client app/browser app communicating over HTTP 1.1 to successfully communicate to the gRPC services with gRPC working on top of HTTP 2 protocol.
But, how is this possible? Where and how does the protocol shift take place?
The answer lies in the proxy used: in this case Envoy.
gRPC-Web is supported by a filter that allows a gRPC-Web client to send requests to Envoy over HTTP/1.1 and get proxied to a gRPC server.
(More support present in Envoy for gRPC: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_protocols/grpc )
So with those concepts arranged rightly in our head, let's now deploy a gRPC Web application and see the actual packets! Follow along.
Step 1: Deploy grpc-Web services
Follow the steps here: https://grpc.io/docs/platforms/web/quickstart/
>git clone https://github.com/grpc/grpc-web
>cd grpc-web
>docker-compose pull prereqs node-server envoy commonjs-client
>docker-compose up -d node-server envoy commonjs-client
The last command spins up three docker containers: one for envoy, one for the grpc node server and one for common utilities.
Now visit: http://localhost:8081/echotest.html
You'll see the example test app.
Step 2: Send a message "Hello" and intercept the request/response in Burp Suite
First call is to OPTIONS:
The Access-Control-Request-Headers request header is used by browsers when issuing a preflight request to let the server know which HTTP headers the client might send when the actual request is made (such as with setRequestHeader()). The complementary server-side header of Access-Control-Allow-Headers will answer this browser-side header.
Note the values for these headers and you can spot some grpc specific headers:
Access-Control-Request-Headers: content-type,custom-header-1,x-grpc-web,x-user-agent
Access-Control-Allow-Headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
The second call is a POST request to GRPC endpoint
Note that it's HTTP 1.1 , with interesting grpc headers:
Content-Type: application/grpc-web-text
Accept: application/grpc-web-text
X-Grpc-Web: 1
The text "AAAAAAcKBWhlbGxv" is Base64 encoded "Hello"
Now, look at the reponse:
content-type: application/grpc-web-text+proto
grpc-accept-encoding: identity
grpc-encoding: identity
The response value:
AAAAAAcKBWhlbGxvgAAAAmxncnBjLXN0YXR1czowDQpncnBjLW1lc3NhZ2U6T0sNCnNlYy1jaC11YToiIE5vdCBBO0JyYW5kIjt2PSI5OSIsICJDaHJvbWl1bSI7dj0iOTYiDQp4LXVzZXItYWdlbnQ6Z3JwYy13ZWItamF2YXNjcmlwdC8wLjENCnNlYy1jaC11YS1tb2JpbGU6PzANCmN1c3RvbS1oZWFkZXItMTp2YWx1ZTENCnVzZXItYWdlbnQ6TW96aWxsYS81LjAgKFdpbmRvd3MgTlQgMTAuMDsgV2luNjQ7IHg2NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzk2LjAuNDY2NC40NSBTYWZhcmkvNTM3LjM2DQphY2NlcHQ6YXBwbGljYXRpb24vZ3JwYy13ZWItdGV4dA0KeC1ncnBjLXdlYjoxDQpzZWMtY2gtdWEtcGxhdGZvcm06Im1hY09TIg0Kb3JpZ2luOmh0dHA6Ly9sb2NhbGhvc3Q6ODA4MQ0Kc2VjLWZldGNoLXNpdGU6c2FtZS1zaXRlDQpzZWMtZmV0Y2gtbW9kZTpjb3JzDQpzZWMtZmV0Y2gtZGVzdDplbXB0eQ0KcmVmZXJlcjpodHRwOi8vbG9jYWxob3N0OjgwODEvDQphY2NlcHQtbGFuZ3VhZ2U6ZW4tR0IsZW4tVVM7cT0wLjksZW47cT0wLjgNCngtZm9yd2FyZGVkLXByb3RvOmh0dHANCngtcmVxdWVzdC1pZDo1NmE1MWU5OC1hYjlkLTRkNTYtYTE5Mi04NWI1M2ViMDhmMTENCg==
is Base64 decoded as:
hello lgrpc-status:0
grpc-message:OK
sec-ch-ua:" Not A;Brand";v="99", "Chromium";v="96"
x-user-agent:grpc-web-javascript/0.1
sec-ch-ua-mobile:?0
custom-header-1:value1
user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36
accept:application/grpc-web-text
x-grpc-web:1
sec-ch-ua-platform:"macOS"
origin:http://localhost:8081
sec-fetch-site:same-site
sec-fetch-mode:cors
sec-fetch-dest:empty
referer:http://localhost:8081/
accept-language:en-GB,en-US;q=0.9,en;q=0.8
x-forwarded-proto:http
x-request-id:56a51e98-ab9d-4d56-a192-85b53eb08f11
Step 3: What does the traffic look like in Envoy?
Get a shell on the envoy node & run tcpdump.
> tcpdump -vvv -s 0 'tcp port 8080 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
Note the incoming & outgoing request from the proxy:
Note that these are similar values to what we saw in the Burp Suite.
However, one hop is missing, between the first & second log in the image above, the call is converted to HTTP2 and sent over the grpc node server.
How do I know that? Follow the next step.
Step 4: What does the traffic look like in gRPC node?
This is tricky. It might not be intuitive at first but tcpdump won't show you a lot here. HTTP 2 is all about binary, how do you make sense of things you'd see here, because you'd likely see gibberish like this:
Here's a tip, capture the pcap and extract it from the docker container & view it in Wireshark!
> tcpdump -s 0 port 9090 -i eth0 -w grpctraffic.pcap
Then, extract this file to your system to view it in Wireshark.
> docker cp ead29df0b605:/github/grpc-web/net/grpc/gateway/examples/echo/node-server/grpctraffic.pcap .
Now open it in Wireshark! However, note that you'd still not see a lot of details, you'd need to configure your wireshark to understand grpc and http 2.
Follow this to do that: https://grpc.io/blog/wireshark/
Once done, you'd be able to see the traffic in a much more meaningful sense:
Note, this is GRPC over HTTP2.
There are 24 headers in the incoming request to grpc server:
Header: :authority: localhost:8080
Header: :path: /grpc.gateway.testing.EchoService/Echo
Header: :method: POST
Header: :scheme: http
Header: sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="96", "Google Chrome";v="96"
Header: x-user-agent: grpc-web-javascript/0.1
Header: sec-ch-ua-mobile: ?0
Header: custom-header-1: value1
Header: user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36
Header: content-type: application/grpc
Header: accept: application/grpc-web-text
Header: x-grpc-web: 1
Header: sec-ch-ua-platform: "macOS"
Header: origin: http://localhost:8081
Header: sec-fetch-site: same-site
Header: sec-fetch-mode: cors
Header: sec-fetch-dest: empty
Header: referer: http://localhost:8081/
Header: accept-encoding: gzip, deflate, br
Header: accept-language: en-GB,en-US;q=0.9,en;q=0.8
Header: <unknown>:
Header: <unknown>: 9a23e6ee-45f8-42ee-ab18-91d925d9525a
Header: te: trailers
Header: grpc-accept-encoding: identity
Note some of these are populated with values from the HTTP 1.1 request, however some of them are totally HTTP 2 exclusive.
Here's the first look into protobuf message:
Note that it's built up of parts like field number, wiretype, value length, value, as described in the protobuf spec: https://developers.google.com/protocol-buffers/docs/encoding#structure
So, that was the sneak peek into end to end request flow of grpc-Web app. Hope you found it insightful. Should you have any further queries feel free to reach out to me & I'd be happy to engage.
Cheers ~
Comentarios