Introduction
Google Cloud Shell is a product of the Google Cloud Platform (GCP) that provides a number of interesting features, such as an interactive shell environment and an integrated code editor, with the intent of making experimenting and administration of GCP products easier while requiring very little setup.
As part of the Google Vulnerability Reward Program (VRP) I discovered that the code editor of the Cloud Shell was vulnerable to Cross-Site WebSocket Hijacking (CSWSH).
Discovery
When I discovered Google Cloud Shell I became interested in how its integrated editor works. It turned out that the
editor was based on the Theia IDE. I continued to take a look at what requests were sent by the browser when using
Cloud Shell and found out that the editor seemed to be sending requests to a domain of the form
970-dot-123456789-dot-devshell.appspot.com
. Further analysis revealed that the domain name varies for every Cloud
Shell user. Apparently a unique ID is assigned to each Cloud Shell user and the domain
970-dot-$ID-dot-devshell.appspot.com
, which is only accessible by the associated user, is used to address the backend
of that user's Cloud Shell code editor. Each Cloud Shell user is assigned an isolated virtual machine where this and
other Cloud Shell backend services are hosted.
Taking a closer look at the requests being sent by the browser revealed that most of the communication of the editor was done using a WebSocket. When I looked at the messages passed through the WebSocket I realized that inside the socket no authentication seemed to happen.
Figure 1: Messages of the editor component exchanged using a WebSocket.
Having observed the network requests I had seen that a cookie (devshell-proxy-session
) had been set for the domain
970-dot-$ID-dot-devshell.appspot.com
by Google's OAuth process. Another vulnerability, where this cookie, which is
responsible for Cloud Shell's proxy service plays an important role, is described here. At that time I suspected that
the authentication of Cloud Shell's editor might rely only on that cookie. The problem with that, however, is that the
same-origin policy does not apply to WebSocket requests, which means that cookies set for the destination domain would
be included regardless of the origin of the request. This would have made the editor vulnerable to CSWSH.
I decided to verify my assumption by trying to open a WebSocket connection to the Cloud Shell editor's WebSocket endpoint from my browser's console while being on another site. That request failed. While the same-origin policy does not apply for WebSocket requests, the origin of the request is included in WebSocket requests. Apparently, Cloud Shell's proxy service seemed to check the origin of the request and thus prevented CSWSH, so I discarded the idea.
Figure 2: A cross-site WebSocket connection failed to be established.
While I continued investigating how the internals of Google Cloud Shell worked, which is possible since a Cloud Shell
user can escape from the Docker container to the underlying virtual machine, I learned that each Cloud Shell virtual
machine is assigned a public IP address as well as a DNS name of the form devshell-vm-$UUID.cloudshell.dev
, where UUID
differs for each virtual machine instance. It turned out that few ports of the virtual machine were even exposed to the
internet, in particular port 970. It turned out that navigating to https://devshell-vm-$UUID.cloudshell.dev:970/
brought up the same Cloud Shell editor as shown in the main interface of Google Cloud Shell.
Figure 3: The Cloud Shell editor loaded inside Cloud Shell's main interface and using its cloudshell.dev
subdomain.
Sometime later, I decided to try if the endpoint https://devshell-vm-$UUID.cloudshell.dev:970/
was vulnerable to
CSWSH. Having just requested a new virtual machine which was assigned the DNS name devshell-vm-$UUID2.cloudshell.dev
I decided to reuse a browser window which still had the Cloud Shell editor of the previous virtual machine
(https://devshell-vm-$UUID1.cloudshell.dev:970/
) loaded. Like previously, I used the browser's console to try to
establish a connection to the editor's WebSocket endpoint (wss://devshell-vm-$UUID2.cloudshell.dev/services
). This
time, the connection was established. Apparently, CSWSH could be performed when using subdomains of cloudshell.dev
.
Figure 4: A cross-site WebSocket connection established successfully.
Exploitability
Having found that CSWSH worked I went on to determine the potential impact of the vulnerability and find a plausible attack scenario.
Impact
The impact of a potential attack is determined by the harm a potential attacker can cause, which is limited by what can be controlled using the hijacked WebSocket. Researching the details of the Theia IDE revealed that the editor also included a shell. While this shell is disabled in Google Cloud Shell's version of the editor, I found out that it can still be spawned and used by sending the appropriate messages to the editor's WebSocket endpoint. Consequently, a potential attacker could execute arbitrary commands on the assigned user's virtual machine and even obtain access to a Cloud Shell user's GCP resources as the virtual machine is automatically authorized to access these resources.
Attack scenario
Successfully exploiting the described CSWSH vulnerability would require an attacker to have knowledge of a Cloud Shell virtual machine's IP address or DNS name, thus limiting a potential attack to targeting a specific Cloud Shell user while using a specific Cloud Shell virtual machine instance. In most cases, this would require potential attackers to those being on the same network as the Cloud Shell user. After obtaining the required IP address a potential attacker would then have to convince the targeted Cloud Shell user to visit a malicious website controlled by the attacker, which could perform CSWSH against the Cloud Shell virtual machine at the obtained IP address.
While trying to create a proof of concept I was confronted with two problems:
- I was not able to perform CSWSH successfully from any other domain than a subdomain of
cloudshell.dev
. That implied that the malicious page would have to be served from a subdomain ofcloudshell.dev
. - In contrast to the endpoint
970-dot-$ID-dot-devshell.appspot.com
, at which the Cloud Shell editor is usually accessed, a Cloud Shell user is not automatically authenticated to the endpointhttps://devshell-vm-$UUID.cloudshell.dev:970/
when connecting to their Cloud Shell instance. This proofed to be a problem as the missing authentication cookie meant that no WebSocket connection would be established.
To tackle the first problem I came up with the idea that a potential attacker could repurpose their own Cloud Shell
virtual machine to serve the malicious page that would perform the CSWSH. Since each Cloud Shell virtual machine is
assigned a cloudshell.dev
subdomain this means that the WebSocket connection would have been established from an
eligible origin. While accessing endpoints such as https://devshell-vm-$UUID.cloudshell.dev:970/
usually requires
authentication using a cookie, which could render serving the malicious page problematic, a potential attacker could
inhibit that authentication requirement on their own virtual machine due to the ability to escape from the Docker
container to the underlying virtual machine and completely control it.
Trying to understand the authentication mechanism of endpoints at devshell-vm-$UUID.cloudshell.dev
, I observed that
when requesting a URL such as https://devshell-vm-$UUID.cloudshell.dev:970/index.html
without any authentication token
I would be forwarded to Google's OAuth process. Upon completion, I would be taken to
https://devshell-vm-$UUID.cloudshell.dev/_cloudshell/login
where the authentication cookie would be set and eventually
I would be redirected back to https://devshell-vm-$UUID.cloudshell.dev:970/index.html
, the URL I had initially
requested. Further experiments revealed that the URL to be used for redirection in that last step was stored when the
initial request was made. However, sending a request with a spoofed host header in the initial request would result in
being redirected to that spoofed host in the final step. I had discovered an open redirect vulnerability, which solved
the second problem described above, as it would allow a potential attacker to send a Cloud Shell user through Google's
OAuth process to have the cookie set, but eventually redirect them to another page controlled by the potential attacker.
A potential attack scenario can thus be summarized as follows:
- The attacker, wanting to perform CSWSH on the Cloud Shell user associated with
devshell-vm-$UUID2.cloudshell.dev
, obtains a Cloud Shell instance of their own,devshell-vm-$UUID1.cloudshell.dev
. They inhibit the authentication requirement on their own virtual machine and place a malicious page performing CSWSH athttps://devshell-vm-$UUID1.cloudshell.dev:970/page.html
. - The attacker makes a request to
https://devshell-vm-$UUID2.cloudshell.dev:970/page.html
with a spoofed host header ofdevshell-vm-$UUID1.cloudshell.dev
. Not having an authentication token, they will be redirected to Google's OAuth process. - Instead of following the redirection in the previous step the attacker somehow persuades the other Cloud Shell user
into following that redirection link. This will take the user through the OAuth process, set the required cookie and
eventually forward them to the malicious page at
https://devshell-vm-$UUID1.cloudshell.dev:970/page.html
to perform CSWSH successfully.
Fix
After creating a proof of concept for the described attack scenario and reporting the vulnerability to Google it was fixed. This was done by having the virtual machine's server compare the origin header of WebSocket requests with their own host name and reject requests where these would not match.
Take away messages
- When using WebSockets, special care must be taken with respect to authentication. As WebSocket requests do not adhere to the same-origin policy, relying on cookies alone for authentication is not sufficient. Either authentication using the origin header or using a custom protocol performed once the WebSocket is opened are possible options.
- When integrating components, such as the Theia IDE, into larger projects, such as Google Cloud Shell, it is necessary to check whether the component's security measures are also adequate within the context of the larger project or customization is required.
Further information
WebSocket security
- an article about CSWSH by @cschneider4711
- another article about CSWSH by Jesse Somerville
- an article about securing WebSocket endpoints by Eero Helenius
- a post about attacks on WebSockets by Daniel
Google Cloud Shell architecture and further vulnerabilities in Google Cloud Shell
- a video by @LiveOverflow about the architecture and a vulnerability found by @wtm_offensi
- architecture overview and four more vulnerabilities by @wtm_offensi
- information about escaping from Docker container to virtual machine by @SpenGietz
- unrelated vulnerabilites in Google Cloud Shell