Chromecast: Understanding its Potential beyond Video Streaming

Google launched the $35 Chromecast into the US market in July 2013 and it became available in the UK in March 2014.  It’s primarily marketed as a device for streaming video to your TV, and, as such, is usually compared with the Roku streamer, Apple TV and the new Amazon Fire TV.

Once you look under the covers, you discover that such comparisons are misleading: unlike the other devices, the Chromecast is actually much more than just a streaming device for TV.  This became clear when Google launched their Software Development Kit (SDK), allowing developers to build their own custom applications for the Chromecast.  Having had my hands on one since almost the day they became available in the UK, I’ve come to think that it’s one of the most exciting and interesting devices of recent times, and I believe it has huge potential within many market sectors, in particular healthcare.  From the reviews and articles that have been written about the Chromecast, I’ve also concluded that it’s a very poorly-understood piece of technology: I think it’s partly Google’s fault – they deliberately focus attention on its role as a video streamer to the extent that few people have thought to understand how it really works and what it’s really capable of doing.

In this article I’ll explain what’s actually inside the Chromecast, how it works and how applications run on it: you’ll hopefully start to understand why I’m so excited about the Chromecast.  Needless to say, I’ve already extended EWD.js to allow anyone to very rapidly develop Chromecast applications, but more about that in a future article!

The Chromecast is basically a device that runs a stripped-down version of ChromeOS (some say it’s actually more a stripped-down Android OS) and a Chrome browser.  Other than that it has:

• an HDMI output that can deliver the output from Chrome to an HDMI-interfaced TV, monitor or projector;
• a wireless network interface.

So a Chromecast device can run any page of HTML plus JavaScript and CSS that you could run in any Chrome browser.  The difference is that the Chromecast has no direct means of controlling Chrome’s user interface (UI).  Instead, pages are loaded into its Chrome browser indirectly via what’s known as a Sender application, and the UI of the page that’s loaded into the Chrome browser is also controlled via the Sender application.  A Sender application is simply another application that runs on a device that does have a means of controlling its UI.  Google’s Chromecast SDK supports three classes of Sender application that run in:

• Android devices
• iOS devices
• Chrome browsers (eg on a Windows, Mac OS X, Linux or ChromeBook device)

In this article I’m going to focus on the latter, but the principles are similar for Android and iOS devices.  The difference is that the Chrome browser approach allows everything to be done in HTML and JavaScript which, of course, lends itself to using EWD.js.

So, a typical environment for the Chromecast is shown below:

As you can see, the one key additional thing you need to do is install the free Chromecast Extension for Chrome on the machine that you’ll be using for your Sender application(s).  The first step, then is to load the Sender application into Chrome by specifying its URL.  The one requirement is that the Sender application HTML page must load the Chromecast APIs from Google’s web site:

The JavaScript that is specific to the Sender Application you load must specify a Chromecast Receiver Application Id.  A Receiver application is a pre-built “container” page that will run in the Chromecast.  Google provides several pre-built generic ones that are used primarily for video streaming.  However, a registered Chromecast Developer can create a Custom Receiver application.  When such an application is registered and made public using the Google Cast SDK Developer Console, it is allocated a unique ID and any Chromecast device can be instructed to load it by a Sender Application.  I’ve created one for use with EWD.js: its Id is CCBCB93C.

The Chromecast SDK Sender APIs include the function chrome.cast.requestSession().  When this is invoked, the Chromecast Extension opens a pop-down window within Chrome showing all Chromecast devices that are available on the wireless network:

When you click on the Chromecast device you want to use, a message is sent to the Chromecast device to load the Receiver application with the specified Id (in this case CCBCB93C). 

A Receiver application is nothing more than a page of HTML, and the Registered Id simply maps to the URL at which that page resides.  In the case of the EWD.js Chromecast Receiver application, that URL is https://ec2.mgateway.com:8080/ewd/chromecastrx/index.html.  You can, of course, try loading it into any browser, but it won’t do much unless it’s loaded into the Chrome browser in a Chromecast device (actually it is possible to get any Chrome browser to emulate a Chromecast device – itself something that has interesting possibilities).

As you’ll see if you inspect the source of the EWD.js Receiver application page, typically, a Chromecast Receiver application is simply:

• a page into which the Chromecast Receiver JavaScript APIs are loaded
• a “container” page that includes an iframe

The piece of magic that now happens is that the Sender and Receiver applications establish a private WebSocket connection across the wireless network:

The idea, then, is that the Sender application can now act as a proxy UI for the application that runs in the Chromecast, with instructions signalled via WebSocket messages. 

Normally the first message that is sent from the Sender application instructs the Receiver application to set the src attribute of the iframe, causing it to load the actual application that you want to run in the Chromecast.  When you use a Chromecast to watch YouTube videos or Netflix films, this is basically what has happened: the YouTube or Netflix-specific HTML video-streaming application is loaded into the generic Receiver application’s iframe.  In our case, however, the WebSocket message from the Sender application instructs the Receiver application to load an EWD.js application into the iframe:

The really important thing to understand about applications that run within the Chromecast is that they run completely within the Chromecast and can communicate over the Internet.  So, if you’re using a Chromecast for streaming a Netflix film, it’s streaming it directly to the Chromecast: the Sender application’s role is simply one of acting as a UI controller for the application within the Chromecast, eg telling it to start, pause or stop the streaming video.  In other words, the streaming does not take place via the Sender application.

So, if you’re running an EWD.js application on a Chromecast, what you end up with is the following environment:

Both the Sender and Chromecast applications are standard EWD.js applications, communicating via WebSocket messages with their corresponding remote Node.js-based back-ends.  Meanwhile, the Sender and Receiver applications can communicate with each other via WebSocket messages across the local wireless network (and note that this is two-way communication: the Receiver application can send messages back to the Sender application).  Meanwhile, within the Chromecast, the Receiver application “container” can communicate in a two-way fashion with the application running within its iframe using postmessage messaging.

Of course, the Sender and Chromecast applications could be hosted by the same EWD.js/Node.js back-end, and this could be connected to an application such as VistA:

Potential Applications

At this point you might be thinking: “All very interesting, but so what?”  Well, let’s think about what this is providing us with: the $35 Chromecast turns any available TV or HDMI-interfaced monitor into a device that can run a fully interactive application without that TV or monitor needing to be physically directly connected to a computer.  The application that is now running in the Chromecast and being displayed on the TV or monitor can be controlled remotely from another Sender device: a PC, tablet or phone.

So, for example, any TV that’s mounted on a wall in a clinic or hospital can be used as an interactive display for an EWD.js application.  A patient could be shown information during a consultation.  Information about the hospital could be displayed on TVs or monitors mounted around the hospital or clinic without the need for them being wired to dedicated computers, and/or replacing expensive proprietary displays.  The content and its styling and format is completely under the control of the hospital or clinic and easily defined and modified: it’s just standard HTML, JavaScript and CSS!

Furthermore, the WebSocket-based communication of EWD.js applications means that real-time information can be sent to the application running in the Chromecast: so life-signals could be sent in real-time to the Chromecast application, updating a graph or chart in real-time without any polling.

Think further “out of the box” and consider the Sender application’s role as a proxy UI controller for the Chromecast application.  Instead of using a keyboard and mouse, what about the inexpensive LeapMotion controller?  Now the Chromecast application can be controlled by hand gestures, once again all developed using JavaScript via the LeapJS APIs.  In fact, if you have a Chromecast and a LeapMotion controller, you can try this idea out for yourself now.  Simply install the Chromecast Extension on your Chrome browser on your PC or Mac and launch the following URL to load a demonstration Sender application:

http://ec2.mgateway.com/ewd/chromecast/index.html

Login with a username of ‘rob‘ and a password of ‘secret‘.

Load the Chromecast application by clicking the Green Power On button.  Once loaded into the Chromecast, you’ll see a new set of buttons in the Sender application, one of which turns on the LeapMotion controller interface.  Move one hand side to side and up and down and watch the pointer on both the Sender and Chromecast applications (Note: I’m trying to get LeapMotion to fix a bug that currently stops the controller interface working properly if you stop and then restart the LeapMotion interface).  To stop the Chromecast application, just click the Red Power Off button on the Sender application’s UI and the Receiver application automatically unloads itself from the Chromecast.

It’s a pretty simple demo, but consider some potential applications of these techniques:

• turning any TV or monitor into a display for bringing up patient imaging, eg X-Rays, moving them around using swipe gestures (yes, in true Minority Report style!)
• assessing a patient’s dexterity or hand tremors and displaying the results in real-time on a TV or monitor for the patient to also see (thanks to Steve Owen for this idea!)
• All done using a $35 Chromecast and a $90 LeapMotion controller!

Developing your own EWD.js Chromecast Applications

Hopefully I’ve now whetted your appetite and you’re now interested in developing your own applications for the Chromecast!  If so, in my next post I’ll introduce the new Chromecast extensions to EWD.js that makes it a piece of cake: you’ll be able to build sophisticated Chromecast applications in just a matter of a few hours.  You won’t even need to be a registered Chromecast developer: just use my registered EWD.js Receiver application and you can do the rest using standard EWD.js coding.  And if you’re a VistA user, start thinking how its power and functionality can be further enhanced by using the Chromecast.

All the power of EWD.js, coming to a TV near you!