Presented in HotOS 2021.
Authors: Ion Stoica, Scott Shenker
Very exciting paper! This paper suggests steps we can take to overcome this differentiation and help create a more commoditized version of cloud computing, i.e., sky computing. The barriers are more economic than technical.
What is cloud computing? Users have access to massive amounts of computation and storage and charged only for the resources they use.
Cloud providers strive to differentiate themselves through proprietary services.
Previous designs of sky computing focus on particular technical solutions and target specific workloads.
What we need in sky computing:
- A compatibility layer: mask low-level technical differences.
- An intercloud layer: route jobs to the right cloud.
- A peering layer: allow cloud to have agreements with each other about how to exchange services.
Most users interact mostly with higher level management and service interfaces. A growing number of them are supported by open source software (OSS).
On purely technical grounds, achieving a widely usable compatibility layer is easily achievable.
While the compatibility layer has clear benefits for users, the cloud providers may not be interested.
Akin to requiring an Internet user to explicitly select the AS paths for its interdomain traffic.
The intercloud layer must allow users to specify policies about where their jobs should run, but not require users to make low-level decisions about job placement.
Today, most clouds have pricing policies where moving data into a cloud is much cheaper than moving it out. It creates a strong incentive for users to process data in the same cloud in which it currently resides. But in some cases moving jobs is still worthwhile.
Why not clouds enter into reciprocal data peering arrangements, where they agree to allow free exporting of data to each other and to connect with high-speed links?
Once a compatibility layer and an intercloud layer are in place, cloud providers will fall into two categories:
- Stand-alone cloud providers
- Lock customers in with proprietary interfaces and data export fees.
- Large enough to offer a variety of proprietary services.
- Sky cloud providers
- Directly support the compatibility layer.
- Agree to reciprocal data peering with other commodity cloud providers.
- Specialize in supporting one or more services.
Some thoughts from the authors:
- In the long term, we will continue to have both kinds of providers.
- Smaller cloud providers are more expected to embrace the compatibility layer.
- The collective sky can counterbalance the large proprietary clouds and allow them to focus their innovation efforts more narrowly.