논문 제목: Snape Reliable and Low-Cost Computing with Mixture of Spot and On-Demand VMs
시작 시간: 2025-07-21

单词:
Reliable 可信赖的；可靠的
Cost 成本
Mixture 混合的
Demand 需求，需要， On-Demand 按需使用

语法:
“with + 名词” 结构的几种常见用法：
表示工具 / 手段 / 方式（最常见）
- Cut the paper with scissors.（用剪刀剪纸）
- Computing with a mixture of VM types（使用多种 VM 类型进行计算）

表示伴随、携带
- He came with his friends.（他和朋友一起来的）

表示特征 / 状态
- A box with a red ribbon（一个有红色丝带的盒子）

引导伴随状语（更复杂的结构）
- With everyone watching, she began to sing.（在大家都注视的情下，她开始唱歌）

为什么不能用 by？

ABSTRACT 概要

Cloud providers often have resources that are not being fully utilized,
单词：
often 时常（60%):
always 总是，一直（100%）
usually 通常(80%)
frequently 频繁的(70%)

utilized 利用

语法：

be（is/are）+ being + 过去分词（utilized）

例句：The system is being updated.（系统正在被更新。）

主动句（主动进行）They are using the server.（他们正在使用服务器）

被动句（被动进行）The server is being used.（服务器正在被使用）

that 引导定语从句：修饰前面的名词。

例句：I saw a movie that made me cry.（我看了一部让我哭的电影。）

https://www.youtube.com/watch?v=bIAFcgs59Ys&list=PLzraVIBodoVCGBnDE-qGPvl9usAaUcvEo&index=2

✅ 用现在进行时（你现在这样写的）：
The people that are working in the office are very friendly.
强调“正在工作中的那些人”。

✅ 用一般现在时（更常见）：
The people who work in the office are very friendly.
= 在办公室工作的人（不是强调“此时此刻”，而是“平时/通常”的情况）

你的句子：
❌ The woman who live in downstairs is a docter

✅ 正确句子：
The woman who lives downstairs is a doctor.

我要嫁一个我爱的人。
I want to marry with the people who loves me

✅ 推荐改法（说“一个人”）：

I want to marry the person who loves me.

I want to marry with the people who is my love

✅ 更自然的改法：

I want to marry the person who I love.
👉（我想嫁给我爱的人）
I want to marry someone I love and who loves me. 👉（我想嫁给我爱的人，也爱我的人）

and they may offer them at a lower cost to make up for the reduced availability of these resources.
为了弥补这些资源可用性降低的情况，他们可能会以更低的成本提供这些资源。

单词，短语：

看看这个结构：

the reduced availability of these resources

availability = 可用性（名词）

of these resources = 修饰 availability，表示“这些资源的可用性”

这就像是定语从句：

the availability that these resources have
（这些资源拥有的可用性）

但英语里更常用 名词 + 介词短语 的结构来表达这个意思。
比如：

and they may offer these resources at a lower cost to make up for the reduced availability of them. 不是更合适吗

ChatGPT 说：

你的句子语法上是正确的，但在地道的英语表达中，结尾的 “of them” 实际上是 不太自然的。更自然的表达应该是：

✅ ...to make up for the reduced availability of these resources.

为什么 “of them” 不如 “of these resources” 更合适？

1."of them" 是代词 + 介词结构，但在英语中：

• 当先前的名词距离较远、或想让句子更清楚、强调意思时，
• 直接重复那个名词（哪怕之前已经出现）反而更清晰自然。
  ✅ 举例：

There is a shortage of nurses.
Hospitals are increasing wages to make up for the lack of nurses.
❌ Not: to make up for the lack of them.

2.代词 "them" 在 of 后面容易模糊意思
特别是在表达“...的可用性”时：

• availability of them ← 不清晰，“他们的可用性”还是“它们的”？
• 而 availability of these resources ← 明确，就是“这些资源的可用性”。

所以最地道的句子版本是：

...and they may offer these resources at a lower cost to make up for the reduced availability of these resources.

但这又稍微有点重复，所以最自然的写法通常是：

✅ ...and they may offer them at a lower cost to make up for the reduced availability of these resources.

分词作定语是英语中很常见的语法结构：

However, customers may be hesitant to use such offerings (such as spot VMs) as making trade-offs between cost and resource availability is not always straightforward.
然而，客户可能会对使用此类产品（例如 spot 虚拟机）感到犹豫，因为在成本和资源可用性之间做出权衡并不总是那么容易。

单词：

In this work, we propose Snape (Spot On-demand Perfect Mixture), an intelligent framework to optimize the cost and resource availability by dynamically mixing on-demand VMs with spot VMs.
在本研究中，我们提出了 Snape（全称为 Spot On-demand Perfect Mixture），这是一个智能框架，旨在通过动态地混合按需虚拟机和抢占式虚拟机来优化成本与资源可用性。

单词：

Through a detailed characterization based on real production traces, we verify that the eviction of spot VMs is predictable to some extent.
基于真实生产环境的运行轨迹进行详细特性分析后，我们验证了 spot 虚拟机的驱逐在某种程度上是可预测的。

✅ 这个短语 没有用定语从句，而是用了 过去分词短语（非谓语动词）作定语，修饰名词 "characterization"。语义等价于定语从句的简化形式。

Snape also leverages constrained reinforcement learning to adjust the mixture policy online.
Snape 还利用了受限强化学习，在运行时动态调整资源混合策略。

Experiments across different configurations show that Snape achieves 44% savings compared to using only on-demand VMs while maintaining 99.96% availability, which is 2.77% higher than using only spot VMs.
在多种配置下的实验表明，Snape 相比仅使用按需 VM 可节省 44% 成本，同时维持 99.96% 的可用性，这比仅使用 spot VM 高出 2.77%。

1 INTRODUCTION
Large cloud providers typically ensure that their services are always available and reliable by allocating a sufficient amount of computing resources, so that it appears to customers that resources are never in short supply.
大型云服务提供商通常会通过分配足够的计算资源，来确保其服务始终可用且可靠，从而让客户感觉资源从未短缺。

为什么用 typically 不用 usually frequently？

Computing with a mixture of VM types. by allocating a sufficient amount of computing resources 为什么上面用with下面用by?

✅ 更直观的类比：

• I cook with a wok. （我用炒锅炒菜 → 工具）
• I improved the dish by adding soy sauce. （我通过加酱油让菜更好吃 → 方法）

✨ 总结：

• with 说明“你使用了什么来做这件事”（强调参与或成分）；
• by 说明“你通过什么动作/手段实现了目的”（强调过程方式）。

This often leads to underutilized resources, and cloud providers are motivated to monetize excess capacity by selling them at discounted prices to compensate for the degradation in resource availability.
这常常导致资源未被充分利用，云服务提供商因而被激励将多余容量进行变现，通过以折扣价出售这些资源，以补偿资源可用性下降所带来的影响。

Example offerings include spot virtual machine (VM), spot block VM and harvest VM.
示例服务包括：抢占式虚拟机（spot VM）、块状抢占式虚拟机（spot block VM）以及收割型虚拟机（harvest VM）。

The trade-off between cost-effectiveness and resource availability often creates a dilemma for users.
在成本效益与资源可用性之间的权衡，常常让用户陷入两难。

On the one hand, some workloads, such as machine learning training and inferencing, caching, and big data processing, can potentially run at reduced reliability for lower cost.
一方面，一些工作负载（如机器学习训练与推理、缓存、大数据处理）可以在较低的可靠性下运行，以换取更低的成本。

On the other hand, cloud providers, while sharing some information about resource availability such as estimated eviction rates for spot VMs at the region level, do not guarantee resource availability.

另一方面，尽管云服务提供商会共享一些关于资源可用性的信息（例如区域级别的 spot 实例预估回收率），但他们并不保证资源可用性。

Figure 1 shows an incident caused by the unexpected high evictions of spot VMs.
图 1 展示了一起由 Spot 虚拟机意外大规模回收所引发的事件。

For months, prior to the incident, the eviction rate had remained stable, leading service owners to believe it would be safe to use spot VMs with some level of overprovisioning based on the estimated eviction rate.
在该事件发生之前的几个月里，Spot 实例的回收率一直保持稳定，这导致服务拥有者相信，只要基于估算的回收率做一些冗余配置，使用 Spot 实例是安全的。

However, around 4am, the eviction rate suddenly increased, resulting in the eviction of approximately 60% of spot VMs.
然而，在凌晨四点左右，Spot 实例的回收率突然上升，导致大约 60% 的 Spot 虚拟机被回收。

Although there was no noticeable impact at the time due to low traffic, the service health began to drop around 3pm as traffic increased.
尽管当时由于流量较低并未产生明显影响，但随着流量在下午三点左右上升，服务健康状况开始下降。

Figure 1: A real service incident due to unexpected high eviction rates of spot VMs (metrics are normalized and time is shifted to UTC).
图 1：一次由于 Spot 虚拟机意外高回收率引发的真实服务事故（所有指标已归一化，时间已转换为 UTC）。

One natural solution to prevent similar incidents as the above is to deploy services in a VM Scaling Group (VMSG) with both spot and on-demand VMs, so that there would still be enough on-demand VMs to support services even if all spot instances were evicted.
为了防止上述类似事件，一种自然的解决方案是在 VM 扩展组（VMSG）中部署服务，混合使用 Spot 和 On-demand 实例，这样即使所有 Spot 实例都被回收，也仍有足够的 On-demand 实例来支撑服务。

Cloud providers have begun providing such offerings recently as described in Section 2.1.
如第 2.1 节所述，云服务提供商近来也已开始提供这类服务。

However, all of them either support a static mixture ratio of spot and on-demand VMs or allocate spot instances in a greedy fashion, which can be less cost-effective when the eviction rate is low.
然而，它们要么仅支持静态的 Spot 和 On-demand 实例配比，要么以贪婪方式分配 Spot 实例，这在回收率较低时可能成本效率较差。

To better balance cost and resource reliability, this work proposes an intelligent framework Snape (Spot On-demand Perfect Mixture), which dynamically adjusts the number of on-demand and spot VMs by leveraging the fact that the eviction of spot VMs is predictable to some extent.
为了更好地平衡成本与资源的可靠性，本文提出了一个智能框架 Snape（Spot 与 On-demand 的完美混合）。该框架可以动态调整 Spot 和 On-demand 实例的数量，方法是利用这样一个事实：Spot 实例的回收在某种程度上是可以预测的。

As the first step to demonstrate the effectiveness of Snape, we introduce a new VMSG in which the number of in-service VMs is equal to or greater than a specified target number that meets a service level objective (SLO), while allowing the number of spot and on-demand VMs to vary for optimized cost saving.
为了展示 Snape 的有效性，我们首先引入一个新的 VM 扩展组（VMSG）。
在这个组中，正在运行的虚拟机数量始终大于或等于一个预设目标值，该目标满足服务水平目标（SLO）；同时，Spot 和 On-demand 实例的数量可以灵活变化，以实现成本的最优化节省。

We defined the VMSG SLO in Section 3.2 regarding the availability of VMs. Customers provide both the target number of in-service VMs and the Service Level Objectives (SLOs). SLOs can be less strict (e.g., three nines) than those for on-demand VMs (four or five nines).
我们在第 3.2 节中定义了 VMSG（虚拟机扩展组）的服务水平目标（SLO），其重点在于虚拟机的可用性。
客户需要提供两项内容：目标运行虚拟机的数量，以及期望的服务水平目标（SLO）。
相较于按需虚拟机通常要求的更高可用性（例如四个九或五个九），这类 SLO 通常可以设定得较为宽松（例如三个九）。

Obtaining an optimal mixture plan is challenging on several aspects:
获得一个最优的混合策略（即 Spot 与 On-demand 实例的组合）在多个方面都面临挑战：
(1) To predict the evictions of spot VMs, the various factors leading to eviction at both global and local levels need to be studied.

1. 预测 Spot 实例的回收行为非常困难，因为必须研究多个导致回收的因素，这些因素既包括全局层面也包括本地层面。
   (2) The uncertainty associated with eviction rate prediction makes tracking the SLO over time complex; optimal mixture decisions would depend on the available SLO budget and the level of uncertainty in the prediction.
2. 回收率预测中存在的不确定性使得在时间上持续满足 SLO 变得复杂；而最优的混合决策往往依赖于可用的 SLO 预算，以及预测中不确定性的程度。
   (3) A less-optimized “myopic” decision could be made if only short-term eviction prediction results are considered. This can arise when the eviction rate is fast-varying.
3. 如果仅考虑短期预测结果，可能会做出次优的“短视型”决策（myopic decision）；这种情况常见于回收率变化剧烈的场景中。

To address these challenges, we first characterize the eviction behaviors of spot VMs by examining traces collected over three months from a leading cloud provider in Section 3.
为了解决这些挑战，我们首先在第 3 节中，通过分析来自一家领先云服务提供商、为期三个月的使用追踪数据，刻画了 Spot 实例的回收行为特征。

For example, we find that the eviction rate is higher on weekdays and working hours, and spot VMs are more likely to get evicted if collocated with a few on-demand VMs.
例如，我们发现：Spot 实例的回收率在工作日和工作时间更高；并且如果它们与少量按需实例部署在一起，被回收的概率也更高。

Furthermore, to better learn the eviction behaviors and achieve optimal decisions in the long term, we employ the framework of constrained reinforcement learning (CRL) and discuss how CRL can be applied to the current problem.
此外，为了更好地了解 Spot 实例的回收行为，并在长期内实现最优决策，我们采用了受限强化学习（CRL）框架，并探讨了如何将其应用于当前问题。

The overall architecture of Snape and its components are introduced in Section 4, followed by a detailed evaluation in Section 5.
Snape 的整体架构及其组件将在第 4 节中介绍，随后在第 5 节进行详细评估。

Our contributions can be summarized as follows:
• To the best of our knowledge, we are the first to conduct a detailed characterization of the eviction behavior of spot VMs by analyzing real production traces from Microsoft Azure.
• We design a constrained reinforcement learning framework, Snape, which can dynamically adjust the mixture number of on-demand and spot VM to maximize cost savings as well as satisfy the specified SLO provided by users.
• We implement an example VM scaling group under the framework of Snape. We validate its effectiveness in experiments that Snape achieves 44% savings than on-demand VMSG while retaining 99.96% availability.
我们的贡献总结如下：

1. 据我们所知，我们是第一个通过分析 Microsoft Azure 的真实生产追踪数据，详细刻画 Spot 虚拟机回收行为的研究者。
2. 我们设计了一个受限强化学习（CRL）框架 Snape，它可以动态调整按需实例和 Spot 实例的混合比例，以最大化成本节省，同时满足用户设定的 SLO（服务水平目标）。
3. 我们在 Snape 框架下实现了一个示例虚拟机扩展组，并在实验中验证了其有效性：Snape 相较于仅使用按需实例的 VMSG 实现了 44% 的成本节省，同时保持了 99.96% 的可用性。

2 BACKGROUND
2.1 VM Scaling Groups
VM Scaling Group (VMSG) has been offered by all major cloud providers though under different names, namely, AWS ASG, GCP MIG, and Azure VMSS.
A scaling group consists of a group of VMs managed by an autoscaler based on a specific rule (e.g., a predefined schedule) or target metrics such as CPU utilization, request per second, or predicted workload.
虚拟机扩展组（VMSG）已经被所有主流云服务提供商提供，尽管名称不同，例如 AWS 的 ASG、GCP 的 MIG 和 Azure 的 VMSS。
一个扩展组由一组虚拟机组成，这些虚拟机由自动扩展器（autoscaler）管理，该管理可以基于某些规则（例如预定义的调度）或某些目标指标（如 CPU 使用率、每秒请求数或预测负载）来进行。

In its early days, a VMSG was typically composed of homogeneous VMs, i.e., with the same SKU size and in the same mode (spot or on-demand).
在其早期阶段，虚拟机扩展组（VMSG）通常由同构的虚拟机构成，即具有相同的 SKU 规格并运行在相同模式（spot 或 on-demand）下。

Recently, clouds (e.g., Azure and AWS) introduced mixed VMSGs which can flexibly orchestrate VMs of different sizes and modes.
The goal is to benefit from the discount brought by spot instances while fulfilling the basic resource requirement with the on-demand instances.
近年来，云服务提供商（如 Azure 和 AWS）引入了混合型 VMSG，可以灵活编排不同大小和模式的虚拟机。
其目标是：在使用按需实例满足基础资源需求的同时，利用 spot 实例带来的成本折扣。

The user is required to configure the number of minimum on-demand instances and the mixture ratio.
The autoscaler is responsible for maintaining the proportion of each VM type.
用户需要配置最小按需实例的数量以及混合比例。
而自动扩展器（autoscaler）则负责维持每种虚拟机类型的比例。

For example, if the minimum number of on-demand instances is 4 and the mixture ratio is 0.5, then when the scaling group size is 10, there are 7 on-demand instances plus 3 spot instances.
Detailed orchestration behaviors for AWS can be found here.
例如，如果最小按需实例数量为 4，混合比例为 0.5，
那么当扩展组的总大小为 10 时，将包含 7 个按需实例与 3 个 Spot 实例。
有关 AWS 的详细编排行为，请参考此处。

For spot instance evictions, we notice that the auto-restoration is supported, which means that the autoscaler will automatically restart the evicted spot instances if capacity permits.
However, this is not guaranteed especially during peak hours due to constrained capacity. We studied the spot VM auto-restoration which will be detailed in Section 3.2.
我们观察到，Spot 实例被驱逐后，支持自动恢复功能，
这意味着如果容量允许，自动扩展器将会自动重启被驱逐的 Spot 实例。
然而，这一行为并不总是有保障的，特别是在高峰时段，由于容量限制，可能无法进行恢复。
我们研究了该自动恢复机制，并将在第 3.2 节中详细说明。

Third-party orchestrators also exist such as AutoSpotting which greedily allocates spot instances to fulfill the required number of VMs.
When spot instances get evicted, it immediately allocates on-demand instances to ensure the required capacity.
例如 AutoSpotting等第三方调度器也存在，它会贪婪地分配 Spot 实例以满足所需的 VM 数量。
当 Spot 实例被回收时，它会立即分配 On-demand 实例来保障容量。

Then, when the spot instances are restored, AutoSpotting will shut down the on-demand instances.
There is no explicit SLO for VMSG and only instance-level SLO for on-demand VMs now.
We thus define the SLO of a VMSG as an extension of the instance-level SLO, which will be discussed in Section 3.2.
之后，一旦 Spot 实例恢复，AutoSpotting 会关闭那些按需实例。
目前，VMSG 并没有显式的 SLO 规定，而只有适用于按需 VM 的实例级别 SLO。
因此，我们将 VMSG 的 SLO 定义为实例级别 SLO 的扩展，并将在 3.2 节中进行详细说明。

2.2 Constrained Reinforcement Learning
Constrained Reinforcement Learning (CRL) is a kind of Reinforcement Learning algorithm whose goal is not only to maximize long-term cumulative reward but also to satisfy the additional constraints without any heuristics or rules.
受限强化学习（CRL）是一种强化学习算法，它的目标不仅是最大化长期累积奖励，
而且还要在没有任何启发式或规则的情况下满足附加约束条件。

CRL is a key field for real-world RL problems as ensuring the satisfaction of certain constraints is necessary and essential in practice. has seen great success in a variety of industries, including electric grids, networking, robotics, resource allocation, and wireless security.
受限强化学习（CRL）是现实世界强化学习问题中的一个关键领域，
因为在实际应用中，确保满足某些约束条件是必要且至关重要的。
它已经在多个行业中取得了显著成功，包括电网、网络、机器人、资源分配以及无线安全等领域。

Applying CRL to our problem, the objective is to maintain the number of in-service VMs, not dropping below the target number while minimizing the cost of using VMs in the long term.
It is challenging to make such long-term decisions to maintain the target due to the complexity of the system, e.g., the behavior of spot evictions.
将 CRL 应用于本问题时，其目标是维持运行中的虚拟机数量不少于目标值，
同时长期最小化虚拟机使用成本。
然而，要做出这种长期策略决策来维持目标是具有挑战性的，
原因在于系统的复杂性，例如 Spot 实例被回收的行为模式。

For instance, adding extra spot VMs to VMSG does not guarantee the target number but results in additional costs; conversely, always keeping a certain number of on-demand VMs is unnecessary (as discussed further in Section 5).
例如，向 VMSG 中添加额外的 Spot 实例虽然会增加数量，
但并不能保证达到目标虚拟机数，反而会带来额外的成本；
相反地，总是保留一定数量的按需实例也没有必要（这一点将在第 5 节进一步讨论）。

3 CHARACTERISTICS OF SPOT VMS AND EVICTIONS
In this section, we conduct empirical studies to analyze the behaviors of evictions concerning spot VMs and VMSG. To study spot VM evictions, we analyze the sampled traces of the eastern US region from Microsoft Azure.
在本节中，我们通过实证研究来分析与 Spot 虚拟机和 VMSG 相关的回收行为。
为此，我们分析了来自 Microsoft Azure 东部地区的采样日志。

The trace includes spot VMs existed between April 15th 2022 and July 15th 2022.
Note that different regions might exhibit widely different spot eviction behaviors.
For the figures with no scale on the y-axis, the range might not be between 0% and 100%.
这些日志涵盖的时间范围为 2022 年 4 月 15 日至 7 月 15 日。
请注意，不同区域的 Spot 驱逐行为可能差异极大。
此外，对于某些图表，如果 y 轴未标注刻度，其数值范围不一定是 0% 到 100%。

3.1 Spot VM Evictions
Eviction reason. Among all spot VMs in the trace, 99.96% spot VMs are terminated, including 6.05% ‘Evicted’ spot VMs and 93.91% ‘Deleted’ spot VMs (i.e., users shut down spot VMs). It indicates that the majority of spot VMs are deleted, e.g., scheduled workloads on spot VMs are completed. Only ‘Evicted’ spot VMs are considered in the below empirical studies.
驱逐原因。在所有追踪记录中的 Spot 虚拟机中，99.96% 被终止，其中包括 6.05% 被“驱逐”的 Spot 虚拟机，以及 93.91% 被“删除”的 Spot 虚拟机（即用户手动关闭 Spot 虚拟机）。这表明大多数 Spot 虚拟机是被删除的，例如在 Spot 虚拟机上安排的工作负载已完成。下面的实证研究仅考虑被“驱逐”的 Spot 虚拟机。

Evictions result from diverse reasons, including deployments of on-demand VMs, service healing, testing, etc. 90.79% evicted spot VMs are caused by the on-demand VM deployments, i.e., the cloud platform needs the capacity back for on-demand VM deployments. It implies that the status of on-demand VMs could be valuable in predicting spot VM evictions.
驱逐的原因多种多样，包括按需虚拟机的部署、服务修复、测试等。90.79% 的 Spot 虚拟机驱逐是由按需虚拟机部署引起的，即云平台需要收回资源以部署按需虚拟机。这意味着按需虚拟机的状态在预测 Spot 虚拟机驱逐方面可能具有重要价值。

Lifetime. Figure 2(a) shows the lifetime distribution of these spot VMs. 55.31% of the evicted spot VMs and 32.80% of the deleted spot VMs live longer than one hour. In addition, spot VMs have different types [8, 14] that have diverse characteristics for resources (i.e., core-to-memory ratio, disk throughput, etc.) to fulfill different purposes. It is common that VM allocators of cloud providers tightly pack VMs by a single dominant resource (i.e., cores) to simplify capacity management [3, 31, 56]. Thus, we group ‘Evicted’ spot VMs by cores to investigate their lifetime.
生命周期。图 2(a) 展示了这些 Spot 虚拟机的生命周期分布。55.31% 的被驱逐 Spot 虚拟机和 32.80% 的被删除 Spot 虚拟机的生命周期超过一小时。此外，Spot 虚拟机有不同的类型 [8, 14]，在资源特性方面（如核数与内存比、磁盘吞吐量等）具有差异，以满足不同的用途。云服务商的虚拟机分配器通常根据某一种主导资源（如 CPU 核数）紧凑地分配虚拟机，以简化容量管理 [3, 31, 56]，这是很常见的做法。因此，我们按核数对被“驱逐”的 Spot 虚拟机进行分组，以研究其生命周期。

Figure 2(b) shows their lifetime grouped by different cores. Spot VMs with larger cores have a short lifetime in general, which matches our expectations. When on-demand VMs need capacity back, nodes with larger spot VMs are more likely to be tightly packed with less remaining empty capacity, resulting in evictions.
图 2(b) 展示了按不同核数分组后的生命周期。总体来看，拥有较多核数的 Spot 虚拟机生命周期较短，这与我们的预期一致。当按需虚拟机需要收回资源时，具有较多核数的 Spot 虚拟机所在节点更有可能被紧凑填充，剩余空闲容量更少，因而更容易被驱逐。

Node status. The cloud system contains a large number of physical servers, i.e., Nodes, that are allocated in different Regions. Each region contains many Clusters and each cluster contains hundreds of nodes [31]. VM allocator places spot and on-demand VMs on nodes, and node status affects the lifetime of spot VMs, e.g., spot VMs in tightly packed nodes are more likely to be evicted leading to a shorter lifetime.

节点状态。云系统包含大量物理服务器，即节点（Nodes），这些节点被分配到不同的区域（Regions）中。每个区域包含多个集群（Clusters），每个集群包含数百个节点 [31]。虚拟机分配器将 spot 虚拟机和按需虚拟机部署到节点上，而节点状态会影响 spot 虚拟机的生命周期，例如，在紧凑部署节点上的 spot 虚拟机更容易被驱逐，从而导致更短的生命周期。

In this paper, we define two metrics, spottiness and stableness, to evaluate the node status.
Spottiness is defined as the difference between spot core percentage and on-demand core percentage of used cores in nodes (including spot and on-demand cores).
在本文中，我们定义了两个指标，spottiness（抢占性） 和 stableness（稳定性），用于评估节点状态。
Spottiness 被定义为节点中已用核心中（包括 spot 核心和按需核心）spot 核心占比与按需核心占比之间的差值。

i.e., “也就是说” 或 “换句话说”

Figure 3(a) shows that spot VMs on nodes with higher spottiness have longer lifetimes.
It indicates that nodes packed with more on-demand cores are more likely to evict spot VMs on them,
and nodes packed with more spot cores or even pure spot cores can host spot VMs for a longer time.
On the other hand, stableness is defined as the variations of remaining core over time.
图 3(a) 显示，具有更高 spottiness（抢占性）的节点上的 spot 虚拟机拥有更长的生命周期。
这表明，部署了更多按需核心的节点更有可能驱逐其上的 spot 虚拟机，
而部署了更多 spot 核心甚至是纯 spot 核心的节点，可以托管 spot 虚拟机更长时间。
另一方面，stableness（稳定性） 被定义为剩余核心数随时间的变化。

Figure 3(a) shows that spot VMs on nodes with higher spottiness have longer lifetimes.
It indicates that nodes packed with more on-demand cores are more likely to evict spot VMs on them,
and nodes packed with more spot cores or even pure spot cores can host spot VMs for a longer time.
On the other hand, stableness is defined as the variations of remaining core over time, i.e.,

where the remaining core is the node core capacity excluding occupied on-demand cores.

图 3(a) 显示，具有更高 spottiness（抢占性）的节点上的 spot 虚拟机拥有更长的生命周期。
这表明，部署了更多按需核心的节点更有可能驱逐其上的 spot 虚拟机；
而部署了更多 spot 核心甚至是纯 spot 核心的节点，可以托管 spot 虚拟机更长时间。
另一方面，stableness（稳定性） 被定义为剩余核心数随时间的变化，
即：
其中，remaining core 是指节点总核心数减去被按需虚拟机占用的核心数。

We classify these nodes to Stable, Intermediate, and Unstable based on stableness
with value in [0.6, 1], [0.4, 0.6), and [0, 0.4).
Figure 3(b) indicates that spot VMs placed on nodes with more core variations will be evicted earlier.
The remaining cores on these nodes are frequently picked to deploy new on-demand VMs,
and spot VMs taking the remaining cores are more likely to be evicted.

我们根据 stableness（稳定性） 将这些节点划分为 Stable（稳定）、Intermediate（中等） 和 Unstable（不稳定） 三类，
其稳定性数值分别位于 [0.6, 1]、[0.4, 0.6) 和 [0, 0.4) 范围内。
图 3(b) 表明，部署在核心变化更频繁的节点上的 spot 虚拟机更早被驱逐。
这些节点上的剩余核心经常被选中用于部署新的按需虚拟机，
而使用这些剩余核心的 spot 虚拟机更容易被驱逐。

Region status.
Moving up to the region level, we analyze the allocated resources and new deployments to examine the region status. As aforementioned, the dominant eviction reason is the on-demand VM deployments. Figure 4 shows the hourly and weekly pattern of allocated resources and new on-demand deployments in order to connect with eviction patterns in the next section.
区域状态。
提升到区域层级后，我们分析了分配的资源和新的部署，以评估区域状态。如前所述，主要的驱逐原因是按需虚拟机（VM）的部署。图4展示了资源分配和新按需部署的小时和每周模式，以便在下一节中与驱逐模式联系起来。
第一段：引入与图示说明

The core utilization (left y-axis) is computed based on the cores of the currently running on-demand VMs over the whole capacity, i.e., excluding resources that have been allocated to existing evictable VMs. The on-demand deployments (right y-axis) are normalized, representing the dynamics of on-demand VM creations over time. Note that we hide the y-axis for confidentiality reasons.
核心使用率（左侧 y 轴）是根据当前正在运行的按需虚拟机的核心数占整个容量的比例计算的，即不包括已分配给可驱逐虚拟机的资源。按需部署（右侧 y 轴）是标准化后的，代表按需虚拟机随时间的创建动态。请注意，我们出于保密原因隐藏了 y 轴。
第二段：核心使用率与部署模式分析

The core utilization is higher during working hours and on weekdays, and the on-demand VM deployments behave in a similar higher pattern on weekdays.
核心使用率在工作时间和工作日较高，按需虚拟机部署在工作日也呈现出相似的高频模式。
第三段：时间模式与行为观察

We observe a peak deployment around 7:00, followed by a rather lower number of deployments with larger variances. On-demand VMs created at this time survive during working hours, leading to fewer creations afterward. The large variances come from diverse requirements, e.g., some services need additional deployments to host extra traffic, and some services have stable traffic during working hours without new deployments.
我们观察到在7:00左右有一个部署高峰，随后部署数量减少，但波动较大。在这个时间创建的按需虚拟机会持续运行整个工作时间，导致之后创建数量减少。波动大的原因是多样化的需求，例如某些服务需要额外的部署来承载额外流量，而某些服务在工作时间内流量稳定，不需要新的部署。

Note that the small peak around 19:00 is caused by regular on-demand deployments at default time (UTC 0:00) shifted to the local time. The hourly pattern figure uses only the weekday data to reduce variances.
需要注意的是，大约在19:00出现的小高峰是由于默认时间（UTC 0:00）的常规按需部署被转换为本地时间导致的。小时模式图只使用了工作日的数据以减少波动。

Eviction rate.
The spot VM lifetime empirical study provides an overview of spot VM evictions within a future time span, and we analyze the spot eviction rate to investigate further the temporal pattern of evictions and its relations with other temporal factors (e.g., core utilization and on-demand deployments in the previous section).
We define the eviction rate as the percentage of the existing spot VMs being evicted within a given time period, e.g., ‘1-hour’ eviction rate is computed by dividing the number of evicted spot VMs within 1 hour over the corresponding currently existing number of spot VMs.
驱逐率。
对 spot VM 生命周期的实证研究提供了未来时间范围内 spot VM 被驱逐情况的概览，我们分析了 spot 驱逐率，以进一步研究驱逐的时间模式及其与其他时间因素（例如上一节中的核心利用率和按需部署）的关系。
我们将驱逐率定义为在给定时间段内被驱逐的现有 spot VM 所占的百分比。例如，“1 小时”驱逐率是通过将 1 小时内被驱逐的 spot VM 数量除以当时现有的 spot VM 总数计算得出。

Figure 5 shows the eviction rate with different time horizons (i.e., 1-, 2-, 3-, 6-, 12-, 24-hour) over time.
The eviction rate is higher with a longer time horizon, and eviction rates are higher during working hours and on weekdays.
In particular, the hourly patterns of eviction rates with short time horizons (1, 2, and 3 hours) follow the hourly patterns of allocated resources and on-demand deployments (Figure 4), showing higher eviction rates during working hours (between 07:00 to 17:00).
On the other hand, the hourly patterns of eviction rates with long time horizons (6, 12, and 24 hours) show higher eviction rates around the start of working hours, capturing evictions within longer time range that covers working hours.
图5展示了不同时间跨度（即 1、2、3、6、12、24 小时）下随时间变化的驱逐率。
时间跨度越长，驱逐率越高；并且在工作时间和工作日的驱逐率更高。
特别是，短时间跨度（1、2、3 小时）的驱逐率小时模式与资源分配和按需部署（见图4）相符，在工作时间段（07:00 至 17:00）期间驱逐率较高。
另一方面，长时间跨度（6、12、24 小时）的驱逐率小时模式在工作时间开始时段附近较高，捕捉到的是覆盖整个工作时间的较长时间范围内的驱逐行为。

3.2 VMSG Characteristics
Virtual Machine Scaling Group (VMSG) manages a group of VMs that application workloads are automatically distributed across them through a load balancer.
虚拟机扩展组（VMSG） 管理着一组虚拟机，应用工作负载会通过负载均衡器自动分布到这些虚拟机上。

VMSG SLO definition. Service Level Objectives (SLOs) measure the service quality from the perspective of users. Unlike on-demand VMs with high availability (99.99%-99.999%) [9, 13], the spot VMSG has no SLO as spot VMs are evictable with no guarantee of availability. We provide a definition of VMSG SLO based on the availability of in-service VMs:

Spot VMSG SLO. Spot VMSG could enable spot VM auto-restoration [12], i.e., spot VMs are restored after evictions to meet the requirement of VM capacity. Figure 6 (a) shows the distribution of the spot VMSG Availability, excluding short-lived VMSG (less than one day).
Spot VMSG 服务等级目标（SLO）。Spot VMSG 可以实现 Spot 虚拟机的自动恢复 [12]，即在被回收后恢复 Spot 虚拟机，以满足虚拟机容量的需求。图 6 (a) 显示了 Spot VMSG 可用性的分布情况，不包括生命周期少于一天的 VMSG。

Around 50% spot VMSG have near 99% availability with the spot restoration, still, other spot VMSGs have lower availability due to large evictions.
Figure 6 (b) shows the restoration time, and 90% spot VMs could be restored within one hour.
Figure 7 shows two cases where the VMSG has high and low VMSG availability due to differences in evictions and restoration time. Some spot VMs in the high availability VMSG have no evictions.
大约 50% 的 Spot VMSG 在启用 Spot 恢复功能后拥有接近 99% 的可用性，然而，其他 Spot VMSG 的可用性则较低，这是由于大量的回收操作造成的。
图 6(b) 显示了恢复时间，90% 的 Spot 虚拟机可以在一小时内恢复。
图 7 展示了两个案例，其中 VMSG 的高可用性或低可用性取决于回收次数和恢复时间的差异。在高可用性的 VMSG 中，一些 Spot 虚拟机从未被回收。

3.3 Spot VM Price
We investigate the spot VM price changing intervals as shown in Figure 8(a). 80% spot VMs keep the price for around one month before changing to a new price.
Figure 8(b) shows the savings of using spot VMs instead of on-demand VMs, and spot VMs can achieve 60% ∼ 90% savings.
我们研究了 Spot 虚拟机价格的变动周期，如图 8(a) 所示。80% 的 Spot 虚拟机在更改为新价格之前，会保持当前价格大约一个月。
图 8(b) 显示了使用 Spot 虚拟机替代按需虚拟机的节省效果，Spot 虚拟机最多可节省 60% ∼ 90% 的成本。

3.4 High-Level Takeaways
We summarize a few insights from the above empirical study:

• The region-level eviction rate shows a temporal pattern that it is higher on peak hours, e.g., weekdays and working hours, with high resource utilization and a large amount of on-demand VM deployments. These eviction patterns show the possibility of eviction prediction considering the features of time, resource utilization, and deployments.

• Spot VMs intend to have lower eviction risks if located in nodes with high spottiness and stableness. It inspires us to consider local VMSG features when learning dynamic mixture policy, including the node features where VMs in this VMSG locate.

• Around half of spot VMSGs have 90% availability which motivates us to design a mixture VMSG to improve the availability considering the large cost savings of using spot VMs.
  我们从上述实证研究中总结出以下几点见解：
  第一条：
  区域级别的回收率呈现时间上的变化模式，在高峰时段（例如工作日和工作时间）会更高，这与资源利用率高和大量按需虚拟机部署有关。这些回收模式表明，可以考虑时间、资源利用率和部署特征来进行回收预测。
  第二条：
  如果 Spot 虚拟机位于具有高 spottiness（抢占特性）和稳定性的节点中，则通常具有较低的回收风险。这启发我们在学习动态混合策略时，应考虑本地 VMSG 的特征，包括该 VMSG 中虚拟机所在节点的特征。
  第三条：
  大约一半的 Spot VMSG 拥有 90% 的可用性，这促使我们设计混合型 VMSG，以在大幅节省成本的同时提升可用性。