Multi Armed Bandit

• \(t=1,2,\cdots,N\)
• \(k\) arm
• at \(t\), choose arm \(i\), receive reward \(X_{i,t}\sim{}D_i\)(iid unknown distribution)
• Policy: \(\pi=\{i(t),t=1,2,\cdots,N\}\)
• Goal: \(\max\mathbb{E}\{\sum_{t=1}^NX_{i,t}\}\)
• \({\rm OPT}=N\max_i\mu_i=N\mu,\mu_i=\mathbb{E}\{X_{i,t}\}\)
• \({\rm Reg}={\rm OPT}-\mathbb{E}\{\sum_{t=1}^NX_{i,t}|\pi\}\)

Introduction

• Single Hop System, \(L\) Queues
• \(Q_l(t=1)=(Q_l(t)+A_l(t)-S_l(t))^+=Q_l(t)+A_l(t)-S_l(t)+U_l(t)\)
• \(U_l(t)=\max\{0,S_l(t)-A_l(t)-Q_l(t)\}\)

Queue Control Problem

\(A_1(t),A_2(t),Q_1(t),Q_2(t),\mu_1(t),\mu_2(t)\)

• \(A_i(t)\) is iid, \(\mathbb{E}\{A_i(t)\}=\lambda_i\), \(A_i(t)\in[0,A_{\max}]\)
• \(S_i(t)\) is condition of link \(i\), \(S_i(t)\in\{0,1\}\)
• \(P_{xy}=P_r\{S_1(t)=x,S_2(t)=y\}\)
• At every time, serve \(Q_1(t)\) or \(Q_2(t)\)
• \(\mu_i(t)=1\text{ if } S_i(t)=1,Q_i(t)>0\quad0\text{ otherwise}\)
• Goal: stabilize both queues

\(~\)

Queue

Definition

• Arrival: \(A(t)\)
• Cumulative arrival: \[X[t_1,t_2]=\int_{t_1}^{t_2}A(t)dt\]
• Service: \(\mu(t)\)
• Cumulative departure: \[Y[t_1,t_2]=\int_{t_1}^{t_2}Y(t)dt\leq\int_{t_1}^{t_2}\mu(t)dt\]
• \(Y(t)=Q(t)~if~Q(t)>0\quad0~otherwise\)
• \(Q(t)=X[0,t]-Y[0,t],Q(0)=0\)

Wasserstein Distance

Definition

Consider, general functions \(f\) and \(g\), the Wasserstein distance is \[\min_{\text{all map }T}\{\sum_{\text{all movements of }T}\text{distance moved}\times\text{amount moved}\}\] For \(f:X\rightarrow{}R^+,g:Y\rightarrow{}R^+\), the distance can be formulated as \[W_p(f,g)=\left(\inf_{T\in\mathcal{M}}\int{}|x-T(x)|^pf(x)dx\right)^{1/p}\] where \(\mathcal{M}\) is the set of all maps that rearrange the distribution \(f\) into \(g\).

Closed

Closed set

A set \(\mathcal{C}\) is closed if it contains its boundary: \[x^k\in\mathcal{C},x^k\rightarrow\bar{x}\Rightarrow\bar{x}\in\mathcal{C}\]

• the intersection of closed sets is closed
• the union of a finite number of closed sets is closed
• inverse under linear mapping \[\{x|Ax\in\mathcal{C}\}\] is closed if \(\mathcal{C}\) is closed

Subgradient

\(g\) is a subgradient of a convex function \(f\) at \(x\in{}dom~f\) if \[f(y)\geq{}f(x)+g^\top(y-x)\quad\text{for all }y\in{}dom~f\]

概述

定义

• Failure Mode, Effects and Criticality analysis
• 归纳分析方法：分析系统中每个设备所有可能的故障模式及对 系统造成的所有可能影响，并按每个故障模式的严重程度及发 生概率予以分类。
  • 一种自下而上的归纳分析方法
  • FMEA和CA

Linear Programming

\[\text{(P) }\min_xc^Tx,s.t.Ax=b,x\geq0\] \[\text{(D) }\min_yb^Ty,s.t.A^Ty+s=c,s\geq0\]

Strong duality

\[c^Tx=b^T\Leftrightarrow{}x^Ts=0\Leftrightarrow{}x_is_i=0\]

维修性特征量

维修分布函数/维修度

产品从故障开始到修理完毕经历的时间 \(Y\) \[M(t)=P(Y\leq{}t)\]

修复率

尚未修复的产品在单位时间内修复完成 \[\mu(t)=\frac{m(t)}{1-M(t)}\]

平均修复时间

\[MTTR=\int_0^\infty{}tm(t)dt=\int_0^\infty{}tdM(t)\]