Hanjie's Blog

一只有理想的羊驼

罗汉杰. 磁力计数据的处理方法和装置[P]. 中国专利: CN115327452A, 2022.11.11.

背景介绍

磁力计也叫地磁、磁感器,它拥有三个正交方向的霍尔传感器,能够测量出三个方向的磁场强度。由于地球的磁场像一个条形磁体一样由磁南极指向磁北极,通过合成三个方向的霍尔传感器磁场强度读数,可以计算出出设备的航向和姿态。磁力计被广泛地应用于手机,飞行器,机器人等设备中。

然而在一般情况下,地球的磁场十分微弱。如果磁场计周围存在磁性物质或者可以影响局部磁场强度的物质存在,会对磁场造成干扰,使得地磁读取数据产生偏差。再加上受制造安装工艺、敏感轴电气性不一致及零点偏移等因素影响,磁力仪本身存在三轴非正交、敏感轴灵敏度不一致及零偏误差等性能缺陷,使得磁测的准确性受到较大影响。

magnetometer1

而各种传感器都有各自的坐标系。在实际应用中,往往需要将多种传感器的坐标对齐到同一个坐标系下。例如,我们一般将磁力计的坐标系对齐到加速度传感器的坐标系下。但由于安装误差的存在,磁力计坐标系与加速度坐标系没有相互重合,存在一个旋转误差。

magnetometer2

因此,我们在使用磁力计前,一方面需要对其进行标定工作,获得真实的地磁方向数据;另一方面将地磁计的坐标系与其他坐标系进行对齐,以方便后续使用。

磁力计误差模型

磁力计误差一般包括硬铁误差,软铁误差,尺度误差,三轴非正交误差和零偏误差。

硬铁误差:永磁铁或者磁化金属会对磁场产生额外的磁场干扰,其干扰磁场的大小和相对于载体的方向一般保持不变。我们称这种干扰导致的误差为硬铁误差\(\textbf{b}_{n}\)。硬铁误差\(\textbf{b}_{n}\)相当于在磁场上添加一个偏移向量,我们定义为一个3×1向量(\(𝑇\)符号表示矩阵转置):

\[\begin{equation} \label{eq:mag1} \textbf{b}_{n}=[b_{n,x},b_{n,y},b_{n,z}]^{T} \end{equation}\]

软铁误差:软铁磁场是由铁磁材料如铁、镍、PCB 板等对受地磁场或电磁场的磁化而产生的。软铁磁场会随时间和载体航向的变化而变化。我们定义软铁磁场造成的误差\(\textbf{A}_{s}\)为一个3×3矩阵:

\[\begin{equation} \label{eq:mag2} \textbf{A}_{s}=\left[{\begin{array}{c c c}{a_{00}}&{a_{01}}&{a_{02}} \newline {a_{10}}&{a_{11}}&{a_{12}} \newline {a_{20}}&{a_{21}}&{a_{22}}\end{array}}\right] \end{equation}\]

尺度误差\(\textbf{S}\)为一个3×3矩阵:

\[\begin{equation} \label{eq:mag3} \textbf{S}=\left[{\begin{array}{c c c}{s_{x}}&{0}&{0} \newline {0}&{s_{y}}&{0} \newline {0}&{0}&{s_{z}}\end{array}}\right] \end{equation}\]

三轴非正交误差\(\textbf{N}\)为一个3×3矩阵:

\[\begin{equation} \label{eq:mag4} \textbf{N}=\left[{\begin{array}{c c c}{n_{x,x}}&{n_{y,x}}&{n_{z,x}} \newline {n_{x,y}}&{n_{y,y}}&{n_{z,y}} \newline {n_{x,z}}&{n_{y,z}}&{n_{z,z}}\end{array}}\right] \end{equation}\]

零偏误差\(\textbf{b}_{m}\)为一个3×1向量:

\[\begin{equation} \label{eq:mag5} \textbf{b}_{m}=[b_{m,x},b_{m,y},b_{m,z}]^{T} \end{equation}\]

综上所述,磁力计的误差模型为:

\[\begin{equation} \label{eq:mag6} \textbf{h}=\textbf{S}\textbf{N}(\textbf{A}_{s}\textbf{h}' + \textbf{b}_{n}) + \textbf{b}_{m} \end{equation}\]

其中,\(\textbf{h}=[h_{x},h_{y},h_{z}]^{T}\)为磁力计未经标定的直接读取值,\(\textbf{h}'=[h_{x}',h_{y}',h_{z}']^{T}\)为磁力计经过标定后的地磁真实值,简化公式\(\eqref{eq:mag6}\)

\[\begin{equation} \label{eq:mag7} \textbf{h}'=\textbf{A}(\textbf{h} - \textbf{b}) \end{equation}\]

其中,\(\textbf{A}=(\textbf{S}\textbf{N}\textbf{A}_{s})^{-1}\)\(\textbf{b}=\textbf{S}\textbf{N}\textbf{b}_{n}+\textbf{b}_{m}\)为未知的误差模型参数。通过对磁力计进行标定,我们可以得到参数\(\textbf{A}\)\(\textbf{b}\)的值,然后我们可以使用公式\(\eqref{eq:mag7}\),将读取的,带误差的测量值\(\textbf{h}\)恢复为真实值\(\textbf{h}'\)

椭圆体拟合

假设空间中有\(𝑛\)个三维点集\(\lbrace \textbf{X}_{i} = [X_{i}, Y_{i}, Z_{i}]^{T} \mid 0 \le i < n \rbrace\),我们希望能够找到一个最佳椭圆体去拟合这些三维点。描述椭圆体的一般方程为:

\[\begin{equation} \label{eq:mag8} 𝑎𝑋^{2} +𝑏𝑌^{2} +𝑐𝑍^{2} + 2𝑑𝑋𝑌 + 2𝑒𝑋𝑍 + 2𝑓𝑌𝑍 + 2𝑔𝑋 + 2h𝑌 + 2𝑖𝑍 + 𝑗 = 0 \end{equation}\]

上式可以写为矩阵形式:

\[\begin{equation} \label{eq:mag9} [X \quad Y \quad Z] \left[{\begin{array}{c c c}{a}&{d}&{e} \newline {d}&{b}&{f} \newline {e}&{f}&{c}\end{array}}\right] \left[{\begin{array}{c}{X} \newline {Y} \newline {Z}\end{array}}\right] + [X \quad Y \quad Z] \left[{\begin{array}{c}{2g} \newline {2h} \newline {2i}\end{array}}\right] + j = 0 \end{equation}\]

其中,\(𝑎, 𝑏, 𝑐, 𝑑, 𝑒, 𝑓, 𝑔, h, 𝑖, 𝑗\)为描述椭圆体的参数,表示了椭圆体的中心位置,轴方向,和旋转等信息。椭圆体拟合,就是给定一个三维点集,求取满足公式\(\eqref{eq:mag9}\)的椭圆体的参数\(𝑎, 𝑏, 𝑐, 𝑑, 𝑒, 𝑓, 𝑔, h, 𝑖, 𝑗\)。椭圆体的拟合方法有很多,这里我们使用经典的最小二乘法。

设定未知的参数向量\(\textbf{v} = [𝑎, 𝑏, 𝑐, 𝑑, 𝑒, 𝑓, 𝑔, h, 𝑖, 𝑗]^{T}\),对于每一个点\(\textbf{X}_{i} = [X_{i}, Y_{i}, Z_{i}]^{T}\),定义一个临时向量\(\textbf{P}_{i}\)

\[\begin{equation} \label{eq:mag10} \textbf{P}_{i} = (X_{i}^{2}, Y_{i}^{2}, Z_{i}^{2}, 2Y_{i}Z_{i}, 2X_{i}Z_{i}, 2X_{i}Y_{i}, 2X_{i}, 2Y_{i}, 2Z_{i}, 1)^{T} \end{equation}\]

定义10×n矩阵\(\textbf{D} = (\textbf{P}_{0}, \textbf{P}_{1}, \cdots, \textbf{P}_{n-1})\),它包含了所有的三维点集的信息。

定义6×6大的临时矩阵\(\textbf{C}_{1}\)

\[\begin{equation} \label{eq:mag11} \textbf{C}_{1} = \left[{\begin{array}{c c c c c c}{-1}&{1}&{1}&{0}&{0}&{0} \newline {1}&{-1}&{1}&{0}&{0}&{0} \newline {1}&{1}&{-1}&{0}&{0}&{0} \newline {0}&{0}&{0}&{-4}&{0}&{0} \newline {0}&{0}&{0}&{0}&{-4}&{0} \newline {0}&{0}&{0}&{0}&{0}&{-4}\end{array}}\right] \end{equation}\]

定义10×10大的约束矩阵\(\textbf{C}\)

\[\begin{equation} \label{eq:mag12} \textbf{C} = \left[{\begin{array}{c c}{\textbf{C}_{1}}&{\textbf{0}_{6×4}} \newline {\textbf{0}_{4×6}}&{\textbf{0}_{4×4}}\end{array}}\right] \end{equation}\]

根据椭圆体的几何特性,拟合的椭圆体,在满足公式\(\eqref{eq:mag9}\)的前提下,还需满足约束公式\(\textbf{v}^{T}\textbf{C}\textbf{v}=1\)。将椭圆体拟合问题变成最小二乘优化问题,即求取\(\textbf{v}\),使得:

\[\begin{equation} \label{eq:mag13} \min_{\textbf{v}} \lVert \textbf{D} \textbf{v} \rVert ^{2}, 并且\textbf{v}^{T}\textbf{C}\textbf{v}=1 \end{equation}\]

使用增广拉格朗日乘子法,从公式\(\eqref{eq:mag13}\)能够得到:

\[\begin{align} \textbf{D}\textbf{D}^{T}\textbf{v} &= \lambda \textbf{C}\textbf{v} \label{eq:mag14} \newline \textbf{v}^{T}\textbf{C}\textbf{v} &= 1 \label{eq:mag15} \end{align}\]

其中,\(\lambda\)为拉格朗日乘数。\(\textbf{D}\textbf{D}^{T}\)为一个10×10大的矩阵,\(\textbf{v}\)为一个10×1的向量。分解矩阵\(\textbf{D}\textbf{D}^{T}\)和向量\(\textbf{v}\)为:

\[\begin{equation} \label{eq:mag16} \textbf{D}\textbf{D}^{T} = \left[{\begin{array}{c c}{\textbf{S}_{11}}&{\textbf{S}_{12}} \newline {\textbf{S}_{12}}&{\textbf{S}_{22}}\end{array}}\right] \end{equation}\]

\[\begin{equation} \label{eq:mag17} \textbf{v} = \left[{\begin{array}{c}{\textbf{v}_{1}} \newline {\textbf{v}_{2}}\end{array}}\right] \end{equation}\]

其中\(\textbf{S}_{11}\)\(\textbf{S}_{12}\)\(\textbf{S}_{22}\)大小为6×6,6×4和4×4,向量\(\textbf{v}_{1}\)\(\textbf{v}_{2}\)大小为6和4。公式\(\eqref{eq:mag14}\)可以改写为:

\[\begin{align} \textbf{C}_{1}^{-1}(\textbf{S}_{11}-\textbf{S}_{12}\textbf{S}_{22}^{-1}\textbf{S}_{12}^{T})\textbf{v}_{1} &= \lambda \textbf{v}_{1} \label{eq:mag18} \newline \textbf{v}_{2} &= - \textbf{S}_{22}^{-1}\textbf{S}_{12}^{T}\textbf{v}_{1} \label{eq:mag19} \end{align}\]

对公式\(\eqref{eq:mag18}\)中的矩阵\(\textbf{C}_{1}^{-1}(\textbf{S}_{11}-\textbf{S}_{12}\textbf{S}_{22}^{-1}\textbf{S}_{12}^{T})\textbf{v}_{1}\)进行奇异值分解,求取特征值和特征向量,则\(\textbf{v}_{1}\)的值为最大特征值所对应的特征向量。然后将\(\textbf{v}_{1}\)代入公式\(\eqref{eq:mag19}\)求取\(\textbf{v}_{2}\),最终椭圆体的参数向量\(v\)可得。

磁力计标定

本磁力计标定方法包括两个部分,一方面是求取磁力计的误差模型,用来获取真实的地磁方向数据;另外一个方面将磁力计坐标系对齐到加速度坐标系下。

我们先采集数据。假定载体上有已经经过标定的加速度计和未经标定的磁力计。旋转载体,然后静置,采集在该静置姿态下的,已标定的加速度计读数\(\textbf{a}\)(由于是在静置姿态下,所以该读数即为重力加速度)和未经标定的磁力计读数\(\textbf{h}\)。重复以上操作\(𝑛\)次,获取载体在各种姿态下(尽量覆盖所有的方向)的重力加速度数据集\(\lbrace \textbf{a}_{i} = [a_{i,x}, a_{i,y}, a_{i,z}]^{T} \mid 0\le i < n \rbrace\)和地磁数据集\(\lbrace \textbf{h}_{i} = [h_{i,x}, h_{i,y}, h_{i,z}]^{T} \mid 0\le i < n \rbrace\)\(𝑛\)为数据集的大小。

磁力计误差标定

公式\(\eqref{eq:mag7}\)为磁力计的误差模型,其中\(\textbf{h}\)为未经标定的磁力计值,\(\textbf{h}'\)为标定后的真实值。由于在实际应用中,一般只需要磁力计的矢量方向用来计算设备的航向角,而不关心磁场的大小,因此会对\(\textbf{h}'\)进行归一化处理,即有\(\textbf{h}'^{T} \textbf{h}'= {\lVert \textbf{h}' \rVert }^2\)。代入公式\(\eqref{eq:mag7}\)并展开,有:

\[\begin{equation} \label{eq:mag20} \textbf{h}^{T} \textbf{Q} \textbf{h} + \textbf{h}^{T} \textbf{n} + d = 0 \end{equation}\]

其中\(\textbf{Q}=\textbf{A}^{T}\textbf{A}\),\(\textbf{n}=-2\textbf{Q}\textbf{b}\),\(d=\textbf{b}^{T}\textbf{Q}\textbf{b} - 1\)

对比公式\(\eqref{eq:mag20}\)和椭圆体公式\(\eqref{eq:mag9}\),我们发现磁力计误差模型\(\eqref{eq:mag20}\)相当于一个椭圆体公式,即磁力计的读值\(\textbf{h}\)会落在了一个椭圆体上(见下图):

mag_calibration1

去掉硬铁误差,软铁误差,尺度误差,零点误差等误差而得到的磁力计真实值\(\textbf{h}'\),应该落在一个圆心在原点的球体上。将未经标定的磁力计数据集\(\lbrace \textbf{h}_{i} \mid 0\le i < n \rbrace\)当作椭圆体上的三维点集\(\lbrace \textbf{X}_{i} \mid 0\le i < n \rbrace\),使用上述的椭圆体拟合法求取参数向量\(\textbf{v} = [𝑎, 𝑏, 𝑐, 𝑑, 𝑒, 𝑓, 𝑔, h, 𝑖, 𝑗]^{T}\)。根据公式\(\eqref{eq:mag9}\)和公式\(\eqref{eq:mag20}\),我们能够获得磁力计的误差模型参数\(\textbf{Q}\)\(\textbf{n}\)\(d\)

\[\begin{equation} \label{eq:mag21} \textbf{Q} = \left[{\begin{array}{c c c}{a}&{d}&{e} \newline {d}&{b}&{f} \newline {e}&{f}&{c}\end{array}}\right], \quad \textbf{n} = \left[{\begin{array}{c}{2g} \newline {2h} \newline {2i}\end{array}}\right], \quad d = j \end{equation}\]

实际上通过公式\(\eqref{eq:mag20}\)计算获得的误差模型参数\(\textbf{Q}\)\(\textbf{n}\)\(d\)并非是真实值,它们与真实值\(\overline{\textbf{Q}}\)\(\overline{\textbf{n}}\)\(\overline{d}\)之间存在一个矢量\(\alpha\)值,即\(\overline{\textbf{Q}} = \alpha \textbf{Q}\)\(\overline{\textbf{n}} = \alpha \textbf{n}\)\(\overline{d} = \alpha d\)\(\textbf{Q}\)\(\textbf{n}\)\(d\)乘以一个任意矢量\(\alpha\)值都会满足公式\(\eqref{eq:mag20}\),所以我们通过公式\(\eqref{eq:mag20}\)获 得的\(\textbf{Q}\)\(\textbf{n}\)\(d\)并非是唯一值)。我们将\(\alpha\)设定为尺度变量,根据定义有\(\overline{\textbf{n}}=-2\overline{\textbf{Q}}\textbf{b}\),\(\overline{d} = \textbf{b}^{T}\overline{\textbf{Q}} \textbf{b} - 1\),进一步得出:

\[\begin{equation} \nonumber \begin{aligned} 1 &= \textbf{b}^{T}\overline{\textbf{Q}} - \overline{d} \newline &= (-0.5\overline{\textbf{n}}^{T}\overline{\textbf{Q}}^{-1}) \overline{\textbf{Q}} (-0.5\overline{\textbf{n}}^{T}\overline{\textbf{Q}}^{-1}) - \overline{d} \newline &= 0.25 \overline{\textbf{n}}^{T}\overline{\textbf{Q}}^{-1} \overline{\textbf{n}} - \overline{d} \newline &= \alpha(0.25 \textbf{n}^{T}\textbf{Q}^{-1}\textbf{n}-d) \end{aligned} \end{equation}\]

\[\begin{equation} \label{eq:mag22} \longrightarrow \quad \alpha = \frac{4}{\textbf{n}^{T}\textbf{Q}^{-1}\textbf{n}-d} \end{equation}\]

根据\(\overline{\textbf{n}}= − 2\overline{\textbf{Q}}\textbf{b}\)\(\overline{\textbf{Q}} = \alpha\textbf{Q}\)\(\overline{\textbf{n}} = \alpha\textbf{n}\),有:

\[\begin{align} b &= -\frac{1}{2} \overline{\textbf{Q}}^{-1}\overline{\textbf{n}} \nonumber \newline &= -\frac{1}{2} \textbf{Q}^{-1} \textbf{n} \label{eq:mag23} \end{align}\]

根据\(\overline{\textbf{Q}}=\overline{\textbf{A}}^{T}\overline{\textbf{A}}\)\(\overline{\textbf{Q}}=\alpha\textbf{Q}\),有:

\[\begin{equation} \label{eq:mag24} \textbf{A} = (\alpha \textbf{Q})^{\frac{1}{2}} \end{equation}\]

则标定后的磁力计值\(\textbf{h}'=\textbf{A}(\textbf{h} - \textbf{b})\),为一个半径为1的球体上的点,如下图:

mag_calibration2

坐标系对齐

设定有3×3旋转矩阵\(\textbf{R}\),能够将磁力计坐标系下的,经过标定的地磁向量\(\textbf{h}'\)对齐到加速度坐标系下,即:

\[\begin{equation} \label{eq:mag25} \textbf{h}^{a} = \textbf{R}\textbf{h}' \end{equation}\]

在局部地理环境下,地磁向量\(\textbf{h}'\)与重力加速度\(\textbf{a}\)有固定的夹角\(\theta\),根据几何关系有:

\[\begin{equation} \label{eq:mag26} \cos(\theta) - \frac{\textbf{a}^{T} \textbf{R} \textbf{h}'}{ {\lVert \textbf{a} \rVert}{\lVert \textbf{h}' \rVert} } = 0 \end{equation}\]

mag_calibration3

我们使用高斯-牛顿优化方法求取旋转矩阵\(\textbf{R}\)。为方便后续计算,我们使用李代数\(\boldsymbol{ \phi } = [{\phi}_{1}, {\phi}_{2}, {\phi}_{3}]\)表示旋转矩阵\(\textbf{R}\),李代数\(\boldsymbol{ \phi }\)与旋转矩阵\(\textbf{R}\)有以下关系\(\textbf{R}(\boldsymbol{ \phi })={e}^{\boldsymbol{ \phi }^{\wedge}}\)\(e\)为自然常数,\(\wedge\)符号表示以下操作:

\[\begin{equation} \label{eq:mag27} \boldsymbol{ \phi }^{\wedge} = ([{\phi}_{1}, {\phi}_{2}, {\phi}_{3}])^{\wedge} = \begin{bmatrix} {0}&{-\phi_{3}}&{\phi_{2}} \newline {\phi_{3}}&{0}&{-\phi_{1}} \newline {-\phi_{2}}&{\phi_{1}}&{0} \end{bmatrix} \end{equation}\]

根据公式\(\eqref{eq:mag26}\),对重力加速度数据集\(\lbrace \textbf{a}_{i} \mid 0\le i < n \rbrace\)和标定后的地磁数据集\(\lbrace \textbf{h}_{i}' = \textbf{A} (\textbf{h}_{i} - \textbf{b}) \mid 0\le i < n \rbrace\)建立代价函数\(L(\textbf{x})\),有:

\[\begin{equation} \label{eq:mag28} L(\textbf{x}) = L([\boldsymbol{ \phi },k]^{T})=\frac{1}{2}\sum_{i=0}^{n-1}(k-\overline{\textbf{a}}_{i}^{T}\textbf{R}(\boldsymbol{ \phi })\overline{\textbf{h}}_{i})^2 \end{equation}\]

其中\(\textbf{x}=[\boldsymbol{ \phi },k]^{T}=[{\phi}_{1}, {\phi}_{2}, {\phi}_{3}, k]^{T}\)\(k=\cos(\theta)\)\({\overline{\textbf{a}}_{i}} = {\textbf{a}_{i}} / {\lVert \textbf{a}_{i} \rVert}\)\({\overline{\textbf{h}}_{i}} = {\textbf{h}_{i}'} / {\lVert \textbf{h}_{i}' \rVert}\)。我们希望通过最小化代价函数,即\(\min_{L(\textbf{x})}\),计算得到\(\textbf{x}\),来求取未知的旋转矩阵\(\textbf{R}(\boldsymbol{ \phi })\)和夹角变量\(k\)

对于数据集\(\lbrace \textbf{a}_{i} \mid 0\le i < n \rbrace\)\(\lbrace \textbf{h}_{i}' \mid 0\le i < n \rbrace\),有n×4雅可比矩阵\(J(\textbf{x})\)

\[\begin{equation} \label{eq:mag29} J(\textbf{x}) = \left[{\begin{array}{c c } {-(\textbf{R}(\boldsymbol{ \phi }) \overline{\textbf{h}}_{0})^{T}\overline{\textbf{a}}_{0}^{\wedge}}&{1} \newline {-(\textbf{R}(\boldsymbol{ \phi }) \overline{\textbf{h}}_{1})^{T}\overline{\textbf{a}}_{1}^{\wedge}}&{1} \newline {\vdots}&{\vdots} \newline {-(\textbf{R}(\boldsymbol{ \phi }) \overline{\textbf{h}}_{n-1})^{T}\overline{\textbf{a}}_{n-1}^{\wedge}}&{1} \end{array}}\right] \end{equation}\]

和n×1残差矩阵\(F(\textbf{x})\)

\[\begin{equation} \label{eq:mag30} F(\textbf{x}) = \left[{\begin{array}{c} {k - \overline{\textbf{a}}_{0}^{T} \textbf{R}(\boldsymbol{ \phi }) \overline{\textbf{h}}_{0}} \newline {k - \overline{\textbf{a}}_{1}^{T} \textbf{R}(\boldsymbol{ \phi }) \overline{\textbf{h}}_{1}} \newline {\vdots} \newline {k - \overline{\textbf{a}}_{n-1}^{T} \textbf{R}(\boldsymbol{ \phi }) \overline{\textbf{h}}_{n-1}} \end{array}}\right] \end{equation}\]

高斯-牛顿优化方法的算法流程如下:

  1. 输入重力加速度数据集\(\lbrace \textbf{a}_{i} \mid 0\le i < n \rbrace\)和标定后的地磁数据集\(\lbrace \textbf{h}_{i}' \mid 0\le i < n \rbrace\);初始化待求\(x=[0, 0, 0, 0]^{T}\)
  2. 根据公式\(\eqref{eq:mag29}\)\(\eqref{eq:mag30}\),计算雅可比矩阵\(J(\textbf{x})\)和残差矩阵\(F(\textbf{x})\)
  3. 计算增量\(\Delta \textbf{x} = -(J(\textbf{x})^{T}J(\textbf{x}))^{-1}J(\textbf{x})^{T}F(\textbf{x})\)
  4. 更新\(\textbf{x} = \textbf{x} + \Delta \textbf{x}\)
  5. 如果\(\lVert \Delta \textbf{x} \rVert > 1e^{-8}\),则返回步骤c;如果否,则说明算法收敛,\(\textbf{x}\)为结果值,进入到下一个步骤。
  6. 因为\(\textbf{x}=[\boldsymbol{ \phi },k]^{T}=[{\phi}_{1}, {\phi}_{2}, {\phi}_{3}, k]^{T}\),根据\(\textbf{R}(\boldsymbol{ \phi })={e}^{\boldsymbol{ \phi }^{\wedge}}\)得到旋转矩阵\(\textbf{R}\)

最终标定结果

通过步骤1,我们获得磁力计的误差模型参数\(\textbf{A}\)\(\textbf{b}\)。根据\(\eqref{eq:mag7}\),可以将带误差的磁力计原始测量值\(\textbf{h}\)转换为真实的地磁值\(\textbf{h}'\)

通过步骤2,我们获得地磁计坐标系与加速度计坐标系之间的旋转关系\(\textbf{R}\), 并且根据公式\(\eqref{eq:mag25}\),将地磁向量\(\textbf{h}'\),对齐到加速度坐标系下,\(\textbf{h}^{a}\)为最终标定的结果。


开源代码:https://github.com/HanjieLuo/drone/tree/master/Client/magnetometer_calibration

罗汉杰. 磁力计数据的处理方法和装置[P]. 中国专利: CN115327452A, 2022.11.11.

安装

安装Node.js

1
brew install node
1
2
node -v         
v18.11.0

安装Hexo 1

1
npm install -g hexo-cli
1
2
npm info hexo-cli version
4.3.0

Blog文件夹下初始化Hexo:

1
2
3
cd Blog
hexo init
npm install

升级Hexo 2

进入带有package.json文件的Hexo文件夹

1
npm update

安装/升级NexT主题 3 4

Blog文件夹下:

1
npm install hexo-theme-next@latest

修改Blog/_config.yml文件:

1
theme: next

测试:

1
2
hexo clean
hexo s --debug

浏览器打开http://localhost:4000

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
hexo -v

INFO Validating config
INFO ==================================
███╗ ██╗███████╗██╗ ██╗████████╗
████╗ ██║██╔════╝╚██╗██╔╝╚══██╔══╝
██╔██╗ ██║█████╗ ╚███╔╝ ██║
██║╚██╗██║██╔══╝ ██╔██╗ ██║
██║ ╚████║███████╗██╔╝ ██╗ ██║
╚═╝ ╚═══╝╚══════╝╚═╝ ╚═╝ ╚═╝
========================================
NexT version 8.13.1
Documentation: https://theme-next.js.org
========================================
hexo: 6.3.0
hexo-cli: 4.3.0
os: darwin 21.6.0 12.6

node: 18.11.0
v8: 10.2.154.15-node.12
uv: 1.44.2
zlib: 1.2.11
brotli: 1.0.9
ares: 1.18.1
modules: 108
nghttp2: 1.50.0
napi: 8
llhttp: 6.0.10
openssl: 1.1.1q
cldr: 41.0
icu: 71.1
tz: 2022a
unicode: 14.0

配置

在Hexo中有两份主要的配置文件,其名称都是_config.yml。 其中,一份位于站点根目录下,主要包含Hexo本身的配置;另一份位于主题目录下/node_modules/hexo-theme-next/_config.yml,这份配置由主题作者提供,主要用于配置主题相关的选项。为了描述方便,在以下说明中,将前者称为站点配置文件, 后者称为主题配置文件

主题配置文件

主题配置文件文件/node_modules/hexo-theme-next/_config.yml中修改:

Schemes

1
scheme: Pisces

language

1
language: en
1
2
3
4
5
6
7
8
9
10
11
12
13
# Usage: `Key: /link/ || icon`
# Key is the name of menu item. If the translation for this item is available, the translated text will be loaded, otherwise the Key name will be used. Key is case-sensitive.
# Value before `||` delimiter is the target link, value after `||` delimiter is the name of Font Awesome icon.
# External url should start with http:// or https://
menu:
home: / || fa fa-home
about: /about/ || fa fa-user
tags: /tags/ || fa fa-tags
categories: /categories/ || fa fa-th
archives: /archives/ || fa fa-archive
#schedule: /schedule/ || fa fa-calendar
#sitemap: /sitemap.xml || fa fa-sitemap
#commonweal: /404/ || fa fa-heartbeat

手动创建页面tags:

1
hexo new page "tags"

修改Blog/source/tags/index.md

1
2
3
4
5
6
---
title: tags
date: 2016-01-28 18:44:44
type: "tags"
comments: false
---
1
hexo new page "categories"

修改Blog/source/categories/index.md

1
2
3
4
5
6
---
title: categories
date: 2016-01-28 19:18:13
type: "categories"
comments: false
---

手动创建页面categories:

1
hexo new page "categories"

修改Blog/source/categories/index.md

1
2
3
4
5
6
---
title: categories
date: 2016-01-28 19:18:13
type: "categories"
comments: false
---

手动创建页面about:

1
hexo new page "about"

修改Blog/source/about/index.md

1
2
3
4
5
---
title: about
date: 2016-01-28 19:43:39
comments: false
---
1
2
3
sidebar:
position: left
display: always

头像

放置在source/images/目录下,配置为:avatar: /images/avatar.png

1
2
3
4
5
6
# Sidebar Avatar
avatar:
# Replace the default image and set the url here.
url: /images/avatar.png
# If true, the avatar will be displayed in circle.
rounded: true

Google Analytics

1
2
3
4
5
6
7
# Google Analytics
# See: https://analytics.google.com
google_analytics:
tracking_id: G-FKR2744S26
# By default, NexT will load an external gtag.js script on your site.
# If you only need the pageview feature, set the following option to true to get a better performance.
only_pageview: false

社交链接

1
2
3
4
5
social:
GitHub: https://github.com/HanjieLuo || fab fa-github
Linkedin: https://www.linkedin.com/in/hanjie-luo-89602197 || fa-brands fa-linkedin-in
YouTube: https://www.youtube.com/luohanjie || fab fa-youtube
Bilibili: https://space.bilibili.com/319752 || fa-brands fa-bilibili

disqus

1
2
3
4
disqus:
enable: true
shortname: luohanjie
count: true

代码高亮

1
2
3
4
5
6
7
8
9
10
11
12
13
14
codeblock:
# Code Highlight theme
# All available themes: https://theme-next.js.org/highlight/
theme:
light: stackoverflow-light
dark: stackoverflow-dark
prism:
light: prism
dark: prism-dark
# Add copy button on codeblock
copy_button:
enable: false
# Available values: default | flat | mac
style:

访客统计、访问次数统计、文章阅读次数统计

1
2
3
4
5
6
7
8
9
10
# Show Views / Visitors of the website / page with busuanzi.
# For more information: http://ibruce.info/2015/04/04/busuanzi/
busuanzi_count:
enable: true
total_visitors: true
total_visitors_icon: fa fa-user
total_views: true
total_views_icon: fa fa-eye
post_views: true
post_views_icon: far fa-eye

本地显示的数字是随机的,部署到服务器上面就好了。

站点建立时间

1
2
3
footer:
# Specify the year when the site was setup. If not defined, current year will be used.
since: 2016
1
npm install hexo-generator-searchdb

Blog\_config.yml添加:

1
2
3
4
5
search:
path: search.xml
field: post
content: true
format: html

修改/node_modules/hexo-theme-next/_config.yml:

1
2
3
4
5
6
7
8
9
10
11
12
13
# Local search
# Dependencies: https://github.com/next-theme/hexo-generator-searchdb
local_search:
enable: true
# If auto, trigger search by changing input.
# If manual, trigger search by pressing enter key or search button.
trigger: auto
# Show top n results per article, show all results by setting to -1
top_n_per_article: 1
# Unescape html strings to the readable one.
unescape: false
# Preload the search data when the page loads.
preload: false

mathjax 5

1
2
3
4
brew install pandoc

npm uninstall hexo-renderer-marked
npm install hexo-renderer-pandoc
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Math Formulas Render Support
# Warning: Please install / uninstall the relevant renderer according to the documentation.
# See: https://theme-next.js.org/docs/third-party-services/math-equations
# Server-side plugin: https://github.com/next-theme/hexo-filter-mathjax
math:
# Default (false) will load mathjax / katex script on demand.
# That is it only render those page which has `mathjax: true` in front-matter.
# If you set it to true, it will load mathjax / katex script EVERY PAGE.
every_page: true

mathjax:
enable: true
# Available values: none | ams | all
tags: ams

katex:
enable: false
# See: https://github.com/KaTeX/KaTeX/tree/master/contrib/copy-tex
copy_tex: false

CDN

1
2
3
4
5
vendors:
internal: local
plugins: cdnjs
# Custom CDN URL
custom_cdn_url: https://lib.baomitu.com/${cdnjs_name}/${version}/${cdnjs_file}

Favicon

准备一张260x260以上大的图片:

favicon

用软件转换成svg格式,改名为logo.svg。在realfavicongenerator或者websiteplanet(It allows to create favicons from pictures that are up to 5 MB. thanks to Virgy)上生成favicon_package包,包括favicon-16x16-next.png(由favicon-16x16.png改名),favicon-32x32-next.png(由favicon-32x32.png改名),apple-touch-icon-next.png(由apple-touch-icon.png改名)。将以上4张图片复制到Blog/node_modules/hexo-theme-next/source/images/进行替换。

站点配置文件

站点配置文件文件_config.yml中修改:

网站

1
2
3
4
5
6
7
title: Hanjie's Blog
subtitle: 一只有理想的羊驼
description: ''
keywords:
author: Hanjie Luo
language: en
timezone: ''

网址

1
2
3
4
5
6
7
8
# URL
## Set your site url here. For example, if you use GitHub Page, set url as 'https://username.github.io/project'
url: http://luohanjie.com
permalink: :year-:month-:day/:urlname.html
permalink_defaults:
pretty_urls:
trailing_index: true # Set to false to remove trailing 'index.html' from permalinks
trailing_html: true # Set to false to remove trailing '.html' from permalinks

文章的Formatter:

1
2
3
4
5
6
7
title: Mac上安装和配置Hexo博客
urlname: install-and-create-a-blog-with-hexo-on-mac
categories: [Tech, Web]
date: 2022-10-25 16:08:00
tags: [Web, Mac, Hexo, Next]
published: true
---

文章

1
2
3
4
5
6
7
8
# Writing
new_post_name: :title.md # File name of new posts
filename_case: 1
highlight:
enable: true
line_number: false
auto_detect: true
tab_replace:

分页

1
2
3
4
index_generator:
path: ''
per_page: 3
order_by: -date

hexo-renderer-markdown-it

支持footnote

可以通过将markdown渲染器替换为hexo-renderer-markdown-it,使得支持footnote功能6。必须要先卸载原先的渲染器,然后安装:

1
2
npm un hexo-renderer-marked --save
npm i hexo-renderer-markdown-it --save

然后在网站\_config.yml文件中添加:

1
2
3
markdown:
plugins:
- markdown-it-footnote

然后发现每一条footnote间都有一行间隔。想要去掉的话,可以在/Blogs//Users/luohanjie/Workspace/Web/Blog/node_modules/hexo-theme-next/source/css/_common/components/post/post-footer.styl中添加:

1
2
3
.footnote-item p {
margin-bottom: 0
}

hexo-renderer-markdown-it还支持很多其他的功能,具体可以到官网了解。

html

The html setting defines whether or not HTML content inside the document should be escaped or passed to the final result.

1
2
3
markdown:
render:
html: true # Doesn't escape HTML content

支持插入pdf

1
npm install --save hexo-pdf

主题配置文件文件/node_modules/hexo-theme-next/_config.yml中修改:

1
2
3
4
pdf:
enable: true
# Default height
height: 500px

使用:

1
2
3
4
5
6
7
8
# Normal PDF
{% pdf http://7xov2f.com1.z0.glb.clouddn.com/bash_freshman.pdf %}

#Google drive
{% pdf https://drive.google.com/file/d/0B6qSwdwPxPRdTEliX0dhQ2JfUEU/preview %}

#Slideshare
{% pdf http://www.slideshare.net/slideshow/embed_code/key/8Jl0hUt2OKUOOE %}

自定义字体

修改/node_modules/hexo-theme-next/_config.yml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
font:
enable: true

# Uri of fonts host, e.g. https://fonts.googleapis.com (Default).
host: https://cdn.baomitu.com

# Font options:
# `external: true` will load this font family from `host` above.
# `family: Times New Roman`. Without any quotes.
# `size: x.x`. Use `em` as unit. Default: 1 (16px)

# Global font settings used for all elements inside <body>.
global:
external: true
family: Noto Serif SC
size:

# Font settings for site title (.site-title).
title:
external: true
family: Noto Serif SC
size:

# Font settings for headlines (<h1> to <h6>).
headings:
external: true
family: Noto Serif SC
size:

# Font settings for posts (.post-body).
posts:
external: true
family: Noto Serif SC

# Font settings for <code> and code blocks.
codes:
external: true
family: Source Code Pro

编辑主题的Blog/node_modules/hexo-theme-next/source/css/_variables/base.styl文件:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
// Font size
$font-size-base = (hexo-config('font.enable') and hexo-config('font.global.size') is a 'unit') ? unit(hexo-config('font.global.size'), em) : 1em;
$font-size-smallest = .55em;
$font-size-smaller = .6125em;
$font-size-small = .675em;
$font-size-medium = 0.8em;
$font-size-large = 0.925em;
$font-size-larger = 1.15em;
$font-size-largest = 1.7em;


// Headings font size
$font-size-headings-step = .125em;
$font-size-headings-base = (hexo-config('font.enable') and hexo-config('font.headings.size') is a 'unit') ? unit(hexo-config('font.headings.size'), em) : 1.425em;

sitemap

1
npm install hexo-generator-sitemap --save

服务器部署 7

使用Git Hook自动部署到vps上。

创建用户

服务器上,创建git用户并且赋予权限:

1
2
3
adduser git
chmod 740 /etc/sudoers
nano /etc/sudoers
1
2
3
# User privilege specification
root ALL=(ALL:ALL) ALL
git ALL=(ALL:ALL) ALL
1
chmod 440 /etc/sudoers

建立密钥

本地Mac上,查看密钥(没有的话需要创建一个):

1
ls -al ~/.ssh

创建钥匙密钥:

1
2
3
4
cd ~/.ssh 
ssh-keygen -t rsa -C "你的邮箱" // 执行这个命令会提示输入用于保存的密钥名和口令之类的,都不填

touch ~/.ssh/config
1
2
3
4
Host *
AddKeysToAgent yes
UseKeychain yes
IdentityFile ~/.ssh/id_rsa
1
ssh-add -K ~/.ssh/id_rsa

复制公匙:

1
pbcopy < ~/.ssh/id_rsa.pub

服务器上:

1
2
3
4
cd /home/git                   //切换到git用户目录
mkdir .ssh //创建.ssh目录
cd .ssh //进入.ssh目录
nano authorized_keys //将本地的公钥复制到authorized_keys文件里
1
2
3
chmod 700 /home/git
chmod 700 /home/git/.ssh #只有拥有者有读、写、执行权限
chmod 600 /home/git/.ssh/authorized_keys #只有拥有者有读写权限
1
nano /etc/ssh/sshd_config
1
2
3
4
// 取消这些注释
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
1
sudo /etc/init.d/ssh restart

关闭git用户的shell权限,设置后git用户可以通过ssh正常使用git服务,但无法登录shell:

1
nano /etc/passwd
1
git:x:1001:1001:,,,:/home/git:/bin/bash 改成 git:x:1001:1001:,,,:/home/git:/usr/bin/git-shell

Git仓库

服务器上:

1
2
3
4
5
6
apt-get install git

cd /home/git //切换到git用户目录
mkdir blog.git //创建仓库目录,以blog.git为例
cd blog.git //进入仓库目录
git init --bare //使用--bare参数初始化为裸仓库,这样创建的仓库不包含工作区

修改/home/git/blog.git目录的用户组权限为git:git

1
2
3
sudo chown git:git -R /home/git/.ssh
sudo chown git:git -R /home/git/.ssh/authorized_keys
sudo chown git:git -R /home/git/blog.git

本地Mac上测试:

1
2
3
ssh -v git@ip

git clone ssh://git@ip:port/home/git/blog.git

Git Hooks自动部署

在本地编辑Markdown文章,然后使用Git推送到VPS的Git仓库。Git Hooks实际上就是当Git仓库收到最新的push时,将Git仓库接受到的内容复制到VPS上的网站目录内。相当于完成了手动将public文件夹复制到VPS的网站根目录里的操作。

建立网站根目录:

1
2
3
cd /home/git
mkdir blog
sudo chown git:git -R /home/git/blog

创建post-receive文件:

1
2
cd /home/git/blog.git/hooks     //切换到hooks目录下
nano post-receive //创建post-receive文件并编辑
1
2
3
4
5
6
7
8
#!/bin/bash
GIT_REPO=/home/git/blog.git
TMP_GIT_CLONE=/tmp/blog
PUBLIC_WWW=/home/git/blog
rm -rf ${TMP_GIT_CLONE}
git clone $GIT_REPO $TMP_GIT_CLONE
rm -rf ${PUBLIC_WWW}/*
cp -rf ${TMP_GIT_CLONE}/* ${PUBLIC_WWW}
1
chmod +x post-receive

部署Nginx

服务器上:

1
2
3
sudo apt-get install nginx

sudo vim /etc/nginx/conf.d/blog.conf

写入:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
server {
listen 80;
server_name example.com; # 自己的域名

root /home/git/blog; # 刚才说的路径
access_log /var/log/nginx/blog_access.log;
error_log /var/log/nginx/blog_error.log;

# 这里是针对静态资源文件做个缓存
location ~* \.(?:ico|css|js|gif|jpe?g|png)$ {
expires 1d;
add_header Pragma public;
add_header Cache-Control "public";
}

# 如果使用了hexo-pdf插件,并且将文件放到服务器上,则还需添加
location ~* \.(pdf)$ {
add_header "Access-Control-Allow-Origin" "http://nagland.github.io";
expires 1d;
}

# 这里就是把请求转给我们的静态文件了
location / {
root /home/git/blog;
if (-f $request_filename) {
rewrite ^/(.*)$ /$1 break;
}
}
}

由于nginx的运行用户没有权限访问网站所在的目录,检查:

1
/etc/nginx/nginx.conf

可以将user的值改为git

重启:

1
service nginx restart

Hexo远程部署

本地Blog文件夹下:

1
npm install hexo-deployer-git --save

站点配置文件文件_config.yml中修改:

1
2
3
4
deploy:
type: git
repo: git@ip:blog.git
branch: master
1
git clone ssh://git@ip:port/home/git/blog.git
1
2
3
hexo clean
hexo generate
hexo deploy

Unity: 2021.2.10f1c1 silicon OS: macOS Monterey 12.1 Compiler: Clang 13.0.0 arm64-apple-darwin21.1.0

一个简单的Demo

新建一个c++文件unity_api.cpp:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#if defined(__CYGWIN32__)
#define UNITY_CPP_INTEROP_DLL_BRIDGE __declspec(dllexport) __stdcall
#elif defined(WIN32) || defined(_WIN32) || defined(__WIN32__) || defined(_WIN64) || defined(WINAPI_FAMILY)
#define UNITY_CPP_INTEROP_DLL_BRIDGE __declspec(dllexport) __stdcall
#elif defined(__MACH__) || defined(__ANDROID__) || defined(__linux__) || defined(__QNX__)
#define UNITY_CPP_INTEROP_DLL_BRIDGE
#else
#define UNITY_CPP_INTEROP_DLL_BRIDGE
#endif

extern "C" {
UNITY_CPP_INTEROP_DLL_BRIDGE float Add(const float a,
const float b) {
return a + b;
}
}

CMakeLists.txt:

1
2
3
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wall -O3 -D__ARM_NEON__ -DENABLE_NEON -Wno-unused-result -march=armv8-a+fp+simd+crypto -DCMAKE_APPLE_SILICON_PROCESSOR=arm64")

add_library(unity_api SHARED unity_api.cpp)

编译生成libunity_api.dylib,并且放到UnityProject/Assets/Plugins/macOS文件夹下,并且在Unity中对库进行设置:

unity_inspector

编写Unity的C#脚本PluginImport.cs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
using System;
using System.Runtime.InteropServices;
using UnityEngine;

public class PluginImport : MonoBehaviour
{
[DllImport("unity_api", CallingConvention = CallingConvention.Cdecl)]
public static extern float Add(float a, float b);

// Start is called before the first frame update
void Start()
{
Debug.Log(Add(1, 2));
}

}

并且将脚本PluginImport.cs挂靠到Unity的Main Camera下,并且运行,可以在Console下看到结果:

unity_simple_test_output

使用OpenCV库

修改c++文件unity_api.cpp:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
#include <opencv2/opencv.hpp>

#if defined(__CYGWIN32__)
#define UNITY_CPP_INTEROP_DLL_BRIDGE __declspec(dllexport) __stdcall
#elif defined(WIN32) || defined(_WIN32) || defined(__WIN32__) || defined(_WIN64) || defined(WINAPI_FAMILY)
#define UNITY_CPP_INTEROP_DLL_BRIDGE __declspec(dllexport) __stdcall
#elif defined(__MACH__) || defined(__ANDROID__) || defined(__linux__) || defined(__QNX__)
#define UNITY_CPP_INTEROP_DLL_BRIDGE
#else
#define UNITY_CPP_INTEROP_DLL_BRIDGE
#endif



extern "C" {
UNITY_CPP_INTEROP_DLL_BRIDGE float Add(const float a,
const float b) {
return a + b;
}

UNITY_CPP_INTEROP_DLL_BRIDGE bool ToGray(const char *color_img_file,
const char *gray_img_file) {

cv::Mat img = cv::imread(std::string(color_img_file));
if (img.empty()) {
return false;
}

cv::Mat img_gray;
cv::cvtColor(img, img_gray, cv::COLOR_BGR2GRAY);
cv::imwrite(std::string(gray_img_file), img_gray);
return true;
}
}

CMakeLists.txt:

1
2
3
4
5
6
7
8
9
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -Wall -O3 -D__ARM_NEON__ -DENABLE_NEON -Wno-unused-result -march=armv8-a+fp+simd+crypto -DCMAKE_APPLE_SILICON_PROCESSOR=arm64")

find_package(OpenCV REQUIRED)

include_directories(${OpenCV_INCLUDE_DIRS})
link_directories(${OpenCV_LIBRARY_DIRS})

add_library(unity_api SHARED unity_api.cpp)
target_link_libraries(unity_api ${OpenCV_LIBS})

编译生成libunity_api.dylib,并且放到UnityProject/Assets/Plugins/macOS文件夹下。修改Unity的C#脚本PluginImport.cs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
using System;
using System.Runtime.InteropServices;
using UnityEngine;

public class PluginImport : MonoBehaviour
{
[DllImport("unity_api", CallingConvention = CallingConvention.Cdecl)]
public static extern float Add(float a, float b);

[DllImport("unity_api", CallingConvention = CallingConvention.Cdecl)]
public static extern bool ToGray(string color_img_file, string gray_img_file);

// Start is called before the first frame update
void Start()
{
Debug.Log(ToGray("/Users/luohanjie/Downloads/lenna_color.png", "/Users/luohanjie/Downloads/lenna_gray.png"));
}

}

并且将脚本PluginImport.cs挂靠到Unity的Main Camera下,并且运行,可以得到的灰度图lenna_gray.png

lenna_gray

通过Unity调用网络摄像头

在Unity的Main Camera下添加一个Raw Image,命名为Background并且设置TagbackgroundCanvas节点中中将Render Mode设置为Screen Space - Camera,并且将Main Camera节点拖拽到Render Camera上。

background1

Background节点(Raw Image)中,将LeftTopPos ZRightBottom都设置为0。为了使得显示的图片比例正确,添加Aspect Ratio Fitter并且Aspect Mode设置为Fit In Parent

background2

Main Camera下添加脚本WebCamera.cs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using UnityEngine;
using UnityEngine.UI;
using System.Runtime.InteropServices;


public class WebCamera : MonoBehaviour
{
public string cam_name = "FaceTime HD Camera";
public int cam_width = 1280;
public int cam_heidht = 720;
public int cam_fps = 30;
public AspectRatioFitter aspect_fitter;
private WebCamTexture web_cam_tex = null;
private bool is_init = false;

// Start is called before the first frame update
void Start()
{
WebCamDevice[] devices = WebCamTexture.devices;
for (int i = 0; i < devices.Length; i++)
{
if (devices[i].name == cam_name)
{
is_init = true;
Debug.Log("Connect camera to " + cam_name);
break;
}
}

if (is_init == false) {
Debug.Log("Can not find " + cam_name);
return;
}

web_cam_tex = new WebCamTexture(cam_name, cam_width, cam_heidht, cam_fps);

if (web_cam_tex == null) {
is_init = false;
Debug.Log ("Can not new WebCamTexture");
return;
}

GameObject.FindWithTag("background").GetComponent<RawImage>().texture = web_cam_tex;

web_cam_tex.Play();
}

void OnStart (Texture preview) {
aspect_fitter.aspectRatio = preview.width / (float)preview.height;
}

// Update is called once per frame
void Update()
{

}
}

然后将Background节点拖拽到Main Camera节点中,Web Camera (Script)项目下的Aspect_fitter项目上:

background1

点击运行可以看到结果:

webcam_unity

将c++ OpenCV的Mat传输到Unity并且显示

c++文件unity_api.cpp:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
#if defined(__CYGWIN32__)
#define UNITY_CPP_INTEROP_DLL_BRIDGE __declspec(dllexport) __stdcall
#elif defined(WIN32) || defined(_WIN32) || defined(__WIN32__) || defined(_WIN64) || defined(WINAPI_FAMILY)
#define UNITY_CPP_INTEROP_DLL_BRIDGE __declspec(dllexport) __stdcall
#elif defined(__MACH__) || defined(__ANDROID__) || defined(__linux__) || defined(__QNX__)
#define UNITY_CPP_INTEROP_DLL_BRIDGE
#else
#define UNITY_CPP_INTEROP_DLL_BRIDGE
#endif

#include <opencv2/opencv.hpp>

cv::Mat img_;

extern "C" {
UNITY_CPP_INTEROP_DLL_BRIDGE bool ReadImage(const char *color_img_file,
unsigned char*& rgb24_pixels,
int& width,
int& height,
int& size) {

img_ = cv::imread(std::string(color_img_file));
if (img_.empty()) {
return false;
}

// Unity中图片坐标系原点在图片的坐下角,与OpenCV中的定义不一样。
// 为了显示正确,需要flip操作。
cv::flip(img_, img_, 0);
cv::cvtColor(img_, img_, cv::COLOR_BGR2RGB);

rgb24_pixels = img_.data;
width = img_.cols;
height = img_.cols;
size = img_.cols * img_.rows * img_.channels();
return true;
}
}

Unity的C#脚本PluginImport.cs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using UnityEngine;
using UnityEngine.UI;
using System.Runtime.InteropServices;

public class PluginImport : MonoBehaviour
{
[DllImport("unity_api", CallingConvention = CallingConvention.Cdecl)]
public static extern bool ReadImage(string color_img_file, ref IntPtr data, ref int width, ref int height, ref int size);

// Start is called before the first frame update
void Start()
{
IntPtr data = IntPtr.Zero;
int width = 0;
int height = 0;
int size = 0;

bool flag = ReadImage("/Users/luohanjie/Downloads/lenna_color.png", ref data, ref width, ref height, ref size);

if (!flag || size <= 0) return;

byte[] buffer = new byte[size];

Texture2D tex = new Texture2D(width, height, TextureFormat.RGB24, false, false);

System.Runtime.InteropServices.Marshal.Copy(data, buffer, 0, size);
tex.LoadRawTextureData(buffer);
tex.Apply();

RawImage background = GameObject.FindWithTag("background").GetComponent<RawImage>();
background.texture = tex;
}
}

在Unity的Main Camera下添加一个Raw Image,命名为Background并且设置Tagbackground

将脚本PluginImport.cs挂靠到Unity的Main Camera下,并且运行:

mat2unity

从c++读取Unity物体的Pose

c++文件unity_api.cpp:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
#if defined(__CYGWIN32__)
#define UNITY_CPP_INTEROP_DLL_BRIDGE __declspec(dllexport) __stdcall
#elif defined(WIN32) || defined(_WIN32) || defined(__WIN32__) || defined(_WIN64) || defined(WINAPI_FAMILY)
#define UNITY_CPP_INTEROP_DLL_BRIDGE __declspec(dllexport) __stdcall
#elif defined(__MACH__) || defined(__ANDROID__) || defined(__linux__) || defined(__QNX__)
#define UNITY_CPP_INTEROP_DLL_BRIDGE
#else
#define UNITY_CPP_INTEROP_DLL_BRIDGE
#endif

extern "C" {
UNITY_CPP_INTEROP_DLL_BRIDGE void GetPose(float *pose_out) {
// 假定我们有一个右手系的pose = [x, y, z, qw, qx, qy, qz],表示物体在摄像头坐标系下(opencv定义)的位姿。
std::array<float, 7> pose;

// 从OpenCV的右手系转到Unity中使用的左手系
pose[1] = -pose[1]; //y
pose[4] = -pose[4]; //qx
pose[6] = -pose[6]; //qz

memcpy(pose_out, pose.data(), 7 * sizeof(float));
}
}

Unity的C#脚本PluginImport.cs:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using UnityEngine;
using UnityEngine.UI;
using System.Runtime.InteropServices;

public class PluginImport : MonoBehaviour
{
[DllImport("unity_api", CallingConvention = CallingConvention.Cdecl)]
public static extern void GetPose([MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 0)] float[] pose);

// Start is called before the first frame update
void Start()
{
float[] obj_pose = new float[7]; // x, y, z, qw, qx, qy, qz
GetPose(obj_pose);

//方法1,根据读取的Pose设置Unity中物体的姿态
GameObject obj = GameObject.FindWithTag("obj_tag_name");
obj.transform.position = new Vector3((float)obj_pose[0], (float)obj_pose[1], (float)obj_pose[2]);
obj.transform.rotation = new Quaternion((float)obj_pose[4], (float)obj_pose[5], (float)obj_pose[6], (float)obj_pose[3]);

//方法2,根据读取的Pose设置Unity中摄像头的姿态
Camera cam_obj = GameObject.FindWithTag("MainCamera").GetComponent<Camera>();
cam_obj.transform.position = new Vector3((float)obj_pose[0], (float)obj_pose[1], (float)obj_pose[2]);
cam_obj.transform.rotation = new Quaternion((float)obj_pose[4], (float)obj_pose[5], (float)obj_pose[6], (float)obj_pose[3]);
}
}
0%