Skip to content

Commit

Permalink
updates
Browse files Browse the repository at this point in the history
  • Loading branch information
huataihuang committed Jul 29, 2020
1 parent d65a5b9 commit de6c347
Show file tree
Hide file tree
Showing 19 changed files with 259 additions and 27 deletions.
9 changes: 9 additions & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,10 @@
* [Intel Skylake/Kaby Lake处理器系列简介](os/linux/kernel/cpu/skylake_kaby_lake_processors.md)
* [Intel Skylake/Kaby Lake处理器超线程bug及处理](os/linux/kernel/cpu/skylake_kaby_lake_processors_broken_hyper-threading.md)
* [dmidecode分析](os/linux/kernel/cpu/dmidecode.md)
* ["unchecked MSR access error"](os/linux/kernel/cpu/unchecked_msr_access_error.md)
* [内核调度](os/linux/kernel/scheduler/README.md)
* [Linux内核调度基础知识](os/linux/kernel/scheduler/linux_kernel_scheduler_basic.md)
* [unthrottle_cfs_rq错误修复](os/linux/kernel/scheduler/fix_unthrottle_cfs_rq.md)
* [内核模块](os/linux/kernel/module/README.md)
* [内核模块参数](os/linux/kernel/module/kernel_module_parameters.md)
* [内核模式设置KMS](os/linux/kernel/module/kms.md)
Expand Down Expand Up @@ -609,6 +613,7 @@
* [在CentOS中部署SR-IOV](virtual/intel/vt-d/implement_sr-iov_in_centos.md)
* [内存虚拟化:扩展页表EPT](virtual/intel/ept/README.md)
* [I/O虚拟化:SR-IOV](virtual/intel/sr-iov/README.md)
* [SR-IOV技术介绍](virtual/intel/sr-iov/introduce_sr-iov.md)
* [I/O虚拟化:数据直接I/O(DDIO)](virtual/intel/ddio/README.md)
* [网络数据虚拟化:DPDK](virtual/intel/dpdk/README.md)
* [如何判断网卡是否支持DPDK技术](virtual/intel/dpdk/determine_ethernet_adapter_support_dpdk.md)
Expand Down Expand Up @@ -939,6 +944,8 @@
* [快速起步的一个例子:猜数字](develop/java/a_guess_game_example.md)
* [jshell](develop/java/shell.md)
* [groovy](develop/java/groovy/README.md)
* [安装groovy](develop/java/groovy/install_groovy.md)
* [groovy简单语法](develop/java/groovy/groovy_simple_grammar.md)
* [Java Trouble Shoot](develop/java/troubleshoot/README.md)
* [Druid连接池"druid.pool.GetConnectionTimeoutException"](develop/java/troubleshoot/druid_pool_get_connection_timeout_excetpion.md)
* [Go](develop/go/README.md)
Expand Down Expand Up @@ -981,6 +988,7 @@
* [使用MultiROM在Nexus 5上安装多操作系统](develop/android/startup/install_ubuntu_touch_and_android_through_multirom.md)
* [LineageOS](develop/android/lineageos/README.md)
* [构建hammerhead(Nexus 5)的LineageOS ROM](develop/android/lineageos/build_lineageos_for_hammerhead.md)
* [构建marlin(Pixel XL)的LineageOS ROM](develop/android/lineageos/build_lineageos_for_marlin.md)
* [在hammerhead(Nexus 5)上安装LineageOS](develop/android/lineageos/install_lineageos_on_hammerhead.md)
* [在hammerhead(Nexus 5)构建Android Go](build_nexus_5_android_go.md)
* [我的Android设备](develop/android/device/README.md)
Expand Down Expand Up @@ -1423,6 +1431,7 @@
* [Spark](big_data/spark/README.md)
* [快速起步](big_data/spark/startup/README.md)
* [单机安装部署Spark](big_data/spark/startup/deploy_spark_single_node.md)
* [Spark机器学习](big_data/spark/startup/machine_learning/README.md)
* [Elasticsearch](big_data/elasticsearch/README.md)
* [ESK简介](big_data/elasticsearch/introduce_esk.md)
* [ESK快速安装](big_data/elasticsearch/install_esk.md)
Expand Down
5 changes: 5 additions & 0 deletions big_data/spark/machine_learning/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Spark机器学习

# GPU加速Spark

* [nvidia提供GPU加速Spark支持](https://www.nvidia.com/en-us/deep-learning-ai/solutions/data-science/apache-spark-3/)
6 changes: 6 additions & 0 deletions develop/android/lineageos/build_lineageos_for_marlin.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
我购买了一台二手的Pixel XL,将实践自己编译LineageOS系统。

# 参考

* [Build lineageos for marlin](https://wiki.lineageos.org/devices/marlin/build)
* [marlin设备的LineageOS相关信息](https://wiki.lineageos.org/devices/marlin)
6 changes: 6 additions & 0 deletions develop/android/startup/android_develop_environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,12 @@ Running Intel® HAXM installer
HAXM installation failed. To install HAXM follow the instructions found at: https://software.intel.com/android/articles/installation-instructions-for-intel-hardware-accelerated-execution-manager-mac-os-x
```

# 配置离线构建环境

Android Studio默认需要连接Google才能使用,因为使用过程中需要下载很多组件。但是在瓷器国,在线更新非常麻烦,需要搭建翻墙梯子,效率很低。

Android Studio也提供了离线部署开发环境的方法,请参考 [配置离线构建依赖项](https://developer.android.com/studio/intro/studio-config#offline)

# 参考

* [怎么在安卓手机上注册谷歌账号? ](https://www.zhihu.com/question/26397758)
Expand Down
2 changes: 1 addition & 1 deletion develop/java/groovy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Groovy使用Java编写,语法和Java类似。Groovy的代码能够和Java代码和好结合,也能用于扩展现有代码。相对于Java,Groovy的代码灵活性有非常显著的提升,并且可以直接使用其他Java语言编写的库。

在W3Cschool网站有一个非常[简明groovy教程](https://www.w3cschool.cn/groovy/groovy_overview.html)
在W3Cschool网站有一个非常[简明groovy教程](https://www.w3cschool.cn/groovy/groovy_overview.html),我主要参考这个简明手册学习。

简书上的 [Groovy简介和一点资源](https://www.jianshu.com/p/5897ecef546f)

Expand Down
Empty file.
19 changes: 19 additions & 0 deletions develop/java/groovy/groovy_split_string.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
groovy可以使用 `|` 来表示 `` ,在字符串分隔时可以使用

```groovy
String[]tokens = pdfName.split("-|\\.");
```

可以将 `AA.BB-CC-DD.zip` 切分成

```
AA
BB
CC
DD
zip
```

# 参考

* [Use String.split() with multiple delimiters](https://stackoverflow.com/questions/5993779/use-string-split-with-multiple-delimiters)
73 changes: 73 additions & 0 deletions develop/java/groovy/install_groovy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# 安装groovy

## 下载软件zip包安装

[groovy官网](http://www.groovy-lang.org/) 下载 [apache-groovy-sdk-3.0.5.zip](https://dl.bintray.com/groovy/maven/apache-groovy-binary-3.0.5.zip) 安装方法参考 [Install Groovy](http://groovy-lang.org/install.html)

```bash
unzip apache-groovy-sdk-3.0.5.zip -d /usr/local
```

上述指定解压缩目录到 `/usr/local` 下,就可以通过 `/usr/local/bin/groovy` 来使用了。

> Groovy 3.0运行需要Java 6+到Java 8,目前在Java 9 snapshots上运行还有一些问题。建议使用Java 8。[macOS上使用多个JDK版本](https://cloud-atlas.readthedocs.io/zh_CN/latest/macos_ios/studio/multi_jdk_on_macos.html)
> 如果想要将Groovy嵌入到应用程序,需要使用合适的 maven repositories或者 [JCenter maven repository](https://oss.jfrog.org/oss-release-local/org/codehaus/groovy)
例如在maven配置中添加:

```xml
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy</artifactId>
<version>3.0.5</version>
```

如果是使用gradle构建,则添加

```
'org.codehaus.groovy:groovy:3.0.5'
```

## 通过SDKMAN!安装

* 更为简单的安装方法是使用脚本安装

```bash
curl -s get.sdkman.io | bash
```

* 然后开启一个新的终端,或者输入命令:

```bash
source "/Users/huatai/.sdkman/bin/sdkman-init.sh"
```

* 最后执行以下命令安装最新的stable Groovy:

```bash
sdk install groovy
```

> 通过 SDKMAN!安装不需要root权限,安装以后执行程序位于 `/Users/huatai/.sdkman/candidates/groovy/current/bin/groovy`
# 运行groovy

通过运行一些简单的脚本或者交互ide来验证groovy安装是否成功。

* 启动groovy shell:

```bash
groovysh
```

* 或者启动一个groovyConsole图形交互控制台:

```bash
groovyConsole
```

* 或者执行任何Groovy脚本:

```bash
groovy SomeScript
```
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,36 @@
* 使用`ping -i 0.1 xxx.x.x.x`来加快ping包频率,可以快速反映出网络是否有性能问题。(默认ping间隔1秒,对于网络抖动不容易发现问题)
* 使用`tcpdump -n host xxx.x.x.x and icmp`检查两边服务器上ping包,当出现响应延迟时,记录下ping包的id,然后根据id检查两边tcpdump的时间戳,对比时间戳差异就可以知道在那段出现延迟。

案例命令

```
tcpdump -nni eth0 icmp and host IP1 and host IP2
```

写入抓包文件

```
tcpdump -w /tmp/icmp.pcap -nni eth0 icmp
```

如果只抓包100mb

```
tcpdump -C 100 -w /tmp/icmp.pcap -nni eth0 icmp
```

还可以重复抓多个100mb的包,例如以下是存储50个文件(也就是 5 gb)

```
tcpdump -W 50 -C 100 -w /tmp/icmp.pcap -nni eth0 icmp
```

存储的抓包文件可以使用Wireshark查看,显示过滤:

```
ip.addr==IP and not icmp.resp_in and icmp.type==8
```

# 案例

* A主机:192.168.1.109
Expand Down Expand Up @@ -49,4 +79,8 @@ sudo tcpdump -n -i eth0 host 192.168.1.109 and icmp

返回的包中间网络时间: `20:47:02.864966 - 20:47:02.863233 = 1.733 ms`

可以确定从 `192.168.4.135``192.168.1.109` 出现了较大延迟,可以检查一下这个链路方向网络交换机以及链路。
可以确定从 `192.168.4.135``192.168.1.109` 出现了较大延迟,可以检查一下这个链路方向网络交换机以及链路。

# 参考

* [Using tcpdump to verify ICMP polling.](https://www.ibm.com/support/pages/using-tcpdump-verify-icmp-polling)
28 changes: 28 additions & 0 deletions os/linux/kernel/cpu/unchecked_msr_access_error.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
ECS虚拟机启动中有关于MSR报错,可能和处理器idle有关,后续需要分析

```
[Sat Jun 20 23:42:04 2020] intel_idle: MWAIT substates: 0x2020
[Sat Jun 20 23:42:04 2020] intel_idle: v0.4.1 model 0x55
[Sat Jun 20 23:42:04 2020] tsc: Marking TSC unstable due to TSC halts in idle states deeper than C2
[Sat Jun 20 23:42:04 2020] unchecked MSR access error: RDMSR from 0x1fc at rIP: 0xffffffff9a861db3 (native_read_msr+0x3/0x30)
[Sat Jun 20 23:42:04 2020] Call Trace:
[Sat Jun 20 23:42:04 2020] intel_idle_cpu_online+0x6c/0xf0
[Sat Jun 20 23:42:04 2020] cpuhp_invoke_callback+0x8d/0x500
[Sat Jun 20 23:42:04 2020] ? sort_range+0x20/0x20
[Sat Jun 20 23:42:04 2020] cpuhp_thread_fun+0xcb/0x130
[Sat Jun 20 23:42:04 2020] smpboot_thread_fn+0xc5/0x160
[Sat Jun 20 23:42:04 2020] kthread+0x112/0x130
[Sat Jun 20 23:42:04 2020] ? kthread_flush_work_fn+0x10/0x10
[Sat Jun 20 23:42:04 2020] ret_from_fork+0x35/0x40
[Sat Jun 20 23:42:04 2020] unchecked MSR access error: WRMSR to 0x1fc (tried to write 0x0000000000000000) at rIP: 0xffffffff9a861f14 (native_write_msr+0x4/0x20)
[Sat Jun 20 23:42:04 2020] Call Trace:
[Sat Jun 20 23:42:04 2020] intel_idle_cpu_online+0x83/0xf0
[Sat Jun 20 23:42:04 2020] cpuhp_invoke_callback+0x8d/0x500
[Sat Jun 20 23:42:04 2020] ? sort_range+0x20/0x20
[Sat Jun 20 23:42:04 2020] cpuhp_thread_fun+0xcb/0x130
[Sat Jun 20 23:42:04 2020] smpboot_thread_fn+0xc5/0x160
[Sat Jun 20 23:42:04 2020] kthread+0x112/0x130
[Sat Jun 20 23:42:04 2020] ? kthread_flush_work_fn+0x10/0x10
[Sat Jun 20 23:42:04 2020] ret_from_fork+0x35/0x40
[Sat Jun 20 23:42:04 2020] intel_idle: lapic_timer_reliable_states 0x2
```
23 changes: 0 additions & 23 deletions os/linux/kernel/cpu/x2apic.md

This file was deleted.

1 change: 1 addition & 0 deletions os/linux/kernel/scheduler/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
内核调度
56 changes: 56 additions & 0 deletions os/linux/kernel/scheduler/fix_unthrottle_cfs_rq.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
Linux系统日志

```bash
[Tue Jul 21 07:02:43 2020] ------------[ cut here ]------------
[Tue Jul 21 07:02:43 2020] rq->tmp_alone_branch != &rq->leaf_cfs_rq_list
[Tue Jul 21 07:02:43 2020] WARNING: CPU: 11 PID: 0 at kernel/sched/fair.c:368 unthrottle_cfs_rq+0x274/0x290
[Tue Jul 21 07:02:43 2020] Modules linked in: nft_chain_nat_ipv6 nf_conntrack_ipv6 nf_nat_ipv6 xt_statistic ipt_REJECT nf_reject_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_defrag_ipv6 nft_chain_route_ipv4 xt_comment xt_mark xt_nat veth xt_conntrack ipt_MASQUERADE nf_conntrack_netlink nft_counter xt_addrtype nft_compat nft_chain_nat_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack nf_tables nfnetlink br_netfilter bridge stp llc overlay fuse xfs libcrc32c intel_rapl_msr intel_rapl_common nfit libnvdimm crct10dif_pclmul crc32_pclmul cirrus ghash_clmulni_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops drm joydev virtio_balloon pcspkr i2c_piix4 ip_tables ext4 mbcache jbd2 ata_generic ata_piix libata virtio_net crc32c_intel net_failover virtio_console serio_raw virtio_blk failover
[Tue Jul 21 07:02:43 2020] Features: eBPF/cgroup
[Tue Jul 21 07:02:43 2020] CPU: 11 PID: 0 Comm: swapper/11 Kdump: loaded Not tainted 4.18.0-193.6.3.el8_2.x86_64 #1
[Tue Jul 21 07:02:43 2020] Hardware name: Alibaba Cloud Alibaba Cloud ECS, BIOS 8c24b4c 04/01/2014
[Tue Jul 21 07:02:43 2020] RIP: 0010:unthrottle_cfs_rq+0x274/0x290
[Tue Jul 21 07:02:43 2020] Code: 8a 05 00 e9 ff fe ff ff 31 db 80 3d 0b 5b 2d 01 00 0f 85 cf fe ff ff 48 c7 c7 58 3d c9 ae c6 05 f7 5a 2d 01 01 e8 26 02 fc ff <0f> 0b 48 85 db 0f 85 d2 fe ff ff e9 ac fe ff ff e8 a7 89 05 00 e9
[Tue Jul 21 07:02:43 2020] RSP: 0018:ffff9dffdfac3e70 EFLAGS: 00010082
[Tue Jul 21 07:02:43 2020] RAX: 0000000000000000 RBX: ffff9dfb69998400 RCX: 0000000000000000
[Tue Jul 21 07:02:43 2020] RDX: 000000000000002d RSI: ffffffffaf81af4d RDI: 0000000000000046
[Tue Jul 21 07:02:43 2020] RBP: ffff9dfb38190c00 R08: ffffffffaf81af20 R09: 000000000000002d
[Tue Jul 21 07:02:43 2020] R10: 0000000000000000 R11: 000000008000000b R12: ffff9dffdf9a9c40
[Tue Jul 21 07:02:43 2020] R13: 0000000000000002 R14: 0000000000000001 R15: 0000000000000001
[Tue Jul 21 07:02:43 2020] FS: 0000000000000000(0000) GS:ffff9dffdfac0000(0000) knlGS:0000000000000000
[Tue Jul 21 07:02:43 2020] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Tue Jul 21 07:02:43 2020] CR2: 00007fb1159234f4 CR3: 000000052760a005 CR4: 00000000003606e0
[Tue Jul 21 07:02:43 2020] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[Tue Jul 21 07:02:43 2020] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[Tue Jul 21 07:02:43 2020] Call Trace:
[Tue Jul 21 07:02:43 2020] <IRQ>
[Tue Jul 21 07:02:43 2020] distribute_cfs_runtime+0xc3/0x140
[Tue Jul 21 07:02:43 2020] sched_cfs_period_timer+0x108/0x210
[Tue Jul 21 07:02:43 2020] ? sched_cfs_slack_timer+0xe0/0xe0
[Tue Jul 21 07:02:43 2020] __hrtimer_run_queues+0x100/0x280
[Tue Jul 21 07:02:43 2020] hrtimer_interrupt+0x100/0x220
[Tue Jul 21 07:02:43 2020] smp_apic_timer_interrupt+0x6a/0x140
[Tue Jul 21 07:02:43 2020] apic_timer_interrupt+0xf/0x20
[Tue Jul 21 07:02:43 2020] </IRQ>
[Tue Jul 21 07:02:43 2020] RIP: 0010:cpuidle_enter_state+0xbc/0x420
[Tue Jul 21 07:02:43 2020] Code: e8 49 f0 a3 ff 80 7c 24 13 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 37 03 00 00 31 ff e8 db fd a9 ff fb 66 0f 1f 44 00 00 <45> 85 e4 0f 88 6c 02 00 00 49 63 cc 4c 8b 3c 24 4c 2b 7c 24 08 48
[Tue Jul 21 07:02:43 2020] RSP: 0018:ffffba3ec31dfe68 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13
[Tue Jul 21 07:02:43 2020] RAX: ffff9dffdfae9c40 RBX: ffffffffaef2a540 RCX: 00000000ffffffff
[Tue Jul 21 07:02:43 2020] RDX: 00094dc87e68fb16 RSI: 00000000000043fa RDI: 0000000000000000
[Tue Jul 21 07:02:43 2020] RBP: ffff9dffdfaf3800 R08: 000ba13b3114dac0 R09: 000000000000037c
[Tue Jul 21 07:02:43 2020] R10: ffff9dffdfae8be0 R11: ffff9dffdfae8bc0 R12: 0000000000000003
[Tue Jul 21 07:02:43 2020] R13: ffffffffaef2a678 R14: 0000000000000003 R15: 0000000000000000
[Tue Jul 21 07:02:43 2020] ? cpuidle_enter_state+0x97/0x420
[Tue Jul 21 07:02:43 2020] cpuidle_enter+0x2c/0x40
[Tue Jul 21 07:02:43 2020] do_idle+0x236/0x280
[Tue Jul 21 07:02:43 2020] cpu_startup_entry+0x6f/0x80
[Tue Jul 21 07:02:43 2020] start_secondary+0x1a7/0x200
[Tue Jul 21 07:02:43 2020] secondary_startup_64+0xb7/0xc0
[Tue Jul 21 07:02:43 2020] ---[ end trace 069b932bbbef0c26 ]---
```

这个报错是CFS调度器相关,可能和 [[PATCH] sched/fair: fix unthrottle_cfs_rq for leaf_cfs_rq list](https://lkml.org/lkml/2020/5/11/999) 有关,待学习分析

# 参考

* [[PATCH] sched/fair: fix unthrottle_cfs_rq for leaf_cfs_rq list](https://lkml.org/lkml/2020/5/11/999)
* [Linux Kernel Scheduler Basics](https://josefbacik.github.io/kernel/scheduler/2017/07/14/scheduler-basics.html)
5 changes: 5 additions & 0 deletions os/linux/kernel/scheduler/linux_kernel_scheduler_basic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@


# 参考

* [Linux Kernel Scheduler Basics](https://josefbacik.github.io/kernel/scheduler/2017/07/14/scheduler-basics.html)
6 changes: 6 additions & 0 deletions os/linux/storage/filesystem/xfs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
XFS是Red Hat主要支持的文件系统,稳定性和性能都非常出色,通常在数据库存储中广泛使用。

# 参考手册

* [xfs - layout, mount options, and supported file attributes for the XFS filesystem](https://www.linux.org/docs/man5/xfs.html)
* [mkfs.xfs(8) — Linux manual page](https://man7.org/linux/man-pages/man8/mkfs.xfs.8.html)
2 changes: 1 addition & 1 deletion server/ipmi/acpi_error_and_node_exporter.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ chip "power_meter-acpi-0"
ignore power1
```

此时再次使用 `sensors` 命令检查,可以看到输出有所不同,一斤关闭了 `power1` :
此时再次使用 `sensors` 命令检查,可以看到输出有所不同,已经关闭了 `power1` :

```
power_meter-acpi-0
Expand Down
8 changes: 7 additions & 1 deletion storage/glusterfs/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,7 @@
# GlusterFS
在CentOS上部署GlusterFS以及Ceph等存储解决方案,当前都采用 [CentOS Storage Special Interest Group (SIG) ](https://wiki.centos.org/SpecialInterestGroup/Storage) 部署,这是一个确保CentOS完善运行不同存储解决方案的整体架构。提供了软件仓库以及文档支持。

# CentOS SIG文档

* [Documentation home for GlusterFS in Storage SIG](https://wiki.centos.org/SpecialInterestGroup/Storage/Gluster)
* [GlusterFS Storage Cluster on CentOS 7](https://wiki.centos.org/HowTos/GlusterFSonCentOS) - 早期的CentOS 7部署Gluster
* [SIG Gluster Quickstart](https://wiki.centos.org/SpecialInterestGroup/Storage/gluster-Quickstart)
1 change: 1 addition & 0 deletions virtual/intel/sr-iov/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
SR-IOV即sigle-root I/O virtualization,是虚拟化技术中将物理设备切分成多个隔离的虚拟设备提供给虚拟机或容器使用的技术。
Empty file.

0 comments on commit de6c347

Please sign in to comment.