Archive for the ‘工具类’ Category

Hadoop生态主要是基于GitHub和jira构建开源社区,今年希望可以参与进去,记录使用方法如下。

由于每个开源项目的要求都可能不同,所以在开始之前,必须先阅读其Contribute页面,一般从README里就可以找到链接。

JIRA操作

当有了代码修改想法时,不论是bugfix,还是功能改进,都可以到项目JIRA页面上,提交一个issue,用英文大致描述想要做的事情。这里需要注意两点:

  1. 提交issue前,先用英文关键词search一下,确认该功能没有实现、且没有其他人提交类似issue
  2. issue粒度最好足够细,一个独立的小功能就好,类似我们敏捷中的task卡片。

如果想自己动手提交代码,可以将该issue assign给自己。真正动手时,可以将issue状态修改为 in progress。

GitHub 操作

在正式coding前,得在github上找到该项目,点击fork按钮fork出一个自己的分支,这样后续在这个分支上的工作只要不被管理员merge回去,就不会有任何影响。

git操作

这时可以建立本地代码仓库了:

git clone https://<path-to-your-repo> <your-local-prj-name> 将远端项目拉取到本地

cd <your-local-prj-name>

git checkout -b <new-branch-name> 建立一个分支,该issue相关的功能都会在这个分支里进行(git与svn不同,git的分支很轻量级,可以认为是功能隔离的单位)

git push origin <new-branch-name>提交新分支,这时github上自己的project下这个分支可见、但为空。

这时可以在新分支上编码了,完成并通过自测后,可以先提交到分支上

git add <new-files>

git commit -m ‘comments’

git push origin <new-branch-name> 这时代码在新分支上可见,但放心,不会影响到社区版本

如果编码持续了一段时间,可能需要从社区版本更新代码下来

git remote add upstream https://<path-to-public-repo> 添加社区版本为upstream源

git fetch upstream 下载社区版本的更新到本地隐藏目录

git checkout master 切换到master分支

git merge upstream/master 合并代码到本地master分支

git push origin master 将合并的结果提交到自己远端的master分支

GitHub操作

这时需要让别人看到自己的代码了,在github自己的project页面上,点击 pull request发起请求。在收到别人回复时,可以进行交流、重复代码修改过程。

JIRA操作

这时还需要将pull request与JIRA关联起来,点击issue页面More/Link,添加一个Web Link,将pull reqeust的uri填进去,link text可以写PR #xxx。

最后,当pull request完成,即被merge回社区或被彻底拒绝后,可以在git里删除分支,并且关闭issue。

IDEA Intellij小技巧和插件 一文中简单介绍了一下IdeaVim插件。在这里详细总结一下这个插件在日常编程中的一些常用小技巧。供有兴趣使用这个插件,但对Vim还不十分熟悉的朋友参考。当然基本的hjkl移动光标和几种常见模式等等基本概念就略过不提了。

为了确保只包含常用操作,这里提到的技巧都没有从现成文档里抄,而是凭记忆列出(不常用自然就不记得了)。估计会有所遗漏,慢慢再补充。

1. 切换Vim模拟器状态

这个插件允许设置一个快捷键一键开启或关闭,在切换模式时会同时自动切换keymap,十分方便。默认键位是Ctrl+Alt+V,但这个键位覆盖了很常用的“抽取局部变量”功能,建议重设,在setting->keymap中查找VIM Emulator即可。

由于开启和关闭状态分别使用两套keymap,因此两套都需要设定。可以把两套keymap下的都设为一样的键,也就是用同一个键切换。但个人建议设为不同的键,这样能清楚知道当前处于那种模式中。并且,如果在开启Vim的插入模式下关闭Vim模拟器,下次进入时仍然是插入模式,比较混乱(因为你关闭模拟器就是为了使用默认keymap输入大段代码,重新开启Vim模拟器就是为了使用普通模式下的命令)。因此建议把Vim keymap中的Exit Insert Mode设为与另一个keymap的Vim Emulator相同的键(也就是进入Vim模拟器的快捷键)。例如,我使用的设定是:
Default keymap -> Vim Emulator : Ctrl+;     (用Ctrl+分号开启Vim模拟器)
Vim keymap -> Vim Emulator : Ctrl+,    (用Ctrl+逗号关闭Vim模拟器)
Vim keymap -> Vim Emulator : Ctrl+;    (用Ctrl+分号退出插入模式,进入普通模式)
这样,在任何时候只要连按两下ctrl+分号,就能保证必定在Vim模拟器的普通模式中。


2. ScrollOff 参数

启动Intellij后在Vim模拟器下输入命令 :set so=5 可以令屏幕滚动时在光标上下方保留5行预览代码(也就是光标会在第5行触发向上滚动,或者在倒数第5行触发向下滚动)。在代码窗口比较狭小时(例如单步跟踪调试时)非常方便。可惜仅在Vim模拟器开启时有效。

3. 行号定位
普通模式下输入 行号G 或 :行号<回车> 都能快速定位到某一行。区别在于前者在输入行号时屏幕上没有任何提示,后者则在Vim命令输入框中可以看到输入过程。(题外话:Sublime Text 2也是用 :行号 来快速定位到某行,应该是沿用了Vim的习惯)

4. 进入修改
进入插入模式的方式有很多,直接选用合适的方式进入插入模式比进入后再用箭头键移动光标要好。常用的有:
o – 在当前行下方插入新行并自动缩进
O – 在当前行上方插入新行并自动缩进 (普通模式下的大写字母命令用 shift+字母键 输入,下同)
i – 在当前字符左方开始插入字符
a – 在当前字符右方开始插入字符
I – 光标移动到行首并进入插入模式
A – 光标移动到行尾并进入插入模式
s – 删除光标所在字符并进入插入模式
S – 删除光标所在行并进入插入模式
c<范围> – 删除光标所在位置周围某个范围的文本并进入插入模式。关于范围请看第5点,常用的组合有:caw – 删除一个单词包括它后面的空格并开始插入; ciw – 删除一个单词并开始插入; ci” – 删除一个字符串内部文本并开始插入; c$ – 从光标位置删除到行尾并开始插入; ct字符 – 从光标位置删除本行某个字符之前(保留该字符)并开始插入。等等。
C – 删除光标位置到行尾的内容并进入插入模式 (相当于c$)
r – 修改光标所在字符,然后返回普通模式
R – 进入覆盖模式

5. 范围操作
某些普通模式的动作命令后面可以追加一些表示范围的指令,表示该动作将作用在整个范围上。这类命令常用的有:
d<范围> – 删除一定范围内的文本
c<范围> – 删除一定范围内的文本并进入插入模式
y<范围> – 将范围内的文本放入0号和”号注册栏
v<范围> – 选择范围内的文本
=<范围> – 自动缩进范围内的文本
gU<范围> – 将范围内的字符转换为大写
gu<范围> – 将范围内的字符转换为小写
><范围> – 将范围中的内容缩进一格
<<范围> – 将范围中的内容取消缩进一格

常用的范围指令有:
空格 – 光标所在位置字符。(例如 gU空格 – 将光标位置字符转为大写)
重复某些动作命令 – 光标所在行。 (例如dd删除一行,yy复制一行,cc删除一行文本并开始插入,>> 当前行缩进一格,==自动缩进当前行)
$ – 从光标位置到行尾
^ – 从光标位置到行首,不包含缩进空白
0 – 从光标位置到行首,包含缩进空白
gg – 从光标位置到文件开头
G – 从光标位置到文件结尾
% – 从光标位置到另一边匹配的括号
f<字符> – 从光标位置到光标右边某个字符首次出现的位置,包括该字符
F<字符> – 从光标位置到光标左边某个字符首次出现的位置,包括该字符
t<字符> – 从光标位置到光标右边某个字符首次出现的位置,包括该字符
F<字符> – 从光标位置到光标左边某个字符首次出现的位置,包括该字符
/正则表达式 – 从光标位置到下一个匹配正则表达式的位置(跨行)
?正则表达式 – 从光标位置到上一个匹配正则表达式的位置(跨行)
aw – 一个单词加一个空格 (a可理解为“一个”,下同)
iw – 一个单词 (i可理解为in,下同)
a” – 一个字符串包括双引号
i” – 一个字符串内部文本
a< – 一组< >包含的文本,包括< >号本身
同理类推: i<, a[, i[, a(, i(
注意:真正vim中的it范围(一对xml标签内部)在ideaVim中不生效。

用/或?命令查找时,正则表达式默认大小写敏感,如果需要不敏感,可以在正则表达式开始处加上\c标志。例如 /\cabc 可以匹配到 ABC。下面提到的:s命令同样适用。

6. 选择文本
在Vim中,选择文本需要进入“可视模式”(Visual Mode),这个名称比较奇怪,它的来由据说是因为在Vim的前身Vi中,选择区域是不可见的。在Vim中选择区域会高亮显示,因此称为“可视模式”。
v – 进入字符选择模式, V – 进入行选择模式, Ctrl+v – 进入块选择模式。
进入相应模式后移动光标即可选中文本。过程中可按o键令光标在选区两端切换。
在块选择模式中选中多行,然后按I或A后输入文本,再退出插入模式,所输入的文本将自动加入到每一行的开头或结尾。

7. 复制粘贴
在Vim模式下,复制粘贴并不直接使用系统的剪贴板,而是使用Vim提供的多个“寄存器”,每个寄存器都以一个字符来表示。关于寄存器的详细说明可以看这里 http://blah.blogsome.com/2006/04/27/vim_tut_register/ (随便google的一个网页),这里简单列一些常用的操作技巧 (注意,vim使用双引号”来作为选择寄存器的命令,因此下文中的双引号均指在普通模式下按双引号键):

a)用y命令将文本存入寄存器后,如果想在别处替换原有内容,可以先用v命令选中原有内容,然后用p命令粘贴。但第一次粘贴后,默认的寄存器将被替换为刚刚删除的内容。如果要再次粘贴之前复制的内容,需要使用 “0p 命令组合来复制。也可以进入插入模式后用 Ctrl+r 0 来复制,例如 ciw<Ctrl+r>0 命令组合将用粘贴内容替换光标处的一个单词,并停留在插入模式。

b)在Windows下,寄存器 + 和 * 都代表系统剪贴板,可以互换使用,选一个顺手的即可。例如 “+yy 命令组合可将当前行复制到系统剪贴板。 ci”<Ctrl+r>* 命令组合则将系统剪贴板的内容替换字符串的内部文本。

c) 寄存器1至9记录之前九次的删除大段文本,每次超过一行的删除操作都会导致这9个寄存器的内容发生位移,最近删除的文本会存入寄存器1。但只有删除超过1行时才会影响寄存器1至9,行内的删除内容则会被存入寄存器-(减号)。如果用q命令录制宏时不涉及跨行删除,可以在宏中直接使用这9个寄存器来暂存文本。(在Vim中,复制内容与录制宏共享同一套寄存器,因此我习惯把字母寄存器留给宏使用)

d) 普通模式下小写p把寄存器内容复制到当前位置之后,大写P把寄存器内容复制到当前位置之前。

e) 使用 :regs 命令可以列出当前所有寄存器的内容

8.  一些插入模式下的常用快捷键
Ctrl+h – 删除光标左边字符
Ctrl+w – 删除光标左边的单词
Ctrl+y – 复制上方的一个字符
Ctrl+e – 复制下方的一个字符
Ctrl+r 0 – 插入前一次用y命令寄存的内容
Ctrl+r * – 插入系统剪贴板的内容
Ctrl+r <寄存器名称> – 插入指定寄存器的内容
Ctrl+a – 插入前一次插入模式所键入的内容
Ctrl+o – 执行一个普通模式下的命令然后返回插入模式。 例如 Ctrl+o A 相当于按 End键, Ctrl+o I相当于按Home键

9. 退出插入模式
退出插入模式可以用 ESC 键,但键位太远。其实也可以用 Ctrl+[ 键退出插入模式 。当然也可以用第1点自定义的Ctrl+;快捷键,但这不是标准vim按键,会养成不良习惯,不建议使用。

10. 重复操作
普通模式下按. (小数点)可重复上一次的修改操作
& – 重复上一次的:s替换命令
@@ – 重复上一次执行的宏

11. 跳转
Ctrl+] 跳转到当前标识符的定义位置 (相当于在当前光标位置的单词上按住ctrl用鼠标点击)
Ctrl+o 回退一步 (go back)
Ctrl+i 前进一步 (go forward)
`. 跳转到之前修改位置
“ 在前一次跳转位置与当前位置间切换
行号G 或 :行号<回车>  跳转到某一行
gg 跳转到文件开头
G  跳转到文件末尾
H  跳转到屏幕顶端(如果设置了set so=n,则跳转到第n行)
L  跳转到屏幕底端(如果设置了set so=n,则跳转到倒数第n行)
M  跳转到屏幕中间
f 或 F 跳转到本行某个字符,小写f向右查找,大写F向左查找。用;或,在匹配间切换
t 或 T 跳转到本行某个字符之前,小写t向右查找,大写T向左查找。用;或,在匹配间切换
/正则表达式  跳转到下一个匹配。用n或N在匹配间切换。
?正则表达式  跳转到上一个匹配。用n或N在匹配间切换。
(结合前面第5点,你也许注意到了,在指定范围时,使用跳转命令将指定一个从光标位置到跳转目标的区域)

12 书签
在普通模式下按 m<小写字母> 即可定义书签,按 `<字母> 则可跳转到某个书签的精确位置,按 ‘<字母>可跳转到某个书签所在行的行首(用来录制宏时比较有用)。最常用的自然是mm, mn, mj, mk, ml这几个顺手的键位。
真正的vim中的全局书签 m<大写字母> 在目前IdeaVim版本中不生效。需要定义全局书签可以使用Idea原本的 F11 + 数字 方式

13 文本替换
使用 :s/正则表达式/替换文本/ 可在本行内替换首次出现的匹配
使用 :s/正则表达式/替换文本/g 在本行内替换所有出现的匹配
使用 :%s/正则表达式/替换文本/g 在当前文件内替换所有出现的匹配

在可视模式下选中文本后,使用:'<,’>s/正则表达式/替换文本/g 命令可在选中区域中替换文本。其中'<,’>部分在可视模式下,按:冒号后自动加入,直接输入s命令即可。但有效区域只能以行为单位。真正Vim中的 \%V 标志在IdeaVim中不生效。

11 代码折叠
zo – 打开折叠
zc – 关闭折叠

14 宏定义
在IdeaVim中定义宏比Idea自带的宏功能要轻量许多。按在普通模式下 q<寄存器名称> 即可开始把后续按键序列录制到指定寄存器中(寄存器参考前面第7条)。录制完毕进入普通模式再按q键即可停止录制。之后用 @<寄存器名称> 即可重放。需要注意的是宏和复制粘贴共用一套寄存器,因此在录制宏时就注意不要把当前宏正在使用的寄存器用来复制了。寄存器内容是自动保存的,重启Idea仍然生效。但IdeaVim没有导出宏独立保存的功能。因此最好把用来保存宏的寄存器和用来复制粘贴的寄存器分开,不要同一个寄存器有时用来记录宏,有时用来复制粘贴。我的习惯是键盘左手区用来保存一些长期使用的宏(比如说我有一个宏专门用来把pom.xml中的版本号抽取到property区域,原来的位置则改用${property}引用)。右手区的hjklnm键用来保存一些临时宏。yuiop五个寄存器保留用来复制粘贴。如果录制的宏不涉及删除大段代码,寄存器1至9也可以用来进行复制粘贴。

执行一次宏后,可以用@@命令重复上一次执行的宏。

在Idea中录制宏时,如果触发了代码自动完成,在自动完成列表启动的状态输入的字符不会被记录。因此最好在Setting -> Code Completion -> Autopopup code completion中把延迟设为500ms以上或干脆关掉。在录制宏的过程中避免触发代码自动完成功能。

录制一些长期有效的宏时,开始录制后,最好先用0,^,T, F, $等命令把光标对齐到行首行末或某个特定起始位置(比如说用 F” 跳转到字符串的左边引号),再用一个f或/指令跳转到操作位置,这样的宏就不用必须把光标放在某个特定字符才能使用了。

15. 一些常用组合技
全选: ggvG
调换两个字符位置: xp
复制一行: yyp
调换两行位置: ddp
插入模式下到行尾继续输入(相当于End键): Ctrl+o A 或 Ctrl+[ A
插入模式下到行首继续输入(相当于Home键): Ctrl+o I 或 Ctrl+[ I
到类定义位置(适用于正确缩进的public,protected类) : ?^p回车

16. 一些在目前版本已知没有实现的一些常用Vim功能
(如果对Vim不熟悉可以跳过这节)
a)let命令 (没有let命令就无法导出/导入寄存器内容,也就是无法导入宏)
b):g命令 (在文本处理中很有用的一个命令,在编程中倒是不那么常用)
c)!命令 (执行shell命令)
d)大部分正则表达式标记 (例如 \%V, \v 等等)
e) 某些多键命令双击最后一个字符表示作用于当前行。例如在Vim中gUU可以把当前行转换为大写,在IdeaVim中无效,实现同样功能可以先用V命令选中当前行,再用gU转换为大写。
f)关于窗口操作的大部分命令 (Ctrl+w系列命令, :split等)
g)所有Vim脚本插件 (不过大部分可以用Idea自身的功能和插件来补偿)

zz from: http://kidneyball.iteye.com/blog/1828427

昨天升级了wordpress到新版本,结果Developer Formatter在插入代码时失败,第一个报错信息是:

 Javascript |  copy code |? 
1
Uncaught TypeError: undefined is not a function

firebug排查定位其错误应该是在wp-content/plugins/devformatter/devinterface.php生成的js文件,调用execInstanceCommand方法出错。

解决方法如下:

#vim wp-content/plugins/devformatter/devinterface.php,找到execInstanceCommand哪一行,修改为:

 Javascript |  copy code |? 
1
      if(HtmlEditor){
2
        edInsertContent(edCanvas, DevFmt_ContentStart + DevFmt_TheContent + DevFmt_ContentEnd);
3
      }else{
4
        alert(DevFmt_ContentStart + DevFmt_TheContent + DevFmt_ContentEnd);
5
        tinyMCE.execCommand('mceReplaceContent', false,
6
          switchEditors.wpautop(DevFmt_ContentStart + DevFmt_TheContent + DevFmt_ContentEnd));
7
      }

第二个问题是,插入带空格的代码后,页面上出现大量DVFMTSC字样,修改wp-content/plugins/devformatter/devfmt_editor.js文件如下:

 Javascript |  copy code |? 
1
2
block = block.replace(/{{DVFMTSC}}/gi, '<!--DVFMTSC-->&').replace(/\n/gi, "<br />");
3
 
4
修改为:
5
block = block.replace(/{{DVFMTSC}}/gi, '&').replace(/\n/gi, "<br />");

参考:

  • http://stackoverflow.com/questions/22813970/typeerror-window-tinymce-execinstancecommand-is-not-a-function

sar命令

我很喜欢用这个命令来查看/监控系统的整体情况。以下引自http://linux.die.net/man/1/sar,并加以解释。

常用参数组合

查看内存

free, sar -r 1, sar -B 1,ps aux

查看cpu和负载

sar -u ALL 1 , sar -P ALL 1,  sar -q 1 , sar -w 1

查看磁盘

sar -b 1,  sar -d 1, sar -v 1

查看网络

sar -n DEV等

man手册及解释

Name

sar – Collect, report, or save system activity information.

Synopsis

sar [ -A ] [ -b ] [ -B ] [ -C ] [ -d ] [ -h ] [ -i interval ] [ -m ] [ -p ] [ -q ] [ -r ] [ -R ] [ -S ] [ -t ] [ -u [ ALL ] ] [ -v ] [ -V ] [ -w ] [ -W ] [ -y ] [ -n { keyword [,…] | ALL } ] [ -I { int[,…] | SUM | ALL | XALL } ] [ -P { cpu [,…] | ALL } ] [ -o [ filename ] | -f [ filename ] ] [ -s [ hh:mm:ss ] ] [ -e [hh:mm:ss ] ] [ interval [ count ] ]

Description

The sar command writes to standard output the contents of selected cumulative activity counters in the operating system. The accounting system, based on the values in the count and interval parameters, writes information the specified number of times spaced at the specified intervals in seconds. If the interval parameter is set to zero, the sar command displays the average statistics for the time since the system was started. If the interval parameter is specified without the count parameter, then reports are generated continuously. The collected data can also be saved in the file specified by the -o filename flag, in addition to being displayed onto the screen. If filename is omitted, sar uses the standard system activity daily data file, the /var/log/sa/sadd file, where the dd parameter indicates the current day. By default all the data available from the kernel are saved in the data file.

The sar command extracts and writes to standard output records previously saved in a file. This file can be either the one specified by the -f flag or, by default, the standard system activity daily data file.

Without the -P flag, the sar command reports system-wide (global among all processors) statistics, which are calculated as averages for values expressed as percentages, and as sums otherwise. If the -P flag is given, the sar command reports activity which relates to the specified processor or processors. If -P ALL is given, the sar command reports statistics for each individual processor and global statistics among all processors.  可以查看所有CPU的整体情况(不带-P参数),也可以查看指定处理器(-P CPU-NUM),还可以查看指定CPU和汇总情况(-P ALL)。对于应用层而言,可以查看程序是否均衡使用了多CPU并行能力。

You can select information about specific system activities using flags. Not specifying any flags selects only CPU activity. Specifying the -A flag is equivalent to specifying -bBdqrRSvwWy -I SUM -I XALL -n ALL -u ALL -P ALL.

The default version of the sar command (CPU utilization report) might be one of the first facilities the user runs to begin system activity investigation, because it monitors major system resources. If CPU utilization is near 100 percent (user + nice + system), the workload sampled is CPU-bound.

If multiple samples and multiple reports are desired, it is convenient to specify an output file for the sar command. Run thesar command as a background process. The syntax for this is:

sar -o datafile interval count >/dev/null 2>&1 &

All data is captured in binary form and saved to a file (datafile). The data can then be selectively displayed with the sarcommand using the -f option. Set the interval and count parameters to select count records at interval second intervals. If the count parameter is not set, all the records saved in the file will be selected. Collection of data in this manner is useful to characterize system usage over a period of time and determine peak usage hours.

Note: The sar command only reports on local activities.

Options

-A

This is equivalent to specifying -bBdqrRSuvwWy -I SUM -I XALL -n ALL -u ALL -P ALL.

-b

Report I/O and transfer rate statistics. The following values are displayed:  物理存储介质的I/O监控。

tps

Total number of transfers per second that were issued to physical devices. A transfer is an I/O request to a physical device. Multiple logical requests can be combined into a single I/O request to the device. A transfer is of indeterminate size.

对物理存储每秒发起的读写请求数目。以下细分为读请求数目rtps,写请求数目wtps。

rtps

Total number of read requests per second issued to physical devices.

wtps

Total number of write requests per second issued to physical devices.

bread/s

Total amount of data read from the devices in blocks per second. Blocks are equivalent to sectors with 2.4 kernels and newer and therefore have a size of 512 bytes. With older kernels, a block is of indeterminate size.

每秒读入的blocks数目,2.4内核及以上,block size = sector size = 512B。下面可以看到写blocks数目。

bwrtn/s

Total amount of data written to devices in blocks per second.

-B

Report paging statistics. Some of the metrics below are available only with post 2.5 kernels. The following values are displayed: 页交换相关数据。

pgpgin/s

Total number of kilobytes the system paged in from disk per second. Note: With old kernels (2.2.x) this value is a number of blocks per second (and not kilobytes).

2.2内核以上,是每秒从磁盘换入的数据量,以KB为单位。

pgpgout/s

Total number of kilobytes the system paged out to disk per second. Note: With old kernels (2.2.x) this value is a number of blocks per second (and not kilobytes).

2.2内核以上,是每秒从磁盘换出的数据量,以KB为单位。

fault/s

Number of page faults (major + minor) made by the system per second. This is not a count of page faults that generate I/O, because some page faults can be resolved without I/O.

每秒系统产生的页中断数目。注意,页中断不一定导致I/O。下面的major faults才必定会导致从disk加载数据到memory。

majflt/s

Number of major faults the system has made per second, those which have required loading a memory page from disk.

每秒系统产生的major中断数目,会导致从disk加载内存页。

pgfree/s

Number of pages placed on the free list by the system per second.

每秒系统释放的空闲内存页数目。

pgscank/s

Number of pages scanned by the kswapd daemon per second.

每秒被kswapd扫描的内存页数目。

pgscand/s

Number of pages scanned directly per second.

每秒被直接扫描的内存页数目。

pgsteal/s

Number of pages the system has reclaimed from cache (pagecache and swapcache) per second to satisfy its memory demands.

每秒系统认为内存不足的次数,包括页内存和swap内存。

%vmeff

Calculated as pgsteal / pgscan, this is a metric of the efficiency of page reclaim. If it is near 100% then almost every page coming off the tail of the inactive list is being reaped. If it gets too low (e.g. less than 30%) then the virtual memory is having some difficulty. This field is displayed as zero if no pages have been scanned during the interval of time.

-C

When reading data from a file, tell sar to display comments that have been inserted by sadc.

-d

Report activity for each block device (kernels 2.4 and newer only). When data is displayed, the device specificationdev m-n is generally used ( DEV column). m is the major number of the device. With recent kernels (post 2.5), n is the minor number of the device, but is only a sequence number with pre 2.5 kernels. Device names may also be pretty-printed if option -p is used (see below). Values for fields avgqu-sz, await, svctm and %util may be unavailable and displayed as 0.00 with some 2.4 kernels. Note that disk activity depends on sadc options “-S DISK” and “-S XDISK” to be collected. The following values are displayed:

tps

Indicate the number of transfers per second that were issued to the device. Multiple logical requests can be combined into a single I/O request to the device. A transfer is of indeterminate size.

与-b选项的tps含义相同。

rd_sec/s

Number of sectors read from the device. The size of a sector is 512 bytes.

与-b选项的bread/s含义相同。

wr_sec/s

Number of sectors written to the device. The size of a sector is 512 bytes.

与-b选项的wread/s含义相同。

avgrq-sz

The average size (in sectors) of the requests that were issued to the device.

每个请求的平均传输数据量,以sectors为单位。sector size = 512B。

avgqu-sz

The average queue length of the requests that were issued to the device.

请求排队的平均长度。如果出现磁盘等硬件问题,排队长度极大可能性会剧烈上升。

await

The average time (in milliseconds) for I/O requests issued to the device to be served. This includes the time spent by the requests in queue and the time spent servicing them.

请求平均处理时间,单位是ms。包含了排队时间和真实的处理时间。

svctm

The average service time (in milliseconds) for I/O requests that were issued to the device.

请求平均的真实处理时间,单位是ms。与await相减就可以得到排队时间了。在我们的hbase服务器上,await – svctm会达到3ms+-,而负载较轻的web服务器只有0.0几ms。

%util

Percentage of CPU time during which I/O requests were issued to the device (bandwidth utilization for the device). Device saturation occurs when this value is close to 100%.

CPU时间中用于IO耗时的占比。越趋近100%,代表设备使用越饱和。

-e [ hh:mm:ss ]

Set the ending time of the report. The default ending time is 18:00:00. Hours must be given in 24-hour format. This option can be used when data are read from or written to a file (options -f or -o ).

-f [ filename ]

Extract records from filename (created by the -o filename flag). The default value of the filename parameter is the current daily data file, the /var/log/sa/sadd file. The -f option is exclusive of the -o option.

-h

Display a short help message then exit.

-i interval

Select data records at seconds as close as possible to the number specified by the interval parameter.

-I { int [,…] | SUM | ALL | XALL }

Report statistics for a given interrupt. int is the interrupt number. Specifying multiple -I int parameters on the command line will look at multiple independent interrupts. The SUM keyword indicates that the total number of interrupts received per second is to be displayed. The ALL keyword indicates that statistics from the first 16 interrupts are to be reported, whereas the XALL keyword indicates that statistics from all interrupts, including potential APIC interrupt sources, are to be reported. Note that interrupt statistics depend on sadc option “-S INT” to be collected.

-m

Report power management statistics. Note that these statistics depend on sadc option “-S POWER” to be collected. The following value is displayed:

MHz

CPU clock frequency in MHz.

-n { keyword [,…] | ALL }

Report network statistics.

Possible keywords are DEVEDEVNFSNFSDSOCKIPEIPICMPEICMPTCPETCPUDPSOCK6IP6,EIP6ICMP6EICMP6 and UDP6.

With the DEV keyword, statistics from the network devices are reported. The following values are displayed:

监控网络情况,简单使用方式:sar -n DEV 1 1

IFACE

Name of the network interface for which statistics are reported.

rxpck/s

Total number of packets received per second.

txpck/s

Total number of packets transmitted per second.

rxkB/s

Total number of kilobytes received per second.

txkB/s

Total number of kilobytes transmitted per second.

rxcmp/s

Number of compressed packets received per second (for cslip etc.).

txcmp/s

Number of compressed packets transmitted per second.

rxmcst/s

Number of multicast packets received per second.

With the EDEV keyword, statistics on failures (errors) from the network devices are reported. The following values are displayed:

IFACE

Name of the network interface for which statistics are reported.

rxerr/s

Total number of bad packets received per second.

txerr/s

Total number of errors that happened per second while transmitting packets.

coll/s

Number of collisions that happened per second while transmitting packets.

rxdrop/s

Number of received packets dropped per second because of a lack of space in linux buffers.

txdrop/s

Number of transmitted packets dropped per second because of a lack of space in linux buffers.

txcarr/s

Number of carrier-errors that happened per second while transmitting packets.

rxfram/s

Number of frame alignment errors that happened per second on received packets.

rxfifo/s

Number of FIFO overrun errors that happened per second on received packets.

txfifo/s

Number of FIFO overrun errors that happened per second on transmitted packets.

With the NFS keyword, statistics about NFS client activity are reported. The following values are displayed:

call/s

Number of RPC requests made per second.

retrans/s

Number of RPC requests per second, those which needed to be retransmitted (for example because of a server timeout).

read/s

Number of ‘read’ RPC calls made per second.

write/s

Number of ‘write’ RPC calls made per second.

access/s

Number of ‘access’ RPC calls made per second.

getatt/s

Number of ‘getattr’ RPC calls made per second.

With the NFSD keyword, statistics about NFS server activity are reported. The following values are displayed:

scall/s

Number of RPC requests received per second.

badcall/s

Number of bad RPC requests received per second, those whose processing generated an error.

packet/s

Number of network packets received per second.

udp/s

Number of UDP packets received per second.

tcp/s

Number of TCP packets received per second.

hit/s

Number of reply cache hits per second.

miss/s

Number of reply cache misses per second.

sread/s

Number of ‘read’ RPC calls received per second.

swrite/s

Number of ‘write’ RPC calls received per second.

saccess/s

Number of ‘access’ RPC calls received per second.

sgetatt/s

Number of ‘getattr’ RPC calls received per second.

With the SOCK keyword, statistics on sockets in use are reported (IPv4). The following values are displayed:

totsck

Total number of sockets used by the system.

tcpsck

Number of TCP sockets currently in use.

udpsck

Number of UDP sockets currently in use.

rawsck

Number of RAW sockets currently in use.

ip-frag

Number of IP fragments currently in use.

tcp-tw

Number of TCP sockets in TIME_WAIT state.

With the IP keyword, statistics about IPv4 network traffic are reported. Note that IPv4 statistics depend on sadc option “-S SNMP” to be collected. The following values are displayed (formal SNMP names between square brackets):

irec/s

The total number of input datagrams received from interfaces per second, including those received in error [ipInReceives].

fwddgm/s

The number of input datagrams per second, for which this entity was not their final IP destination, as a result of which an attempt was made to find a route to forward them to that final destination [ipForwDatagrams].

idel/s

The total number of input datagrams successfully delivered per second to IP user-protocols (including ICMP) [ipInDelivers].

orq/s

The total number of IP datagrams which local IP user-protocols (including ICMP) supplied per second to IP in requests for transmission [ipOutRequests]. Note that this counter does not include any datagrams counted in fwddgm/s.

asmrq/s

The number of IP fragments received per second which needed to be reassembled at this entity [ipReasmReqds].

asmok/s

The number of IP datagrams successfully re-assembled per second [ipReasmOKs].

fragok/s

The number of IP datagrams that have been successfully fragmented at this entity per second [ipFragOKs].

fragcrt/s

The number of IP datagram fragments that have been generated per second as a result of fragmentation at this entity [ipFragCreates].

With the EIP keyword, statistics about IPv4 network errors are reported. Note that IPv4 statistics depend on sadc option “-S SNMP” to be collected. The following values are displayed (formal SNMP names between square brackets):

ihdrerr/s

The number of input datagrams discarded per second due to errors in their IP headers, including bad checksums, version number mismatch, other format errors, time-to-live exceeded, errors discovered in processing their IP options, etc. [ipInHdrErrors]

iadrerr/s

The number of input datagrams discarded per second because the IP address in their IP header’s destination field was not a valid address to be received at this entity. This count includes invalid addresses (e.g., 0.0.0.0) and addresses of unsupported Classes (e.g., Class E). For entities which are not IP routers and therefore do not forward datagrams, this counter includes datagrams discarded because the destination address was not a local address [ipInAddrErrors].

iukwnpr/s

The number of locally-addressed datagrams received successfully but discarded per second because of an unknown or unsupported protocol [ipInUnknownProtos].

idisc/s

The number of input IP datagrams per second for which no problems were encountered to prevent their continued processing, but which were discarded (e.g., for lack of buffer space) [ipInDiscards]. Note that this counter does not include any datagrams discarded while awaiting re-assembly.

odisc/s

The number of output IP datagrams per second for which no problem was encountered to prevent their transmission to their destination, but which were discarded (e.g., for lack of buffer space) [ipOutDiscards]. Note that this counter would include datagrams counted in fwddgm/s if any such packets met this (discretionary) discard criterion.

onort/s

The number of IP datagrams discarded per second because no route could be found to transmit them to their destination [ipOutNoRoutes]. Note that this counter includes any packets counted in fwddgm/s which meet this ‘no-route’ criterion. Note that this includes any datagrams which a host cannot route because all of its default routers are down.

asmf/s

The number of failures detected per second by the IP re-assembly algorithm (for whatever reason: timed out, errors, etc) [ipReasmFails]. Note that this is not necessarily a count of discarded IP fragments since some algorithms can lose track of the number of fragments by combining them as they are received.

fragf/s

The number of IP datagrams that have been discarded per second because they needed to be fragmented at this entity but could not be, e.g., because their Don’t Fragment flag was set [ipFragFails].

With the ICMP keyword, statistics about ICMPv4 network traffic are reported. Note that ICMPv4 statistics depend on sadc option “-S SNMP” to be collected. The following values are displayed (formal SNMP names between square brackets):

imsg/s

The total number of ICMP messages which the entity received per second [icmpInMsgs]. Note that this counter includes all those counted by ierr/s.

omsg/s

The total number of ICMP messages which this entity attempted to send per second [icmpOutMsgs]. Note that this counter includes all those counted by oerr/s.

iech/s

The number of ICMP Echo (request) messages received per second [icmpInEchos].

iechr/s

The number of ICMP Echo Reply messages received per second [icmpInEchoReps].

oech/s

The number of ICMP Echo (request) messages sent per second [icmpOutEchos].

oechr/s

The number of ICMP Echo Reply messages sent per second [icmpOutEchoReps].

itm/s

The number of ICMP Timestamp (request) messages received per second [icmpInTimestamps].

itmr/s

The number of ICMP Timestamp Reply messages received per second [icmpInTimestampReps].

otm/s

The number of ICMP Timestamp (request) messages sent per second [icmpOutTimestamps].

otmr/s

The number of ICMP Timestamp Reply messages sent per second [icmpOutTimestampReps].

iadrmk/s

The number of ICMP Address Mask Request messages received per second [icmpInAddrMasks].

iadrmkr/s

The number of ICMP Address Mask Reply messages received per second [icmpInAddrMaskReps].

oadrmk/s

The number of ICMP Address Mask Request messages sent per second [icmpOutAddrMasks].

oadrmkr/s

The number of ICMP Address Mask Reply messages sent per second [icmpOutAddrMaskReps].

With the EICMP keyword, statistics about ICMPv4 error messages are reported. Note that ICMPv4 statistics depend on sadc option “-S SNMP” to be collected. The following values are displayed (formal SNMP names between square brackets):

ierr/s

The number of ICMP messages per second which the entity received but determined as having ICMP-specific errors (bad ICMP checksums, bad length, etc.) [icmpInErrors].

oerr/s

The number of ICMP messages per second which this entity did not send due to problems discovered within ICMP such as a lack of buffers [icmpOutErrors].

idstunr/s

The number of ICMP Destination Unreachable messages received per second [icmpInDestUnreachs].

odstunr/s

The number of ICMP Destination Unreachable messages sent per second [icmpOutDestUnreachs].

itmex/s

The number of ICMP Time Exceeded messages received per second [icmpInTimeExcds].

otmex/s

The number of ICMP Time Exceeded messages sent per second [icmpOutTimeExcds].

iparmpb/s

The number of ICMP Parameter Problem messages received per second [icmpInParmProbs].

oparmpb/s

The number of ICMP Parameter Problem messages sent per second [icmpOutParmProbs].

isrcq/s

The number of ICMP Source Quench messages received per second [icmpInSrcQuenchs].

osrcq/s

The number of ICMP Source Quench messages sent per second [icmpOutSrcQuenchs].

iredir/s

The number of ICMP Redirect messages received per second [icmpInRedirects].

oredir/s

The number of ICMP Redirect messages sent per second [icmpOutRedirects].

With the TCP keyword, statistics about TCPv4 network traffic are reported. Note that TCPv4 statistics depend on sadc option “-S SNMP” to be collected. The following values are displayed (formal SNMP names between square brackets):

active/s

The number of times TCP connections have made a direct transition to the SYN-SENT state from the CLOSED state per second [tcpActiveOpens].

passive/s

The number of times TCP connections have made a direct transition to the SYN-RCVD state from the LISTEN state per second [tcpPassiveOpens].

iseg/s

The total number of segments received per second, including those received in error [tcpInSegs]. This count includes segments received on currently established connections.

oseg/s

The total number of segments sent per second, including those on current connections but excluding those containing only retransmitted octets [tcpOutSegs].

With the ETCP keyword, statistics about TCPv4 network errors are reported. Note that TCPv4 statistics depend on sadc option “-S SNMP” to be collected. The following values are displayed (formal SNMP names between square brackets):

atmptf/s

The number of times per second TCP connections have made a direct transition to the CLOSED state from either the SYN-SENT state or the SYN-RCVD state, plus the number of times per second TCP connections have made a direct transition to the LISTEN state from the SYN-RCVD state [tcpAttemptFails].

estres/s

The number of times per second TCP connections have made a direct transition to the CLOSED state from either the ESTABLISHED state or the CLOSE-WAIT state [tcpEstabResets].

retrans/s

The total number of segments retransmitted per second – that is, the number of TCP segments transmitted containing one or more previously transmitted octets [tcpRetransSegs].

isegerr/s

The total number of segments received in error (e.g., bad TCP checksums) per second [tcpInErrs].

orsts/s

The number of TCP segments sent per second containing the RST flag [tcpOutRsts].

With the UDP keyword, statistics about UDPv4 network traffic are reported. Note that UDPv4 statistics depend on sadc option “-S SNMP” to be collected. The following values are displayed (formal SNMP names between square brackets):

idgm/s

The total number of UDP datagrams delivered per second to UDP users [udpInDatagrams].

odgm/s

The total number of UDP datagrams sent per second from this entity [udpOutDatagrams].

noport/s

The total number of received UDP datagrams per second for which there was no application at the destination port [udpNoPorts].

idgmerr/s

The number of received UDP datagrams per second that could not be delivered for reasons other than the lack of an application at the destination port [udpInErrors].

With the SOCK6 keyword, statistics on sockets in use are reported (IPv6). Note that IPv6 statistics depend on sadc option “-S IPV6″ to be collected. The following values are displayed:

tcp6sck

Number of TCPv6 sockets currently in use.

udp6sck

Number of UDPv6 sockets currently in use.

raw6sck

Number of RAWv6 sockets currently in use.

ip6-frag

Number of IPv6 fragments currently in use.

With the IP6 keyword, statistics about IPv6 network traffic are reported. Note that IPv6 statistics depend on sadc option “-S IPV6″ to be collected. The following values are displayed (formal SNMP names between square brackets):

irec6/s

The total number of input datagrams received from interfaces per second, including those received in error [ipv6IfStatsInReceives].

fwddgm6/s

The number of output datagrams per second which this entity received and forwarded to their final destinations [ipv6IfStatsOutForwDatagrams].

idel6/s

The total number of datagrams successfully delivered per second to IPv6 user-protocols (including ICMP) [ipv6IfStatsInDelivers].

orq6/s

The total number of IPv6 datagrams which local IPv6 user-protocols (including ICMP) supplied per second to IPv6 in requests for transmission [ipv6IfStatsOutRequests]. Note that this counter does not include any datagrams counted in fwddgm6/s.

asmrq6/s

The number of IPv6 fragments received per second which needed to be reassembled at this interface [ipv6IfStatsReasmReqds].

asmok6/s

The number of IPv6 datagrams successfully reassembled per second [ipv6IfStatsReasmOKs].

imcpck6/s

The number of multicast packets received per second by the interface [ipv6IfStatsInMcastPkts].

omcpck6/s

The number of multicast packets transmitted per second by the interface [ipv6IfStatsOutMcastPkts].

fragok6/s

The number of IPv6 datagrams that have been successfully fragmented at this output interface per second [ipv6IfStatsOutFragOKs].

fragcr6/s

The number of output datagram fragments that have been generated per second as a result of fragmentation at this output interface [ipv6IfStatsOutFragCreates].

With the EIP6 keyword, statistics about IPv6 network errors are reported. Note that IPv6 statistics depend on sadc option “-S IPV6″ to be collected. The following values are displayed (formal SNMP names between square brackets):

ihdrer6/s

The number of input datagrams discarded per second due to errors in their IPv6 headers, including version number mismatch, other format errors, hop count exceeded, errors discovered in processing their IPv6 options, etc. [ipv6IfStatsInHdrErrors]

iadrer6/s

The number of input datagrams discarded per second because the IPv6 address in their IPv6 header’s destination field was not a valid address to be received at this entity. This count includes invalid addresses (e.g., ::0) and unsupported addresses (e.g., addresses with unallocated prefixes). For entities which are not IPv6 routers and therefore do not forward datagrams, this counter includes datagrams discarded because the destination address was not a local address [ipv6IfStatsInAddrErrors].

iukwnp6/s

The number of locally-addressed datagrams received successfully but discarded per second because of an unknown or unsupported protocol [ipv6IfStatsInUnknownProtos].

i2big6/s

The number of input datagrams that could not be forwarded per second because their size exceeded the link MTU of outgoing interface [ipv6IfStatsInTooBigErrors].

idisc6/s

The number of input IPv6 datagrams per second for which no problems were encountered to prevent their continued processing, but which were discarded (e.g., for lack of buffer space) [ipv6IfStatsInDiscards]. Note that this counter does not include any datagrams discarded while awaiting re-assembly.

odisc6/s

The number of output IPv6 datagrams per second for which no problem was encountered to prevent their transmission to their destination, but which were discarded (e.g., for lack of buffer space) [ipv6IfStatsOutDiscards]. Note that this counter would include datagrams counted in fwddgm6/s if any such packets met this (discretionary) discard criterion.

inort6/s

The number of input datagrams discarded per second because no route could be found to transmit them to their destination [ipv6IfStatsInNoRoutes].

onort6/s

The number of locally generated IP datagrams discarded per second because no route could be found to transmit them to their destination [unknown formal SNMP name].

asmf6/s

The number of failures detected per second by the IPv6 re-assembly algorithm (for whatever reason: timed out, errors, etc.) [ipv6IfStatsReasmFails]. Note that this is not necessarily a count of discarded IPv6 fragments since some algorithms can lose track of the number of fragments by combining them as they are received.

fragf6/s

The number of IPv6 datagrams that have been discarded per second because they needed to be fragmented at this output interface but could not be [ipv6IfStatsOutFragFails].

itrpck6/s

The number of input datagrams discarded per second because datagram frame didn’t carry enough data [ipv6IfStatsInTruncatedPkts].

With the ICMP6 keyword, statistics about ICMPv6 network traffic are reported. Note that ICMPv6 statistics depend on sadc option “-S IPV6″ to be collected. The following values are displayed (formal SNMP names between square brackets):

imsg6/s

The total number of ICMP messages received by the interface per second which includes all those counted by ierr6/s [ipv6IfIcmpInMsgs].

omsg6/s

The total number of ICMP messages which this interface attempted to send per second [ipv6IfIcmpOutMsgs].

iech6/s

The number of ICMP Echo (request) messages received by the interface per second [ipv6IfIcmpInEchos].

iechr6/s

The number of ICMP Echo Reply messages received by the interface per second [ipv6IfIcmpInEchoReplies].

oechr6/s

The number of ICMP Echo Reply messages sent by the interface per second [ipv6IfIcmpOutEchoReplies].

igmbq6/s

The number of ICMPv6 Group Membership Query messages received by the interface per second [ipv6IfIcmpInGroupMembQueries].

igmbr6/s

The number of ICMPv6 Group Membership Response messages received by the interface per second [ipv6IfIcmpInGroupMembResponses].

ogmbr6/s

The number of ICMPv6 Group Membership Response messages sent per second [ipv6IfIcmpOutGroupMembResponses].

igmbrd6/s

The number of ICMPv6 Group Membership Reduction messages received by the interface per second [ipv6IfIcmpInGroupMembReductions].

ogmbrd6/s

The number of ICMPv6 Group Membership Reduction messages sent per second [ipv6IfIcmpOutGroupMembReductions].

irtsol6/s

The number of ICMP Router Solicit messages received by the interface per second [ipv6IfIcmpInRouterSolicits].

ortsol6/s

The number of ICMP Router Solicitation messages sent by the interface per second [ipv6IfIcmpOutRouterSolicits].

irtad6/s

The number of ICMP Router Advertisement messages received by the interface per second [ipv6IfIcmpInRouterAdvertisements].

inbsol6/s

The number of ICMP Neighbor Solicit messages received by the interface per second [ipv6IfIcmpInNeighborSolicits].

onbsol6/s

The number of ICMP Neighbor Solicitation messages sent by the interface per second [ipv6IfIcmpOutNeighborSolicits].

inbad6/s

The number of ICMP Neighbor Advertisement messages received by the interface per second [ipv6IfIcmpInNeighborAdvertisements].

onbad6/s

The number of ICMP Neighbor Advertisement messages sent by the interface per second [ipv6IfIcmpOutNeighborAdvertisements].

With the EICMP6 keyword, statistics about ICMPv6 error messages are reported. Note that ICMPv6 statistics depend on sadc option “-S IPV6″ to be collected. The following values are displayed (formal SNMP names between square brackets):

ierr6/s

The number of ICMP messages per second which the interface received but determined as having ICMP-specific errors (bad ICMP checksums, bad length, etc.) [ipv6IfIcmpInErrors]

idtunr6/s

The number of ICMP Destination Unreachable messages received by the interface per second [ipv6IfIcmpInDestUnreachs].

odtunr6/s

The number of ICMP Destination Unreachable messages sent by the interface per second [ipv6IfIcmpOutDestUnreachs].

itmex6/s

The number of ICMP Time Exceeded messages received by the interface per second [ipv6IfIcmpInTimeExcds].

otmex6/s

The number of ICMP Time Exceeded messages sent by the interface per second [ipv6IfIcmpOutTimeExcds].

iprmpb6/s

The number of ICMP Parameter Problem messages received by the interface per second [ipv6IfIcmpInParmProblems].

oprmpb6/s

The number of ICMP Parameter Problem messages sent by the interface per second [ipv6IfIcmpOutParmProblems].

iredir6/s

The number of Redirect messages received by the interface per second [ipv6IfIcmpInRedirects].

oredir6/s

The number of Redirect messages sent by the interface by second [ipv6IfIcmpOutRedirects].

ipck2b6/s

The number of ICMP Packet Too Big messages received by the interface per second [ipv6IfIcmpInPktTooBigs].

opck2b6/s

The number of ICMP Packet Too Big messages sent by the interface per second [ipv6IfIcmpOutPktTooBigs].

With the UDP6 keyword, statistics about UDPv6 network traffic are reported. Note that UDPv6 statistics depend on sadc option “-S IPV6″ to be collected. The following values are displayed (formal SNMP names between square brackets):

idgm6/s

The total number of UDP datagrams delivered per second to UDP users [udpInDatagrams].

odgm6/s

The total number of UDP datagrams sent per second from this entity [udpOutDatagrams].

noport6/s

The total number of received UDP datagrams per second for which there was no application at the destination port [udpNoPorts].

idgmer6/s

The number of received UDP datagrams per second that could not be delivered for reasons other than the lack of an application at the destination port [udpInErrors].

The ALL keyword is equivalent to specifying all the keywords above and therefore all the network activities are reported.

-o [ filename ]

Save the readings in the file in binary form. Each reading is in a separate record. The default value of the filenameparameter is the current daily data file, the /var/log/sa/sadd file. The -o option is exclusive of the -f option. All the data available from the kernel are saved in the file (in fact, sar calls its data collector sadc with the option “-S ALL”. Seesadc(8) manual page).

-P { cpu [,…] | ALL }

Report per-processor statistics for the specified processor or processors. Specifying the ALL keyword reports statistics for each individual processor, and globally for all processors. Note that processor 0 is the first processor.

-p

Pretty-print device names. Use this option in conjunction with option -d. By default names are printed as dev m-nwhere m and n are the major and minor numbers for the device. Use of this option displays the names of the devices as they (should) appear in /dev. Name mappings are controlled by /etc/sysconfig/sysstat.ioconf.

-q

Report queue length and load averages. The following values are displayed:

负载情况。

runq-sz

Run queue length (number of tasks waiting for run time).

等待执行的队列长度。

plist-sz

Number of tasks in the task list.

总任务数目。

ldavg-1

System load average for the last minute. The load average is calculated as the average number of runnable or running tasks (R state), and the number of tasks in uninterruptible sleep (D state) over the specified interval.

1分钟负载情况,包括了 可执行、正在执行,以及不可中断的休眠状态的进程。

ldavg-5

System load average for the past 5 minutes.

ldavg-15

System load average for the past 15 minutes.

-r

Report memory utilization statistics. The following values are displayed:

内存使用情况监控。

kbmemfree

Amount of free memory available in kilobytes.

kbmemused

Amount of used memory in kilobytes. This does not take into account memory used by the kernel itself.

内存用量,未考虑内核使用的内存。

%memused

Percentage of used memory.

kbbuffers

Amount of memory used as buffers by the kernel in kilobytes.

内核使用的buffer大小,单位是KB。关于buffer和cache的区别,请参考Understanding free command in Linux/UnixOverview of memory management。But in future if any application want to use these buffers/cache, Linux will free it for you。

kbcached

Amount of memory used to cache data by the kernel in kilobytes.

内核使用的cache大小,单位是KB。But in future if any application want to use these buffers/cache, Linux will free it for you。

kbcommit

Amount of memory in kilobytes needed for current workload. This is an estimate of how much RAM/swap is needed to guarantee that there never is out of memory.

保证系统正常运行所需的内存,这是一个预测值,单位是KB。

%commit

Percentage of memory needed for current workload in relation to the total amount of memory (RAM+swap). This number may be greater than 100% because the kernel usually overcommits memory.

kbactive

Amount of active memory in kilobytes (memory that has been used more recently and usually not reclaimed unless absolutely necessary).

活跃内存用量,如果非必须,不会被reclaimed。

kbinact

Amount of inactive memory in kilobytes (memory which has been less recently used. It is more eligible to be reclaimed for other  purposes).

非活跃内存用量。

-R

Report memory statistics. The following values are displayed:

也是内存相关监控。

frmpg/s

Number of memory pages freed by the system per second. A negative value represents a number of pages allocated by the system. Note that a page has a size of 4 kB or 8 kB according to the machine architecture.

系统每秒释放的内存页数目。负值代表系统 申请的内存页 > 释放的内存页。内存页的大小可能是4kB或8KB。

bufpg/s

Number of additional memory pages used as buffers by the system per second. A negative value means fewer pages used as buffers by the system.

每秒用于buffer用途的内存页。

campg/s

Number of additional memory pages cached by the system per second. A negative value means fewer pages in the cache.

每秒用于cache用途的内存页。

-s [ hh:mm:ss ]

Set the starting time of the data, causing the sar command to extract records time-tagged at, or following, the time specified. The default starting time is 08:00:00. Hours must be given in 24-hour format. This option can be used only when data are read from a file (option -f ).

-S

Report swap space utilization statistics. The following values are displayed:

kbswpfree

Amount of free swap space in kilobytes.

kbswpused

Amount of used swap space in kilobytes.

%swpused

Percentage of used swap space.

kbswpcad

Amount of cached swap memory in kilobytes. This is memory that once was swapped out, is swapped back in but still also is in the swap area (if memory is needed it doesn’t need to be swapped out again because it is already in the swap area. This saves I/O).

%swpcad

Percentage of cached swap memory in relation to the amount of used swap space.

-t

When reading data from a daily data file, indicate that sar should display the timestamps in the original locale time of the data file creator. Without this option, the sar command displays the timestamps in the user’s locale time.

-u [ ALL ]

Report CPU utilization. The ALL keyword indicates that all the CPU fields should be displayed. The report may show the following fields:

CPU使用监控。也是默认监控,即sar -u  == sar。

%user

Percentage of CPU utilization that occurred while executing at the user level (application). Note that this field includes time spent running virtual processors.

%usr

Percentage of CPU utilization that occurred while executing at the user level (application). Note that this field does NOT include time spent running virtual processors.

%nice

Percentage of CPU utilization that occurred while executing at the user level with nice priority.

%system

Percentage of CPU utilization that occurred while executing at the system level (kernel). Note that this field includes time spent servicing hardware and software interrupts.

%sys

Percentage of CPU utilization that occurred while executing at the system level (kernel). Note that this field does NOT include time spent servicing hardware or software interrupts.

%iowait

Percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.

虽然CPU是空闲的,但还有未完成的IO请求的占比。

%steal

Percentage of time spent in involuntary wait by the virtual CPU or CPUs while the hypervisor was servicing another virtual processor.

%irq

Percentage of time spent by the CPU or CPUs to service hardware interrupts.

硬中断的处理时间占比。

%soft

Percentage of time spent by the CPU or CPUs to service software interrupts.

软中断的处理时间占比。

%guest

Percentage of time spent by the CPU or CPUs to run a virtual processor.

%idle

Percentage of time that the CPU or CPUs were idle and the system did not have an outstanding disk I/O request.

CPU是空闲的,且没有未完成的IO请求。与%iowait的区别请注意。

Note: On SMP machines a processor that does not have any activity at all (0.00 for every field) is a disabled (offline) processor.

-v

Report status of inode, file and other kernel tables. The following values are displayed:

文件系统相关数据。

dentunusd

Number of unused cache entries in the directory cache.

file-nr

Number of file handles used by the system.

inode-nr

Number of inode handlers used by the system.

pty-nr

Number of pseudo-terminals used by the system.

-V

Print version number then exit.

-w

Report task creation and system switching activity.

任务调度数据。

proc/s

Total number of tasks created per second.

cswch/s

Total number of context switches per second.

-W

Report swapping statistics. The following values are displayed:

pswpin/s

Total number of swap pages the system brought in per second.

pswpout/s

Total number of swap pages the system brought out per second.

-y

Report TTY device activity. The following values are displayed:

rcvin/s

Number of receive interrupts per second for current serial line. Serial line number is given in the TTY column.

xmtin/s

Number of transmit interrupts per second for current serial line.

framerr/s

Number of frame errors per second for current serial line.

prtyerr/s

Number of parity errors per second for current serial line.

brk/s

Number of breaks per second for current serial line.

ovrun/s

Number of overrun errors per second for current serial line.

Note that with recent 2.6 kernels, these statistics can be retrieved only by root.

Environment

The sar command takes into account the following environment variables:

S_TIME_FORMAT

If this variable exists and its value is ISO then the current locale will be ignored when printing the date in the report header. The sar command will use the ISO 8601 format (YYYY-MM-DD) instead.

S_TIME_DEF_TIME

If this variable exists and its value is UTC then sar will save its data in UTC time (data will still be displayed in local time). sar will also use UTC time instead of local time to determine the current daily data file located in the /var/log/sadirectory. This variable may be useful for servers with users located across several timezones.

Examples

sar -u 2 5

Report CPU utilization for each 2 seconds. 5 lines are displayed.

sar -I 14 -o int14.file 2 10

Report statistics on IRQ 14 for each 2 seconds. 10 lines are displayed. Data are stored in a file called int14.file.

sar -r -n DEV -f /var/log/sa/sa16

Display memory and network statistics saved in daily data file ‘sa16′.

sar -A

Display all the statistics saved in current daily data file.

Bugs

/proc filesystem must be mounted for the sar command to work.

All the statistics are not necessarily available, depending on the kernel version used.

Files

/var/log/sa/sadd

Indicate the daily data file, where the dd parameter is a number representing the day of the month.

/proc contains various files with system statistics.

Author

Sebastien Godard (sysstat <at> orange.fr)

See Also

sadc(8)sa1(8)sa2(8)sadf(1)isag(1)pidstat(1)mpstat(1)iostat(1)vmstat(8)

 

存储型服务器

首先以一台存储服务器为例,进行内存相关分析。

free命令输出如下,首先需关注的是65953012B=62GB,但该系统物理内存是64GB,这是因为total不包含内核使用的内存。如果有硬件保留了部分内存,那也未计算在内。

从第二行可用看到,若视buf和cached都为可复用,则总可用内存还有54G,比较充足。但如果不包含buf和cached,完全free的内存只有268M,是非常小的。而尤其是cached是将常用磁盘数据缓存,如果频繁flush掉这块内存,对性能是有影响的,即本来读内存的操作变成了读磁盘的操作。那如何确定内存真的够使呢?往下看。

sar -r 1 5 查看内存整体使用情况如下,前面几列数据与free一致,关注kbactive和kbinact 字段,也表明如果发生memory reclaim,还有30G+ inact不经常使用的内存可用,一般不会触及经常active的内存。

sar -B 1 查看paging信息,下图是从中截取的一段。可用看到pgpgin/s在一段时间内持续为1024,而pgpgout/s是间歇性的突增。fault/s虽然较大,majflt/s为0,代表虽然发生了页中断,但主要是在虚拟内存到物理内存的映射上,基本没有从disk到内存的中断,所以性能可以接受。这也是跟该服务器承载业务相关,其上运行hbase, hdfs, zookeeper, thrift以及ETL C++进程,写多读很少,所以在没有发生split、merge的时候,基本没有从disk到磁盘的数据读入。

sar -R 1 10,这段时间新申请的内存页大于释放的内存页,且新增了少量用于buffer的内存页,新增了较多用于cached的内存页。

统计所有进程的vsz和rss用量,vsz达71G,rss仅9G。可以看到vsz虚拟内存已经超过了62G系统总内存,归功于虚拟内存系统,才可以使总内存看起来增加了。而rss内存小于free命令输出第二行的 -buffers/cache的11G,显然也不会包含buffer和cached内存。

$ ps axu | perl -ne ‘BEGIN{$vsz=0; $rss=0;} split; $vsz+=$_[4]; $rss+=$_[5];END{print $vsz .”\t”. $rss .”\n”;}’

71522748 9625152

其中vsz用量top10的进程如下,可以看到大部分进程的rss远远小于vsz,因为Linux针对用户进程的malloc内存会尽量延迟分配,而代码段、数据段等也只会使用到时,才依赖缺页中断从磁盘加载到内存中,否则只是虚拟内存的逻辑空间地址而已。

  1. thrift 4,901,424 486,352
  2. hbase region server 4,789,688 4,485,900
  3. hbase master 4,672,580 232,412
  4. zookeeper 4,588,768 85,508
  5. ETL C++ 1,885,824 787,656
  6. ETL C++ 1,862,092 774,448
  7. ETL C++ 1,682,696 414,276
  8. hadoop datanode 1,416,264 176,760
  9. hadoop datanode 1,376,536 177,436
  10. 监控平台进程 996,048 14,372

计算型Nginx服务器

再以一台Nginx服务器为例,分析内存。其buffer用量较少,仅290M,buffer主要用于块设备的读写缓存,应该也是由于nginx服务面向http请求,其数据吞吐与HBase相比较小。

sar -r 1 5:

sar -B 1:

sar -R 1 10,平均而言这段时间申请的内存页要多于释放的内存页,没有新增用于buffer的内存页,新增少量用于cached的内存页。

统计所有进程的vsz和rss用量,vsz仅18G,rss仅3G,可以看到这台服务器内存非常充沛。

$ ps axu | perl -ne ‘BEGIN{$vsz=0; $rss=0;} split; $vsz+=$_[4]; $rss+=$_[5];END{print $vsz .”\t”. $rss .”\n”;}’
18630732 3119256

 

 

参考资料

http://www.linuxatemyram.com/play.html

http://www.linuxhowtos.org/System/Linux%20Memory%20Management.htm

http://www.linuxnix.com/2013/05/find-ram-size-in-linuxunix.html

http://en.wikipedia.org/wiki/Paging

http://www.win.tue.nl/~aeb/linux/lk/lk-9.html

http://blog.csdn.net/dlutbrucezhang/article/details/9058583

http://oss.org.cn/kernel-book/

2013-09-11

1.1 敏捷联盟

敏捷这个词被滥用了很久,部分人认为站会、自动部署、没计划、没文档就是敏捷了。其实这都是片面且偏激的看法。从敏捷宣言的解读就可以看出,在强调个体能力和沟通的同时,它也不忽视工具。而在强调代码的同时,也不忽视文档,只是不要求“面面俱到”的文档,而是维护系统原理和架构文档,细节留给代码。而响应变化胜过遵循计划方面,我理解,在互联网行业里,从RD的角度来看,由于来自PM或用户的需求是多变的,所以无法生造出一个长远(横跨几个月)且不变的计划;但从PM或项目经理的角度,得有谱,得规划出产品的长远意义和走向,否则项目将失去灵魂。

1.2  原则

对于我们来说,与其规划一个耗时数月的产品,不如拆分为小功能,花上一两周作出第一版(需要有核心的产品价值和可快速迭代的架构),快速上线(不要局限于一个入口,尤其是需要有我们完全可控的入口),然后收集用户反馈、分析用户行为,持续升级和推广。这里不局限于一个入口,是有感于之前创业阶段,把宝都压在淘宝APP上,完全受制于对方的政策。可快速迭代的架构,我理解必须是内聚和解耦且拥有全面自动化回归case的,这样才可以放心的对某一些子功能动手术。所以,这里强调的是迭代规划、架构、人、数据监控与分析、推广

敏捷中人的作用尤其重要!对程序员的要求有:

  • 视软件如己出,为自己做事,且相信产品会为用户、世界带来极大的价值(后者可能对于我这样的人比较有意义)。只有这样,才有持续的动力 提高自己的技能、避免坏味并写出高质量的代码、主动发现有问题的点并修复它、自驱动领取力所能及的task等等。
  • 言出必行,由于强调的是面对面的交谈,文档、邮件仅作为备忘录记录大事件,所以对于细节更多是通过人的自律来保证的(在敏捷初始,可能还是需要通过细致的TODO list来保证吧?在涉及团队间交互时,接口文档还是必不可少的。)其实,个人觉得,这是不分行业的,是基本的要求,就是按时按质完成工作。

进度的评估,以可用功能完成进度为准,不包含调研、设计、文档、基础lib库的进展。因为后者都太虚无,PM或用户看不到真正的效果,也无法准确的验收进度。这也从另一方面,强迫敏捷开发团队将需求拆分为可独立上线的子功能,否则进度一直都是0%!

最后,每隔一段时间,敏捷团队需要坐下来回顾实施过程中的经验和困难,并作出调整。这一方面是积累,另一方面也可以看出敏捷的原则不是定死的,而是可以根据团队的情况,灵活应用。

 

2013-9-12

2.1.3 短交付周期

极限编程里有“发布计划”、“迭代计划”两个概念。前者是多个完整的story,进行一次发布或上线,持续3个月。后者虽然也由一个或多个story组成,但仅完成开发、测试并持续集成至版本仓库,不发布,持续2周。这种发布的频率可能是产生自传统软件行业,以通信行业为例,一个完整的系统交付可能持续数年甚至更长,每3个月做一个版本升级已经很快。但个人感觉虽然频率并不适合互联网,但思想仍可以借鉴。

3个月的发布计划,会迫使需求提出者作出较为长远的规划,避免需求无目的的堆积。而将多个有组织的需求,再拆分为较小的开发周期,并在此周期内保持需求的稳定,可以使开发者不至于因为需求的频繁变动而乱了节奏。但只要需求未被纳入开发迭代,提出者就可以对需求进行调整,也保持了快速应变的能力。

应用到互联网行业,我们是否仍然是做3个月的规划,2个周的迭代。改变的是迭代结束就上线,将效果交给最终的用户去评判,并加入监控和分析,对于影响较大的紧急反馈在RB分支里在几天内立刻修复并上线,而其他反馈并入随后的迭代里。

2.1.12 简单的设计、2.1.13 重构

简单的设计、不提前设计,这个在现实中如何平衡?而这两点,其实也是基于“重构”来的。简单的设计,在面对新需求,需要变更代码、lib库甚至架构时,如何处之?答案是有全面自动化case保证的重构和高质量的工程师!但现实中,100%全面的case是不存在的,人的方面更是变化多端。所以,个人觉得,需要折中。

  • 欢迎重构,但尽量把重构放在本迭代中,不把坏味代码留到发布版本里,减少后面为了消除代码坏味而进行的重构。
  • 通过数据推测性能需求,通过对需求提出者的诘问推测功能需求,为可预测的将来做准备。
  • 在需要对已有功能做重构时,化整为零,每重构完一个小功能就build,确保无误;并且尽量采取小流量的方式先试点再推广。

2.1.14 隐喻

TODO

这篇blog几乎跟什么深奥的技术没有关系,甚至可能略显小儿科。

核心是,程序员不应该浪费时间在重复的劳动上,除非这重复工作确实不可避免,并且意义足够重大,比如说拯救地球-,-。而最近的工作中,发现几个违反了懒惰规则的地方,写了几个小工具,记录一下。

  1. phpunit的testSuite.php入口文件
  2. api使用的参数检查checker类
  3. 新修改代码的批量语法检查
  4. 脚本替换

phpunit的testSuite.php入口文件

在一个项目里,发现phpunit的入口文件testSuite.php中,定义了一个继承自PHPUnit_Framework_TestSuite的类,该类的构造函数中,hard code了一些测试用例的声明。每次有新测试用例,都会把之前旧用例注释掉。

//require_once ‘libs/ATest.php';
//$this->addTestSuite(‘ATest’);

require_once ‘libs/BTest.php';
$this->addTestSuite(‘BTest’);

由于是多人开发,还会出现冲突。其实只要一个小小的改动:

      public function __construct() {
          $this->setName('TestsSuite');

          global $argv;
          if (($cnt=count($argv)) > 2){
              for($idx=2; $idxaddTestFile($argv[$idx]);
              }
          }
      }

使用的时候,传入用例的文件名称即可(还可以利用linux的文件名自动补完):

phpunit TestsSuite.php ‘cases/ATest.php’

api使用的参数检查checker类

严格意义上来说,每个方法都不应该信任调用者传来的参数,而进行严格的参数检查。但这真是个体力活!导致的结果,要不然就重复的编码,要不然就是直接省略,把参数检查的责任交给下一级方法或者引入安全问题。其实可以抽象一下。

写了个简单的checker类,调用者配置待检查的参数是否必须存在,若存在使用什么检查handler(比如_check_string检查是多长的字符串,_check_unsignedint检查必须是正整数等),如果不符合要求,返回什么错误码和错误提示。

于是,参数检查不但变得简单,而且该配置本身还起到了代码注释的作用!

新修改代码的批量语法检查

打字快了的时候,难免没有手误,在测试和提交前,可以利用php -l来检查文件里是否有语法错误。但是每次都手工执行,多麻烦啊!写了一个脚本,通过svn st命令获取到新修改文件的名称,再检查所有以”.php”结尾的文件(如果有其他后缀的php文件,也可以添加)。

#!/bin/bash

### 检查新修改的php文件,是否有语法错误(svn st)

svnmodified=$(svn st)

# 错误个数
errnum=0

for line in $svnmodified;do
        pos=${#line}-4
        tail=${line:$pos}

        if [ "$tail" == ".php" ];then
                php -l $line

                if [ $? -ne 0 ];then
                        ((errnum=errnum+1))
                fi
        fi
done

echo -e "\n\n####### Total errors: $errnum ######\n\n" >&2

我们还可以进一步,把它和svn想绑定,每次svn ci的时候,都自动执行一遍。

脚本替换

有时项目里,会有大量文件的内容替换工作,比如这次,我需要将某一类老的方法调用替换到新方法上去,共计500多处,如果手工修改,估计就直接放弃了。

通过分析新老调用的区别,其实用sed命令,可以很容易实现,最后,用了一行代码:

sed -i’.bak’ “s/OldLog::Log\s*(\s*\([^,]\{1,\},\)\{2\}/NewLog::notice(/g” `grep  “OldLog::Log” -rwl * | grep -v svn | grep -v tags`

完成之后,svn diff检查结果,再略微手工调整,工作量大大减少。

结语

程序员的工作,可以很快乐,但乐子是自己找的。

nginx能够通过upstream的方式,把请求分配到不同的phpcgi服务上。

#phpcgi/etc/php-fpm.conf
listen = 10.241.133.144:9000   # 本地ip,非127.0.0.1
listen.allowed_clients = 10.241.133.137 # nginx server ip,非127.0.0.1

#nginx/conf/nginx.conf
upstream multiphp{
  server 10.241.133.144:9000 weight=1;
  server 127.0.0.1:9000 weight=1;
}
server{
  ……
  location ~ .*\.php{
    fastcgi_pass  multiphp;
    #fastcgi_pass  127.0.0.1:9000;
    fastcgi_index index.php;
    fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    include fastcgi.conf;
  }
}

2011-12-31  开发中,积分系统;builder模式改造抽奖领奖流程

2011-12-31  待上线,修改代码,catch异常,即使tokyo崩溃,项目也可运行

2011-12-30  13:30人为操作导致tokyo server崩溃,数据丢失;15点左右,恢复一部分,使网页可运行;17点,全部恢复

今天在火丁笔记上看到介绍了Linux运维利器之ClusterShell,试用了一次,正好符合我上线代码的需求!

我们的环境如下:2台db server,4台web server,全部得通过跳板机登录,且跳板机不转发ssh命令(至少我目前没发现怎样操作),之前仅有2台web server的时候,我都是通过上线脚本,操控svn进行代码的更新。但是现在增长到4台之后,就显得麻烦了!

利用clustershell正好可以解决这个问题!在其中一台server上(我选择了一台web server),安装clustershell(它依赖于python),并且建立conf文件,在/etc/clustershell/groups中配置db和web服务器组:

[admin@v080027 clustershell-1.5.1]$ cat /etc/clustershell/groups
db: 192.168.80.103 192.168.80.104
web: 192.168.80.105 192.168.80.106 192.168.80.27 192.168.80.49
并且建立该server到其他机器的免密码ssh登录方式。

这样,就可以方便的管理其他服务器了!

交互的问题:因为上线前,都是要先比对新旧代码,确定无误后,输入“yes”,才正式merge代码的!由于上线脚本是自己写的,会read stdin的yes or no,所以只需要echo “yes” | ./push.sh 即可。当然,在这之前,需要先进入某一台服务器,查看push.sh的diff输出,确定无误!

虽然简陋,但是暂时可以满足我的需求。所以不再进一步学习了

深入理解linux的权限设置和SUID,SGID以及粘滞位

我们知道文件的权限可以用三个八进制数字表示。其实文件的权限应该用四个八进制来表示,不过用 ls -l 命令时,只显示三个罢了。那个没有显示的八进制数字其实是第一个,它用来设定一些特殊权限。这个八进制数字的三个位是:

SUID SGID sticky-bit

它们的含义是:

SUID 当设置了

SUID 位的文件被执行时,该文件将以所有者的身份运行,也就是说无论谁来执行这个文件,他都有文件所有者的特权。如果所有者是 root 的话,那么执行人就有超级用户的特权了。这时该位将变成一个安全漏洞,因此不要轻易设置该位。

SGID 与上面的内容类似。文件运行时,运行者将具有所属组的特权。

sticky-bit sticky 位要求操作系统既是在可执行程序退出后,仍要在内存中保留该程序的映象。这样做是为了节省大型程序的启动时间。但是会占用系统资源。因此设置该位,不如把程序写好。

set uid ;set gid;sticky bit区别

每一个文件有所有者及组编号,set uid ;set gid可以改变用户对文件具有的权限:写和执行.

setuid: 在执行时具有文件所有者的权限.
setgid: 设置目录. 一个目录被标上setgid位,此目录下创建的文件继承该目录的属性.
sticky bit: 该位可以理解为防删除位. 设置sticky bit位后,就算用户对目录具有写权限,但也只能添加文件而不能删除文件。
如何设置:

操作这些标志与操作文件权限的命令是一样的, 都是 chmod. 有两种方法来操作,
1) chmod u+s temp — 为temp文件加上setuid标志. (setuid 只对文件有效,U=用户)
chmod g+s tempdir — 为tempdir目录加上setgid标志 (setgid 只对目录有效,g=组名)
chmod o+t temp — 为temp文件加上sticky标志 (sticky只对文件有效)

2) 采用八进制方式. 这一组八进制数字三位的意义如下,
abc
a – setuid位, 如果该位为1, 则表示设置setuid
b – setgid位, 如果该位为1, 则表示设置setgid
c – sticky位, 如果该位为1, 则表示设置sticky

设置后, 可以用 ls -l 来查看. 如果本来在该位上有x, 则这些特殊标志显示为小写字母 (s, s, t). 否则, 显示为大写字母 (S, S, T)
如:

rwsrw-r– 表示有setuid标志 (rwxrw-r–:rwsrw-r–)
rwxrwsrw- 表示有setgid标志 (rwxrwxrw-:rwxrwsrw-)
rwxrw-rwt 表示有sticky标志 (rwxrw-rwx:rwxrw-rwt)

理解文件权限

所谓的文件权限,是指对文件的访问权限,包括对文件的读、写、删除、执行。Linux 是一个多用户操作系统,它允许多个用户同时登录和工作。因此 Linux 将一个文件或目录与一个用户和组联系起来。请看下面的例子:

drwxr-xr-x 5 root root 1024 Sep 13 03:27 Desktop

与文件权限相关联的是第一、第三、第四个域。第三个域是文件的所有者,第四个域是文件的所属组,而第一个域则限制了文件的访问权限。在这个例子中,文件的所有者是 root,所属的组是 root,文件的访问权限是 drwxr-xr-x。对于文件和目录讲,每个文件和目录都有一组权限标志和它们结合在一起,在上例中就是第一个域中的内容。下面来仔细分析这个域中各个符号的意义:

该域由 10 个字符组成,可以把它们分为四组,具体含义分别是:

d rwx r-x r-x

文件类型 所有者权限标志 组权限标志 其他用户权限标志

其中:

文件类型:第一个字符。由于 Linux 系统对与设备、目录、文件都当作是文件来处理,因此该字符表明此文件的类型,字符与对应的意义如下表:

文件标志

文件类型

例子

普通文件

数据文件、

ASCII 纯文本文件、程序

d 目录

/bin
b 块设备

/dev/hda(第一个 IDE 硬盘)
c 字符设备

/dev/ttyS1(与 DOS 种的串口等同)
s 套接字

/dev/log
p 命名管道

/dev/initctl(与“|”等同)
l 符号链接

/dev/modem->/dev/ttyS1

权限标志:

对每个文件或目录都有 4 类不同的用户。每类用户各有一组读、写和执行(搜索)文件的访问权限,这 4 类用户是:

root:系统特权用户类,既 UID = 0 的用户。

owner:拥有文件的用户。

group:共享文件的组访问权限的用户类的用户组名称。

world:不属于上面 3 类的所有其他用户。

作为 root,他们自动拥有了所有文件和目录的全面的读、写和搜索的权限,所以没有必要明确指定他们的权限。其他三类用户则可以在耽搁文件或者目录的基础上别授权或撤消权限。因此对另外三类用户,一共 9 个权限位与之对应,分为 3 组,每组 3 个,分别用 r、w、x 来表示,分别对应 owner、group、world。

权限位对于文件和目录的含义有些许不同。每组 3 个字符对应的含义从左至右的顺序,对于文件来说是:读文件的内容(r)、写数据到文件(w)、做为命令执行该文件(x)。对于目录来说是:读包含在目录中的文件名称(r)、写信息到目录中去(增加和删除索引点的连接)、搜索目录(能用该目录名称作为路径名去访问它所包含的文件或子目录)。具体来说就是:

1. 有只读权限的用户不能用 cd 进入该目录;还必须有执行权限才能进入。

2. 有执行权限的用户只有在知道文件名并拥有该文件的读权限的情况下才可以访问目录下的文件。

3. 必须有读和执行权限才可以使用 ls 列出目录清单,或使用 cd 进入目录。

4. 如用户有目录的写权限,则可以创建、删除或修改目录下的任何文件或子目录,既是该文件或子目录属于其他用户。

修改文件权限

首先讲修改文件的所有权,使用 chown 和 chgrp 命令:

chown new_user file or directory:修改文件或目录的所有者。

chgrp new_group file or directory:修改文件或目录的所属组。

这里需要注意的是:普通用户不能将文件或目录的所有权交与他人,只有 root 有这一权限。但是普通用户有权改变文件或目录的所属组。

由于每类用户的权限都是由 rwx 三位组成,因此可以用三个八进制数字表示文件的访问权。一个八进制数字可以用三个二进制数字表示,那么与其对应,权值为 4 的位对应 r,权值为 2 的位对应 w,权值为 1 的位对应 x。对于一类用户,将这三位与其对应的权值相乘求和,就可以得出对该类用户的访问权限。

改变文件访问权限的命令是 chmod,格式是:

chmod permission file_name

比如 chmod 764 a.txt,它表示对于文件的所有者,具有对该文件读、写、执行的权限。对于文件所属组的用户,拥有读、写的权限。对于其他用户,只有读权限。

这里需要注意的是:文件的创建者是唯一可以修改该文件访问权限的普通用户,另外一个可以修改文件访问权限的用户是 root。

还有一种表示方法,就是用字符串来设定文件访问权限。其中读用 r 表示,写用 w 表示,执行用 x 表示;所有者用 u 表示,组用户用 g 表示,其他用户用 o 表示,所有用户用 a 表示。那么上面例子就写成西面的模样:

chmod a+r,u+w,u+x,g+w a.txt

理解 SUID 和 SGID

在上面的章节中,我们知道了文件的权限可以用三个八进制数字表示。其实文件的权限应该用四个八进制来表示,不过用 ls -l 命令时,只显示三个罢了。那个没有显示的八进制数字其实是第一个,它用来设定一些特殊权限。这个八进制数字的三个位是:

SUID SGID sticky-bit

它们的含义是:

SUID 当设置了

SUID 位的文件被执行时,该文件将以所有者的身份运行,也就是说无论谁来执行这个文件,他都有文件所有者的特权。如果所有者是 root 的话,那么执行人就有超级用户的特权了。这时该位将变成一个安全漏洞,因此不要轻易设置该位。

SGID 与上面的内容类似。文件运行时,运行者将具有所属组的特权。

sticky-bit sticky 位要求操作系统既是在可执行程序退出后,仍要在内存中保留该程序的映象。这样做是为了节省大型程序的启动时间。但是会占用系统资源。因此设置该位,不如把程序写好。

zz from: http://hi.baidu.com/%B5%DA%C8%FD%B4%FA%B9%BA%CE%EF/blog/item/647c6b8bbe33af17c8fc7af2.html