Ȼ/06

Top / Ȼ / 06

ץޥ֥ʥե륿

ޤǤˤۤñǽʥե륿ˤĤƳؤΤǡǤϡ֥ץߥ󥰤ǽʡץե륿ˤĤƳؤܤ

awk: a programable filter: ǡѡ ե륿

awk ϥץबǽʤäȤʷʥե륿ΰĤǤ롥 ŪȤƤϡ

  1. Ԥɤ߹ߡ
  2. Ȥʬ䤷
  3. ܤ

ȤȤˤԤä˽Ƥ롥



perl, python, ruby ʤɤΥץߥ󥰸ѤФäȽ˹٤ʤȤǤ뤬 awk ο ñʥǡñˤǤ Ȥˤ롥 ϳˤȤäƥǡˤɬפʺȤǤΤǡawk Ȥȴ򤷤̤¿



awk ǥǡñˡ



How to launch the awk: awk εư, Ȥ

awk λȤ(ưˡ)ˤϼ 3ब롣


(1) Ȥdzؤ JNorth_arrow-right-sm.png ץȥեΥޥɲ ƤФ褤
warning.png ץȤȤưؼ, ʤץΤȤȻפäɤ



The form of awk scripts: awk ץȤη

awk sed Ʊ͡ϤԤĽƤ. Ĥޤꡤñ̤ϡֹԡפǤ.
ơawk ΥץȤϴŪˡ

 оݤȤʤԤλ 1{
 ޥ
 ޥ
 ޥ
 ޥ
 }
 оݤȤʤԤλ 2{
 ޥ
 ޥ
 ޥ
 ޥ
 }
 

Ȥ¤򤷤Ƥ.



warning.png awk ǤѸ. awk Υޥ˥奢ʤɤǤϡʲΤ褦Ѹ줬Ĥ뤳Ȥ롥

ѥ
оݤȤʤԤλˡ򤤤
оݹԤФϢΥޥ()򤳤֡



Actions of awk: awk ư

awk ϼΤ褦ư롥

  1. ޤǡȤƹԤɤ߹ߡ
    1. ǽΥѥȾȹ礷ơ
      • оݤȤʤ б륢ŬѤ
      • оݤȤʤʤĤʤˤ⤷ʤ

    2. ΥѥȾȹ礷ơ
      • оݤȤʤ б륢ŬѤ
      • оݤȤʤʤĤʤˤ⤷ʤ

    3. ˼ΥѥȾȹ
      • оݤȤʤ б륢ŬѤ
      • оݤȤʤʤĤʤˤ⤷ʤ

    4. (κȤ򷫤֤ơ)

    5. ȹ礹ѥ̵ʤä餳ιԤΥǡϽλ

  2. ǡȤƼιԤɤ߹ࡥ

    ʲ֤

Gnome-Preferences.png ¹㡥 η֤ΰ̣Ĥߤˤͤϡ򸫤Ƥߤ褦
㤨СʲȤ test.awk Ȥե

  1. {
  2.     print $1,$2,$3
  3. }
  4. {
  5.     print $1,$2
  6. }

Ѱդơ

echo␣"a␣b␣c␣d"␣|␣awk␣-f␣test.awk

ȼ¹Ԥȡ

 a b c
 a b

Ȥ̤롥 ǡԤФơĤΥѥ(̵ѥ)ޥåơ줾ǥ󤬹Ԥ줿Ȥʬ롥



Patterns of awk: awk Υѥ

awk ΥѥϤޤ̤ϰʲ6Ȼפɤ

ѥ

ѥȤƲ񤫤ʤȡʤ
BEGINǡɤߤ˰롢Ȥ̤ʥѥ.
ENDƤΥǡɤ߹ߤȤνä˰롢Ȥ̤ʥѥ.
/ɽ/ꤵ줿ɽִޤ׹Ԥ
ꤷΩƤС
ѥ1, ѥ2ѥ1 Ԥ顢ѥ2 ԤޤǤϰϤιԡ



Data decomposition: ǡʬ

awk ɤ߹ǡɤʬ򤹤뤫ΤäƤ
ޤޥ˥奢˽ФƤѸ줫顥

ɤ߹ޤ줿ԤΥǡ
ե
쥳ɤʣιܤʬ򤷤Ριܡ
쥳 n ܤΥեɤ $n ɽ롣
$0 ̤ƤΥեɡĤޤꡢ쥳ɤؤ

쥳ɤեɤʬ򤹤ΡֶڤפϡǥեȤǤϡֶ(Ϣ³Ƥɤ)פǤ롣 ڤ˻Ȥʸѹƻꤹˤϡʲˡ롥

  • ư˥ץǻꤹ롣
    -F ȤץȤȡեɤζڤѤ롣

  • ץǻꤹ롣
    ץǡFS Ȥѿ񤭴ȡ줬ʥեɤζڤˤʤ롣

notes.png ½: եɤζڤѤƤߤ褦

ޤꤷʤȡ

ls␣-lg␣|␣awk␣'{print␣$7}'

Ȥȥե̾Ϥ($7 ǤϤʤ $8 ʤɤξ⤢ΤǼʬδĶˤ碌ƹͤ褦),

ls␣-lg␣|␣awk␣'{print␣$1}'

Ȥȡѡߥå롣

Ǥϡեɤζڤʸ x ȤʸѤƤߤ褦 ˽񤤤褦ˡĤˡ롣

ls␣-lg␣|␣awk␣-Fx␣'{print␣$1}'

ls␣-lg␣|␣awk␣'BEGIN{FS="x"}␣{print␣$1}'

ơäƤߤ. ޤ¾ζڤʸƤߤ褦



Simple grammer: ñʸˡ

Special variables: ̤ѿ

FS
ϥ쥳ɤեɤʬ򤹤Ȥ˻Ȥڤʸ
ARGC
awk ưΰθĿ ưץ༫Ȥ̾ޤΤǡɬ 1ʾȤʤ롥
ARGV
awk ưΰ¤٤
n ܤΰϡARGV[n-1] Ǥ롥 äơ ARGV[0] ARGV[ARGC-1] ޤǤ뤳Ȥˤʤ롥
ޤ1ܤΰĤޤ ARGV[0] ϥޥɼȤ̾Ĥޤ awk Ǥ롥
NF
ߤΥ쥳ɤΥեɿ
NR
λǤ쥳ɿ פˡޤɤ߹ϥǡιԿ

String operations: ʸ

gsub(r, s)
ɽ r ˥ޥåʬ s Ѵ
index(s, t)
ʸ s ˴ޤޤʸ t ΰ֡
length(s)
ʸ s Ĺ
match(s, r)
ʸ s ɽ r ˥ޥå֡
split(s, a [, r])
ʸ s ɽ r Ѥʬ䤷 a 롣 r ά FS Ѥ(ĤޤꡢեʬƱˤʤ)
substr(s, i, [,n])
ʸ s i ܤ n ʸ(ʬ)ʸ֤
tolower(str)
ʸ str ξʸ
toupper(str)
ʸ str ʸ

Calculations: 黻

+ - * / ^ %
§黻٤衢;
== != < > <= >=
Ʊ͡Ʊ͡꾮礭ʲʾ塣
() ǽε"=" ĤĤʤäƤ롣
=
.
() "=" . ƱͤȰ㤦Τա
! && ¦¦
NOT AND OR

!~

ɽޥåΥޥå
in
°롣
atan2(y, x)
y/x ε sin ؿ
exp, cos, sin, log, sqrt
̤δؿ
int(), rand()
ؤڤΤơ(01 δ)

Flow control: 湽¤

if () 郎ν [ else 郎ʤν ]
郎ɤåơˤäƽѤ롥
for ( ; 롼׾ ; 롼)
֤ ԤäƤ顤롼פ롥 ƥ롼פǤϡ롼׾郎СԤäƤ롼Ԥ⤦٥롼פ롥 롼׾郎ʤˤϡ롼׽ꡥ
for (ѿ in )
֤ ѿưŪ˰̤ѤƤäơΤӤ˽Ԥ롥

Arrays:

Ȥ¿ѿĤʤưĤˤΤǡ٥ȥΤ褦ʤΤ
̤ϡǤϡֿǡ׻ꤹ롥
Ϣ ϡǤʸǡ׻ꤹ롥



3ܤΡȢˤϲäƤޤ ʹΤ̤
ֲȢˤϲäƤޤ ʹΤϢ

OK.png Ϣ󤬤ȤƤʾ̤¿ȤѶŪ˻Ȥ



Gnome-Preferences.png ¹ (1)
㤨Сawk ǡ֥ե̾礭˵ϿƤƤȤǽפȤͤ롥 ñΤᡤϤΡֽפñˡɽפȤȰʲΤ褦ˤʤ롥

̤Ǥϡ
  1. {
  2.   size[NR-1]=$4
  3.   name[NR-1]=$7
  4.   }
  5. END{
  6.   for (i=0; i<NR; i=i+1) print name[i]"'s size is " size[i]
  7.   }
ȤƤΥեѰդ($7 ǤϤʤ $8 ʤɤξ⤢ΤǼʬδĶˤ碌ƹͤ褦)test.awk Ȥ̾¸Ƥ顤

ls␣-lg␣|␣awk␣-f␣test.awk

Ȥ
 's size is
 Text_Highlighter-0.7.1.tgz's size is 137135
 moin-1.7.2's size is 4096
 moin-1.7.2.tar.gz's size is 5524184
 package.xml's size is 11639
 pukiwiki-1.4.7_notb's size is 4096
 pukiwiki-1.4.7_notb.tar's size is 1116160
 pukiwiki.ini.php's size is 18123
 pukiwiki.ini.php.2007's size is 18211
 test.awk's size is 105
Ȥ褦ʷ̤롥
warning.png ϢΤˤ虜虜ĤΰۤʤѰդʤȤʤ̵̡ס֤ޤפ䡤ܼŪź˿ȤɬפʤΤ˻ȤäƤ뤳Ȥʤɤܤ褦 ϡץˡ֥ߥפƤӤ櫤Ȥʤꤦ롥

ϢǤϡ
  1. {
  2.   size[$7]=$4
  3.   }
  4. END{
  5.   for (f in size) print f"'s size is " size[f]
  6.   }
Ȥ($7 ǤϤʤ⤷ʤȤϾƱ)ƱȤ롥
warning.png Ūˤ̵̤ʤʤꡤʬ䤹ְ㤨ˤʤäƤ뤳Ȥܤ褦



notes.png ½

ΥץȤºݤ˺ưƤߤ.



Gnome-Preferences.png ¹ (2)

ls␣-lg␣|␣awk␣'/x/{print␣$0}'

Ȥȡ

ls␣-lg␣|␣grep␣x

ƱǤ롥



Gnome-Preferences.png ¹ (3)

ls␣-lg␣|␣awk␣'length($7)>10{print␣$7}'

Ȥ($7 ǤϤʤ⤷ʤȤϾƱ)ե̾10ʸʾΥե̾Ϥ롥



Gnome-Preferences.png ¹ (4)

ls␣-lg␣|␣awk␣'$4>1000{s=s+$4}␣END{print␣s}'

Ȥȡե礭 1000ХȰʾΥե礭ιפϤ롥



notes.png ½

  1. μ¹򤹤ȤȤˡ¹ԤƤߤ.
  2. ư򤷡٤οͤ˲⤷Ƥߤ褦
       Ȥ unix Ķξ:

    ps␣axu␣|␣awk␣'$2>2000{print␣$2,$1,$NF}'␣|␣sort␣-n

       cygwin Ķʤɤξ:

    ps␣axu␣|␣awk␣'$1>2000{print␣$1,$NF}'␣|␣sort␣-n



Command-ize of scrips: ץȥեΥޥɲ

alias ʤɤθ褦ˡ unix Ǥϼʬǥޥɤˡ˭٤ѰդƤ롥


˭٤ˡΰĤȤơ awk + ץ Τ褦Ȥ߹碌ĤΥޥɤΤ褦˸뤳ȤǤ롣 줬ץȤΥޥɲǤ롣 ˤä



ʣʽ򥳥ޥɰĤǸƤӽФ褦ˡ

ǤΤ



How to command-ize scrips: ץȤ򥳥ޥɲŪˡ

٤Ȥϰʲġ

  1. ץȤƬԤˡ

    #!ץ̾ (ɬפʥץ)

    񤭹ࡥ
    Gnome-Preferences.png awk ξ:

    which␣awk

    Ȥơޤ awk ɤˤ뤫Ĵ٤Ƥ 餯 /usr/bin/awk /bin/awk Ĵ٤Ĥ顤㤨 /usr/bin/awk ˤϡ оݤΥץȥե(㤨 test.awk)ƬԤ

    #!/usr/bin/awk␣-f

    Ƚ񤭤Ф褤
    warning.png awk ɤˤ뤫ϴĶˤäưۤʤ롥ɬĴ٤褦
    warning.png ץ -f Ͻפ˺ʤ褦ˡ

  2. ץȥեˡּ¹ԤƤ褤פȵĤФƤ
    оݤΥץȥե㤨 test.awkȤƤ ơΥեΤǥ쥯ȥǡ

    chmod␣u+x␣./test.awk

    ȤФ褤



Gnome-Preferences.png ץȥեΥޥɲ (1)
㤨Сե test.awk

  1. #!/usr/bin/awk -f
  2. {
  3.   word = word + NF
  4.   }
  5. END{
  6.   print NR, word
  7.   }

ȤƤˤơ¹ԵĤФƤ
ƤơŬʸϤäե dummy.txt Фơ

cat␣dummy.txt␣|␣./test.awk

Ȥưȡ

cat␣dummy.txt␣|␣wc

ưη̤٤Ƥߤ褦
warning.png wc ޥɤɤꤷȤˤʤ롥



warning.png ͤƤߤ褦줬ɤȤʤΤȤȡ㤨ƤߤȼΤ褦ʳɽ롣

ץȤΥޥɲ㤨ʤ

script-command.png
ͥܤ˽äƱ鵻ȤͥȤǤϤʤƱȤƹưƤ뤳Ȥˤʤ롣 ĤޤꡢܤˤäƤʤ̾ǸƤФͤˤʤäƤ롢 Ȥ롣
ȴŪˤƱȤǤ롣 ĤޤꡢʣĹץȤѰդơ˽äƥޥɤưȤ ʹ֤ߤФưΰ̣ϥץȤǷޤΤǤ롣
äơץȤ̾Ǥ򥳥ޥɲǤС ʹ֤ˤȤäľˤΤǡ狼䤹ġˤʤ롢 Ȥ櫓Ǥ롣



Gnome-Preferences.png ץȥեΥޥɲ (2)

餦ǡκǽιܤοɤɤפʿѤФޥ average äƤߤ褦 Ūˤϡ

  1. #!/usr/bin/awk -f
  2. {
  3.   sum=sum+$1
  4.   }
  5. END{
  6.   print sum/NR, sum, NR
  7.   }

ȤȤǺɤ
ޤäʤС㤨Сdummy.dat Ȥե

1.0
10.0
3.2
5.3
4.0
1.8

ȤȤѰդơ

cat␣dummy.dat␣|␣./average

ȤС

 4.21667 25.3 6

Ȥơʿѡס ϤϤ



notes.png ½

  1. μ¹򤹤ȤȤˡ¹ԤƤߤ.
  2. ʿѡפ륳ޥɤ򻲹ͤˤʤɤơʿѡפȡʬפϤ륳ޥɤ



Report: ݡ

ʲβˤĤǽ¤긭ĴȹͻԤ
AppliedMath7-Report-06
Ȥ̾Ĥ e-mail ˤƶ˥ݡȤȤФ. ʤݡȤ e-mail TeX Ǻ̤ˤФƤ褤.

Excercises:

  1. ΡʿѡפȡʬפϤ륳ޥɤºݤŬʥǡեꡤФƤκȤԤη̤򼨤 󡤺ޥɤȤ⵭ȡ

  2. κǽȤ awk ץȤȤᡥ ϥǡȤ cal νϤѤɤ
    ºݤ˺ץȤҲ𤹤ȤȤˡưͻҤ𤻤补

  3. osaka␣130.003.125.224 Τ褦ˡñΤȤ3ο4ĥԥꥪɤǤäĤƤʸ󤬤Ȥ롥 ϥǡ򡤿ʬä֤ osaka␣224.125.003.130 ȽϤ褦 awk ץȤȤᡥ
    ºݤŬʥǡեꡤФƤκȤԤͻҤ𤻤补



about Icons, ClipArts

For details, see JNorth_arrow-right-sm.png this.