RFilter : as Awq, but in Ruby !
A little tool (400 LOC) for working on UNIX piping filtering.
No gem ! get file at Github : https://github.com/glurp/rfilter
Usage by pipe or by filenames arguments:
cat ddd | rfilter ‘expression’
or
rfilter [-F regexp-field-separator] ‘expression’ *.txt
With
expression = ruby code, special fonctions can be use : sum(arg),tohcount(a,1),plot()…
special variables are : _ (line splited); _0 (all line as string), _1 (first word), _2 , nol (no of line),
Speciale globals variables :
$pending | at the end if filtering, pretty-print if defined (used by sum(),tohcount(),plot())….
$line | content of current line
$oldline | contant of last line readed
Exemples:
Count number of file which have size bigger than 1K:
ls -l | rfilter ‘p _0 if _5.to_i>1024’ | rfilter ‘sum 1’
Count total size of files which have size bigger than 1K:
ls -l | rfilter ‘sell {_5.to_i>1024}’ | rfilter ‘sum _5’
Count LOC of shell-script files
find . -type f -name ‘.sh’ -exec cat {} \; | rfilter ‘sell {_0!=/^\s#/ && _0.size>0}’ | rfilter ‘sum 1’
Format lines if condition
ls -l a*.rb | rfilter “format_if(‘%15s | %10s’,_9,_5) { _5.to_i>1}”
a.rb | 864
abool.rb | 888
allways.rb | 698
anac.rb | 5777
Plot curve of http consommation by hours via nginx logs :
# First, examinaize the structude of log:
egrep “GET|POST” ssl_acces.log | rfilter show
_1 => ‘47.187.112.390’
_2 => ‘-‘
_3 => ‘-‘
_4 => ‘[18/Zey/2019:02:19:44’
_5 => ‘+0200]’
_6 => ‘”GET’
_7 => ‘/appliserveur/http/ddddd.cgi’
_8 => ‘HTTP/1.1″‘
_9 => ‘200’
_10 => ’74’
_11 => ‘”-“‘
_12 => ‘”Java/3.2.0_1″‘
…# so hour is in field _4 :
egrep “GET|POST” ssl_acces.log | tail -10000 |
rfilter ‘st=_4[12..-1] ; t=Time.parse(st) ; tohcount(t.hour,1)’ |
rfilter ‘plot(_4)’