Short documentation for JWM / Linux

Last Update: 01.07.2019 for version 2.3.0

Contents


Table of Contents
  1. Short documentation for JWM / Linux
    1. Contents
    2. Introduction
    3. Author
    4. Tools Summary
    5. History
    6. Installation
    7. Requirements for JWM / Linux
      1. Required GCC version
      2. Linux Distributions tested
        1. Old Linux Distributions tested (History)
      3. add_os_packages.sh - Script to add missing packages
    8. Using JWM / Linux
      1. Creating a JWM / Linux session
      2. Using JWM commands via jwm.sh
    9. JWM / Linux manServer
    10. Directories in the JWM / Linux archive
    11. Libraries included in JWM / Linux
    12. Tools included in JWM / Linux
    13. Scripts and files include in JWM / Linux
      1. Develop Templates
    14. Scripts and files for the JWM / Linux Environment
    15. Aliase in JWM / Linux
    16. Examples
      1. ack
      2. arping
      3. atop
      4. bmon
      5. bonnie++
      6. busybox
        1. busybox_httpd.sh
      7. bvi
      8. bwm-ng
      9. cdpinfo
      10. cdpr
      11. checksec
      12. collectl
        1. collectl - List detailed output
        2. collectl - List processes in tree view
        3. collectl - View process I/O
        4. collectl - View with timestamps
      13. CPUHog
      14. cpumon
      15. cpustat
      16. darkstat
        1. darkstat.sh
      17. dhtest
      18. dmidecode
      19. dropwatch
      20. dstat
      21. fiemap
      22. fio
      23. fnotifystat
      24. forkstat
      25. eventstat
      26. fspy
      27. health-check
      28. hdperf
      29. horst
      30. hostscope
      31. hping2
      32. htop
      33. httpry
      34. ibmonitor
      35. iftop
      36. inxi
      37. ifstat
      38. inotifywait / inotifywatch
      39. Intel Performance Counter Monitor
      40. inq
      41. ioping
      42. iotop
      43. iozone
      44. ipaudit
      45. ipband
      46. iperf
      47. iperf3
      48. ipscan
      49. iptraf
      50. iptraf-ng
      51. iptstate
      52. itop
      53. JDiskReport
      54. jnettop
      55. justniffer
      56. latencytop
      57. lxtx
      58. lynis
      59. manServer
      60. memconf
      61. memstat
      62. monitor_system.sh
      63. mpathstat.py
      64. mtr
      65. nano
      66. netdata
        1. netdata.sh
      67. nethogs
      68. ngrep
      69. nicstat
      70. nload
      71. nmap
      72. netio
      73. netperf
      74. netsniff-ng
      75. nmon
        1. nmonchart
        2. nmonanalyzer
      76. numatop
      77. oprofile
      78. p0f
      79. pagemon
      80. pchar
      81. perftools
        1. exec_snoop_wrapper.sh
        2. open_snoop_wrapper.sh
        3. opensnoop1
      82. pigz
      83. pktstat
      84. powertop
      85. powerstat
      86. procdump
      87. prtdiag
      88. pt-diskstats
      89. scanlogd
      90. screen
      91. smartmontools
      92. smem
      93. smemstat
      94. sockperf
      95. stress
      96. stress-ng
      97. sysfsutils
      98. sysstat tools
      99. t50
      100. tcpdump
      101. tcptrack
      102. tecmint_monitor.sh
      103. testssl.sh
      104. tiptop
      105. trace-cmd
      106. tracebox
      107. wavemon
    17. Compiling JWM / Linux
      1. make_tools.sh usage
      2. How to add another tool
      3. How to add another OS library
      4. Check for missing libarires
    18. Create your own tar archive
    19. Hints for Compiling the tools
      1. Compiling justniffer with a boost staging area
      2. Error message : "undefined reference to clock_gettime"
      3. Error message : "error: ‘GENL_ID_GENERATE’ undeclared"
      4. Error message: //lib64/libtinfo.so.5: error adding symbols: DSO missing from command line
      5. Error message:  ... undefined reference to `__builtin_bswap16'
      6. Error message: ... error: called object ‘major’ is not a function or function pointer
      7. Error message: error: /lib64/libncurses.so.5: undefined symbol: _nc_putchar
    20. Binaries for other OS
    21. Some howtos for Linux internal tools
      1. Using the ftrace interface with shell commands
      2. Using nmcli to view the WLAN status
    22. Sample output of the compile scripts
      1. Example output of compiling the tools in parallel sessions
      2. Example output of compiling the tools sequential
    23. History archive
    24. Download
    25. Links to other resources
    26. Links to libraries neccessary for JWM
    27. Other tools not included
    28. Tools not included anymore

 


Introduction

JWM / Linux is a collection of various performance and analyze tools for Linux. In addition,  JWM / Linux contains some useful general Linux tools.  These are


JWM / Linux is distributed via tar.gz file so that it can be installed by any user on a Linux machine.

All tools included are compiled and tested under Linux for x86 with 64 Bit (see the list of tested distributions below). For most of the tools the source code is included.

JWM / Linux contains a lot of tools that do the same thing (see here); e.g. there is more then one top replacement, there are multiple tools to monitor the network bandwidth, etc.  Just select the tool that is best for the task you're doing.

Most of the tools included are command line tools for ssh or telnet sessions, some use an internal webserver as user interface, and some are tools with an X Server GUI.

Unfortunately there is no such thing like a "linux binary" that runs on all Linux distributions. Therefore in most cases you must recompile all tools for your distribution.
You can use the script ./compile_scripts/check_for_missing_libs.sh  to check if the libraries required by the tools in JWM / Linux exist in your Linux installation.

Please note that some of the tools included need root privileges (most of them are in the ./sbin directory).

To check for a new version of JWM / Linux and updated documentation visit the JWM / Linux home page http://bnsmb.de/linux/JWM_Linux_documentation.html.


Be aware that this page now heavily uses the "new" html tag <details></details>. Therefor you should use a browser that supports this tag.

<details> Test
If you can see this text and can NOT hide it by clicking on <details> Test above your browser does not support the tag <details></details>.



Author

The collection was created by : Bernd Schemmer

email: Bernd.Schemmer@gmx.de

Homepage: http://bnsmb.de

Only small changes neccessary to compile the tools on all distributions tested were done for the source code of some tools. All changes done are documented in the entries in the table Tools included in JWM / Linux.




Tools Summary


Find below a list of all tools contained in JWM / Linux sorted by usage:


Usage
Tools
General tools
ack, busybox, bvi, pigz, screen, manServer nano
General Monitoring tools collectl, dstat, hostscope, netdata, oprofile, lxtx


Hardware information tools dmidecode, inxi, memconf, prtdiag, tecmint_monitor.sh, inq, inxi
Software information tools inxi
Security tools lynis, testssl, checksec


Process List
atop, htop, nmon, cpustat
Process monitoring tools forkstat, health-check, procdump


Kernel monitoring tools eventstat, itop, latencytop, numatop, perftools, powertop, powerstat, sysstat tools, tiptop, trace-cmd
CPU Monitoring tools CPUMonIntel Performance Counter Monitor


Memory Monitoring tools smem, smemstat, pagemon, memstat


Network Monitor tools
bwm-ng, darkstat, ibmonitor, iftop, ifstat, ipaud, ipband, iptraf, jnettop, mtr, nethogs, nicstat, nload, pchar, pktstat, tracebox, bmon, iptstate, iptraf-ng, tcptrack, httpry scanlogd
Network sniffer tools
cdpinfo, cdpr, dhest, ipscan, ngrep, nmap, p0f, justniffer, netsniff-ng, tcpdump, dropwatch
Network test tools hping2, t50, sockperf, arping
WLAN monitoring horst, wavemon


Disk Performance Test bonnie++, fio, hdperf, iozone, ioping
Network performance test tools netio, netperf, iperf3, iperf
Performance help tools
CPUHog, stress, stress-ng, fio


File Info tools
fiemap
File access monitoring tools
fnotifystat, fspy, inotifywait
Storage access monitoring tools
iotop, pt-diskstats
Disk Monitoring tools smartmontools


Tools with WebGUI darkstat, netdata, manServer



History


Date
Version
Comment
01.07.2019 /bs
2.3.0
added sockperf 3.5
added httpry 0.1.8
added trace-cmd 2.7
added nano 3.1
added monitor_systems.sh 1.0.0
added checksec 1.10.0
added ioping 1.1
added scanlogd 2.2.7
added arping 2.19
added dropwatch 1.5

added some binaries (e.g for iperf) for other OS

New versions
new version: stress-ng 0.09.57
new version: lynis 2.7.4
new version: dmidecode 3.2
new version: sysstat tools 12.1.5
new version: forkstat 0.02.09
new version: eventstat 0.04.05
new version: smemstat 0.02.03
new version: iozone 3_487
new version: bwm-ng 0.6.2
new version: busybox 1.30.1
new version: powerstat 0.02.18
new version: collectl 4.3.1
new version: netdata 1.14.0
new version: inxi 3.0.33
new version: atop 2.4.0
new version: stress-ng 0.09.59.1
new version: health-check 0.03.01
new version: nmon 16j
new version: ack 3.0

removed: unfs3


01.09.2018/bs 2.2.11 added testssl.sh 3.0beta
added ack 3 beta

New versions

new version: ack 2.24
new version: CPUhog 2.1.3
new version: cpustat 00.02.05
new version: dhtest 1.5
new version: eventstat 0.04.04
new version: forkstat 0.02.05
new version: health-check  0.03.00
new version: lynis 2.6.7
new version: scriptt_mini.sh
new version: horst 5.1
new version: hostscope 4.0
new version: htop 2.2.0
new version: inxi 3.0.21
new version: inq 9.0.0.0
new version: iozone 3_482
new version: ipscan  3.5.3
new version: oprofile 1.3.0
new version: pigz 2.4
new version: powertop 2.9
new version: procdump 1.0.1
new version: smemstat 0.02.01
new version: stress-ng 0.9.37
new version: t50 5.8.0
new version: smartmontools 6.6
new version: sysstat tools 12.0.1
new version: perftools 19.12.2017
new version: nmap 7.70
new version: netsniff-ng 0.6.4
new version: busybox 1.29.2

removed: saidar

added a patched version of the opensnoop script from the perftools
added the wrapper script  open_snoop_wrapper.sh for the script opensnoop from the perftools
added the wrapper script exec_snoop_wrapper.sh for the script execsnoop from the perftools 

fixes:

start_compiling.sh was rewritten because the script did not process all kind of symbolic links for the libraries correct
fixed a lot of compile scripts in the directory ${JWM_DIR}/compile_scripts so that they can be used with a non-standard compiler in Centos 6.9
the compile scripts printed a wrong error message in case of an error if called direct - fixed
added the "new" directories from Mint 19 to the library search path in start_compiling
the script to start collectl, scripts/collectl, did not work -- fixed

03.01.2018/bs 2.2.10 added the kornshell scrip template scriptt_mini.sh ( in the directory ${JWM_DIR}/develop)

added procdump 1.0.0
added memstat 1.0.0

New versions

new version: inxi 2.3.53
new version: lynis 2.5.8
new version: health-check 0.02.08
new version: ack 2.20
new version: collectl 4.3.0
new version: htop 2.0.2
new version: hostscope 2.3.0
new version: stress-ng 0.09.07
new version: eventstat 0.04.02
new version: memconf v3.14
new version: netsniff-ng 0.6.3
new version: iperf3 3.6.0

fixed some code in the Perl script for the MAN Server that the new Perl version does not like:

Changes in manServer.pl

[JWM session] [xtrnaw7@t540p /var/tmp/jwm]$ diff /var/tmp/jwm/scripts/manServer.pl /var/tmp/jwm/scripts/manServer.pl.org
585c585
<     s,\\\{,,g;
---
>     s,\\{,,g;
1959c1959
<             elsif ($col =~ m/T\{/) #}
---
>             elsif ($col =~ m/T{/) #}
1961c1961
<                 $col =~ s/T\{//; #}
---
>                 $col =~ s/T{//; #}
[JWM session] [xtrnaw7@t540p /var/tmp/jwm]$





...
see the History archive for the previous updates

27.10.2013/bs 1.0.0 first public release
see also the README file in the tar archive
.


Installation


JWM / Linux is delivered in a compressed tar archive.

The name of the tar archive is

jwml_<archive_version>_<distname>_<distrelease>_<cpu_architecture>_<date>.tar.gz

e.g.

jwml_v100_Mint_14_Nadia_3.5.0-17-generic_x86_64_2013-10-26.tar.gz

This is JWM / Linux version v1.0.0 compiled under "MINT 14 Nadia" with kernel patch "3.5.0-17" for "x86_64" created on "2013/10/26".

or

jwml_v226_Fedora_release_23_(Twenty_Three)_4.7.7-100.fc23.x86_64_x86_64_2016-10-26.tar.gz

This is JWM / Linux version 2.2.6 compiled under "Fedora release 23_(Twenty Three)" with kernel patch "4.7.7-100" for "x86_64" created on "2016/10/26".


To use JWM / Linux  download the tar file for your distribution and unpack it using tar with the options -xzf in an empty directory.

If there is no archive with precompiled binaries for your distribution just compile the sources yourself.


Note:

Some of the tools included need a fixed target location while building the binaries. To make sure that every user can install the tools the target directory used for the precompiled binaries is /var/tmp/jwm (/var/tmp is writable by everyone on a Linux machine). Therefor you should install the precompiled tools into the directory  /var/tmp/jwm to be sure that all included binaries work.

This directory is called ${JWM_DIR} or JWM_DIR in the rest of this document.

To change the target location you should compile the tools.



Requirements for JWM / Linux


The tools in JWM / Linux should run on every modern 64 Bit Linux for x86 . Not so common libraries used by the tools are included in the archive.

In addition, the source code for the tools is included in the archive (if available) so that you can create your own binaries.

The script to execute the compile tools is written in ksh -- therefor /bin/ksh is required to compile the tools.

Some make scripts expect sed to be in /usr/bin but in new Linux distributions sed is in /bin. To get around this issue just create a symoblic link:

sudo ln -s /bin/sed /usr/bin/sed

The same is true for mkdir:

sudo ln -s /bin/mkdir /usr/bin/mkdir



To compile the tools the necessary compile tools (gcc, g++, c++, make, bison, flex, yacc) , the kernel headers, and some library header files must be installed.
If the configure script for a tool fails just check the output of the configure script for missing header files, install them, and restart compiling.

You may use the script ./compile_scripts/ add_os_packages.sh to install the missing OS packages for the supported Linux distributions.


The environment is written for bash or ksh.

See also the notes for the tools in the table with the descriptions and the Hints for Compiling the tools below.


Required GCC version

Some of the tools require a recent version of GCC to compile. Therefor you should update the GCC before compiling the tools.

example:

To update the GCC on Centos 6.9 do

sudo yum install centos-release-scl
sudo yum update
sudo yum install devtoolset-7-gcc-c++

Results:

[xtrnaw7@centos69 ~]$ gcc --version | head -1
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23)
[xtrnaw7@centos69 ~]$

[xtrnaw7@centos69 ~]$ scl --list
devtoolset-7
[xtrnaw7@centos69 ~]$

[xtrnaw7@centos69 ~]$ scl enable devtoolset-7 bash
[xtrnaw7@centos69 ~]$

[xtrnaw7@centos69 ~]$ gcc --version | head -1
gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)
[xtrnaw7@centos69 ~]$




Linux Distributions tested

JWM / Linux was compiled on these Linux Distributions:


JWM Version
Date
Linux Distribution
Kernel version gcc version Supported by
add_os_packages?
Comment
2.3.0
16.06.2019
Fedora 30
5.1.8
gcc (GCC) 9.1.1 20190503 (Red Hat 9.1.1-1)
yes
all packages can be compiled; this is the main distribution for development since 01.01.2019

2.3.0 16.06.2019
RedHat 7 3.10.0
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36)

yes
all packages can be compiled except these ones:

t50

2.3.0 16.06.2019 Centos 6.10 2.6.32
gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)
yes
some tools do not compile anymore with the standard gcc in Centos 6.9
Therefor I installed and used the GCC v7:

gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)

(see Required GCC version)

all packages can be compiled except these ones:

health-check  (kernel not supported)
fnotifystat  (kernel not supported)

justniffer must be compiled manually -- see here








2.3.0 16.06.2019 Mint 19
4.15.0
gcc (Ubuntu 7.4.0-1ubuntu1~18.04) 7.4.0
yes
all packages can be compiled
2.3.0 16.06.2019 Debian 3.16.0
gcc (Debian 4.9.2-10+deb8u2) 4.9.2
yes all packages can be compiled







2.3.0 20.06.2019
OpenSuse LEAP
42.3
4.4.180
gcc (SUSE Linux) 4.8.5
yes
all packages can be compiled except these ones:

t50

2.3.0
20.06.2019
OpenSUSE LEAP 15.2
4.12.14
gcc (SUSE Linux) 7.4.1 20190424 [gcc-7-branch revision 270538]

gcc (SUSE Linux) 8.2.1 20180831 [gcc-8-branch revision 264010]


yes
all packages can be compiled except these ones:

horst
hostscope
iptraf
justniffer
tracebox

2.3.0
16.06.2019
OpenSuse Tumbleweed
 5.1.7
gcc (SUSE Linux) 9.1.1 20190520 [gcc-9-branch revision 271396]
yes
all packages can be compiled except these ones:

horst
hostscope
iptraf
justniffer
tracebox









2.3.0 16.06.2019 Antergos (ArchLinux)
5.1.9
gcc (GCC) 8.3.0
yes
To compile some of the tools a modification of a source file is neccessary :

#include <sys/sysmacros.h>

must be added to some of the source files (for details why see below)

Details:
atop add the line

#include <sys/sysmacros.h>

to the file

showsys.c

fio add the line

#include <sys/sysmacros.h>

to the file

os/os-linux.h

fnotifystat add the line

#include <sys/sysmacros.h>

to the file

fnotifystat.c



Note [04.01.2019]

These modifications are also neccesary for other Linux distributions with a recent gcc version
so they are done by the compile scripts in JWM / Linux now.

Note [16.06.2019]

fio will not compile anymore due to a change in the GLUSTER api (Note: you can install the fio package from the ArcLinux repositories if required - see also here: https://github.com/axboe/fio/issues/781)











Old Linux Distributions tested (History)

The old versions of JWM / Linux were compiled on these Linux Distributions:

JWM / Linux distributions tested

JWM Version
Date
Linux Distribution
Kernel version gcc version Supported by
add_os_packages?
Comment
2.2.11
22.08.2018
Fedora 27 4.17.14
gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6)
yes
all packages can be compiled; this is the main distribution for development as of 19.08.2018
2.2.11 22.08.2018
RedHat 7 3.10.0
gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
yes
all packages can be compiled except these ones:

t50

2.2.11 22.08.2018
Centos 6.9 2.6.32
gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)

installed per default but not used is:

gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23)

yes
some tools do not compile anymore with standard gcc in Centos 6.9
Therefor I installed and used the GCC v7:

gcc (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)

(see Required GCC version)

all packages can be compiled except these ones:

health-check  (kernel not supported)
fnotifystat  (kernel not supported)

justniffer must be compiled manually -- see here

2.2.11
22.08.2018
Mint 19
4.15.0
gcc (Ubuntu 7.3.0-16ubuntu3) 7.3.0
yes
all packages can be compiled
2.2.11
25.08.2018
OpenSuse LEAP
4.4.104
gcc (SUSE Linux) 4.8.5
yes
all packages can be compiled except these ones:

t50

2.2.11
25.08.2018
Debian
3.16.0
gcc (Debian 4.9.2-10+deb8u1) 4.9.2
yes
all packages can be compiled
2.2.11
25.08.2018
Antergos (ArchLinux)
4.18.4.0
gcc (GCC) 8.2.0

yes
all packages can be compiled except these ones

unfs3

To compile some of the tools a modification of a source file is neccessary :

#include <sys/sysmacros.h>

must be added to some of the source files (for details why see below):

Details:
atop add the line

#include <sys/sysmacros.h>

to the file

showsys.c

fio add the line

#include <sys/sysmacros.h>

to the file

os/os-linux.h

fnotifystat add the line

#include <sys/sysmacros.h>

to the file

fnotifystat.c




2.2.11
29.08.2018
Fedora 23
4.8.13
gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6)
yes
all packages can be compiled
2.2.10 26.12.201 Fedora 27 4.14.7 gcc (GCC) 7.2.1 20170915 (Red Hat 7.2.1-2) yes all packages can be compiled; this is the main distribution for development as of 26.12.2017
2.2.10 26.12.201 RedHat 7 3.10.0 gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16) yes all packages can be compiled except these ones:

t50
2.2.10 26.12.201 entos 6.9 2.6.32 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) yes all packages can be compiled except these ones:

t50

health-check  (kernel not supported)
fnotifystat  (kernel not supported)














2.2.9 08.10.2017 Antergos (Arch Linux based Distribution) 4.13.4 gcc (GCC) 7.2.0 yes all packages can be compiled
2.2.9 08.10.2017 Fedora 25 4.12.14 gcc version 6.4.1 20170727 (Red Hat 6.4.1-1) (GCC) yes all packages can be compiled; this is the main distribution for development as of 14.08.2017
2.2.9 08.10.2017 Fedora 23 4.8.13 gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6) yes all packages can be compiled
2.2.9 08.10.2017 RHEL 7.4 (Maipo) 3.10.0 gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-16) yes all packages can be compiled
2.2.9 08.10.2017 Open Suse LEAP 42 4.4.87
gcc (SUSE Linux) 4.8.5
yes all packages can be compiled
2.2.9 08.10.2017 Ubuntu 16.04.3 LTS 4.10.0 gcc (Ubuntu 5.4.0-06ubuntu1-16.04.4) 5.4.0  20160609 yes all packages can be compiled
2.2.9
08.10.2017
Linux Mint 18
4.4.0
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
yes
all packages can be compiled
2.2.9
08.10.2017
Debian 8.6
3.16.0
gcc (Debian 4.9.2-10) 4.9.2
yes
all packages can be compiled except these ones:

t50

2.2.9 08.10.2017 Centos (RHEL 6.9) 2.6.32 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) yes all packages can be compiled except these ones:

health-check  (kernel not supported)
fnotifystat  (kernel not supported)

t50 can not be compiled with the default gcc from this Linux distribution

2.2.8 07.04.2017 Fedora 25 4.9.13
yes all packages can be compiled; this is the main distribution for development as of 07.04.2017
2.2.8 08.04.2017 Apricity (Arch Linux based Distribution) 4.10.8
yes all packages can be compiled
2.2.8 08.04.2017 Mint 17 3.19.0
yes all packages can be compiled
2.2.8 08.04.2017 OpenSUSE 12 3.7.10
yes all packages can be compiled except these ones:

horst
hostscope (can be compiled with a current gcc version)
  Note:  Use the binaries from the tar files ${JWM_DIR}/hostscope_jwm_rhel6.tar for this distribution

2.2.8
07.04.2017
Centos 6.9
2.6.32

yes
all packages can be compiled except these ones:

health-check  (kernel not supported)
fnotifystat  (kernel not supported)
hostscope (can be compiled with a current gcc version)
  Note:  Use the binaries from the tar files ${JWM_DIR}/hostscope_jwm_rhel6.tar for this distribution
t50 can not be compiled with the default gcc from this Linux distribution




JWM Version
Date
Linux Distribution
Kernelversion Supported by
add_os_packages?
Comment
2.2.7 25.12.2016 Fedora 23 4.8.13 yes all packages can be compiled; this is the main distribution for development as of 27.12.2016
2.2.7 25.12.2016 Fedora 24 4.8.14 yes all packages can be compiled
2.2.7 27.12.2016 Fedora 25 4.8.15 yes all packages can be compiled
2,2.7 25.12.2016 Mint 18 4.4.0 yes all packages can be compiled
2.2.7 26.12.2016 Mint 17 3.19.0 yes all packages can be compiled
2.2.7 27.12.2016 Debian 8 3.16.0 yes all packages can be compiled
2.2.7 25.12.2016 OpenSUSE 42.2 4.4.36 yes all packages can be compiled
2.2.7 26.12.2016 OpenSUSE 12.3 3.7.10 yes all packages can be compiled
2.2.7 26.12.2016 OpenSUSE 13.2 3.16.6 yes all packages can be compiled
2.2.7 25.12.2016 Oracle Linux Server release 6.5 3.8.13 yes
all packages can be compiled except these ones:

health-check  (kernel not supported)
fnotifystat  (kernel not supported)
hostscope (can be compiled with a current gcc version)
  Note:  Use the binaries from the tar files ${JWM_DIR}/hostscope_jwm_rhel6.tar for this distribution

to compile bmon you must download, compile, and install libnl3 and libconfuse (both libraries are not in the repositories for this distribution)

to compile justniffer you must download boost (the boost in the repositories for this distribution is to old)

see also Libraries neccessary for JWM
2.2.7 26.12.2016 Red Hat Enterprise Linux Workstation release 6.8 (Santiago) 2.6.32 yes
all packages can be compiled except these ones:

health-check  (kernel not supported)
fnotifystat  (kernel not supported)
hostscope  (can be compiled with a current gcc version)
  Note:  Use the binaries from the tar files ${JWM_DIR}/hostscope_jwm_rhel6.tar for this distribution

to compile justniffer you must download boost (the boost in the repositories for this distribution is to old)

t50 only compiles with a current version of gcc; the gcc 4.4.7 does not like pragma statements inside of functions

see also Libraries neccessary for JWM
2.2.7 30.12.2016 Fedora 13 2.6.33 yes all packages can be compiled except these ones:

health-check  (kernel not supported)
fnotifystat  (kernel not supported)
hostscope (can be compiled with a current gcc version)
  Note:  Use the binaries from the tar files ${JWM_DIR}/hostscope_jwm_rhel6.tar for this distribution

to compile bmon you must download, compile, and install libnl3  (the libraries are not in the repositories for this distribution)

to compile justniffer you must download boost (the boost in the repositories for this distribution is to old)

to compile oprofile and stress-ng a new version of the binutils is required

t50 only compiles with a current version of gcc; the gcc 4.4.7 does not like pragma statements inside of functions

see also Libraries neccessary for JWM




add_os_packages.sh - Script to add missing packages

The script ./compile_scripts/add_os_packages.sh can be used to install the missing packages to compile all tools in JWM / Linux.

Usage:

[26.12.2016 16:19 ] ### The logfile used is /var/tmp/add_os_packages.sh.log
 add_os_packages.sh

 Function: add missing OS packages for compiling JWM / Linux
 
 Usage:    add_os_packages.sh [-v|--verbose] [-q|--quiet] [-f|--force] [start|stop|restart|status]

 Parameter:
   -v - verbose mode
   -q - quiet mode
   -f - force execution
   -d - dry-run mode, only print the commands to execute

[26.12.2016 16:19 ] ### The logfile used was /var/tmp/add_os_packages.sh.log
[26.12.2016 16:19 ] ### add_os_packages.sh ended at Mon Dec 26 16:19:17 CET 2016
[xtrnaw7@t540p /var/tmp/jwm/compile_scripts]$

add_os_packages.sh supports Linux distributions based on

add_os_packages.sh installs all packages that are missing after an initial installation of the distribution without any modifications.

The script asks the user for confirmation before starting the installation.

Note:

The scripts assumes a "standard installation" of the Linux distribution  - so there may be other libraries or tools missing to compile all tools . Just check the error messages of the compile script and install the missing packages manually.



Using JWM / Linux




Creating a JWM / Linux session


To use JWM / Linux change to the directory with JWM / Linux (the JWM_DIR) and issue

. ./init_session

e.g.

screenshot:


[xtrnaw7@t540p /var/tmp/jwm]$ . ./init_session
JWM_DIR is "/var/tmp/jwm"
[bash] Starting environment for "JMW / Linux v230 created on Fedora_release_30_(Thirty) 5.1.8-300.fc30.x86_64 x86_64 at 2019-06-15" ...
[bash] Initializing JWM environment ...
[bash] JWM environment initialized. The tools available are:
/var/tmp/jwm/bin:
atop          cpulimit    fspy        htop          jnettop      ndiff      nping                opjitconv    procdump     screen     tiptop
atopsar       cpumon      get_device  httpry        justniffer   netio      ocount               opreport     protoc       smemstat   total
bmon          cpustat     get_driver  ibmonitor     kernelshark  netperf    opannotate           pcap-config  prtdiag      sockperf   tracebox
bmore         dlist_test  get_module  ifstat        luatracebox  netserver  oparchive            perf.data    prtdiag.cfg  stress     trace-cmd
bon_csv2html  dropwatch   hdperf      inotifywait   makelocal    nicstat    op-check-perfevents  pidstat      ptiptop      stress-ng  trace-graph
bon_csv2txt   dstat       horst       inotifywatch  memstat      nload      operf                pigz         rawtime      sysstat    trace-view
bvi           fiemap      horst.sh    iostat        mpstat       nmap       opgprof              pit_server   rnano        systool    uninstall_ndiff
bwm-ng        fileop      hostscope   iozone        nano         nmon       ophelp               plugins      rvnamed      tapestat   unpigz
cifsiostat    fio         hping2      iperf3        ncat         nmonchart  opimport             powerstat    rvnamed-ng   tcptrack   wavemon

/var/tmp/jwm/bin1:
CPUhog.jar  CPUhog.sh  inq  inq.LinuxAMD64  inq.LinuxI386  iodump  iperf  ipscan  ipscan-linux64-3.5.3.jar  latencytop-gui  latencytop-tui  lxtx  pchar_for_linux26

/var/tmp/jwm/sbin:
arping        cdpr       flowtop       ifpps    ipstrings   mausezahn    numatop     pcm-memory.x  pcm-sensor.x  scanlogd  tcpdump.4.9.2         zcav
astraceroute  darkstat   fnotifystat   iftop    iptraf      mtr          p0f         pcm-msr.x     pcm-tsx.x     smartctl  trafgen
biosdecode    dhtest     forkstat      ioping   iptraf-ng   nethogs      pagemon     pcm-numa.x    pcm.x         smartd    unfsd
bonnie++      dmidecode  health-check  ipaudit  iptstate    netsniff-ng  pchar       pcm-pcie.x    pktstat       t50       update-smart-drivedb
bpfc          eventstat  hostscoped    ipband   latencytop  ngrep        pcm-core.x  pcm-power.x   powertop      tcpdump   vpddecode

/var/tmp/jwm/scripts:
ack                              collectl_top                      fio_tests.sh            iotop.sh              memconf                pt-diskstats
ack-2.24-single-file.pl          collectl_top_io                   hostscope.sh            itop                  memconf-v3.14.pl       rtp_task_desc.inp
ack-v3.0.0                       compile_justniffer_with_boost.sh  init_infoscale.include  jdiskreport           mle.sh                 show_kernel_memory
addtimestamp                     darkstat.sh                       init_infoscale.sh       jwm.sh                monitor_system.sh      smem
busybox_httpd.sh                 exec_snoop_wrapper.sh             inxi                    manServer_107_ma1.pl  mpathstat.py           start_stop_manserver.sh
cdpinfo                          execute_on_all_hosts.sh           iotop                   manServer_107.pl      netdata.sh             tecmint_monitor.sh
check_network_adapter_config.sh  execute_scripts.sh                iotop-0.4.4             manServer.pl          opensnoop1             testssl.sh
collectl                         execute_tasks.sh                  iotop-0.6               manServer.pl.org      open_snoop_wrapper.sh  view_vmware_server_configuration.sh

Additional general Linux tools are in the directory /var/tmp/jwm/busybox

Additional performance monitoring tools are in /var/tmp/jwm/perf-tools

[JWM session] [xtrnaw7@t540p /var/tmp/jwm]$







Notes:

init_session sets the environment variables PATH, LD_LIBRARY_PATH, MANPATH, PYTHONPATH, and PS1 and defines some useful aliase.

If you want to use the already installed Linux libraries instead of the libraries contained in JWM / Linux set and export the variable LD_LIBRARY_PATH before calling init_session.




Using JWM commands via jwm.sh


To only execute a single command from the JWM / Linux environment without starting a JWM / Linux shell you can use the script ${JWM_DIR}/scripts/jwm.sh.

jwm.sh creates a temporary JWM / Linux environment and then executes the command. The usage for jwm.sh is

[xtrnaw7@t540p /var/tmp/jwm]$ /var/tmp/jwm/scripts/jwm.sh -h
[18.11.2016 17:08 ] ### jwm.sh started at Fri Nov 18 17:08:53 CET 2016
[18.11.2016 17:08 ] ### The logfile used is /var/tmp/jwm.sh.log
 jwm.sh

 Function: exeucte a command in a temporary JWM / Linux environment
 
 Usage:    jwm.sh [-v|--verbose] [-q|--quiet] [-f|--force] command [parameter]

 Parameter:
   -v - verbose mode
   -q - quiet mode
   -f - force execution

[18.11.2016 17:08 ] ### The logfile used was /var/tmp/jwm.sh.log
[18.11.2016 17:08 ] ### jwm.sh ended at Fri Nov 18 17:08:53 CET 2016
[xtrnaw7@t540p /var/tmp/jwm]$


You should create a symbolic link for jwm.sh in a directory that is in your standard path, e.g

sudo ln -s /var/tmp/jwm/scripts/jwm.sh /usr/bin/jwm

Then you can use jwm.sh like this

[xtrnaw7@t540p /var/tmp/jwm]$  jwm  nicstat 1 1
[18.11.2016 17:29 ] ### jwm started at Fri Nov 18 17:29:22 CET 2016
[18.11.2016 17:29 ] ### The logfile used is /var/tmp/jwm.log
    Time      Int   rKB/s   wKB/s   rPk/s   wPk/s    rAvs    wAvs %Util    Sat
17:29:22  enp0s25   12.01    3.06    9.34    6.22  1316.5   504.8  0.01   0.00
17:29:22       lo    2.74    2.74   41.09   41.09   68.30   68.30  0.00   0.00
17:29:22   wlp4s0    0.00    0.00    0.00    0.00   112.5   145.5  0.00   0.00
[18.11.2016 17:29 ] ### The logfile used was /var/tmp/jwm.log
[18.11.2016 17:29 ] ### jwm ended at Fri Nov 18 17:29:22 CET 2016
[xtrnaw7@t540p /var/tmp/jwm]$


Note: Use jwm.sh with the parameter -v to get some more information for trouble shooting:
[xtrnaw7@t540p /var/tmp/jwm]$ jwm -v nicstat 1 1
[29.08.2018 13:32 ] ### jwm started at Wed Aug 29 13:32:54 CEST 2018
[29.08.2018 13:32 ] ### The logfile used is /var/tmp/jwm.log
[29.08.2018 13:32 ] INFO: SCRIPTNAME is "jwm"
[29.08.2018 13:32 ] INFO: SCRIPTDIR is "/tools/scripts"
[29.08.2018 13:32 ] INFO: REAL_SCRIPTNAME is "/data/tools/scripts/jwm"
[29.08.2018 13:32 ] INFO: REAL_SCRIPTDIR is "/data/tools/scripts"
[29.08.2018 13:32 ] INFO: WORKING_DIR is "/var/tmp/jwm"
[29.08.2018 13:32 ] INFO: RUNNING_IN_TERMINAL_SESSION is "0" (0 = yes, 1 = no)
[29.08.2018 13:32 ] INFO: CUR_USER_ID is "1000"
[29.08.2018 13:32 ] INFO: CUR_USER_NAME is "xtrnaw7"
[29.08.2018 13:32 ] INFO: CUR_GROUP_ID is "1000"
[29.08.2018 13:32 ] INFO: CUR_GROUP_NAME is "xtrnaw7"
[29.08.2018 13:32 ] INFO: ACTION is ""
[29.08.2018 13:32 ] INFO: Using the JWM directory "/var/tmp/jwm"
[29.08.2018 13:32 ] INFO: PATH is now /var/tmp/jwm/scripts:/var/tmp/jwm/bin:/var/tmp/jwm/sbin:/tools/scripts:/tools/bin:/tools/sbin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/home/xtrnaw7/.local/bin:/home/xtrnaw7/bin
[29.08.2018 13:32 ] INFO: LD_LIBRARY_PATH is now /var/tmp/jwm/lib:
[29.08.2018 13:32 ] INFO: MANPATH is now /var/tmp/jwm/man:/tools/man:
[29.08.2018 13:32 ] INFO: PYTHONPATH is now :/var/tmp/jwm/lib/python2.7/site-packages/
[29.08.2018 13:32 ] INFO: Executing now "nicstat 1 1" ...
    Time      Int   rKB/s   wKB/s   rPk/s   wPk/s    rAvs    wAvs %Util    Sat
13:32:54  enp0s25   20.84    8.38   18.73   12.40  1139.6   692.1  0.02   0.00
13:32:54       lo    0.00    0.00    0.01    0.01   77.56   77.56  0.00   0.00
[29.08.2018 13:32 ] ### The logfile used was /var/tmp/jwm.log
[29.08.2018 13:32 ] ### jwm ended at Wed Aug 29 13:32:54 CEST 2018
[xtrnaw7@t540p /var/tmp/jwm]$

 




JWM / Linux manServer


JWM / Linux contains a simple manServer that can be used to view the man pages in a web browser that listens on localhost:7777.

To use it start a JWM / Linux session

cd /var/tmp/jwm

. init_session


and start the JWM / Linux manServer

[JWM session] [xtrnaw7@t540p /var/tmp/jwm/scripts]$ start_stop_manserver.sh start
[04.11.2016 21:01 ] ### start_stop_manserver.sh started at Fri Nov  4 21:01:38 CET 2016
[04.11.2016 21:01 ] ### The logfile used is /var/tmp/start_stop_manserver.sh.log
[04.11.2016 21:01 ] Starting the JWM manserver ...
export MANPATH="/var/tmp/jwm/share/man:/tools/man::/var/tmp/jwm/perf-tools/man"   
nohup /var/tmp/jwm/scripts/manServer.pl -s 7777 >/tmp/manserver_jwm.log 2>&1 &
echo $! >"/tmp/manserver_jwm.pid"
[04.11.2016 21:01 ] Use "http://localhost:7777" or "http://t540p:7777" to connect to the JWM manserver
[04.11.2016 21:01 ] ### The logfile used was /var/tmp/start_stop_manserver.sh.log
[04.11.2016 21:01 ] ### start_stop_manserver.sh ended at Fri Nov  4 21:01:38 CET 2016
[JWM session] [xtrnaw7@t540p /var/tmp/jwm/scripts]$



Now you can use your Web Browser to view the man pages, e.g.




To add other man pages to the JWM / Linux manServer and use another port use :

MANPATH=$MANPATH:/usr/share/man ./start_stop_manserver.sh --port 7788 start

example
[JWM session] [xtrnaw7@t540p /var/tmp/jwm]$ MANPATH=$MANPATH:/usr/share/man start_stop_manserver.sh --port 7788 start
[12.01.2019 17:23 ] ### start_stop_manserver.sh started at Sat Jan 12 17:23:05 CET 2019
[12.01.2019 17:23 ] ### The logfile used is /var/tmp/start_stop_manserver.sh.log
[12.01.2019 17:23 ] Starting the JWM manserver ...
export MANPATH="/var/tmp/jwm/share/man:/tools/man::/var/tmp/jwm/perf-tools/man:/usr/share/man"   
nohup /var/tmp/jwm/scripts/manServer.pl -s 7788 >/tmp/jwm_manserver_7788.log 2>&1 &
echo $! >"/tmp/jwm_manserver_7788.pid"
[12.01.2019 17:23 ] Use "http://localhost:7788" or "http://t540p:7788" to connect to the JWM manserver
[12.01.2019 17:23 ] ### The logfile used was /var/tmp/start_stop_manserver.sh.log
[12.01.2019 17:23 ] ### start_stop_manserver.sh ended at Sat Jan 12 17:23:05 CET 2019
[JWM session] [xtrnaw7@t540p /var/tmp/jwm]$






start_stop_manserver.sh details
Use the parameter status to check the status of the JWM manserver:

[JWM session] [xtrnaw7@t540p /var/tmp/jwm/scripts]$ start_stop_manserver.sh status
[04.11.2016 21:02 ] ### start_stop_manserver.sh started at Fri Nov  4 21:02:11 CET 2016
[04.11.2016 21:02 ] ### The logfile used is /var/tmp/start_stop_manserver.sh.log
[04.11.2016 21:02 ] The JWM manserver already running; the PID is 8378
[04.11.2016 21:02 ] The JWM manserver is listening to the ports:
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 0.0.0.0:7777            0.0.0.0:*               LISTEN      8378/perl          
[04.11.2016 21:02 ] ### The logfile used was /var/tmp/start_stop_manserver.sh.log
[04.11.2016 21:02 ] ### start_stop_manserver.sh ended at Fri Nov  4 21:02:11 CET 2016
[JWM session] [xtrnaw7@t540p /var/tmp/jwm/scripts]$



[JWM session] [xtrnaw7@t540p /var/tmp/jwm/scripts]$ start_stop_manserver.sh stop
[04.11.2016 21:04 ] ### start_stop_manserver.sh started at Fri Nov  4 21:04:37 CET 2016
[04.11.2016 21:04 ] ### The logfile used is /var/tmp/start_stop_manserver.sh.log
[04.11.2016 21:04 ] The JWM manserver is not running
[04.11.2016 21:04 ] ### The logfile used was /var/tmp/start_stop_manserver.sh.log
[04.11.2016 21:04 ] ### start_stop_manserver.sh ended at Fri Nov  4 21:04:37 CET 2016
[JWM session] [xtrnaw7@t540p /var/tmp/jwm/scripts]$


The usage for the JWM mansever start/stop script is:

[JWM session] [xtrnaw7@t540p /var/tmp/jwm/scripts]$ start_stop_manserver.sh --help
[04.11.2016 21:13 ] ### start_stop_manserver.sh started at Fri Nov  4 21:13:13 CET 2016
[04.11.2016 21:13 ] ### The logfile used is /var/tmp/start_stop_manserver.sh.log
 start_stop_manserver.sh

 Function: start or stop the manserver in JWM / Linux
 
 Usage:    start_stop_manserver.sh [-v|--verbose] [-q|--quiet] [-f|--force] [-p|--port port] [start|stop|restart|status]

 Parameter:
   -v - verbose mode
   -q - quiet mode
   -f - force execution
   -p - port for the JWM manserver, the default port is 7777

[04.11.2016 21:13 ] ### The logfile used was /var/tmp/start_stop_manserver.sh.log
[04.11.2016 21:13 ] ### start_stop_manserver.sh ended at Fri Nov  4 21:13:13 CET 2016
[JWM session] [xtrnaw7@t540p /var/tmp/jwm/scripts]$





Directories in the JWM / Linux archive


Directory name
Directory contents
Comment
bin
binaries w/ source

bin1
binaries w/o source

busybox
busybox binary and links

checksec*
checksec
collectl*
collectl

compile_scripts
scripts for compiling the tools

data/*
data files for the tools

develop
script templates and other development files

etc
config files for the tools

htdocs
webserver based applications and html pages

include
include files

info
.info files

iotop*
iotop (various versions)

jdiskreport* JDiskReport
lib
libraries for the tools

lib64
symbolic link to lib

libexec add files for the tools
lynis*
lynis installation

man
symbolic link to ./share/man

netdata*
netdata installation

new
staging directory for new tools

other_os
this directory contains some binaries for other OS

perf-tools
directory with the perftools

sbin
binaries that need root privileges

scripts
well, scripts

share
docs, manpages, etc for the tools

src
source for the tools

testssl* directories with the tool testssl.sh
workdir
data files and log files for compiling the tools




Libraries included in JWM / Linux

Because not all libraries necessary for the tools are installed on all machines (especially on machines in production environments) some libraries are included in JWM / Linux. The libraries are in the directory ${JWM_DIR}/lib.

As of this writing these libraries are:

Library
Version
Description
Source
libpcap
1.0.0
necessary library for some of the network related tools

Note:

Version 1.0.0 is used here because newer versions do not compile out-of-the box on some Linux Releases.
http://www.tcpdump.org/
libprotobuf
libprotobuf-lite
libprotoc
9.0.1
used by hostscope











The make script for JWM / Linux also copies some OS libraries necessary for some of the tools to the JWM / Linux lib directory. The libraries to copy are listed in the file

${JWM_DIR}/compile_scripts/OS_LIBS

The libraries to copy are [as of 22.08.2018]; (you may change the file if the libraries are already on the target machines for JWM / Linux ]:

Library
Comment
libcrypto.so.10*

libdb-4.7*

libdb-5.3*

libtinfo*
libnuma*
liblua*
liblua5.2*

liblua-5.1*

liblua-5.2*

liblua-5.3*

libnet*

libnl*
libconfuse*

libnl-3*
libnl-route-3*
libnl-genl-3*
libsctp1*
libnetfilter_conntrack*
libboost_regex*
libboost_regex-mt*
libboost_iostreams*
libboost_program_options*
libboost_program_options-mt*
liburcu*
libcli*
libnet*
libsctp1*









see How to add another OS Library for details for the copy process

Note:

To suppress the copying of the libraries create the semaphor file ${JWM_DIR}/lib/_DO_NOT_COPY_LIBS_ prior to calling the make script (or remove the file ${JWM_DIR}/compile_scripts/OS_LIBS).

 


Tools included in JWM / Linux


Tool
Version
Last
Update
Interface
Description
Source
ack
3.0
2019
cli
ack is a grep replacement written in Perl

"Ack is highly portable, and will run on any machine running Perl 5.8.8 or higher."



http://beyondgrep.com/
arping
2.19
2017
cli
Arping is a util to find out if a specific IP address on the LAN is 'taken'
and what MAC address owns it.
http://github.com/ThomasHabets/arping

atop
2.4.0
2019
cli
top like system monitor

Hints for compiling atop
If you get the error message

cc -O2 -I. -Wall          -c -o photosyst.o photosyst.c
photosyst.c: In function ‘lvmmapname’:
photosyst.c:1466:19: error: called object ‘major’ is not a function or function pointer
     dmp->major  = major(statbuf.st_rdev);
                   ^~~~~
photosyst.c:1421:25: note: declared here
 lvmmapname(unsigned int major, unsigned int minor,
            ~~~~~~~~~~~~~^~~~~

add the line

#include <sys/sysmacros.h>

to the file

photosyst.c



http://www.atoptool.nl/
bvi
1.4rc0
2014
cli
bvi is a vi like editor for binary files
http://bvi.sourceforge.net/
bmon 4.0 2017 cli "bmon is a monitoring and debugging tool to capture networking related statistics and prepare them visually in a human friendly way. It features various output methods including an interactive curses user interface and a programmable text output for scripting."

https://github.com/tgraf/bmon/
bonnie++
1.04 UNRELEASED
2017
cli
bonnie++ is a disk performance test tool
http://www.coker.com.au/bonnie++/
busybox
1.30.1
2019
cli BusyBox: The Swiss Army Knife of Embedded Linux

Note:

The config for busybox in the tar archive is the default config without the i2c tools. The make script does not create a new config.

Use

make menuconfig

in the busybox source directory to select which busybox components should be compiled.


Hints for some of the errors I got while compiling busybox on older Linux versions
Error message ‘MTD_FILE_MODE_RAW’ undeclared


-> disable "Miscellaenous Utilities/nandwrite" and "Miscellaenous Utilities/nanddump"

Error message  ‘BLKSECDISCARD’ undeclared


-> disable "Linux System Utilities/blkdiscard"

Error message  sync.c:(.text.sync_main+0x7a): undefined reference to `syncfs'


-> disable "Core Utils/sync"

Error message  nsenter.c:(.text.nsenter_main+0x1b6): undefined reference to `setns'


-> disable "Linux System Utilties/nsenter"

The config without these tools is saved in the file rhel6_config in the busybox source directroy. Use "make menuconfig" in the busybox source directory to use this config.


http://www.busybox.net
bwm-ng
0.6.2
2018
cli Bandwidth Monitor NG is a small and console-based live network and disk-io bandwidth monitor
http://sourceforge.net/projects/bwmng/
cdpinfo
1.2
2012
cli A perl script to  listen for Cisco Discovery Protocol (CDP) packets and print out key values such as switch, port and vlan
https://github.com/wildajet/scripts/blob/master/cdpinfo
cdpr
2.4
2013
cli Cisco Discovery Protocol Reporter - view and decode CDP packets
http://sourceforge.net/projects/cdpr/
checksec 1.10.0
2018
cli
Test security features implemented in the running Linux version
https://github.com/slimm609/checksec.sh
http://www.trapkit.de/tools/checksec.html
collectl
4.3.1
2018
cli powerfull performance monitoring tool
http://collectl.sourceforge.net/

CPUHog
2.1.3
2018
GUI
Java app. to utilise (waste) as much CPU as it can. The application allows the user to create multiple Java threads, each performing a CPU-intensive task tasks with a selectable memory footprint. The options let you 'thrash' the aspect you want. https://sourceforge.net/projects/cpuhog/
cpumon
2.4
2014
cli monitor cpu usage
http://www.softndesign.org/codes/c/cpumon/
cpustat
0.01.28
2015
cli cpustat periodically dumps out the current CPU utilisation statistics of running processes. cpustat has been optimised to have a minimal CPU overhead and typically uses about 35% of the CPU compared to top. cpustat also includes some simple statistical analysis options that can help characterise the way CPUs are being loaded. https://github.com/ColinIanKing/cpustat
darkstat
3.0.719
2015
WebGUI
Captures network traffic, calculates statistics about usage, and serves reports over HTTP.

Use darkstat.sh to start or stop the darkstat daemon.

http://unix4lyfe.org/darkstat/

dhtest
1.5
2018
cli DHCP Test client
https://github.com/saravana815/dhtest
dmidecode
3.2
2019
cli dmidecode reports information about your system's hardware as described in your system BIOS according to the SMBIOS/DMI standard (see a sample output) http://www.nongnu.org/dmidecode/
dstat
0.7.3
2017
cli Resource statistic tool
http://dag.wieers.com/home-made/dstat/
dropwatch
1.5
2009
cli
list dropped network packages
https://github.com/nhorman/dropwatch
eventstat 0.04.05
2018
cli eventstat periodically dumps out the current kernel event state. It keeps track of current events and outputs the change in events on each output update. The tool requires sudo to run since it needs to write to /proc/timer_stats to start and stop the event monitoring. http://kernel.ubuntu.com/~cking/eventstat/
fiemap

2017
cli
Reads file extents using the FIEMAP ioctl
https://github.com/ColinIanKing/fiemap
fio
2.1.10
2017
cli benchmark tool

Hints for Compiling fio
If you get these errors

/usr/bin/ld: diskutil.o: in function `get_device_numbers':
/var/tmp/jwm/src/fio-2.1.10/diskutil.c:184: undefined reference to `major'
/usr/bin/ld: /var/tmp/jwm/src/fio-2.1.10/diskutil.c:185: undefined reference to `minor'
/usr/bin/ld: /var/tmp/jwm/src/fio-2.1.10/diskutil.c:163: undefined reference to `major'

add

#include <sys/sysmacros.h>

to the file 

os/os-linux.h




http://freecode.com/projects/fio
fnotifystat 0.01.17
2017
cli Fnotifystat is a program that dumps the file system activity in a given period of time.

Note:
The make script aborts the compilation if the file fanotify.h is not found in /usr/include or one of the sub directories. This file is missing in older Linux versions

Hints for compiling fnotifystat
If you get the error message

cc -Wall -Wextra -DVERSION='"0.01.17"' -O2 fnotifystat.o -o fnotifystat
/usr/bin/ld: fnotifystat.o: in function `fnotify_get_filename.constprop.3':
fnotifystat.c:(.text+0xbad): undefined reference to `minor'

add the line

#include <sys/sysmacros.h>

to the file

fnotifystat.c


http://kernel.ubuntu.com/~cking/fnotifystat/
forkstat
0.02.09
2018
cli forkstat is a program that logs process fork(), exec() and exit() activity. It is useful for monitoring system behaviour and to track down rogue processes that are spawning off processes and potentially abusing the system.

Note that forkstat uses the Linux netlink connector to gather process activity and this may miss events if the system is overly busy. Netlink connector also requires root privilege.
http://kernel.ubuntu.com/~cking/forkstat/
fspy
0.1.1



2010
cli fspy is an easy to use linux filesystem activity monitoring tool which is meant to be small, fast and to handle system resources conservative. you can apply filters, use diffing and your own output format in order to get the best results.
https://www.security-database.com/toolswatch/fspy-v0-1-1-linux-filesystem.html
hdperf
1.5
2013
cli harddrive performance benchmark
http://sourceforge.net/projects/hdperf/
health-check 0.03.01
2019
cli The health-check tool monitors processes in various ways to help identify areas where it is consuming too many resources. One can trace one or more processes (including all their threads and child processes too) for a full story of system activity.

Note:
The make script aborts the compilation if the file fanotify.h is not found in /usr/include or one of the sub directories. This file is missing in older Linux versions


http://kernel.ubuntu.com/~cking/health-check/
horst 5.1 2016 cli horst is a small, lightweight IEEE802.11 WLAN analyzer with a text interface. https://github.com/br101/horst
hostscope
4.0
2018
cli, network
hostscope displays key system metrics of Linux hosts, such as detailed CPU load, speed and temperature, I/O rates of network interfaces, I/O rates of disks, and user process summary information. All metrics are multicast on the LAN, if wanted, and clients can switch between multiple hosts on the network.

http://www.maier-komor.de/hostscope.html
hping2
2.0.0
2006
cli command-line oriented TCP/IP packet assembler/analyzer https://sourceforge.net/projects/hping2/
htop
2.2.0
2018
cli top like system monitor
http://htop.sourceforge.net/
httpry
0.1.8.0
2014
cli
a HTTP logging and information retrieval tool

https://github.com/jbittel/httpry#
ibmonitor
1.4
2006
cli ibmonitor is an interactive linux console application which shows
bandwidth consumed and total data transferred on all interfaces.
http://ibmonitor.sourceforge.net/index.html
ifstat
1.1
2004
cli report network interface bandwidth
http://gael.roualland.free.fr/ifstat/
iftop
1.0pre4
2014
cli report network interface bandwidth http://www.ex-parrot.com/pdw/iftop/
inxi
3.0.33
2018
cli inxi - the universal, portable, system information tool for console and irc.

http://smxi.org/

inotify
3.14
2010
cli

inotify-tools is a C library and a set of command-line programs for Linux providing a simple interface to inotify. These programs can be used to monitor and act upon filesystem events. ...
The programs are written in C and have no dependencies other than a Linux kernel supporting inotify.

inotify-tools 3.14 is the latest version, released on the 7th of March 2010.

inotify is a file change notification system in the Linux kernel, available since version 2.6.13

To check whether your own kernel version supports Inotify as well, you can run the following command:

% grep INOTIFY_USER /boot/config-$(uname -r)
CONFIG_INOTIFY_USER=y
https://github.com/rvoicilas/inotify-tools/wiki

see also:

http://www.infoq.com/articles/inotify-linux-file-system-event-monitoring
inq
9.0.0.0

2018
cli disk inquiry program from EMC,

Notes:
The archive contains only the 32bit binary and the 64bit binary
The 32 Bit Version is still 7.6.2.0


ftp://ftp.emc.com/pub/symm3000/inquiry/

(no source code available)
Intel PCM
2.11
2012
cli Intel Performance Counter Monitor
https://software.intel.com/en-us/articles/intel-performance-counter-monitor/
ioping
1.1
2018
cli
simple disk I/0 latency measuring tool
https://github.com/koct9i/ioping
iotop
0.4.4
2013
cli iotop is a Python program with a top like UI used to show of behalf of which process is the I/O going on.

Be aware of the requirements for this iotop version:

"It requires Python >= 2.5 (or Python >= 2.4 with the ctypes module) and a Linux kernel >= 2.6.20 with the CONFIG_TASK_DELAY_ACCT CONFIG_TASKSTATS,
CONFIG_TASK_IO_ACCOUNTING and CONFIG_VM_EVENT_COUNTERS options on."

http://guichaz.free.fr/iotop/
iotop
0.6
2013
cli iotop is a Python program with a top like UI used to show of behalf of which process is the I/O going on.

Be aware of the requirements for this iotop version:

"It requires Python >= 2.7 and a Linux kernel >= 2.6.20 with the CONFIG_TASK_DELAY_ACCT CONFIG_TASKSTATS,
CONFIG_TASK_IO_ACCOUNTING and CONFIG_VM_EVENT_COUNTERS options on."

http://guichaz.free.fr/iotop/
iozone
3_487
2019
cli IOzone is a filesystem benchmark tool
http://iozone.org/
ipaudit
1.1
2016
cli IPAudit monitors network activity on a network by host, protocol and port. http://ipaudit.sourceforge.net/
ipband
0.8.1
2008
cli ipband is a pcap based IP traffic monitor. http://ipband.sourceforge.net/
iperf3 3.6.0 2018 cli iperf3 is a tool for active measurements of the maximum achievable bandwidth on IP networks. It supports tuning of various parameters related to timing, protocols, and buffers. For each test it r./src/bwm-ngeports the bandwidth, loss, and other parameters.

Note: A binary version of the "old" iperf 2.0.5 that should run on most Linux distributions is in the directory ./bin1.

https://github.com/esnet/iperf
ipscan
3.5.3
2018
GUI
A cross-platform network scanner written in Java that is fast and simple to use
https://sourceforge.net/projects/ipscan/
iptraf
3.0.0
2005
cli IPTraf is a console-based network statistics utility for Linux. http://iptraf.seul.org/index.html
iptraf-ng 1.1.4 2013 cli iptraf-ng is a fork of the original iptraf-3.0.0
https://fedorahosted.org/iptraf-ng/

18.08.2018: The URL does not work anymore
iptstate 2.2.6 2016 cli IPTState is a top-like interface to your netfilter connection-tracking table. https://www.phildev.net/iptstate/index.shtml
itop

2012
cli itop monitors the interrupts of a system in real time
http://www.void.gr/kargig/blog/2012/06/24/interrupts-top-like-utility-for-linux/
JDiskReport 1.4.1 .2014 GUI "JDiskReport enables you to understand how much space the files and directories consume on your disk drives, and it helps you find obsolete files and folders.

The tool analyses your disk drives and collects several statistics which you can view as overview charts and details tables."

http://www.jgoodies.com/freeware/jdiskreport/
jnettop
0.13
2006
cli jnettop is a traffic visualiser, which captures traffic going through the host it is running from and displays streams sorted by bandwidth they use.
https://sourceforge.net/projects/jnettop/files/
justniffer 0.5.14 2014 cli justniffer is a network sniffer

Hints for compiling justniffer
Compiling justniffer requires boost  version 1.46 or newer.
If you do not want to install a new boost version you can compile justniffer using a boost staging area.

See also the hint below regarding GENL_ID_GENERATE.

http://justniffer.sourceforge.net/

https://sourceforge.net/projects/justniffer/?source=directory
latencytop
0.5
2009
cli view process latencieshttpry http://www.latencytop.org

[27.10.2013/bs: The web site is not available]

http://git.infradead.org/latencytop.git
lxtx 1.1.12 2016 cli A simple command line program, like vmstat, sar, or iostat.

Note:
There is no source code available for this tool.

http://ontune.us/lxtx/
lynis
2.7.0
2018
cli Lynis is a security auditing tool for UNIX derivatives like Linux, macOS, BSD, Solaris, AIX, and others https://cisofy.com/download/lynis/
manServer
1.07 ma1
2006
WebGUI
manServer - convert manual pages to HTML for viewing with a web browser. Use start_stop_manserver.sh to start or stop the JWM / Linux manServer.

A copy of this page is in the directory ./htdocs.

Note: the download link on that page does NOT work.

Working Download Link (05.11.2016)

http://www.filewatcher.com/m/manServer.pl.63487-0.html
memconf
3.14
2017
cli Identify sizes of memory modules installed on a Solaris, Linux, FreeBSD or HP-UX workstation or server. http://sourceforge.net/projects/memconf/
memstat 1.0.0 ? cli memstat is a small proc-based utility designed to help the system administrator figure out what's consuming memory https://www.gnu.org/software/hurd/user/tlecarrour/memstat.html
monitor_system.sh
1.0.0
2018
cli
monitor_system.sh is a simple wrapper script to start one or more monitor commands -- for further details see the documentation for the script.
http://bnsmb.de/Linux/monitor_system.html
mpathstat.py 1.1 2016 cli Python script to sort the output of iostat for better monitoring of mulitpathed devices http://prefetch.net/blog/index.php/2016/10/24/displaying-multi-path-statistics-on-linux-systems/
mtr
0.92
2017
cli mtr combines the functionality of the 'traceroute' and 'ping' programs in a single network diagnostic tool. http://www.bitwizard.nl/mtr/
nano
3.1
2018
cli
nano is a small visual editor
https://nano-editor.org/
netdata
1.14.0
2019
WebGUI real time performance monitoring in a Web dashboard

Very short netdata usage help
To use netdata start the daemon:

/var/tmp/jwm/netdata/usr/bin/netdata

(or use the script netdata.sh)

Then connect via WebBrowser to

http://127.0.0.1:19999/

To view the config use:

http://127.0.0.1:19999/netdata.conf

Note:

Please make sure that all files in ${JWM_DIR}/netdata are owned by the user executing netdata  


https://github.com/firehol/netdata
nethogs
0.8.0
2015
cli
NetHogs is a small 'net top' tool https://github.com/raboof/nethogs#readme
netio
1.32
2012
cli
network benchmark tool

Note:

For 10 GB connections netio is not that useful; use one of the other tools (for example netperf instead)

https://web.ars.de/netio/






netperf
2.7.0
2015
cli network benchmark tool
http://www.netperf.org/netperf
netsniff-ng 0.6.4 2018 cli netsniff-ng is a free Linux networking toolkit, a Swiss army knife for your daily Linux network plumbing if you will.

Hints for compiling netsniff-ng
netsniff-ng does not compile with the libraries and headers from JWM / Linux . You must install the latest version of the libraries to compile netsniff-ng.

Compiling the version 0.6.3 will fail with the error message

staging/tools.c:485:10: error: ‘UINT64_MAX’ undeclared (first use in this function)

To fix this issue add the missing statement

#include <stdint.h>

to the file

./staging/tools.c


http://netsniff-ng.org/
ngrep
1.47
2006
cli grep for network traffic
http://ngrep.sourceforge.net/
nicstat
1.95
2014
cli Network traffic statics utility for Solaris and Linux http://sourceforge.net/projects/nicstat/
nload
0.7.4
2012
cli nload is a console application which monitors network traffic and bandwidth usage in real time. It visualizes the in- and outgoing traffic using two graphs and provides additional info like total amount of transfered data and min/max network usage.
http://www.roland-riegel.de/nload/index.html
nmap
7.70
2017
cli network port scanner
http://nmap.org/
nmon
16j
2019
cli Nigel's performance Monitor for Linux

Hints for compiling nmon

The source code for nmon version 15+ uses functions from the math library but does not link the necessary lib for that, so compiling nmon ends with an error:

cc -o nmon_power_rhel3 lmon.c -g -O2 -D JFS -D GETUSER -Wall -D LARGEMEM -lncurses -g -D POWER
/tmp/ccLo9PxQ.o: In function `main':
/var/tmp/jwm/src/nmon/lmon.c:5616: undefined reference to `ceilf'
collect2: error: ld returned 1 exit status

To get around this edit the makefile and change the line

LDFLAGS=-lncurses -g

to

LDFLAGS=-lncurses -g -lm



Included are also nmonchart:

"nmonchart is a Korn shell script for AIX or Linux to convert nmon collected files to a webpage (.html) including Google Chart and JavaScript to display the top 53+ AIX and Linux Performance Graphs and configuration details"

and the Java Nmon Analyzer

"The Java Nmon analyser is a free, open source analyser tool which is helpful in analyzing performance data captured using the nmon performance tool. and real-time monitor the Linux/AIX 's performance base on nmon."

There are other tools to use the data collected by nmon - see the list of other tools not included in JWM / Linux below.

nmon and nmon chart:

http://nmon.sourceforge.net/pmwiki.php

nmon Analyzer:

https://sourceforge.net/projects/jnmonanalyser/
numatop
1.0.4
2015
cli NumaTOP is an observation tool for runtime memory locality characterization and analysis of processes and threads running on a NUMA system. It helps the user characterize the NUMA behavior of processes and threads and identify where the NUMA-related performance bottlenecks reside.

NumaTOP Requirements
NumaTOP requires a special patch which supports PEBS Load Latency functionality.

The patch has not been integrated in 3.8 (probably it will be integrated
in 3.9).

Numatop is supported on Intel Xeon processors: 5500-series, 6500/7500-series,
5600 series, E7-x8xx-series, and E5-16xx/24xx/26xx/46xx-series.

E5-16xx/24xx/26xx/46xx-series had better be updated to latest CPU microcode
(microcode must be 0x618+ or 0x70c+).


https://01.org/numatop
oprofile 1.3.0 2018 cli OProfile is a system-wide profiler for Linux systems, capable of profiling all running code at low overhead

Note:

The binary operf will only be compiled if the kernel supports the neccessary events.

Note 24.12.2016 (for version 1.1.0)
Added the patch from

https://patchwork.openembedded.org/patch/122253/

so that oprofile 1.1.0 also compiles with GCC-6


http://oprofile.sourceforge.net/news/
p0f
3.09b
2016
cli P0f is a tool that utilizes an array of sophisticated, purely passive traffic fingerprinting mechanisms to identify the players behind any incidental TCP/IP communications (often as little as a single normal SYN) without interfering in any way. Version 3 is a complete rewrite of the original codebase, incorporating a significant number of improvements to network-level fingerprinting, and introducing the ability to reason about application-level payloads (e.g., HTTP).
http://lcamtuf.coredump.cx/p0f3/
pagemon
0.01.10
2016
cli "pagemon is an interactive memory/page monitoring tool allowing one to browse the memory map of an active running process on Linux.
https://github.com/ColinIanKing/pagemon
pchar
1.5
2005
cli pchar is a tool to characterize the bandwidth, latency, and loss of links along an end-to-end path through the Internet.

Hints for compiling pchar
If compiling pchar with make fails with saying something like "SO_BSDCOMPAT not defined" temporary add

#define SO_BSDCOMPAT    14

to the file

/usr/include/asm-generic/socket.h

The compile script in JWM / Linux adds this statement to the include file pc.h from pchar if neccessary.

The usage of the abs function in the pchar source file ResultTable.cc is not allowed with GCC 7.x anymore. Therefor I corrected the source file:

[xtrnaw7@t61p pchar-1.5]$ diff ResultTable.cc ResultTable.cc.org
913,921c913,917
< //        residuals[l] = abs(partialmins[i] -
< //                   ((currentslope *
< //                     column2size(i) /
< //                     slopescale) +
< //                    currentintercept));
<         residuals[l] = partialmins[i] > ((currentslope * column2size(i) / slopescale ) + currentintercept ) ?
<             partialmins[i] -  ((currentslope * column2size(i) / slopescale ) + currentintercept ) :
<             ((currentslope * column2size(i) / slopescale ) + currentintercept ) - partialmins[i] ;
<
---
>         residuals[l] = abs(partialmins[i] -
>                    ((currentslope *
>                      column2size(i) /
>                      slopescale) +
>                     currentintercept));
938,939c934
< //        ys[l] = abs(partialmins[i] - mediany);
<         ys[l] = partialmins[i] > mediany ? partialmins[i] - mediany : mediany - partialmins[i] ;
---
>         ys[l] = abs(partialmins[i] - mediany);
[xtrnaw7@t61p pchar-1.5]$


[xtrnaw7@t61p pchar-1.5]$ ls -ltr ResultTable.cc*
-rw-r--r-- 1 xtrnaw7 users 26718 Aug 13 11:35 ResultTable.cc        # source w/ corrections for gcc 7.x
-rw-r--r-- 1 xtrnaw7 users 26109 Aug 13 11:42 ResultTable.cc.org    # original source file
-rw-r--r-- 1 xtrnaw7 users 26718 Aug 13 11:42 ResultTable.cc.gc7x  
# source w/ corrections for gcc 7.x
[xtrnaw7@t61p pchar-1.5]$




http://www.kitchenlab.org/www/bmah/Software/pchar/
perftools
(2017/12/19)
2017
cli Performance analysis tools based on Linux perf_events (aka perf) and ftrace (Brendan Gregg)
https://github.com/brendangregg/perf-tools
pigz
2.4.0
2017
cli parallel gzip
http://zlib.net/pigz/
pktstat
1.8.5
2012
cli pktstat displays a real-time list of active connections seen on a network interface, and how much bandwidth is being used by what.

It partially decodes HTTP and FTP protocols to show what filename is being transferred, as well as X11 application names. Entries hang around on the screen for a few seconds so you can see what just happened, in short a very useful tool to figure out network connection in real time.

http://linuxpoison.blogspot.de/2010/12/top-like-utility-to-monitor-network.html
powertop
2.9
2017
cli

PowerTOP is a Linux tool to diagnose issues with power consumption and power management.

In addition to being a diagnostic tool, PowerTOP also has an interactive mode where the user can experiment various power management settings for cases where the Linux distribution has not enabled these settings.

Hints for compiling powertop 2.8 and older

Neccessary changes in main.cpp to compile in GCC 7.x:

[xtrnaw7@t540p /var/tmp/jwm/src/powertop-2.6.1/src]$ diff main.cpp main.cpp.org
420c420
<                 sprintf(workload, "%s", optarg ? optarg : (char *)'\0' );
---
>                 sprintf(workload, "%s", optarg ? optarg :'\0' );
[xtrnaw7@t540p /var/tmp/jwm/src/powertop-2.6.1/src]$


https://01.org/powertop/
powerstat
0.02.18
2018
cli "Powerstat measures the power consumption of a machine using the battery stats or the Intel RAPL interface. The output is like vmstat but also shows power consumption statistics. At the end of a run, powerstat will calculate the average, standard deviation and min/max of the gathered data." https://github.com/ColinIanKing/powerstat
procdump 1.0.1 2018 cli ProcDump is a Linux reimagining of the classic ProcDump tool from the Sysinternals suite of tools for Windows.  ProcDump provides a convenient way for Linux developers to create core dumps of their application based on performance triggers.

Hints for compiling procdump

Changes for the source code and make file done are:

[xtrnaw7@rhel7-001 procdump-1.0.0]$ diff Makefile Makefile.org
3c3
< CFLAGS=-I ./include -pthread -lrt
---
> CFLAGS=-I ./include -pthread
[xtrnaw7@rhel7-001 procdump-1.0.0]$

[xtrnaw7@t540p /var/tmp/jwm/src/procdump-1.0.0/src]$ diff ProcDumpConfiguration.c ProcDumpConfiguration.c.org
461,462c461
<     int i ;
<     for (i = 0; i < self->nThreads; i++) {
---
>     for (int i = 0; i < self->nThreads; i++) {
588,589c587,588
<     int i ;
<     for (i = 0; i < strLen; i++) {
---
>
>     for (int i = 0; i < strLen; i++) {
[xtrnaw7@t540p /var/tmp/jwm/src/procdump-1.0.0/src]$

40p /var/tmp/jwm/src/procdump-1.0.0/src]$ diff CoreDumpWriter.c CoreDumpWriter.c.org
184c184
<         int j;        
---
>                 
186c186
<         for(j = 0; j < i; j++){
---
>         for(int j = 0; j < i; j++){


https://github.com/Microsoft/ProcDump-for-Linux
prtdiag
1.2
2003
cli prtdiag is a little script that displays the hardware configuration and status of a running machine.
http://people.redhat.com/tcallawa/prtdiag/
pt-diskstats 2.2.19
cli view disks I/O . This version is from percona-toolkit 2.2.19.

Percona Toolkit is a collection of advanced command-line tools used by Percona (http://www.percona.com/) support staff to perform a variety of MySQL and system tasks that are too difficult or complex to perform manually.
https://www.percona.com/doc/percona-toolkit/2.1/pt-diskstats.html
scanlogd
2.2.7
2012
cli
scanlogd is a TCP port scan detection tool, originally designed to illustrate various attacks an IDS developer has to deal with, for a Phrack Magazine article. Thus, unlike some of the other port scan detection tools out there, scanlogd is designed to be totally safe to use. https://www.openwall.com/scanlogd/
screen
4.0.3

cli screen is a full-screen window manager that multiplexes a physical terminal between several processes, typically interactive shells.

Hints for compiling screen
To compile screen the include file /usr/include/sys/stropts.h is necessary but in standard Linux distributions this file does not exist. To get around this bug do

sudo mkdir -p /usr/include/sys
sudo touch /usr/include/sys/stropts.h



https://www.gnu.org/software/screen/
smartmontools
6.6
2017
cli smartmontools contains utility programs (smartctl, smartd) to control/monitor storage systems using the Self-Monitoring, Analysis and Reporting Technology System (S.M.A.R.T.) built into most modern ATA and SCSI disks. It is derived from smartsuite. http://sourceforge.net/projects/smartmontools/
smem
1.4
2013
cli

smem is a tool that can give numerous reports on memory usage on Linux systems. Unlike existing tools, smem can report proportional set size (PSS), which is a more meaningful representation of the amount of memory used by libraries and applications in a virtual memory system.

Because large portions of physical memory are typically shared among multiple applications, the standard measure of memory usage known as resident set size (RSS) will significantly overestimate memory usage. PSS instead measures each application's "fair share" of each shared area to give a realistic measure.

https://www.selenic.com/smem/
smemstat
0.02.3
2018
cli Smemstat reports the physical memory usage taking into consideration shared memory. The tool can either report a current snapshot of memory usage or periodically dump out any changes in memory.
http://kernel.ubuntu.com/~cking/smemstat/
sockperf
3.5
2018
cli
sockperf is a network benchmarking utility over socket API that was designed for testing performance (latency and throughput) of high-performance systems (it is also good for testing performance of regular networking systems as well). It covers most of the socket API calls and options. https://github.com/Mellanox/sockperf
stress
1.0.4
2014
cli stress is a deliberately simple workload generator for POSIX systems. It imposes a configurable amount of CPU, memory, I/O, and disk stress on the system. It is written in C, and is free software licensed under the GPLv2. http://people.seas.harvard.edu/~apw/stress/https://sourceforge.net/projects/linux-diag/files/sysfsutils/sysfsutils-2.0.0/
stress-ng
0.09.59.1
2019
cli stress-ng will stress test a computer system in various selectable ways.

Hints for compiling stress-ng
If you get the error message

stress-rdrand.c:49: Error: no such instruction: `rdrand %rax'


then the installed binutils are too old. Update the binutils (see Links_to libraries neccessary for JWM below)  and try again

13.08.2017 /bs

Changes in stress-fp-error.c from version 0.07.14 to successfully compile in ArcLinux with

[xtrnaw7@t61p /var/tmp/jwm/src/stress-ng-0.07.14]$ uname -a
Linux t61p 4.12.6-1-ARCH #1 SMP PREEMPT Sat Aug 12 09:16:22 CEST 2017 x86_64 GNU/Linux
[xtrnaw7@t61p /var/tmp/jwm/src/stress-ng-0.07.14]$ gcc --version
gcc (GCC) 7.1.1 20170630
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[xtrnaw7@t540p /var/tmp/jwm/src/stress-ng-0.07.14]$ diff stress-fp-error.c  stress-fp-error.c.org
30,44d29
< /*
<   neccessary code for ArcLinux (?)  13.08.2017/bs
< */
< #if !defined EDOM
< # define EDOM   33      /* Math argument out of domain of function.  */
< #endif
<
< #if !defined ERANGE
< # define ERANGE 34      /* Math result not representable.  */
< #endif
<
< /*
<   end of work around for ArcLinux
< */
<


18.08.2018 /bs

stress-ng needs a GCC version that supports this:

#pragma GCC diagnostic not allowed inside functions

The GCC version in Centos 6.9 does not support that. Therefor stress-ng must be compiled with a newer GCC version in Centos 6.9  (see Required GCC version on how to install a newer GCC version):

scl enable devtoolset-7 bash
cd /var/tmp/jwm/compile_scripts && ./compile_stress-ng.sh



http://kernel.ubuntu.com/~cking/stress-ng/
sysfsutils 2.1.0 2006 cli This package's purpose is to provide a set of utilities for interfacing with sysfs https://sourceforge.net/projects/linux-diag/files/sysfsutils/
sysstat
12.1.5
2019
cli misc. tools :
cifsiostat mpstat iostat pidstat sysstat tapestat

Hints for compiling sysstat
18.08.2018 /bs

sysstat needs a GCC version that supports the __builtin_bswap16.
The GCC version in Centos 6.9 does not support that. Therefor sysstat must be compiled with a newer GCC version in Centos 6.9 (see Required GCC version on how to install a newer GCC version):

scl enable devtoolset-7 bash
cd /var/tmp/jwm/compile_scripts && ./compile_sysstat.sh



http://sebastien.godard.pagesperso-orange.fr/
t50
5.8.0
2018
cli T50 - Experimental Mixed Packet Injector

T50 (f.k.a. F22 Raptor) is a tool designed to perform "Stress Testing"

https://sourceforge.net/projects/t50/
tcptrack 1.4.2 2004
cli Monitor TCP connections on the network
no home page
tcpdump 4.9.2 2017
cli
a powerful command-line packet analyzer http://www.tcpdump.org/
tecmint_monitor.sh
0.1
2016
cli A Shell Script to Monitor Network, Disk Usage, Uptime, Load Average and RAM Usage in Linux
http://www.tecmint.com/linux-server-health-monitoring-script/

https://github.com/atarallo/TECMINT_MONITOR/
testssl.sh 2.9.5-6 2018 cli testssl.sh is a free command line tool which checks a server's service on any port for the support of TLS/SSL ciphers, protocols as well as recent cryptographic flaws and more. https://testssl.sh/
tiptop
2.3

cli tiptop is a performance monitoring tool for Linux.  It provides a dynamic real-time view of the tasks running in the system. tiptop is very similar to the top utility, but most of the information displayed comes from hardware counters.

Requirements:  Linux 2.6.31+
http://tiptop.gforge.inria.fr/
trace-cmd
2.7
2018
cli
trace-cmd: A front-end for Ftrace

see also Using the ftrace interface with shell

https://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git/


tracebox
git repository cloned 06.03.2014
2014
cli tracebox — A Middlebox Detection Tool

Hints for compiling tracebox
If the configure script complains about a missing file crafterconfig.h.in do

  cd /var/tmp/jwm/src/tracebox/noinst/libcrafter/libcrafter
  touch crafterconfig.h.in


Then execute the configure script again.

If you get an compile error like this

lua.cc:165:28: error: ‘luaL_register’ was not declared in this scope

try compiling tracebox with lua 5.2, e.g:

cd /var/tmp/jwm/src/tracebox
( export LUA_INCLUDE=-I/usr/include/lua5.2 ;  export LUA_LIB=-llua5.2 ; export LUA=/usr/bin/lua5.2 ; ./configure --prefix=/var/tmp/jwm )


http://www.tracebox.org/






wavemon 0.8.0 2008
cli WLAN monitor http://freecode.com/projects/wavemon/



Scripts and files include in JWM / Linux


Script
Purpose
Comment
./scripts/addtimestamp
add a timestamp to every output line of an executable

./scripts/busybox_httpd.sh start or stop the busybox httpd daemon
the htdocs directory for the httpd daemon is ${JWM_DIR}/htdocs
./scripts/collectl
wrapper script to call collectl

./scripts/collectl_top
call collectl with top like output

./scripts/collectl_top_io
call collectl with top like output with I/O

./scripts/darkstat.sh
start or stop the darkstat daemon

./scripts/execute_on_all_hosts.sh
execute a script or command on multiple hosts

./scripts/execute_tasks.sh
a wrapper script to execute tasks defined in an include file
execute_tasks.include is an sample include file for execute_tasks.sh; use

./scripts/execute_tasks.sh -v -h

to get the usage help and documentation for the script

./scripts/execute_scripts.sh
execute one or more executables in parallel or sequential sessions

./scripts/exec_snoop_wrapper.sh a wrapper script for the execsnoop script from the perftools
./scripts/iotop*
shell scripts to start iotop

./scripts/netdata.sh
start or stop the netdata daemon

./scripts/open_snoop_wrapper.sh a wrapper script for the opensnoop script from the perftools
./scripts/opensnoop1 a patched version of the opensnoop script from the perftools
./scripts/start_stop_manserver.sh
start or stop the JWM manServer

./scripts/compile_justniffer_with_boost.sh script to compile justniffer with a boost staging area
./scripts/jwm.sh script to execute a command in a temporay JWM / Linux environment
./compile_scripts/check_for_missing_libs.sh script to check for missing libraries
./compile_scripts/add_os_packages.sh script to install the OS packages required to compile all tools included in JWM / Linux

Note:

The scripts in the directory ./scripts not listed in the table above are part of one of the tools listed below


Develop Templates


These files/scripts can be used as template for new scripts

Script / file
Purpose
Comment
./develop/scriptt_mini.sh
simple kornshell script template with only the most useful functions

./develop/scriptt.sh
complex kornshell script template with lots of predefined variables and functions

./develop/send_monitor_login_mail.sh
a sample script to monitor a log file

./compile_scripts/compile_tool.template
a template for new compile scripts






Scripts and files for the JWM / Linux Environment

This table list the files and scripts that make up the JWM / Linux environment:

File
Type
Purpose
Comment
compile_tools.sh
script
simple wrapper script for make_tools.sh





create_archive.sh
script
create a new archive

init_session
script
start a JWM / Linux session

JWM_DESC
textfile
contains the description of the archive; this file will be (re)created by create_archive.sh before creating the new archive

jwm_env
include script
this is the include script for all other scripts in JWM / Linux that defines the necessary variables and some general functions

JWM_Linux_documentation.html
HTML page
this file

README
textfile
readme file for JWM / Linux

VERSION
textfile
contains the version of the JWM / Linux archive

NO_COMPILE_LIST
textfile
file with the list of packages that should not be compiled

compile_scripts/make_tools.sh
main script to compile the tools

compile_scripts/check_compile_status
script
view the compile status

compile_scripts/compile_<toolname>.sh script
compile the tool <toolname>

compile_scripts/start_compiling.sh
script
help script for the compile process

Note:

This script is used to compile the libraries included in JWM / Linux and copy the requried OS libraries to the JWM / Linux lib directory.

compile_scripts/jwml_include.inc script
include script for the compile scripts

compile_scripts/compile_tool.template
script
template for a compile script

./lib/_DO_NOT_COPY_LIBS_
file
semaphore file to request make_tools.sh NOT to copy the OS libraries

compile_scripts/OS_LIBS
textfile
list of OS libraries that should be copied to the JWM / Linux lib directory




Aliase in JWM / Linux


The JWM / Linux init script init_session defines the following aliase:

Alias
Purpose
Comment
jwm_version
print the version of the installed JWM archive

jwm_tools
list the tools in the current JWM environment

jwm_dir
print the the JWM / Linux base directory

jwm_home
change the working directory to the JWM / Linux base directory







Examples

This section contains the usage and some usage examples for the tools included in JWM / Linux. For more indepth information see either the man page or the documentation on the websites for the tools.


Note:

Most of the examples are copied from the documentation for the tools.


ack

ack is a replacement for grep written in Perl.

ack usage
The usage help is:

[xtrnaw7@t540p /var/tmp/jwm/scripts]$ ./ack --help
Usage: ack [OPTION]... PATTERN [FILES OR DIRECTORIES]

Search for PATTERN in each source file in the tree from the current
directory on down.  If any files or directories are specified, then
only those files and directories are checked.  ack may also search
STDIN, but only if no file or directory arguments are specified,
or if one of them is "-".

Default switches may be specified in an .ackrc file. If you want no dependency
on the environment, turn it off with --noenv.

Example: ack -i select

Searching:
  -i, --ignore-case             Ignore case distinctions in PATTERN
  -S, --[no]smart-case          Ignore case distinctions in PATTERN,
                                only if PATTERN contains no upper case.
                                Ignored if -i or -I are specified.
  -I, --no-ignore-case          Turns on case-sensitivity in PATTERN.
                                Negates -i and --smart-case.
  -v, --invert-match            Invert match: select non-matching lines
  -w, --word-regexp             Force PATTERN to match only whole words
  -Q, --literal                 Quote all metacharacters; PATTERN is literal
  --match PATTERN               Specify PATTERN explicitly. Typically omitted.

Search output:
  -l, --files-with-matches      Only print filenames containing matches
  -L, --files-without-matches   Only print filenames with no matches
  --output=expr                 Output the evaluation of expr for each line
                                (turns off text highlighting)
  -o                            Show only the part of a line matching PATTERN
                                Same as --output='$&'
  --passthru                    Print all lines, whether matching or not
  -m, --max-count=NUM           Stop searching in each file after NUM matches
  -1                            Stop searching after one match of any kind
  -H, --with-filename           Print the filename for each match (default:
                                on unless explicitly searching a single file)
  -h, --no-filename             Suppress the prefixing filename on output
  -c, --count                   Show number of lines matching per file
  --[no]column                  Show the column number of the first match

  -A NUM, --after-context=NUM   Print NUM lines of trailing context after
                                matching lines.
  -B NUM, --before-context=NUM  Print NUM lines of leading context before
                                matching lines.
  -C [NUM], --context[=NUM]     Print NUM lines (default 2) of output context.

  --print0                      Print null byte as separator between filenames,
                                only works with -f, -g, -l, -L or -c.

  -s                            Suppress error messages about nonexistent or
                                unreadable files.


File presentation:
  --pager=COMMAND               Pipes all ack output through COMMAND.  For
                                example, --pager="less -R".  Ignored if output
                                is redirected.
  --nopager                     Do not send output through a pager.  Cancels
                                any setting in ~/.ackrc, ACK_PAGER or
                                ACK_PAGER_COLOR.
  --[no]heading                 Print a filename heading above each file's
                                results.  (default: on when used interactively)
  --[no]break                   Print a break between results from different
                                files.  (default: on when used interactively)
  --group                       Same as --heading --break
  --nogroup                     Same as --noheading --nobreak
  -p, --proximate=LINES         Separate match output with blank lines unless
                                they are within LINES lines from each other.
  -P, --proximate=0             Negates --proximate.
  --[no]underline               Print a line of carets under the matched text.
  --[no]color, --[no]colour     Highlight the matching text (default: on unless
                                output is redirected, or on Windows)
  --color-filename=COLOR
  --color-match=COLOR
  --color-colno=COLOR
  --color-lineno=COLOR          Set the color for filenames, matches, line and
                                column numbers.
  --help-colors                 Show a list of possible color combinations.
  --help-rgb-colors             Show a list of advanced RGB colors.
  --flush                       Flush output immediately, even when ack is used
                                non-interactively (when output goes to a pipe or
                                file).


File finding:
  -f                            Only print the files selected, without
                                searching.  The PATTERN must not be specified.
  -g                            Same as -f, but only select files matching
                                PATTERN.
  --sort-files                  Sort the found files lexically.
  --show-types                  Show which types each file has.
  --files-from=FILE             Read the list of files to search from FILE.
  -x                            Read the list of files to search from STDIN.

File inclusion/exclusion:
  --[no]ignore-dir=name         Add/remove directory from list of ignored dirs
  --[no]ignore-directory=name   Synonym for ignore-dir
  --ignore-file=FILTER:ARGS     Add filter for ignoring files.
  -r, -R, --recurse             Recurse into subdirectories (default: on)
  -n, --no-recurse              No descending into subdirectories
  --[no]follow                  Follow symlinks.  Default is off.

File type inclusion/exclusion:
  --type=X                      Include only X files, where X is a recognized
                                filetype, e.g. --php, --ruby
  --type=noX                    Exclude X files, e.g. --nophp, --no-ruby.
  -k, --known-types             Include only files of types that ack recognizes.
  --help-types                  Display all known types, and how they're defined.

File type specification:
  --type-set=TYPE:FILTER:ARGS   Files with the given ARGS applied to the given
                                FILTER are recognized as being of type TYPE.
                                This replaces an existing definition for TYPE.
  --type-add=TYPE:FILTER:ARGS   Files with the given ARGS applied to the given
                                FILTER are recognized as being type TYPE.
  --type-del=TYPE               Removes all filters associated with TYPE.

Miscellaneous:
  --version                     Display version & copyright
  --[no]env                     Ignore environment variables and global ackrc
                                files.  --env is legal but redundant.
  --ackrc=filename              Specify an ackrc file to use
  --ignore-ack-defaults         Ignore default definitions included with ack.
  --create-ackrc                Outputs a default ackrc for your customization
                                to standard output.
  --dump                        Dump information on which options are loaded
                                and where they're defined.
  --[no]filter                  Force ack to treat standard input as a pipe
                                (--filter) or tty (--nofilter)
  --help                        This help
  --man                         Print the manual.
  --help-types                  Display all known types, and how they're defined.
  --help-colors                 Show a list of possible color combinations.
  --help-rgb-colors             Show a list of advanced RGB colors.
  --thpppt                      Bill the Cat
  --bar                         The warning admiral
  --cathy                       Chocolate! Chocolate! Chocolate!

Filter specifications:
    If FILTER is "ext", ARGS is a list of extensions checked against the
        file's extension.
    If FILTER is "is", ARGS must match the file's name exactly.
    If FILTER is "match", ARGS is matched as a case-insensitive regex
        against the filename.
    If FILTER is "firstlinematch", ARGS is matched as a regex the first
        line of the file's contents.

Exit status is 0 if match, 1 if no match.

ack's home page is at https://beyondgrep.com/

The full ack manual is available by running "ack --man".

This is version v3.0.0 of ack.  Run "ack --version" for full version info.
[xtrnaw7@t540p /var/tmp/jwm/scripts]$






arping


Arping is a util to find out if a specific IP address on the LAN is 'taken' and what MAC address owns it. Sure, you *could* just use 'ping' to find out if it's taken and even if the computer blocks ping (and everything else) you still get an entry in your ARP cache. But what if you aren't on a routable net? Or the host blocks ping (all ICMP even)? Then you're screwed. Or you use arping.


arping usage
[xtrnaw7@t540p /data/download/arping-arping-2.x]$  ./src/arping --help
ARPing 2.19, by Thomas Habets <thomas@habets.se>
usage: arping [ -0aAbdDeFpPqrRuUv ] [ -w <sec> ] [ -W <sec> ] [ -S <host/ip> ]
              [ -T <host/ip ] [ -s <MAC> ] [ -t <MAC> ] [ -c <count> ]
              [ -C <count> ] [ -i <interface> ] [ -m <type> ] [ -g <group> ]
              [ -V <vlan> ] [ -Q <priority> ] <host/ip/MAC | -B>

Options:

    -0     Use this option to ping with source IP address 0.0.0.0. Use this
           when you haven't configured your interface yet.  Note that  this
           may  get  the  MAC-ping  unanswered.   This  is  an alias for -S
           0.0.0.0.
    -a     Audiable ping.
    -A     Only count addresses matching  requested  address  (This  *WILL*
           break  most things you do. Only useful if you are arpinging many
           hosts at once. See arping-scan-net.sh for an example).
    -b     Like -0 but source broadcast source  address  (255.255.255.255).
           Note that this may get the arping unanswered since it's not nor-
           mal behavior for a host.
    -B     Use instead of host if you want to address 255.255.255.255.
    -c count
           Only send count requests.
    -C count
           Only wait for this many replies, regardless of -c and -w.
    -d     Find duplicate replies. Exit with 1 if there are answers from
           two different MAC addresses.
    -D     Display answers as exclamation points and missing packets as dots.
    -e     Like -a but beep when there is no reply.
    -F     Don't try to be smart about the interface name.  (even  if  this
           switch is not given, -i overrides smartness)
    -g group
           setgid() to this group instead of the nobody group.
    -h     Displays a help message and exits.
    -i interface
           Use the specified interface.
    -m type
           Type of timestamp to use for incoming packets. Use -vv when
           pinging to list available ones.
    -q     Does not display messages, except error messages.
    -Q pri 802.1p priority to set. Should be used with 802.1Q (-V).
           Defaults to 0.
    -r     Raw output: only the MAC/IP address is displayed for each reply.
    -R     Raw output: Like -r but shows "the other one", can  be  combined
           with -r.
    -s MAC Set source MAC address. You may need to use -p with this.
    -S IP  Like  -b and -0 but with set source address.  Note that this may
           get the arping unanswered if the target does not have routing to
           the  IP.  If you don't own the IP you are using, you may need to
           turn on promiscious mode on the interface (with -p).  With  this
           switch  you can find out what IP-address a host has without tak-
           ing an IP-address yourself.
    -t MAC Set target MAC address to use when pinging IP address.
    -T IP  Use -T as target address when pinging MACs that won't respond to
           a broadcast ping but perhaps to a directed broadcast.
           Example:
           To check the address of MAC-A, use knowledge of MAC-B and  IP-B.
           $ arping -S <IP-B> -s <MAC-B> -p <MAC-A>
    -p     Turn  on  promiscious  mode  on interface, use this if you don't
           "own" the MAC address you are using.
    -P     Send ARP replies instead of requests. Useful with -U.
    -u     Show index=received/sent instead  of  just  index=received  when
           pinging MACs.
    -U     Send unsolicited ARP.
    -v     Verbose output. Use twice for more messages.
    -V num 802.1Q tag to add. Defaults to no VLAN tag.
    -w sec Specify a timeout before ping exits regardless of how many
packets have been sent or received.
    -W sec Time to wait between pings.
Report bugs to: thomas@habets.se
Arping home page: <http://www.habets.pp.se/synscan/>
Development repo: http://github.com/ThomasHabets/arping
[xtrnaw7@t540p /data/download/arping-arping-2.x]$



arping example
[xtrnaw7@t540p /var/tmp/jwm/sbin]$ ./arping 192.168.1.151
arping: libnet_init(LIBNET_LINK, <null>): libnet_open_link(): UID/EUID 0 or capability CAP_NET_RAW required
arping: you may need to run as root
[xtrnaw7@t540p /var/tmp/jwm/sbin]$ sudo ./arping 192.168.1.151
ARPING 192.168.1.151
60 bytes from 1c:c1:de:83:dc:24 (192.168.1.151): index=0 time=219.432 usec
60 bytes from 1c:c1:de:83:dc:24 (192.168.1.151): index=1 time=224.237 usec
60 bytes from 1c:c1:de:83:dc:24 (192.168.1.151): index=2 time=204.635 usec
^C
--- 192.168.1.151 statistics ---
3 packets transmitted, 3 packets received,   0% unanswered (0 extra)
rtt min/avg/max/std-dev = 0.205/0.216/0.224/0.008 ms
[xtrnaw7@t540p /var/tmp/jwm/sbin]$





atop

atop is a replacement for top.

atop usage
[xtrnaw7@t540p bin]$ ./atop -h
Usage: atop [-flags] [interval [samples]]
        or
Usage: atop -w  file  [-S] [-a] [interval [samples]]
       atop -r [file] [-b hh:mm] [-e hh:mm] [-flags]

    generic flags:
      -a  show or log all processes (i.s.o. active processes only)
      -R  calculate proportional set size (PSS) per process
      -P  generate parseable output for specified label(s)
      -L  alternate line length (default 80) in case of non-screen output
      -f  show fixed number of lines with system statistics
      -F  suppress sorting of system resources
      -G  suppress exited processes in output
      -l  show limited number of lines for certain resources
      -y  show individual threads
      -1  show average-per-second i.s.o. total values

      -x  no colors in case of high occupation
      -g  show general process-info (default)
      -m  show memory-related process-info
      -d  show disk-related process-info
      -n  show network-related process-info
      -s  show scheduling-related process-info
      -v  show various process-info (ppid, user/group, date/time)
      -c  show command line per process
      -o  show own defined process-info
      -u  show cumulated process-info per user
      -p  show cumulated process-info per program (i.e. same name)

      -C  sort processes in order of cpu-consumption (default)
      -M  sort processes in order of memory-consumption
      -D  sort processes in order of disk-activity
      -N  sort processes in order of network-activity
      -A  sort processes in order of most active resource (auto mode)

    specific flags for raw logfiles:
      -w  write raw data to   file (compressed)
      -r  read  raw data from file (compressed)
          special file: y[y...] for yesterday (repeated)
      -S  finish atop automatically before midnight (i.s.o. #samples)
      -b  begin showing data from specified time
      -e  finish showing data after specified time

    interval: number of seconds   (minimum 0)
    samples:  number of intervals (minimum 1)

If the interval-value is zero, a new sample can be
forced manually by sending signal USR1 (kill -USR1 pid_atop)
or with the keystroke 't' in interactive mode.

Please refer to the man-page of 'atop' for more details.