The GNU gProf profiler can be used to get a detailed profile of the program. It gives a an easy to read output which shows how much time is being spent in each function.
Usage
gcc -pg TestProf.c -o TestGprof.out
./TestGprof.out
This will result in a file called "gmon.out" by default. To obtain a profile,
gprof -b TestGprof gmon.out > analysis.out
Reference for all the command line switches for gProf
This tool is basically a memory debugger and a profiler. So it can help analyze performance in two ways,
- Detailed Desc. of Memory Usage and Leaks.
- Deatiled Profile of Program in terms of Instruction Reads per Function.
It has a command line option '--callgrind' which gives a detailed profile of the program and this output is formatted in by an application called Kcachegrind. The profile has a callgraph which is very helpful in determining performance bottlnecks.
The output is much easier to read than gProf but while gProf indicates the amount of time spent in a function in terms of seconds, callgrind does not, as while analyzing the program, it replaces the basic code (such as malloc) with it's own code and this slows down the execution time which is why it cannot give an accurate measurement of the amount of time spent in a function.
Usage
For viewing memory leaks,
gcc -g TestVal.c -o TestVal.out
valgrind --leak-check=full --show-leak-kinds=all TestVal.out
For obtaining a profile,
valgrind --tool=callgrind TestVal.out
To view the output, open the "callgrind.out.____" file with Kcachegrind
Reference for Callgrind options
This is a comprehensive tool that lets you monitor resources like the CPU, Memory, Network and I/O.
- CPU :
sar <sampling_interval> <number_of_samples> - Memory :
sar -r <sampling_interval> <number_of_samples> - Page Stats :
sar -B <sampling_interval> <number_of_samples> - Swap Space :
sar -S <sampling_interval> <number_of_samples>
This is also a very powerful tool, specially for digging into interrupt behavior of the system.
Specifically for interrupts across all cores use mpstat -I ALL <sampling_interval> <number_of_samples>
This command will give a detailed report about the interrupt behaviour of the system,
Linux 4.15.0-70-generic (vishal-Inspiron-7559) Monday 02 December 2019 _x86_64_ (8 CPU)
12:52:10 IST CPU intr/s
12:52:10 IST all 1404.31
12:52:10 IST CPU 0/s 1/s 8/s 9/s 12/s 14/s 16/s 120/s 121/s 122/s 123/s 124/s 125/s 126/s 127/s 128/s 129/s 130/s NMI/s LOC/s SPU/s PMI/s IWI/s RTR/s RES/s CAL/s TLB/s TRM/s THR/s DFR/s MCE/s MCP/s HYP/s ERR/s MIS/s PIN/s NPI/s PIW/s
12:52:10 IST 0 0.00 0.00 0.00 0.01 0.00 0.18 0.05 0.00 0.00 0.00 0.16 0.00 0.00 0.06 0.00 0.00 0.00 0.01 0.00 83.41 0.00 0.00 0.01 0.00 26.32 10.22 7.61 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
12:52:10 IST 1 0.00 0.00 0.00 0.06 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.33 0.00 75.15 0.00 0.00 0.00 0.00 10.66 9.56 7.76 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
12:52:10 IST 2 0.00 0.15 0.00 0.90 0.00 6.14 0.00 0.00 0.00 0.01 5.03 0.00 0.00 0.01 0.00 6.14 0.00 0.62 0.00 74.89 0.00 0.00 0.00 0.00 5.96 9.66 7.86 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
12:52:10 IST 3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 169.58 0.00 0.00 0.02 0.00 0.00 113.23 0.00 0.00 2.87 0.00 5.60 9.18 7.34 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
12:52:10 IST 4 0.00 0.03 0.00 0.00 0.00 0.00 341.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2.56 0.00 82.28 0.00 0.00 0.00 0.00 7.24 9.34 7.48 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
12:52:10 IST 5 0.00 0.00 0.00 0.00 0.00 12.72 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.86 0.00 12.72 0.00 2.03 0.00 80.02 0.00 0.00 0.12 0.00 3.10 9.63 7.72 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
12:52:10 IST 6 0.00 0.15 0.00 0.39 0.00 3.18 0.00 0.00 0.00 0.00 0.00 0.00 0.00 3.88 0.00 3.18 0.00 0.00 0.00 80.40 0.00 0.00 0.17 0.00 2.28 9.64 7.71 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
12:52:10 IST 7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 6.26 0.00 0.00 0.00 0.00 0.00 0.00 10.97 0.00 80.11 0.00 0.00 0.00 0.00 1.96 9.61 7.65 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
12:52:10 IST CPU HI/s TIMER/s NET_TX/s NET_RX/s BLOCK/s IRQ_POLL/s TASKLET/s SCHED/s HRTIMER/s RCU/s
12:52:10 IST 0 14.35 30.23 0.00 0.04 0.16 0.00 0.00 44.65 0.00 21.57
12:52:10 IST 1 12.68 29.69 0.00 0.04 0.00 0.00 0.01 34.31 0.00 20.86
12:52:10 IST 2 11.34 30.78 0.00 0.03 5.01 0.00 0.12 30.99 0.00 21.65
12:52:10 IST 3 165.07 38.97 0.00 0.04 0.00 0.00 0.00 38.98 0.00 26.67
12:52:10 IST 4 22.17 30.01 0.00 0.04 0.00 0.00 0.08 29.04 0.00 21.40
12:52:10 IST 5 20.83 30.38 0.00 0.04 0.00 0.00 0.00 28.45 0.00 21.30
12:52:10 IST 6 21.31 31.47 0.00 0.04 0.00 0.00 0.11 28.97 0.00 22.01
12:52:10 IST 7 9.37 31.46 0.00 0.04 6.25 0.00 0.15 28.58 0.00 22.21
This is specific to docker. This command shows the CPU, RAM, Block I/O, Network I/O and PIDs being used by every active container.
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
cf5a806fae57 socialnetwork_write-home-timeline-service_1 0.01% 4.715MiB / 125.8GiB 0.00% 31.9MB / 15.9MB 0B / 0B 10
0152ad588897 socialnetwork_compose-post-service_1 0.00% 2.117MiB / 125.8GiB 0.00% 25.3MB / 0B 0B / 0B 2
4b161ad2c398 socialnetwork_home-timeline-redis_1 0.17% 19.38MiB / 125.8GiB 0.02% 37MB / 1.15MB 0B / 10.6MB 4
455ddeab0c9d socialnetwork_home-timeline-service_1 0.00% 2.125MiB / 125.8GiB 0.00% 25.2MB / 0B 0B / 0B 2
d808781c0f00 socialnetwork_post-storage-mongodb_1 1.50% 84.89MiB / 125.8GiB 0.07% 34.9MB / 7.02MB 0B / 555MB 39
ee82544471da socialnetwork_user-mongodb_1 1.01% 86.18MiB / 125.8GiB 0.07% 39.5MB / 16.4MB 0B / 541MB 161
ac6c999d8715 socialnetwork_user-timeline-redis_1 0.17% 5.418MiB / 125.8GiB 0.00% 25.9MB / 665kB 0B / 0B 4
7a865f1c9d2d socialnetwork_media-frontend_1 0.00% 17.53MiB / 125.8GiB 0.01% 489kB / 0B 0B / 0B 17
4a5a16b3ecab socialnetwork_media-memcached_1 0.01% 2.48MiB / 125.8GiB 0.00% 25.4MB / 0B 0B / 0B 10
d97a0b994f35 socialnetwork_post-storage-memcached_1 0.01% 2.449MiB / 125.8GiB 0.00% 550kB / 0B 0B / 0B 10
38ba4fa4a694 socialnetwork_user-service_1 64.57% 25.07MiB / 125.8GiB 0.02% 183MB / 138MB 0B / 0B 131
76ef0b850194 socialnetwork_social-graph-redis_1 0.16% 9.184MiB / 125.8GiB 0.01% 39.3MB / 9.01MB 0B / 721kB 4
47d8b8947e8a socialnetwork_unique-id-service_1 63.10% 7.832MiB / 125.8GiB 0.01% 131MB / 87.7MB 0B / 0B 2
4eaa9a343d2a socialnetwork_social-graph-mongodb_1 0.95% 197.5MiB / 125.8GiB 0.15% 71.5MB / 279MB 0B / 691MB 161
758a57cb81c1 socialnetwork_text-service_1 183.58% 10.12MiB / 125.8GiB 0.01% 284MB / 219MB 0B / 0B 9
559272c554bd socialnetwork_media-mongodb_1 1.06% 72.67MiB / 125.8GiB 0.06% 25.2MB / 0B 0B / 529MB 32
4f99b4a388d4 socialnetwork_user-timeline-mongodb_1 0.95% 75.57MiB / 125.8GiB 0.06% 31.6MB / 8.76MB 0B / 542MB 41
7142b2492d1d socialnetwork_write-home-timeline-rabbitmq_1 0.26% 197.9MiB / 125.8GiB 0.15% 29.3MB / 4.43MB 0B / 1.13MB 724
c891b79f89ba socialnetwork_url-shorten-mongodb_1 47.80% 438.8MiB / 125.8GiB 0.34% 170MB / 27.6MB 0B / 1.08GB 47
0f4b54c04a36 socialnetwork_url-shorten-service_1 93.70% 10.12MiB / 125.8GiB 0.01% 113MB / 189MB 0B / 0B 26
30756611b7d6 socialnetwork_user-mention-service_1 68.51% 5.617MiB / 125.8GiB 0.00% 220MB / 231MB 0B / 0B 17
71cb8473af05 socialnetwork_media-service_1 67.09% 7.547MiB / 125.8GiB 0.01% 141MB / 90.5MB 0B / 0B 3
afabd6c2fec7 socialnetwork_post-storage-service_1 0.00% 4.379MiB / 125.8GiB 0.00% 36.8MB / 10.8MB 0B / 0B 3
cddc0746ccf1 socialnetwork_user-timeline-service_1 0.00% 4.539MiB / 125.8GiB 0.00% 10.8MB / 8.67MB 0B / 0B 5
9dbed74f70e5 socialnetwork_social-graph-service_1 0.00% 36.63MiB / 125.8GiB 0.03% 328MB / 107MB 0B / 0B 36
9ea6b2530e7b socialnetwork_compose-post-redis_1 0.17% 7.207MiB / 125.8GiB 0.01% 37MB / 6.63MB 0B / 41kB 4
e2ae099570a2 socialnetwork_nginx-thrift_1 179.67% 77.26MiB / 125.8GiB 0.06% 606MB / 902MB 229kB / 0B 33
5c11005e7626 socialnetwork_url-shorten-memcached_1 0.01% 2.492MiB / 125.8GiB 0.00% 609kB / 0B 0B / 0B 10
a63cb691d2a5 socialnetwork_user-memcached_1 33.06% 6.562MiB / 125.8GiB 0.01% 194MB / 157MB 0B / 0B 10
6ddc7c1f37c1 socialnetwork_jaeger_1 28.17% 3.42GiB / 125.8GiB 2.72% 386MB / 3.75kB 0B / 0B 10