This repository has been archived by the owner on Jan 24, 2023. It is now read-only.
forked from pmacct/pmacct
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathCONFIG-KEYS
2544 lines (2173 loc) · 141 KB
/
CONFIG-KEYS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
SUPPORTED CONFIGURATION KEYS
Both configuration directives and commandline switches are listed below.
A configuration consists of key/value pairs, separated by the ':' char.
Starting a line with the '!' symbol, makes the whole line to be ignored
by the interpreter, making it a comment. Please also refer to QUICKSTART
document and the 'examples/' sub-tree for some examples.
Directives are sometimes grouped, like sql_table and print_output_file:
this is to stress if multiple plugins are running as part of the same
daemon instance, such directives must be casted to the plugin they refer
to - in order to prevent undesired inheritance effects. In other words,
grouped directives share the same field in the configuration structure.
LEGEND of flags:
GLOBAL Can't be configured on individual plugins
NO_GLOBAL Can't be configured globally
NO_PMACCTD Does not apply to 'pmacctd'
NO_UACCTD Does not apply to 'uacctd'
NO_NFACCTD Does not apply to 'nfacctd'
NO_SFACCTD Does not apply to 'sfacctd'
ONLY_PMACCTD Applies only to pmacctd
ONLY_UACCTD Applies only to uacctd
ONLY_NFACCTD Applies only to nfacctd
ONLY_SFACCTD Applies only to sfacctd
MAP Indicates the input file is a map
LIST OF DIRECTIVES:
KEY: debug (-d)
VALUES: [ true | false ]
DESC: Enables debug (default: false).
KEY: debug_internal_msg
VALUES: [ true | false ]
DESC: Extra flag to enable debug of internal messaging between Core process
and plugins. It has to be enabled on top of 'debug' (default: false).
KEY: daemonize (-D) [GLOBAL]
VALUES: [ true | false ]
DESC: Daemonizes the process (default: false).
KEY: aggregate (-c)
VALUES: [ src_mac, dst_mac, vlan, cos, etype, src_host, dst_host, src_net, dst_net,
src_mask, dst_mask, src_as, dst_as, src_port, dst_port, tos, proto, none,
sum_mac, sum_host, sum_net, sum_as, sum_port, flows, tag, tag2, label,
class, tcpflags, in_iface, out_iface, std_comm, ext_comm, lrg_comm,
as_path, peer_src_ip, peer_dst_ip, peer_src_as, peer_dst_as, local_pref,
med, src_std_comm, src_ext_comm, src_lrg_comm, src_as_path, src_local_pref,
src_med, mpls_vpn_rd, mpls_label_top, mpls_label_bottom, mpls_stack_depth,
sampling_rate, src_host_country, dst_host_country, src_host_pocode,
dst_host_pocode, pkt_len_distrib, nat_event, fw_event, post_nat_src_host,
post_nat_dst_host, post_nat_src_port, post_nat_dst_port, timestamp_start,
timestamp_end, timestamp_arrival, export_proto_seqno, export_proto_version ]
FOREWORDS: Individual IP packets are uniquely identified by their header field values (a
rather large set of primitives!). Same applies to uni-directional IP flows, as
they have at least enough information to discriminate where packets are coming
from and going to. Aggregates are instead used for the sole purpose of IP
accounting and hence can be identified by an arbitrary set of primitives.
The process to create an aggregate starting from IP packets or flows is: (a)
select only the primitives of interest (generic aggregation), (b) optionally
cast certain primitive values into broader logical entities, ie. IP addresses
into network prefixes or Autonomous System Numbers (spatial aggregation) and
(c) sum aggregate bytes/flows/packets counters when a new tributary IP packet
or flow is captured (temporal aggregation).
DESC: Aggregate captured traffic data by selecting the specified set of primitives.
sum_<primitive> are compound primitives which sum ingress/egress traffic in a
single aggregate; current limit of sum primitives: each sum primitive is mutual
exclusive with any other, sum and non-sum, primitive. The 'none' primitive
allows to make an unique aggregate which accounts for the grand total of
traffic flowing through a specific interface. 'tag', 'tag2' and 'label' enable
generation of tags when tagging engines (pre_tag_map, post_tag) are in use.
'class' enables L7 traffic classes when Packet/Flow Classification engine
(classifiers) is in use.
NOTES: * Some primitives (ie. tag2, timestamp_start, timestamp_end) are not part of
any default SQL table schema shipped. Always check out documentation related
to the RDBMS in use (ie. 'sql/README.mysql') which will point you to extra
primitive-related documentation, if required.
* List of the aggregation primitives available to each specific pmacct daemon
is available via -a command-line option, ie. "pmacctd -a".
* sampling_rate: if counters renormalization (ie. sfacctd_renormalize) is
enabled this field will report a value of one (1); otherwise it will report
the rate that is passed by the protocol or sampling_map. A value of zero (0)
means 'unknown' and hence no rate is applied to original counter values.
* src_std_comm, src_ext_comm, src_lrg_comm, src_as_path are based on reverse
BGP lookups; peer_src_as, src_local_pref and src_med are by default based on
reverse BGP lookups but can be alternatively based on other methods, for
example maps (ie. bgp_peer_src_as_type). Internet traffic is by nature
asymmetric hence reverse BGP lookups must be used with caution (ie. against
own prefixes).
* Communities (ie. std_comm, ext_comm, lrg_comm) and AS-PATHs (ie. as_path)
are fixed size (96 and 128 chars respectively at time of writing). Directives
like bgp_stdcomm_pattern and bgp_aspath_radius are aimed to keep length of
these strings under control but sometimes this is not enough. While the longer
term approach will be to define these primitives as varchar, the short-term
approach is to re-define default size, ie. MAX_BGP_STD_COMMS MAX_BGP_ASPATH
in network.h, to the desired size (blowing extra memory). This will require
recompiling the binary.
* timestamp_start, timestamp_end and timestamp_arrival should not be mixed
with pmacct support for historical accounting, ie. breakdown of traffic in
time-bins via the sql_history feature; these primitives have the effect of
letting pmacct act as a logger up to the msec level (if reported by the
capturing method). timestamp_start records NetFlow/IPFIX flow start time or
observation; timestamp_end records NetFlow/IPFIX flow end time; finally,
timestamp_arrival records libpcap packet timestamp and sFlow/NetFlow/IPFIX
packet arrival time at the collector.
* export_proto_seqno reports about export protocol (NetFlow, sFlow, IPFIX)
sequence number; due to its potential de-aggregation effect, two main use-
cases are seen as use of this primitive:
1) if using a log type (de-)aggregation method, ie. for security, forensics,
etc., in addition to existing primitives;
2) if using a reporting type aggregation method, it is recommended to split
this primitive in a separate plugin instance instead for sequencing
analysis.
DEFAULT: src_host
KEY: aggregate_primitives [GLOBAL, MAP]
DESC: Expects full pathname to a file containing custom-defined primitives. Once
defined in this file, primitives can be used in 'aggregate' statements. The
feature is currently available only in nfacctd, for NetFlow v9/IPFIX, pmacctd
and uacctd. Examples are available in 'examples/primitives.lst.example'. This
map does not support reloading at runtime.
DEFAULT: none
KEY: aggregate_filter [NO_GLOBAL]
DESC: Per-plugin filtering applied against the original packet or flow. Aggregation
is performed slightly afterwards, upon successful match of this filter.
By binding a filter, in tcpdump syntax, to an active plugin, this directive
allows to select which data has to be delivered to the plugin and aggregated
as specified by the plugin 'aggregate' directive. See the following example:
...
aggregate[inbound]: dst_host
aggregate[outbound]: src_host
aggregate_filter[inbound]: dst net 192.168.0.0/16
aggregate_filter[outbound]: src net 192.168.0.0/16
plugins: memory[inbound], memory[outbound]
...
This directive can be used in conjunction with 'pre_tag_filter' (which, in
turn, allows to filter tags). You will also need to force fragmentation handling
in the specific case in which a) none of the 'aggregate' directives is including
L4 primitives (ie. src_port, dst_port) but b) an 'aggregate_filter' runs a filter
which requires dealing with L4 primitives. For further information, refer to the
'pmacctd_force_frag_handling' directive.
DEFAULT: none
KEY: pcap_filter [GLOBAL, PMACCTD_ONLY]
DESC: This filter is global and applied to all incoming packets. It's passed to libpcap
and expects libpcap/tcpdump filter syntax. Being global it doesn't offer a great
flexibility but it's the fastest way to drop unwanted traffic. It applies only to
pmacctd.
DEFAULT: none
KEY: snaplen (-L) [GLOBAL, NO_NFACCTD, NO_SFACCTD]
DESC: Specifies the maximum number of bytes to capture for each packet. This directive has
key importance when enabling both classification and connection tracking engines. In
fact, some protocols (mostly text-based eg.: RTSP, SIP, etc.) benefit of extra bytes
because they give more chances to successfully track data streams spawned by control
channel. But it must be also noted that capturing larger packet portion require more
resources. The right value need to be traded-off. In case classification is enabled,
values under 200 bytes are often meaningless. 500-750 bytes are enough even for text
based protocols. Default snaplen values are ok if classification is disabled.
DEFAULT: 68 bytes; 128 bytes if compiled with --enable-ipv6
KEY: plugins (-P)
VALUES: [ memory | print | mysql | pgsql | sqlite3 | mongodb | nfprobe | sfprobe | tee |
amqp | kafka ]
DESC: Plugins to be enabled. memory, print, nfprobe, sfprobe and tee plugins are always
included in pmacct executables as they do not contain dependencies on external
libraries. Database (ie. SQL, MongoDB) and messaging ones (ie. amqp, kafka) do
have external dependencies and hence are available only if explicitely configured
and compiled.
memory plugin uses a memory table as backend; then, a client tool, 'pmacct', can fetch
the memory table content; the memory plugin is good for prototype solutions and/or
small environments. mysql, pgsql and sqlite3 plugins output respectively to MySQL,
PostgreSQL and SQLite 3.x (or BerkeleyDB 5.x with the SQLite API compiled-in) tables
to store data. mongodb enables use of the noSQL document-oriented database MongoDB
(requires installation of MongoDB API C driver which is shipped separatedly from the
main MongoDB package; read more in QUICKSTART in the "noSQL (MongoDB) setup examples"
section). print plugin prints output data to flat-files or stdout in JSON, CSV or
tab-spaced formats, or encodes it using the Apache Avro serialization system. amqp
and kafka plugins allow to output data to RabbitMQ and Kafka brokers respectively. All
these plugins, SQL, no-SQL and messaging are good for production solutions and/or
larger scenarios.
nfprobe acts as a NetFlow/IPFIX agent and exports collected data via NetFlow v1/v5/
v9 and IPFIX datagrams to a remote collector. sfprobe acts as a sFlow agent and
exports collected data via sFlow v5 datagrams to a remote collector. Both nfprobe
and sfprobe plugins apply only to pmacctd and uacctd daemons. tee acts as a replicator
for NetFlow/IPFIX/sFlow data (also transparent); it applies to nfacctd and sfacctd
daemons only. Plugins can be either anonymous or named; configuration directives can
be either global or bound to a specific plugins, if named. An anonymous plugin is
declared as 'plugins: mysql' in the config whereas a named plugin is declared as
'plugins: mysql[name]'. Then, directives can be bound specifically to such named
plugin as: 'directive[name]: value'.
DEFAULT: memory
KEY: [ nfacctd_pipe_size | sfacctd_pipe_size | pmacctd_pipe_size | tee_pipe_size ]
DESC: Defines the size of the kernel socket to read (ie. daemons) and write (ie. tee plugin)
traffic data. The socket is highlighted below with "XXXX":
XXXX
[network] ----> [kernel] ----> [core process] ----> [plugin] ----> [backend]
[__________pmacct___________]
On Linux systems, if this configuration directive is not specified default socket size
awarded is defined in /proc/sys/net/core/[rw]mem_default ; the maximum configurable
socket size is defined in /proc/sys/net/core/[rw]mem_max instead. Still on Linux, the
"drops" field of /proc/net/udp or /proc/net/udp6 can be checked to ensure its value
is not increasing.
DEFAULT: Operating System default
KEY: [ bgp_daemon_pipe_size | bmp_daemon_pipe_size ] [GLOBAL]
DESC: Defines the size of the kernel socket used for BGP and BMP messaging. The socket is
highlighted below with "XXXX":
XXXX
[network] ----> [kernel] ----> [core process] ----> [plugin] ----> [backend]
[__________pmacct___________]
On Linux systems, if this configuration directive is not specified default socket size
awarded is defined in /proc/sys/net/core/rmem_default ; the maximum configurable socket
size (which can be changed via sysctl) is defined in /proc/sys/net/core/rmem_max
instead.
DEFAULT: Operating System default
KEY: plugin_pipe_size
DESC: Core Process and each of the plugin instances are run into different processes. To
exchange data, they set up a circular queue (home-grown implementation, referred to
as 'pipe') and highlighted below with "XXXX":
XXXX
[network] ----> [kernel] ----> [core process] ----> [plugin] ----> [backend]
[__________pmacct___________]
This directive sets the total size, in bytes, of such queue. Its default size is set
to 4MB. Whenever facing heavy traffic loads, this size can be adjusted to hold more
data. In the following example, the queue between the Core process and the plugin
'test' is set to 10MB:
...
plugins: memory[test]
plugin_pipe_size[test]: 10240000
...
When enabling debug, log messages about obtained and target pipe sizes are printed.
If obtained is less than target, it could mean the maximum socket size granted by
the Operating System has to be increased. On Linux systems default socket size awarded
is defined in /proc/sys/net/core/[rw]mem_default ; the maximum configurable socket
size (which can be changed via sysctl) is defined in /proc/sys/net/core/[rw]mem_max
instead.
In case of data loss messages containing the "missing data detected" string will be
logged - indicating the plugin affected and current settings.
DEFAULT: 4MB
KEY: plugin_pipe_amqp
VALUES: [ true | false ]
DESC: By defining this directive to 'true', a RabbitMQ broker is used for queueing and
data exchange between the Core Process and the plugins. This is in alternative to
the home-grown circular queue implementation (see plugin_pipe_size description).
This directive, along with all other plugin_pipe_amqp_* directives, can be set
globally or apply on a per plugin basis (ie. it is a valid scenario, if multiple
plugins are instantiated, that some make use of home-grown queueing, while others
use RabbitMQ based queueing). For a quick comparison: while relying on a RabbitMQ
broker for queueing introduces an external dependency (rabbitmq-c library, RabbitMQ
server, etc.), it reduces the amount of setting needed by the home-grown circular
queue implementation. See QUICKSTART for some examples.
DEFAULT: false
KEY: plugin_pipe_kafka
VALUES: [ true | false ]
DESC: By defining this directive to 'true', a Kafka broker is used for queueing and data
exchange between the Core Process and the plugins. This is in alternative to the
home-grown circular queue implementation (see plugin_pipe_size description). This
directive, along with all other plugin_pipe_kafka_* directives, can be set globally
or apply on a per plugin basis (ie. it is a valid scenario, if multiple plugins are
instantiated, that some make use of home-grown queueing, while others use Kafka
based queueing). For a quick comparison: using a Kafka broker gives the same
benefits as using a RabbitMQ broker (read plugin_pipe_amqp); plus, Kafka scales
more than RabbitMQ. See QUICKSTART for some examples.
NOTES: librdkafka does not currently expose the file descriptor(s) used internally so to
allow an external poll() or select() against it/them. This is being worked on as
part of librdkafka issue #429: https://github.com/edenhill/librdkafka/issues/429 .
As a result of that, the plugin_pipe_kafka feature is limited to 'print' plugin as
a proof of concept.
DEFAULT: false
KEY: plugin_buffer_size
DESC: By defining the transfer buffer size, in bytes, this directive enables buffering of
data transfers between core process and active plugins. Once a buffer is filled, it
is delivered to the plugin. Setting a larger value may improve throughput (ie. amount
of CPU cycles required to transfer data); setting a smaller value may improve latency,
especially in scenarios with little data influx. It is disabled by default. If used
with the home-grown circular queue implemetation, the value has to be minor/equal to
the size defined by 'plugin_pipe_size' and keeping a ratio between 1:100 and 1:1000
among the two is considered good practice; the circular queue of plugin_pipe_size size
is partitioned in chunks of plugin_buffer_size; if used with the RabbitMQ broker based
queueing (ie. 'plugin_pipe_amqp: true') this directive sets the frame_max allowed by
the underlying RabbitMQ session.
DEFAULT: Set to the size of the smallest element to buffer
KEY: plugin_pipe_backlog
VALUES: [0 <= value < 100]
DESC: Expects the value to be a percentage. It creates a backlog of buffers on the pipe
before actually releasing them to the plugin. The strategy helps optimizing inter
process communications where plugins are quicker handling data than the Core process.
By default backlog is disabled; as with buffering in general, this feature should be
enabled with caution in lab and low-traffic environments.
DEFAULT: 0
KEY: plugin_pipe_check_core_pid
VALUES: [ true | false ]
DESC: When enabled (default), validates the sender of data at the plugin side. The check
consists in verifying that the sender PID matches the PID of the plugin parent
process. This is useful when plugin_pipe_amqp or plugin_pipe_kafka are enabled and
hence a broker sits between the daemon Core Process and the Plugins. The feature is
not inteded to be a security one; instead its objective is to limit impact of such
things like mis-configurations, daemons started twice with the same configuration,
etc.
DEFAULT: true
KEY: files_umask
DESC: Defines the mask for newly created files (log, pid, etc.) and their related directory
structure. A mask less than "002" is not accepted due to security reasons.
DEFAULT: 077
KEY: files_uid
DESC: Defines the system user id (UID) for files opened for writing (log, pid, etc.); this
is indeed possible only when running the daemon as super-user; by default this is left
untouched. This is also applied to any intermediary directory structure which might be
created.
DEFAULT: Operating System default (current user UID)
KEY: files_gid
DESC: Defines the system group id (GID) for files opened for writing (log, pid, etc.); this
is indeed possible only when running the daemon as super-user; by default this is left
untouched. This is also applied to any intermediary directory structure which might be
created.
DEFAULT: Operating System default (current user GID)
KEY: interface (-i) [GLOBAL, PMACCTD_ONLY]
DESC: Interface on which 'pmacctd' listens. If such directive isn't supplied, a libpcap
function is used to select a valid device. [ns]facctd can catch similar behaviour by
employing the [ns]facctd_ip directives; also, note that this directive is mutually
exclusive with 'pcap_savefile' (-I).
DEFAULT: Interface is selected by by the Operating System
KEY: pcap_savefile (-I) [GLOBAL, PMACCTD_ONLY]
DESC: File in libpcap savefile format from which read data (this is in alternative to binding
to an intervace). The file has to be correctly finalized in order to be read. As soon
as 'pmacctd' is finished with the file, it exits (unless the 'savefile_wait' option is
in place). The directive doesn't apply to [ns]facctd; to replay original NetFlow/sFlow
streams, a tool like TCPreplay can be used instead. The directive is mutually exclusive
with 'interface' (-i).
DEFAULT: none
KEY: interface_wait (-w) [GLOBAL, PMACCTD_ONLY]
VALUES: [ true | false ]
DESC: If set to true, this option causes 'pmacctd' to wait for the listening device to become
available; it will try to open successfully the device each few seconds. Whenever set to
false, 'pmacctd' will exit as soon as any error (related to the listening interface) is
detected.
DEFAULT: false
KEY: savefile_wait (-W) [GLOBAL, PMACCTD_ONLY]
VALUES: [ true | false ]
DESC: If set to true, this option will cause 'pmacctd' to wait indefinitely for a signal (ie.
CTRL-C when not daemonized or 'killall -9 pmacctd' if it is) after being finished with
the supplied libpcap savefile (pcap_savefile). It's particularly useful when inserting
fixed amounts of data into memory tables by keeping the daemon alive.
DEFAULT: false
KEY: promisc (-N) [GLOBAL, PMACCTD_ONLY]
VALUES: [ true | false ]
DESC: If set to true, puts the listening interface in promiscuous mode. It's mostly useful when
running 'pmacctd' in a box which is not a router, for example, when listening for traffic
on a mirroring port.
DEFAULT: true
KEY: imt_path (-p)
DESC: Specifies the full pathname where the memory plugin has to listen for client queries.
When multiple memory plugins are active, each one has to use its own file to communicate
with the client tool. Note that placing these files into a carefully protected directory
(rather than /tmp) is the proper way to control who can access the memory backend.
DEFAULT: /tmp/collect.pipe
KEY: imt_buckets (-b)
DESC: Defines the number of buckets of the memory table which is organized as a chained hash
table. A prime number is highly recommended. Read INTERNALS 'Memory table plugin' chapter
for further details.
DEFAULT: 32771
KEY: imt_mem_pools_number (-m)
DESC: Defines the number of memory pools the memory table is able to allocate; the size of each
pool is defined by the 'imt_mem_pools_size' directive. Here, a value of 0 instructs the
memory plugin to allocate new memory chunks as they are needed, potentially allowing the
memory structure to grow undefinitely. A value > 0 instructs the plugin to not try to
allocate more than the specified number of memory pools, thus placing an upper boundary
to the table size.
DEFAULT: 16
KEY: imt_mem_pools_size (-s)
DESC: Defines the size of each memory pool. For further details read INTERNALS 'Memory table
plugin'. The number of memory pools is defined by the 'imt_mem_pools_number' directive.
DEFAULT: 8192
KEY: syslog (-S)
VALUES: [ auth | mail | daemon | kern | user | local[0-7] ]
DESC: Enables syslog logging, using the specified facility.
DEFAULT: none (logging to stderr)
KEY: logfile
DESC: Enables logging to a file (bypassing syslog); expected value is a pathname
DEFAULT: none (logging to stderr)
KEY: [ amqp_host | plugin_pipe_amqp_host ]
DESC: Defines the AMQP/RabbitMQ broker IP. amqp_* directives refer to the broker used by an
AMQP plugin to purge data out; plugin_pipe_amqp_* directives refer to the broker used
by the core process to send data to plugins.
DEFAULT: localhost
KEY: [ bgp_daemon_msglog_amqp_host | bgp_table_dump_amqp_host | bmp_dump_amqp_host |
bmp_daemon_msglog_amqp_host | sfacctd_counter_amqp_host |
telemetry_daemon_msglog_amqp_host | telemetry_dump_amqp_host ] [GLOBAL]
DESC: See amqp_host. bgp_daemon_msglog_amqp_* directives refer to the broker used by the BGP
thread to stream data out; bgp_table_dump_amqp_* directives refer to the broker used
by the BGP thread to dump data out at regular time intervals; bmp_daemon_msglog_amqp_*
directives refer to the broker used by the BMP thread to stream data out; bmp_dump_amqp_*
directives refer to the broker used by the BMP thread to dump data out at regular time
intervals; sfacctd_counter_amqp_* directives refer to the broker used by sfacctd to
stream sFlow counter data out; telemetry_daemon_msglog_amqp_* directives refer to the
broker used by the Streaming Telemetry thread/daemon to stream data out;
telemetry_dump_amqp_* directives refer to the broker used by the Streaming Telemetry
thread/daemon to dump data out at regular time intervals.
DEFAULT: See amqp_host
KEY: [ amqp_vhost | plugin_pipe_amqp_vhost ]
DESC: Defines the AMQP/RabbitMQ server virtual host; see also amqp_host.
DEFAULT: "/"
KEY: [ bgp_daemon_msglog_amqp_vhost | bgp_table_dump_amqp_vhost | bmp_dump_amqp_vhost |
bmp_daemon_msglog_amqp_vhost | sfacctd_counter_amqp_vhost |
telemetry_daemon_msglog_amqp_vhost | telemetry_dump_amqp_vhost ] [GLOBAL]
DESC: See amqp_vhost; see also bgp_daemon_msglog_amqp_host.
DEFAULT: See amqp_vhost
KEY: [ amqp_user | plugin_pipe_amqp_user ]
DESC: Defines the username to use when connecting to the AMQP/RabbitMQ server; see also
amqp_host.
DEFAULT: guest
KEY: [ bgp_daemon_msglog_amqp_user | bgp_table_dump_amqp_user | bmp_dump_amqp_user |
bmp_daemon_msglog_amqp_user | sfacctd_counter_amqp_user |
telemetry_daemon_msglog_amqp_user | telemetry_dump_amqp_user ] [GLOBAL]
DESC: See amqp_user; see also bgp_daemon_msglog_amqp_host.
DEFAULT: See amqp_user
KEY: [ amqp_passwd | plugin_pipe_amqp_passwd ]
DESC: Defines the password to use when connecting to the server; see also amqp_host.
DEFAULT: guest
KEY: [ bgp_daemon_msglog_amqp_passwd | bgp_table_dump_amqp_passwd |
bmp_dump_amqp_passwd | bmp_daemon_msglog_amqp_passwd |
sfacctd_counter_amqp_passwd | telemetry_daemon_msglog_amqp_passwd |
telemetry_dump_amqp_passwd ]
[GLOBAL]
DESC: See amqp_passwd; see also bgp_daemon_msglog_amqp_host.
DEFAULT: See amqp_passwd
KEY: amqp_routing_key
DESC: Name of the AMQP routing key to attach to published data. Dynamic names are supported
through the use of variables, which are computed at the moment when data is purged to
the backend. The list of variables supported is:
$peer_src_ip Value of the peer_src_ip primitive of the record being processed.
$pre_tag Value of the tag primitive of the record being processed.
$post_tag Configured value of post_tag.
$post_tag2 Configured value of post_tag2.
See also amqp_host.
DEFAULT: 'acct'
KEY: [ bgp_daemon_msglog_amqp_routing_key | bgp_table_dump_amqp_routing_key |
bmp_daemon_msglog_amqp_routing_key | bmp_dump_amqp_routing_key |
sfacctd_counter_amqp_routing_key | telemetry_daemon_msglog_amqp_routing_key |
telemetry_dump_amqp_routing_key ] [GLOBAL]
DESC: See amqp_routing_key; see also bgp_daemon_msglog_amqp_host. Variables supported by
the configuration directives described in this section:
$peer_src_ip Value of the peer_src_ip primitive of the record being processed.
DEFAULT: none
KEY: [ amqp_routing_key_rr | kafka_topic_rr ]
DESC: Performs round-robin load-balancing over a set of AMQP routing keys or Kafka topics.
The base name for the string is defined by amqp_routing_key or kafka_topic. This key
accepts a positive int value. If, for example, amqp_routing_key is set to 'blabla'
and amqp_routing_key_rr to 3 then the AMQP plugin will round robin as follows:
message #1 -> blabla_0, message #2 -> blabla_1, message #3 -> blabla_2, message #4
-> blabla_0 and so forth. This works in the same fashion for kafka_topic. By default
the feature is disabled, meaning all messages are sent to the base AMQP routing key
or Kafka topic (or the default one, if no amqp_routing_key or kafka_topic is being
specified).
DEFAULT: 0
KEY: [ bgp_daemon_msglog_amqp_routing_key_rr | bgp_table_dump_amqp_routing_key_rr |
bmp_daemon_msglog_amqp_routing_key_rr | bmp_dump_amqp_routing_key_rr |
telemetry_daemon_msglog_amqp_routing_key_rr | telemetry_dump_amqp_routing_key_rr ]
[GLOBAL]
DESC: See amqp_routing_key_rr; see also bgp_daemon_msglog_amqp_host.
DEFAULT: See amqp_routing_key_rr
KEY: [ amqp_exchange | plugin_pipe_amqp_exchange ]
DESC: Name of the AMQP exchange to publish data; see also amqp_host.
DEFAULT: pmacct
KEY: [ bgp_daemon_msglog_amqp_exchange | bgp_table_dump_amqp_exchange |
bmp_daemon_msglog_amqp_exchange | bmp_dump_amqp_exchange |
sfacctd_counter_amqp_exchange | telemetry_daemon_msglog_amqp_exchange |
telemetry_dump_amqp_exchange ] [GLOBAL]
DESC: See amqp_exchange
DEFAULT: See amqp_exchange; see also bgp_daemon_msglog_amqp_host.
KEY: amqp_exchange_type
DESC: Type of the AMQP exchange to publish data. Currently only 'direct' and 'fanout' types are
supported; see also amqp_host.
DEFAULT: direct
KEY: [ bgp_daemon_msglog_amqp_exchange_type | bgp_table_dump_amqp_exchange_type |
bmp_daemon_msglog_amqp_exchange_type | bmp_dump_amqp_exchange_type |
sfactd_counter_amqp_exchange_type | telemetry_daemon_msglog_amqp_exchange_type |
telemetry_dump_amqp_exchange_type ] [GLOBAL]
DESC: See amqp_exchange_type; see also bgp_daemon_msglog_amqp_host.
DEFAULT: See amqp_exchange_type
KEY: amqp_persistent_msg
VALUES: [ true | false ]
DESC: Marks messages as persistent so that a queue content does not get lost if RabbitMQ restarts.
Note from RabbitMQ docs: "Marking messages as persistent doesn't fully guarantee that a
message won't be lost. Although it tells RabbitMQ to save message to the disk, there is
still a short time window when RabbitMQ has accepted a message and hasn't saved it yet.
Also, RabbitMQ doesn't do fsync(2) for every message -- it may be just saved to cache and
not really written to the disk. The persistence guarantees aren't strong, but it is more
than enough for our simple task queue."; see also amqp_host.
DEFAULT: false
KEY: [ bgp_daemon_msglog_amqp_persistent_msg | bgp_table_dump_amqp_persistent_msg |
bmp_daemon_msglog_amqp_persistent_msg | bmp_dump_amqp_persistent_msg |
sfacctd_counter_persistent_msg | telemetry_daemon_msglog_amqp_persistent_msg |
telemetry_dump_amqp_persistent_msg ] [GLOBAL]
VALUES: See amqp_persistent_msg; see also bgp_daemon_msglog_amqp_host.
DESC: See amqp_persistent_msg
DEFAULT: See amqp_persistent_msg
KEY: amqp_frame_max
DESC: Defines the maximum size, in bytes, of an AMQP frame on the wire to request of the broker
for the connection. 4096 is the minimum size, 2^31-1 is the maximum; see also amqp_host.
DEFAULT: 131072
KEY: [ bgp_daemon_msglog_amqp_frame_max | bgp_table_dump_amqp_frame_max |
bmp_daemon_msglog_amqp_frame_max | bmp_dump_amqp_frame_max |
sfacctd_counter_amqp_frame_max | telemetry_daemon_msglog_amqp_frame_max |
telemetry_dump_amqp_frame_max ] [GLOBAL]
DESC: See amqp_frame_max; see also bgp_daemon_msglog_amqp_host.
DEFAULT: See amqp_frame_max
KEY: amqp_heartbeat_interval
DESC: Defines the heartbeat interval in order to detect general failures of the RabbitMQ server.
The value is expected in seconds. By default the heartbeat mechanism is disabled with a
value of zero. According to RabbitMQ C API, detection takes place only upon publishing a
JSON message, ie. not at login or if idle. The maximum value supported is INT_MAX (or
2147483647); see also amqp_host.
DEFAULT: 0
KEY: [ bgp_daemon_msglog_amqp_heartbeat_interval | bgp_table_dump_amqp_heartbeat_interval |
bmp_daemon_msglog_amqp_heartbeat_interval | bmp_dump_amqp_heartbeat_interval |
sfacctd_counter_amqp_heartbeat_interval | telemetry_daemon_msglog_amqp_heartbeat_interval |
telemetry_dump_amqp_heartbeat_interval ] [GLOBAL]
DESC: See amqp_heartbeat_interval; see also bgp_daemon_msglog_amqp_host.
DEFAULT: See amqp_heartbeat_interval
KEY: plugin_pipe_amqp_routing_key
DESC: Name of the AMQP routing key to use to send data to a plugin. Currently each plugin
must bind to a different routing key in order to avoid duplications. Dynamic names
are supported through the use of variables, which are computed at startup. The list
of variables supported is:
$core_proc_name Configured core_proc_name.
$plugin_name Configured plugin name (ie. 'foo' if 'plugins: print[foo]')
$plugin_type Plugin type (ie. memory, print, amqp, etc.)
DEFAULT: '$core_proc_name-$plugin_name-$plugin_type'
KEY: plugin_pipe_amqp_retry
DESC: Defines the interval of time, in seconds, after which a connection to the RabbitMQ
server should be retried after a failure is detected; see also amqp_host.
DEFAULT: 60
KEY: [ bgp_daemon_msglog_amqp_retry | bmp_daemon_msglog_amqp_retry |
sfacctd_counter_amqp_retry | telemetry_daemon_msglog_amqp_retry ] [GLOBAL]
DESC: See plugin_pipe_amqp_retry; see also bgp_daemon_msglog_amqp_host.
DEFAULT: See plugin_pipe_amqp_retry
KEY: kafka_topic
DESC: Name of the Kafka topic to attach to published data. Dynamic names are supported by
kafka_topic through the use of variables, which are computed at the moment when data
is purged to the backend. The list of variables supported by amqp_routing_key:
$peer_src_ip Value of the peer_src_ip primitive of the record being processed.
$pre_tag Value of the tag primitive of the record being processed.
$post_tag Configured value of post_tag.
$post_tag2 Configured value of post_tag2.
DEFAULT: kafka_topic: 'pmacct.acct'
KEY: plugin_pipe_kafka_topic
DESC: Name of the Kafka topic to use to send data to a plugin. Currently each plugin must
bind to a different routing key in order to avoid duplications. Dynamic names are
supported through the use of variables, which are computed at startup. The list of
variables supported is:
$core_proc_name Configured core_proc_name.
$plugin_name Configured plugin name (ie. 'foo' if 'plugins: print[foo]')
$plugin_type Plugin type (ie. memory, print, amqp, etc.)
DEFAULT: '$core_proc_name-$plugin_name-$plugin_type'
KEY: plugin_pipe_kafka_retry
DESC: Defines the interval of time, in seconds, after which a connection to the Kafka
broker should be retried after a failure is detected.
DEFAULT: 60
KEY: [ kafka_broker_host | plugin_pipe_broker_host ]
DESC: Defines one or multiple, comma-separated, Kafka brokers. If only a single broker
IP address is defined then the broker port is read via the kafka_broker_port config
directive (legacy syntax); if multiple brokers are defined then each broker port,
if not left to default 9092, is expected as part of this directive, for example:
"broker1:10000,broker2". kafka_* directives refer to the broker used by a kafka
plugin to purge data out; plugin_pipe_kafka_* directives refer to the broker used
by the core process to send data to plugins. When defining multiple brokers, if the
host is IPv4, the value is expected as 'address:port'. If IPv6, it is expected as
'[address]:port'. When defining a single broker, this is not needed as the IPv6
address is detected and wrapped-around '[' ']' symbols.
DEFAULT: 127.0.0.1
KEY: [ kafka_broker_port | plugin_pipe_broker_port ]
DESC: Defines the Kafka broker port. See also kafka_broker_host.
DEFAULT: 9092
KEY: [ kafka_partition | plugin_pipe_kafka_partition ]
DESC: Defines the Kafka broker topic partition ID. RD_KAFKA_PARTITION_UA or ((int32_t)-1)
is to define the configured or default partitioner (slower than sending to a fixed
partition). See also kafka_broker_host.
DEFAULT: 0
KEY: kafka_partition_key
DESC: Defines the Kafka broker topic partition key. A string of printable characters is
expected as value.
DEFAULT: none
KEY: [ kafka_fallback | plugin_pipe_kafka_fallback ]
DESC: Defines the Kafka version to fallback to in order to preserve backward compatibility
as in https://github.com/edenhill/librdkafka/wiki/Broker-version-compatibility .
DEFAULT: none
KEY: [ bgp_daemon_msglog_kafka_broker_host | bgp_table_dump_kafka_broker_host |
bmp_daemon_msglog_kafka_broker_host | bmp_dump_kafka_broker_host |
sfacctd_counter_kafka_broker_host | telemetry_daemon_msglog_kafka_broker_host |
telemetry_dump_kafka_broker_host ] [GLOBAL]
DESC: See kafka_broker_host
DEFAULT: See kafka_broker_host
KEY: [ bgp_daemon_msglog_kafka_broker_port | bgp_table_dump_kafka_broker_port |
bmp_daemon_msglog_kafka_broker_port | bmp_dump_kafka_broker_port |
sfacctd_counter_kafka_broker_port | telemetry_daemon_msglog_kafka_broker_port |
telemetry_dump_kafka_broker_port ] [GLOBAL]
DESC: See kafka_broker_port
DEFAULT: See kafka_broker_port
KEY: [ bgp_daemon_msglog_kafka_topic | bgp_table_dump_kafka_topic |
bmp_daemon_msglog_kafka_topic | bmp_dump_kafka_topic |
sfacctd_counter_kafka_topic | telemetry_daemon_msglog_kafka_topic |
telemetry_dump_kafka_topic ] [GLOBAL]
DESC: See kafka_topic
DEFAULT: none
KEY: [ bgp_daemon_msglog_kafka_topic_rr | bgp_table_dump_kafka_topic_rr |
bmp_daemon_msglog_kafka_topic_rr | bmp_dump_kafka_topic_rr |
telemetry_daemon_msglog_kafka_topic_rr | telemetry_dump_kafka_topic_rr ]
[GLOBAL]
DESC: See kafka_topic_rr
DEFAULT: See kafka_topic_rr
KEY: [ bgp_daemon_msglog_kafka_partition | bgp_table_dump_kafka_partition |
bmp_daemon_msglog_kafka_partition | bmp_dump_kafka_partition |
sfacctd_counter_kafka_partition | telemetry_daemon_msglog_kafka_partition |
telemetry_dump_kafka_partition ] [GLOBAL]
DESC: See kafka_partition
DEFAULT: See kafka_partition
KEY: [ bgp_daemon_msglog_kafka_partition_key |
bgp_table_dump_kafka_partition_key |
bmp_daemon_msglog_kafka_partition_key | bmp_dump_kafka_partition_key |
sfacctd_counter_kafka_partition_key |
telemetry_daemon_msglog_kafka_partition_key |
telemetry_dump_kafka_partition_key ] [GLOBAL]
DESC: See kafka_partition_key
DEFAULT: See kafka_partition_key
KEY: [ bgp_daemon_msglog_kafka_fallback | bgp_table_dump_kafka_fallback |
bmp_daemon_msglog_kafka_fallback | bmp_dump_kafka_fallback |
sfacctd_counter_kafka_fallback | telemetry_daemon_msglog_kafka_fallback |
telemetry_dump_kafka_fallback ] [GLOBAL]
DESC: See kafka_fallback
DEFAULT: See kafka_fallback
KEY: [ bgp_daemon_msglog_kafka_retry | bmp_daemon_msglog_kafka_retry |
sfacctd_counter_kafka_retry | telemetry_daemon_msglog_kafka_retry ] [GLOBAL]
DESC: See plugin_pipe_kafka_retry
DEFAULT: See plugin_pipe_kafka_retry
KEY: pidfile (-F) [GLOBAL]
DESC: Writes PID of Core process to the specified file. PIDs of the active plugins are written
aswell by employing the following syntax: 'path/to/pidfile-<plugin_type>-<plugin_name>'.
This gets particularly useful to recognize which process is which on architectures where
pmacct does not support the setproctitle() function.
DEFAULT: none
KEY: networks_file (-n)
DESC: Full pathname to a file containing a list of networks - and optionally ASN information,
BGP next-hop (peer_dst_ip) and IP prefix labels (read more about the file syntax in
examples/networks.lst.example). Purpose of the feature is to act as a resolver when
network, next-hop and/or peer/origin ASN information is not available through other
means (ie. BGP, IGP, telemetry protocol) or for the purpose of overriding such
information with custom/self-defined one. IP prefix labels rewrite the resolved
source and/or destination IP prefix into the supplied label; labels can be up to 15
characters long.
DEFAULT: none
KEY: networks_file_filter
VALUES [ true | false ]
DESC: Makes networks_file work as a filter in addition to its basic resolver functionality:
networks and hosts not belonging to defined networks are zeroed out.
DEFAULT: false
KEY: networks_file_no_lpm
VALUES [ true | false ]
DESC: Makes a matching IP prefix defined in a networks_file win always, even if it is not
the longest. It applies when the aggregation method includes src_net and/or dst_net
and nfacctd_net (or equivalents) and/or nfacctd_as (or equivalents) configuration
directives are set to 'longest' (or 'fallback'). For example we receive the following
PDU via NetFlow:
SrcAddr: 10.0.8.29 (10.0.8.29)
DstAddr: 192.168.5.47 (192.168.5.47)
[ .. ]
SrcMask: 24 (prefix: 10.0.8.0/24)
DstMask: 27 (prefix: 192.168.5.32/27)
a BGP peering is available and BGP contains the following prefixes: 192.168.0.0/16 and
10.0.0.0/8. Such a scenario is typical when more specifics are not re-distributed in
BGP but are only available in the IGP. A networks_file contains the prefixes 10.0.8.0/24
and 192.168.5.0/24. 10.0.8.0/24 is the same as in NetFlow; but 192.168.5.0/24 (say,
representative of a range dedicated to a specific customer across several locations and
hence composed of several sub-prefies) would not be the longest match and hence the
prefix from NetFlow, 192.168.5.32/27, would be the outcome of the network aggregation
process; setting networks_file_no_lpm to true makes 192.168.5.0/24, coming from the
networks_file, win instead.
DEFAULT: false
KEY: networks_mask
DESC: Specifies the network mask - in bits - to apply to IP address values in L3 header. The
mask is applied sistematically and before evaluating the 'networks_file' content (if
any is specified).
DEFAULT: none
KEY: networks_cache_entries
DESC: Networks Lookup Table (which is the memory structure where the 'networks_file' data is
loaded) is preeceded by a Network Lookup Cache where lookup results are saved to speed
up later searches. NLC is structured as an hash table, hence, this directive is aimed to
set the number of buckets for the hash table. The default value should be suitable for
most common scenarios, however when facing with large-scale network definitions, it is
quite adviceable to tune this parameter to improve performances. A prime number is highly
recommended.
DEFAULT: IPv4: 99991; IPv6: 32771
KEY: ports_file
DESC: Full pathname to a file containing a list of (known/interesting/meaningful) ports (one
for each line, read more about the file syntax into examples/ tree). The directive allows
to rewrite as zero port numbers not matching any port defined in the list. Indeed, this
makes sense only if aggregating on either 'src_port' or 'dst_port' primitives.
DEFAULT: none
KEY: sql_db
DESC: Defines the SQL database to use. Remember that when using the SQLite3 plugin, this
directive refers to the full path to the database file
DEFAULT: 'pmacct'; SQLite 3.x: '/tmp/pmacct.db'
KEY: [ sql_table | print_output_file | mongo_table ]
DESC: In SQL and mongodb plugins this defines the table to use; in print plugin it defines the
file to write output to. Dynamic names are supported through the use of variables, which
are computed at the moment when data is purged to the backend. The list of supported
variables follows:
%d The day of the month as a decimal number (range 01 to 31).
%H The hour as a decimal number using a 24 hour clock (range 00 to 23).
%m The month as a decimal number (range 01 to 12).
%M The minute as a decimal number (range 00 to 59).
%s The number of seconds since Epoch, ie., since 1970-01-01 00:00:00 UTC.
%w The day of the week as a decimal, range 0 to 6, Sunday being 0.
%W The week number of the current year as a decimal number, range
00 to 53, starting with the first Monday as the first day of
week 01.
%Y The year as a decimal number including the century.
$ref Configured refresh time value for the plugin.
$hst Configured sql_history value, in seconds, for the plugin.
$peer_src_ip Record value for peer_src_ip primitive (if primitive is not part of
the aggregation method then this will be set to a null value).
$tag Record value for tag primitive ((if primitive is not part of the
aggregation method then this will be set to a null value).
$tag2 Record value for tag2 primitive ((if primitive is not part of the
aggregation method then this will be set to a null value).
SQL plugins notes:
Time-related variables require 'sql_history' to be specified in order to work correctly
(see 'sql_history' entry in this in this document for further information) and that the
'sql_refresh_time' setting is aligned with the 'sql_history', ie.:
sql_history: 5m
sql_refresh_time: 300
Furthermore, if the 'sql_table_schema' directive is not specified, tables are expected
to be already in place. This is an example on how to split accounted data among multiple
tables basing on the day of the week:
sql_history: 1h
sql_history_roundoff: h
sql_table: acct_v4_%w
The above directives will account data on a hourly basis (1h). Also the above sql_table
definition will make: Sunday data be inserted into the 'acct_v4_0' table, Monday into
the 'acct_v4_1' table, and so on. The switch between the tables will happen each day at
midnight: this behaviour is ensured by the use of the 'sql_history_roundoff' directive.
Ideally sql_refresh_time and sql_history values should be aligned for the dynamic tables
to work; sql_refresh_time with a value smaller than sql_history is also supported; whereas
the feature does not support values of sql_refresh_time greater than sql_history. The
maximum table name length is 64 characters.
Print plugin notes:
* if a non-dynamic filename is selected, content is overwritten to the existing one in
case print_output_file_append is set to false (default). Are supported scenarios where
multiple level of directories need to be created in order to create the target file,
ie. "/path/to/%Y/%Y-%m/%Y-%m-%d/blabla-%Y%m%d-%H%M.txt". Shell replacements are not
supported though, ie. '~' symbol to denote the user home directory. print_history
values are used for time-related variables substitution of dynamic print_output_file
names.
MongoDB plugin notes:
The table name is expected as <database>.<collection> . Default table is test.acct
Common notes:
The maximum number of variables it may contain is 32.
DEFAULT: see notes
KEY: print_output_file_append
VALUES: [ true | false ]
DESC: If set to true, print plugin will append to existing files instead of overwriting. If
appending, and in case of an output format requiring a title, ie. csv, formatted, etc.,
intuitively the title is not re-printed.
DEFAULT: false
KEY: print_output_lock_file
DESC: If no print_output_file is defined (ie. print plugin output goes to stdout), this
directive defined a global lock to serialize output to stdout, ie. in cases where
multiple print plugins are defined or purging events of the same plugin queue up.
By default output is not serialized and a warning message is printed to flag the
condition.
KEY: print_latest_file
DESC: Defines the full pathname to pointer(s) to latest file(s). Dynamic names are supported
through the use of variables, which are computed at the moment when data is purged to the
backend: refer to print_output_file for a full listing of supported variables; time-based
variables are not allowed. Three examples follow:
#1:
print_output_file: /path/to/spool/foo-%Y%m%d-%H%M.txt
print_latest_file: /path/to/spool/foo-latest
#2:
print_output_file: /path/to/spool/%Y/%Y-%m/%Y-%m-%d/foo-%Y%m%d-%H%M.txt
print_latest_file: /path/to/spool/latest/foo
#3:
print_output_file: /path/to/$peer_src_ip/foo-%Y%m%d-%H%M.txt
print_latest_file: /path/to//spool/latest/blabla-$peer_src_ip
NOTES: Update of the latest pointer is done evaluating files name. For correct working of the
feature, responsibility is put on the user. A file is reckon as latest if it is
lexicographically greater than an existing one: this is generally fine but requires
dates to be in %Y%m%d format rather than %d%m%Y. Also, upon restart of the daemon, if
print_output_file is modified to a different location good practice would be to 1)
manually delete latest pointer(s) or 2) move existing print_output_file files to the
new targer location. Finally, if upgrading from pmacct releases before 1.5.0rc1, it is
recommended to delete existing symlinks.
DEFAULT: none
KEY: sql_table_schema
DESC: Full pathname to a file containing a SQL table schema. It allows to create the SQL table
if it does not exist; this directive makes sense only if a dynamic 'sql_table' is in use.
A configuration example where this directive could be useful follows:
sql_history: 5m
sql_history_roundoff: h
sql_table: acct_v4_%Y%m%d_%H%M
sql_table_schema: /usr/local/pmacct/acct_v4.schema
In this configuration, the content of the file pointed by 'sql_table_schema' should be:
CREATE TABLE acct_v4_%Y%m%d_%H%M (
[ ... PostgreSQL/MySQL specific schema ... ]
);
This setup, along with this directive, are mostly useful when the dynamic tables are not
closed in a 'ring' fashion (e.g., the days of the week) but 'open' (e.g., current date).
DEFAULT: none
KEY: sql_table_version (-v)
VALUES [ 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 ]
DESC: Defines the version of the SQL table. SQL table versioning was introduced to achieve two
goals: a) make tables work out-of-the-box for the SQL beginners, smaller installations
and quick try-outs; and in this context b) to allow introduction of new features over
time without breaking backward compatibility. For the SQL experts, the alternative to
versioning is 'sql_optimize_clauses' which allows custom mix-and-match of primitives:
in such a case you have to build yourself custom SQL schemas and indexes. Check in the
'sql/' sub-tree the SQL table profiles which are supported by the pmacct version you are
currently using. It is always adviced to explicitely define a sql_table_version in
order to predict which primitive will be written to which column. All versioning rules
are captured in sql/README.[mysql|sqlite3|pgsql] documents.
DEFAULT: 1
KEY: sql_table_type
VALUES [ original | bgp ]
DESC: BGP-related primitives are divided in legacy and non-legacy. Legacy are src_as, dst_as;
non-legacy are all the rest. Up to "original" tables v5 src_as and dst_as were written
in the same field as src_host and dst_host. From "original" table v6 and if sql_table_type
"bgp" is selected, src_as and dst_as are written in their own field (as_src and as_dst
respectively). sql_table_type is by default set to "original" and is switched to "bgp"
automatically if any non-legacy primitive is in use, ie. peer_dst_ip, as_path, etc. This
directive allows to make the selection explicit and/or circumvent default behaviour.
Apart from src_as and dst_as, regular table versioning applies to all non-BGP related
fields, for example: a) if "sql_table_type: bgp" and "sql_table_version: 1" then the "tag"
field will be written in the "agent_id" column whereas; b) if "sql_table_type: bgp" and
"sql_table_version: 9" instead, then the "tag" field will be written in the "tag" column.
All versioning rules are captured in sql/README.[mysql|sqlite3|pgsql] documents.
DEFAULT: original
KEY: sql_data
VALUES: [ typed | unified ]
DESC: This switch makes sense only when using PostgreSQL plugin and supplied default tables
up to v5: the pgsql scripts in the sql/ tree, up to v5, will in fact create a 'unified'
table along with multiple 'typed' tables. The 'unified' table has IP and MAC addresses
specified as standard CHAR strings, slower and not space savy but flexible; 'typed'
tables sport PostgreSQL own types (inet, mac, etc.), resulting in a faster but more
rigid structure. Since v6 unified mode is being discontinued leading to simplification.
The supplied 'typed' schema can still be customized, ie. to write IP addresses in CHAR
fields because making use of IP prefix labels, transparently to pmacct - making this
configuration switch deprecated.
DEFAULT: typed
KEY: [ sql_host | mongo_host ]
DESC: Defines the backend server IP/hostname
DEFAULT: localhost