diff --git a/CHANGES b/CHANGES index 027c63b8..4da0d8f5 100644 --- a/CHANGES +++ b/CHANGES @@ -1,3 +1,86 @@ +2.13.0-46 | 2023-06-30 12:47:34 -0700 + + * Update man page with changes to other documentation (Tim Wojtulewicz, Corelight) + + * Remove references to bro-pkg.meta and bro-pkg.index (Tim Wojtulewicz, Corelight) + + * Remove references to legacy bro naming (Tim Wojtulewicz, Corelight) + + This intentionally leaves behind references to bro-pkg.meta and __bro_plugin__ + but removes all of the other references to scripts and files that have been + deprecated and/or removed since Zeek 3.0. + + * User-visible warning when bro-pkg.meta is in use. (Arne Welzel, Corelight) + + Relates to #119 + + * bundle/unbundle: Rework built-in dependency check (Arne Welzel, Corelight) + + Extend unbundle to validate dependencies of contained packages and log a + warning if any are not fulfilled or fail to validate. This can happen when + a bundle is created with --nodeps or the target system has different + built-in packages as compared to the source system. + + Built-in packages are included in a bundle's manifest.txt file in a new + [meta] section. + + * manager: Do not include built-in packages in manifest (Arne Welzel, Corelight) + + Bugfix and test for mistakenly recording built-in packages within + the manifest.json file. The built-in information is only ever + requested from Zeek. + + * package: Add PackageVersion (Arne Welzel, Corelight) + + Make checking if a package version fullfills a version_spec re-usable. + + This certainly changes the messages produced, but doesn't seem there + are many tests upset about that. + + * Add built-in package awareness (Arne Welzel, Corelight) + + This change uses Zeek 6.0's --build-info flags to discover the "zkg.provides" + entry and makes ``zkg`` aware of these as installed packages. + + It introduces the following changes + + * A pseudo zeek-builtin:// scheme for git_urls to recognize when a + Package instance relates to a built-in package + * The package source name "zeek-builtin" is now reserved + * A zkg bundle's manifest now contains a new section called `[bundle_builtin]` + listing any packages that are expected to exists as built-in packages on + the target system. The content is derived from the built-in packages on + the system building the bundle. This is somewhat narrow and opinionated, + but at the same time avoids the complexity of any constructed scenarios + one can come up with here. + * A new tracking method "builtin" exists. It should act as version but + makes it explicit that a package is built-in. + + Most commands have been extended to either ignore (purge) or fail + (install, remove, test, load, unload, pin, unpin) when they detect + a built-in package being operated on. The commands list and info + received a ``--include-builtin`` option for explicitly including + built-in packages in the output. + + * zkg: Log warnings by default (Arne Welzel, Corelight) + + Currently, to see warnings generated by zkg a user has to pass a single -v. + Change this behavior by always installing the StreamHandler() with a + warning level so that warnings are displayed by default. + + * package: Add a few __repr__() implementations (Arne Welzel, Corelight) + + Not having __repr__() implemented makes it more difficult than needed + when using print / IPython shell to look around and discover state. + Implement some of them to aid that development style. + + They aren't perfect, but better than not having them at all. + + * _util: Add ZeekInfo helper class (Arne Welzel, Corelight) + + Mostly for collecting executable paths for now, but could also see + the ZEEKPATH collections being done there down the road. + 2.13.0-32 | 2023-06-20 15:55:21 -0700 * Add --ignore-dirty-git option to test and install commands (Tim Wojtulewicz, Corelight) diff --git a/VERSION b/VERSION index 43a762f7..524edf47 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.13.0-32 +2.13.0-46 diff --git a/doc/man/zkg.1 b/doc/man/zkg.1 index c6a89a15..855e1c56 100644 --- a/doc/man/zkg.1 +++ b/doc/man/zkg.1 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH "ZKG" "1" "Jun 20, 2023" "2.13.0-32" "Zeek Package Manager" +.TH "ZKG" "1" "Jun 30, 2023" "2.13.0-46" "Zeek Package Manager" .SH NAME zkg \- Zeek Package Manager . @@ -62,7 +62,7 @@ See \fI\%Config File\fP\&. Store all state in user\(aqs home directory. Precludes \-\-configfile. .TP .B \-\-verbose=0\fP,\fB \-v=0 -Increase program output for debugging. Use multiple times for more output (e.g. \-vvv). +Increase program output for debugging. Use multiple times for more output (e.g. \-vv). .TP .B \-\-extra\-source Add an extra source. @@ -99,7 +99,7 @@ usage: zkg test [\-h] [\-\-version VERSION] [\-\-ignore\-dirty\-git] package [pa .INDENT 7.0 .TP .B package -The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in either "alice/zkg.index" or "alice/bro\-pkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". +The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in "alice/zkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". .UNINDENT .TP .B Options: @@ -134,7 +134,7 @@ usage: zkg install [\-h] [\-\-skiptests] [\-\-nodeps] [\-\-nosuggestions] [\-\-v .INDENT 7.0 .TP .B package -The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in either "alice/zkg.index" or "alice/bro\-pkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". +The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in "alice/zkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". .UNINDENT .TP .B Options: @@ -182,7 +182,7 @@ usage: zkg remove [\-h] [\-\-force] [\-\-nodeps] package [package ...] .INDENT 7.0 .TP .B package -The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in either "alice/zkg.index" or "alice/bro\-pkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". +The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in "alice/zkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". .UNINDENT .TP .B Options: @@ -326,7 +326,7 @@ usage: zkg refresh [\-h] [\-\-aggregate] [\-\-fail\-on\-aggregate\-problems] [\- .INDENT 7.0 .TP .B \-\-aggregate=False -Crawls the urls listed in package source zkg.index (or legacy bro\-pkg.index) files and aggregates the metadata found in their zkg.meta (or legacy bro\-pkg.meta) files. The aggregated metadata is stored in the local clone of the package source that zkg uses internally for locating package metadata. For each package, the metadata is taken from the highest available git version tag or the default branch, like "main" or "master", if no version tags exist +Crawls the urls listed in package source zkg.index files and aggregates the metadata found in their zkg.meta files. The aggregated metadata is stored in the local clone of the package source that zkg uses internally for locating package metadata. For each package, the metadata is taken from the highest available git version tag or the default branch, like "main" or "master", if no version tags exist .TP .B \-\-fail\-on\-aggregate\-problems=False When using \-\-aggregate, exit with error when any packages trigger metadata problems. Normally such problems only cause a warning. @@ -359,7 +359,7 @@ usage: zkg upgrade [\-h] [\-\-skiptests] [\-\-nodeps] [\-\-nosuggestions] [\-\-f .INDENT 7.0 .TP .B package -The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in either "alice/zkg.index" or "alice/bro\-pkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". +The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in "alice/zkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". .UNINDENT .TP .B Options: @@ -431,7 +431,7 @@ usage: zkg unload [\-h] [\-\-force] [\-\-nodeps] package [package ...] .INDENT 7.0 .TP .B package -The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in either "alice/zkg.index" or "alice/bro\-pkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". +The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in "alice/zkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". .UNINDENT .TP .B Options: @@ -464,7 +464,7 @@ usage: zkg pin [\-h] package [package ...] .INDENT 7.0 .TP .B package -The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in either "alice/zkg.index" or "alice/bro\-pkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". +The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in "alice/zkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". .UNINDENT .UNINDENT .SS unpin @@ -487,7 +487,7 @@ usage: zkg unpin [\-h] package [package ...] .INDENT 7.0 .TP .B package -The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in either "alice/zkg.index" or "alice/bro\-pkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". +The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in "alice/zkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". .UNINDENT .UNINDENT .SS list @@ -499,7 +499,7 @@ Outputs a list of packages that match a given category. .sp .nf .ft C -usage: zkg list [\-h] [\-\-nodesc] [{all,installed,not_installed,loaded,unloaded,outdated}] +usage: zkg list [\-h] [\-\-nodesc] [\-\-include\-builtin] [{all,installed,not_installed,loaded,unloaded,outdated}] .ft P .fi .UNINDENT @@ -520,6 +520,9 @@ Possible choices: all, installed, not_installed, loaded, unloaded, outdated .TP .B \-\-nodesc=False Do not display description text, just the package name(s). +.TP +.B \-\-include\-builtin=False +Also output packages that Zeek has built\-in. By default these are not shown. .UNINDENT .UNINDENT .SS search @@ -554,7 +557,8 @@ Shows detailed information/metadata for given packages. If the package is curren .sp .nf .ft C -usage: zkg info [\-h] [\-\-version VERSION] [\-\-nolocal] [\-\-json] [\-\-jsonpretty SPACES] [\-\-allvers] +usage: zkg info [\-h] [\-\-version VERSION] [\-\-nolocal] [\-\-include\-builtin] [\-\-json] [\-\-jsonpretty SPACES] + [\-\-allvers] package [package ...] .ft P .fi @@ -566,7 +570,7 @@ usage: zkg info [\-h] [\-\-version VERSION] [\-\-nolocal] [\-\-json] [\-\-jsonpr .INDENT 7.0 .TP .B package -The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in either "alice/zkg.index" or "alice/bro\-pkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". If a single name is given and matches one of the same categories as the "list" command, then it is automatically expanded to be the names of all packages which match the given category. +The name(s) of package(s) to operate on. The package may be named in several ways. If the package is part of a package source, it may be referred to by the base name of the package (last component of git URL) or its path within the package source. If two packages in different package sources have conflicting paths, then the package source name may be prepended to the package path to resolve the ambiguity. A full git URL may also be used to refer to a package that does not belong to a source. E.g. for a package source called "zeek" that has a package named "foo" located in "alice/zkg.index", the following names work: "foo", "alice/foo", "zeek/alice/foo". If a single name is given and matches one of the same categories as the "list" command, then it is automatically expanded to be the names of all packages which match the given category. .UNINDENT .TP .B Options: @@ -578,6 +582,9 @@ The version of the package metadata to inspect. A version tag, branch name, or .B \-\-nolocal=False Do not read information from locally installed packages. Instead read info from remote GitHub. .TP +.B \-\-include\-builtin=False +Also output packages that Zeek has built\-in. By default these are not shown. +.TP .B \-\-json=False Output package information as JSON. .TP @@ -597,7 +604,7 @@ The default output of this command is a valid package manager config file that c .sp .nf .ft C -usage: zkg config [\-h] [{all,sources,user_vars,state_dir,script_dir,plugin_dir,bin_dir,zeek_dist,bro_dist}] +usage: zkg config [\-h] [{all,sources,user_vars,state_dir,script_dir,plugin_dir,bin_dir,zeek_dist}] .ft P .fi .UNINDENT @@ -610,12 +617,12 @@ usage: zkg config [\-h] [{all,sources,user_vars,state_dir,script_dir,plugin_dir, .B config_param Name of a specific config file field to output. .sp -Possible choices: all, sources, user_vars, state_dir, script_dir, plugin_dir, bin_dir, zeek_dist, bro_dist +Possible choices: all, sources, user_vars, state_dir, script_dir, plugin_dir, bin_dir, zeek_dist .UNINDENT .UNINDENT .SS autoconfig .sp -The output of this command is a valid package manager config file that is generated by using the \fBzeek\-config\fP script that is installed along with Zeek. It is the suggested configuration to use for most Zeek installations. For this command to work, the \fBzeek\-config\fP (or \fBbro\-config\fP) script must be in \fBPATH\fP, unless the \-\-user option is given, in which case this creates a config that does not touch the Zeek installation. +The output of this command is a valid package manager config file that is generated by using the \fBzeek\-config\fP script that is installed along with Zeek. It is the suggested configuration to use for most Zeek installations. For this command to work, the \fBzeek\-config\fP script must be in \fBPATH\fP, unless the \-\-user option is given, in which case this creates a config that does not touch the Zeek installation. .INDENT 0.0 .INDENT 3.5 @@ -638,7 +645,7 @@ Skip any confirmation prompt. .UNINDENT .SS env .sp -This command returns shell commands that, when executed, will correctly set \fBZEEKPATH\fP and \fBZEEK_PLUGIN_PATH\fP (also \fBBROPATH\fP and \fBBRO_PLUGIN_PATH\fP for legacy compatibility) to use scripts and plugins from packages installed by the package manager. For this command to function properly, either have the \fBzeek\-config\fP script (installed by zeek) in \fBPATH\fP, or have the \fBZEEKPATH\fP and \fBZEEK_PLUGIN_PATH\fP (or \fBBROPATH\fP and \fBBRO_PLUGIN_PATH\fP) environment variables already set so this command can append package\-specific paths to them. +This command returns shell commands that, when executed, will correctly set \fBZEEKPATH\fP and \fBZEEK_PLUGIN_PATH\fP to use scripts and plugins from packages installed by the package manager. For this command to function properly, either have the \fBzeek\-config\fP script (installed by zeek) in \fBPATH\fP, or have the \fBZEEKPATH\fP and \fBZEEK_PLUGIN_PATH\fP environment variables already set so this command can append package\-specific paths to them. .INDENT 0.0 .INDENT 3.5 @@ -776,7 +783,7 @@ state_dir = # A subdirectory named "packages" is always created within the # specified path and the package manager will copy the directory # specified by the "script_dir" option of each package\(aqs zkg.meta -# (or legacy bro\-pkg.meta) file there. +# file there. # If left blank or with \-\-user this defaults to /script_dir. # In Zeek\-bundled installations, it defaults to # /share/zeek/site. @@ -789,7 +796,7 @@ script_dir = # A subdirectory named "packages" is always created within the # specified path and the package manager will copy the directory # specified by the "plugin_dir" option of each package\(aqs zkg.meta -# (or legacy bro\-pkg.meta) file there. +# file there. # If left blank or with \-\-user this defaults to /plugin_dir. # In Zeek\-bundled installations, it defaults to # /lib/zeek/plugins. @@ -808,7 +815,8 @@ bin_dir = # The directory containing Zeek distribution source code. This is only # needed when installing packages that contain Zeek plugins that are -# not pre\-built. The legacy name of this option is "bro_dist". +# not pre\-built. This value is generally not needed by most users other +# than plugin developers anymore. zeek_dist = [templates] @@ -819,13 +827,13 @@ default = https://github.com/zeek/package\-template [user_vars] -# For any key in this section that is matched for value interpolation -# in a package\(aqs zkg.meta (or legacy bro\-pkg.meta) file, the corresponding -# value is substituted during execution of the package\(aqs \(gabuild_command\(ga. -# This section is typically automatically populated with the -# the answers supplied during package installation prompts -# and, as a convenience feature, used to recall the last\-used settings -# during subsequent operations (e.g. upgrades) on the same package. +# For any key in this section that is matched for value interpolation in a +# package\(aqs zkg.meta file, the corresponding value is substituted during +# execution of the package\(aqs \(gabuild_command\(ga. This section is typically +# automatically populated with the the answers supplied during package +# installation prompts and, as a convenience feature, used to recall the +# last\-used settings during subsequent operations (e.g. upgrades) on the same +# package. .ft P .fi diff --git a/doc/package.rst b/doc/package.rst index ac48a61c..80c3845d 100644 --- a/doc/package.rst +++ b/doc/package.rst @@ -22,12 +22,6 @@ This is the package's metadata file in INI file format and may contain :ref:`additional fields ` that describe the package as well as how it inter-operates with Zeek, the package manager, or other packages. -.. note:: - - :file:`zkg.meta` is the canonical metadata file name used :program:`since - zkg v2.0`. The previous metadata file name of :file:`bro-pkg.meta` is also - accepted when no :file:`zkg.meta` exists. - .. _package-shorthand-name: Note that the shorthand name for your package that may be used by :ref:`zkg @@ -36,7 +30,7 @@ last component of its git URL. E.g. a package at ``https://github.com/zeek/foo`` may be referred to as **foo** when using :program:`zkg` and a Zeek script that wants to load all the scripts within that package can use: -.. code-block:: bro +.. code-block:: zeek @load foo @@ -229,11 +223,6 @@ though the following step are the essentials needed to create a package. script_dir = scripts/Demo/Rot13 build_command = ./configure && make - .. note:: - - See :ref:`legacy-bro-support` for notes on configuring packages to - support Bro 2.5 or earlier. - #. Add example script code: .. code-block:: console @@ -257,9 +246,7 @@ though the following step are the essentials needed to create a package. *p = (*p - b + 13) % 26 + b; } - BroString* bs = new BroString(1, reinterpret_cast(rot13), - strlen(rot13)); - return new StringVal(bs); + return make_intrusive(strlen(rot13), rot13); %} #. Commit everything to git: @@ -576,11 +563,6 @@ An example :file:`zkg.meta`:: script_dir = scripts/Demo/Rot13 build_command = ./configure && make -.. note:: - - See :ref:`legacy-bro-support` for notes on configuring packages to - support Bro 2.5 or earlier. - The default CMake skeleton for Zeek plugins will use :file:`build/` as the directory for the final/built version of the plugin, which matches the defaulted value of the omitted `plugin_dir` metadata field. @@ -591,31 +573,6 @@ script components, the "plugin" part is always unconditionally loaded by Zeek, but the "script" components must either be explicitly loaded (e.g. :samp:`@load {}`) or the package marked as :ref:`loaded `. -.. _legacy-bro-support: - -Supporting Older Bro Versions -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Plugin skeletons generated before Bro v2.6 and also any packages -that generally want to support such Bro versions need to pass -an additional configuration option such as:: - - build_command = ./configure --bro-dist=%(bro_dist)s && make - -See the :ref:`Value Interpolation ` -section for more information on what the ``%(bro_dist)s`` -string does, but a brief explanation is that it will expand to -a path containing the Bro source-code on the user's system. -For newer versions of Bro, packages are able to work entirely -with the installation path and don't require original source code. - -Also note that other various Zeek scripting and CMake infrastructure may -have changed between Bro v2.6 and Zeek v3.0. So if you plan to support -older version of Bro (before the Zeek rename), then you should keep an eye -out for various things that got renamed. For example, the `zeek_init` event -won't exist in any version before Zeek v3.0, nor will any CMake macros -that start with `zeek_plugin`. - .. _metadata-interpolation: Value Interpolation @@ -624,18 +581,14 @@ Value Interpolation The `build_command field`_ may reference the settings any given user has in their customized :ref:`package manager config file `. -For example, if a metadata field's value contains the ``%(bro_dist)s`` string, +For example, if a metadata field's value contains the ``%(zeek_dist)s`` string, then :program:`zkg` operations that use that field will automatically -substitute the actual value of `bro_dist` that the user has in their local +substitute the actual value of `zeek_dist` that the user has in their local config file. Note the trailing 's' character at the end of the interpolation -string, ``%(bro_dist)s``, is intended/necessary for all such interpolation -usages. Note that :program:`since zkg v2.0`, `zeek_dist` is the canonical name -for `bro_dist` within the :ref:`zkg config file `, -but either one means the same thing and should work. To support older -versions of :program:`bro-pkg`, you'd want to use `bro_dist` in package -metadata files. - -Besides the `bro_dist`/`zeek_dist` config keys, any key inside the +string, ``%(zeek_dist)s``, is intended/necessary for all such interpolation +usages. + +Besides the `zeek_dist` config key, any key inside the `user_vars` sections of their :ref:`package manager config file ` that matches the key of an entry in the package's `user_vars field`_ will be interpolated. @@ -777,24 +730,17 @@ An example :file:`zkg.meta`:: The field is a list of dependency names and their version requirement specifications. -A dependency name may be either `zeek`, `zkg`, `bro`, `bro-pkg`, +A dependency name may be either `zeek`, `zkg`, a full git URL of the package, or a :ref:`package shorthand name `. -- The special `zeek` and `bro` dependencies refers not to a package, - but the version of Zeek that the package requires in order to function. If - the user has :program:`zeek-config` or :program:`bro-config` in their - :envvar:`PATH` when installing/upgrading a package that specifies a `zeek` or - `bro` dependency, then :program:`zkg` will enforce that the requirement is - satisfied. +- The special `zeek` dependency refers not to a package, but the version of Zeek + that the package requires in order to function. If the user has + :program:`zeek-config` in their :envvar:`PATH` when installing/upgrading a + package that specifies a `zeek` dependency, then :program:`zkg` will enforce + that the requirement is satisfied. - .. note:: - - In this context, `zeek` and `bro` mean the same thing -- the - later is maintained for backwards compatibility while the former - became available :program:`since zkg v2.0`. - -- The special `zkg` and `bro-pkg` dependencies refers to the version of the +- The special `zkg` dependency refers to the version of the package manager that is required by the package. E.g. if a package takes advantage of new features that are not present in older versions of the package manager, then it should indicate that so users of those old version @@ -802,13 +748,6 @@ a full git URL of the package, or a :ref:`package shorthand name error/exception, or worse, seeing no errors, but without the desired functionality being performed. - .. note:: - - This feature itself, via use of a `bro-pkg` dependency, is only - available :program:`since bro-pkg v1.2` while a `zkg` dependency is only - recognized :program:`since zkg v2.0`. Otherwise, `zkg` and `bro-pkg` mean - the same thing in this context. - - The full git URL may be directly specified in the `depends` metadata if you want to force the dependency to always resolve to a single, canonical git repository. Typically this is the safe approach to take when listing diff --git a/doc/quickstart.rst b/doc/quickstart.rst index 942ec4b7..dd356bc2 100644 --- a/doc/quickstart.rst +++ b/doc/quickstart.rst @@ -141,7 +141,7 @@ have Zeek automatically load the scripts from all :ref:`installed ` packages that are also marked as ":ref:`loaded `" add: -.. code-block:: bro +.. code-block:: zeek @load packages diff --git a/doc/source.rst b/doc/source.rst index 86705b6b..ae39a5cc 100644 --- a/doc/source.rst +++ b/doc/source.rst @@ -22,12 +22,6 @@ usernames or some unique way of identifying the organization/person that maintains Zeek packages. However, a source is free to use a flat organization with a single, top-level :file:`zkg.index`. -.. note:: - - The magic index file name of :file:`zkg.index` is available :program:`since - zkg v2.0`. For compatibility purposes, the old index file name of - :file:`bro-pkg.index` is also still supported. - After creating a git repo for the package source and adding package index files to it, it's ready to be used by :ref:`zkg `. @@ -36,10 +30,9 @@ to it, it's ready to be used by :ref:`zkg `. Package Index Files ------------------- -Files named :file:`zkg.index` (or the legacy :file:`bro-pkg.index`) are used to -describe the :doc:`Zeek Packages ` found within the package source. -They are simply a list of git URLs pointing to the git repositories of -packages. For example:: +Files named :file:`zkg.index` are used to describe the :doc:`Zeek Packages +` found within the package source. They are simply a list of git URLs +pointing to the git repositories of packages. For example:: https://github.com/zeek/foo https://github.com/zeek/bar @@ -75,9 +68,8 @@ Aggregating Metadata -------------------- The maintainer/operator of a package source may choose to periodically aggregate -the metadata contained in its packages' :file:`zkg.meta` (and legacy -:file:`bro-pkg.meta`) files. The :ref:`zkg refresh ` -is used to perform the task. For example: +the metadata contained in its packages' :file:`zkg.meta` files. The :ref:`zkg +refresh ` is used to perform the task. For example: .. code-block:: console diff --git a/doc/zkg.rst b/doc/zkg.rst index 3af3edd4..d8892037 100644 --- a/doc/zkg.rst +++ b/doc/zkg.rst @@ -1,5 +1,3 @@ -.. _bro-pkg: - .. _zkg: zkg Command-Line Tool @@ -217,8 +215,6 @@ template info :prog: zkg :path: template info -.. _bro-pkg-config-file: - .. _zkg-config-file: Config File diff --git a/testing/baselines/tests.install-bro-pkg-warning/stderr b/testing/baselines/tests.install-bro-pkg-warning/stderr deleted file mode 100644 index 6cfda2a7..00000000 --- a/testing/baselines/tests.install-bro-pkg-warning/stderr +++ /dev/null @@ -1,2 +0,0 @@ -### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. -XXXX-XX-XX XX:XX:XX WARNING Package one/alice/foo is using the legacy bro-pkg.meta metadata file. It will soon stop working unless updated to use zkg.meta instead. Please report this to the package maintainers. diff --git a/testing/baselines/tests.refresh/agg.errout b/testing/baselines/tests.refresh/agg.errout index 0bcb3dec..77a5925b 100644 --- a/testing/baselines/tests.refresh/agg.errout +++ b/testing/baselines/tests.refresh/agg.errout @@ -1,8 +1,8 @@ ### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. XXXX-XX-XX XX:XX:XX DEBUG found source clone of "one" at <...>/one XXXX-XX-XX XX:XX:XX DEBUG refresh "one": pulling <...>/one -XXXX-XX-XX XX:XX:XX WARNING <...>/bro-pkg.meta: missing metadata file -XXXX-XX-XX XX:XX:XX WARNING skipping aggregation of <...>/bad_pkg: bad metadata: missing zkg.meta (or bro-pkg.meta) metadata file +XXXX-XX-XX XX:XX:XX WARNING <...>/zkg.meta: missing metadata file +XXXX-XX-XX XX:XX:XX WARNING skipping aggregation of <...>/bad_pkg: bad metadata: missing zkg.meta metadata file XXXX-XX-XX XX:XX:XX DEBUG metadata refresh: 8 additions (alice/bar, alice/baz, alice/foo, alice/i-have-no-scripts, alice/new_pkg, alice/qux, bob/corge, bob/grault), 0 changes, 0 removals XXXX-XX-XX XX:XX:XX INFO committed package source "one" metadata update XXXX-XX-XX XX:XX:XX DEBUG fetch package one<...>/foo diff --git a/testing/baselines/tests.refresh/agg.fail.errout b/testing/baselines/tests.refresh/agg.fail.errout index cbe7ff0f..43e80165 100644 --- a/testing/baselines/tests.refresh/agg.fail.errout +++ b/testing/baselines/tests.refresh/agg.fail.errout @@ -1,7 +1,7 @@ ### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. XXXX-XX-XX XX:XX:XX DEBUG found source clone of "one" at <...>/one XXXX-XX-XX XX:XX:XX DEBUG refresh "one": pulling <...>/one -XXXX-XX-XX XX:XX:XX WARNING <...>/bro-pkg.meta: missing metadata file -XXXX-XX-XX XX:XX:XX WARNING skipping aggregation of <...>/bad_pkg: bad metadata: missing zkg.meta (or bro-pkg.meta) metadata file +XXXX-XX-XX XX:XX:XX WARNING <...>/zkg.meta: missing metadata file +XXXX-XX-XX XX:XX:XX WARNING skipping aggregation of <...>/bad_pkg: bad metadata: missing zkg.meta metadata file XXXX-XX-XX XX:XX:XX DEBUG metadata refresh: 8 additions (alice/bar, alice/baz, alice/foo, alice/i-have-no-scripts, alice/new_pkg, alice/qux, bob/corge, bob/grault), 0 changes, 0 removals XXXX-XX-XX XX:XX:XX DEBUG fetch package one<...>/foo diff --git a/testing/baselines/tests.refresh/agg.fail.out b/testing/baselines/tests.refresh/agg.fail.out index e7f23ede..ea6e24ec 100644 --- a/testing/baselines/tests.refresh/agg.fail.out +++ b/testing/baselines/tests.refresh/agg.fail.out @@ -2,6 +2,6 @@ Refresh package source: one No membership changes WARNING: Metadata aggregated, but excludes the following packages due to described problems: - <...>/bad_pkg: missing zkg.meta (or bro-pkg.meta) metadata file + <...>/bad_pkg: missing zkg.meta metadata file Refresh installed packages No new outdated packages diff --git a/testing/baselines/tests.refresh/agg.out b/testing/baselines/tests.refresh/agg.out index 1d399d87..81f81633 100644 --- a/testing/baselines/tests.refresh/agg.out +++ b/testing/baselines/tests.refresh/agg.out @@ -2,7 +2,7 @@ Refresh package source: one No membership changes WARNING: Metadata aggregated, but excludes the following packages due to described problems: - <...>/bad_pkg: missing zkg.meta (or bro-pkg.meta) metadata file + <...>/bad_pkg: missing zkg.meta metadata file Pushed aggregated metadata Refresh installed packages No new outdated packages diff --git a/testing/tests/install-bro-pkg-warning b/testing/tests/install-bro-pkg-warning deleted file mode 100644 index bd94ede1..00000000 --- a/testing/tests/install-bro-pkg-warning +++ /dev/null @@ -1,3 +0,0 @@ -# @TEST-EXEC: (cd packages/foo; git mv zkg.meta bro-pkg.meta; git commit -m 'Use bro-pkg.meta') -# @TEST-EXEC: zkg install foo 2> stderr -# @TEST-EXEC: TEST_DIFF_CANONIFIER='sed -r "s/[0-9]{2}/XX/g"' btest-diff stderr diff --git a/zeekpkg/__init__.py b/zeekpkg/__init__.py index 73da0664..d97e9453 100644 --- a/zeekpkg/__init__.py +++ b/zeekpkg/__init__.py @@ -9,7 +9,7 @@ import logging -__version__ = "2.13.0-32" +__version__ = "2.13.0-46" __all__ = ["manager", "package", "source", "template", "uservar"] LOG = logging.getLogger(__name__) diff --git a/zeekpkg/_util.py b/zeekpkg/_util.py index ffdac930..0b967a6d 100644 --- a/zeekpkg/_util.py +++ b/zeekpkg/_util.py @@ -360,9 +360,6 @@ def read_zeek_config_line(stdout): def get_zeek_version(): zeek_config = find_program("zeek-config") - if not zeek_config: - zeek_config = find_program("bro-config") - if not zeek_config: return "" diff --git a/zeekpkg/manager.py b/zeekpkg/manager.py index 4832e799..70c031e1 100644 --- a/zeekpkg/manager.py +++ b/zeekpkg/manager.py @@ -49,7 +49,6 @@ BUILTIN_SOURCE, BUILTIN_SCHEME, METADATA_FILENAME, - LEGACY_METADATA_FILENAME, TRACKING_METHOD_VERSION, TRACKING_METHOD_BRANCH, TRACKING_METHOD_COMMIT, @@ -119,22 +118,15 @@ def populate(self): make_symlink(entry.path, os.path.join(self.clone_dir, entry.name)) def get_subprocess_env(self): - zeekpath = os.environ.get("ZEEKPATH") or os.environ.get("BROPATH") - pluginpath = os.environ.get("ZEEK_PLUGIN_PATH") or os.environ.get( - "BRO_PLUGIN_PATH" - ) + zeekpath = os.environ.get("ZEEKPATH") + pluginpath = os.environ.get("ZEEK_PLUGIN_PATH") if not (zeekpath and pluginpath): zeek_config = find_program("zeek-config") - path_option = "--zeekpath" - - if not zeek_config: - zeek_config = find_program("bro-config") - path_option = "--bropath" if zeek_config: cmd = subprocess.Popen( - [zeek_config, path_option, "--plugin_dir"], + [zeek_config, "--zeekpath", "--plugin_dir"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, bufsize=1, @@ -149,7 +141,7 @@ def get_subprocess_env(self): if not pluginpath: pluginpath = line2 else: - return None, 'no "zeek-config" or "bro-config" found in PATH' + return None, 'no "zeek-config" found in PATH' zeekpath = os.path.dirname(self.script_dir) + os.pathsep + zeekpath pluginpath = os.path.dirname(self.plugin_dir) + os.pathsep + pluginpath @@ -158,8 +150,6 @@ def get_subprocess_env(self): env["PATH"] = self.bin_dir + os.pathsep + os.environ.get("PATH", "") env["ZEEKPATH"] = zeekpath env["ZEEK_PLUGIN_PATH"] = pluginpath - env["BROPATH"] = zeekpath - env["BRO_PLUGIN_PATH"] = pluginpath return env, "" @@ -199,20 +189,19 @@ class Manager: scratch_dir (str): a directory where the package manager performs miscellaneous/temporary file operations - script_dir (str): the directory where the package manager will - copy each installed package's `script_dir` (as given by its - :file:`zkg.meta` or :file:`bro-pkg.meta`). Each package gets a - subdirectory within `script_dir` associated with its name. + script_dir (str): the directory where the package manager will copy each + installed package's `script_dir` (as given by its :file:`zkg.meta`). + Each package gets a subdirectory within `script_dir` associated with + its name. - plugin_dir (str): the directory where the package manager will - copy each installed package's `plugin_dir` (as given by its - :file:`zkg.meta` or :file:`bro-pkg.meta`). Each package gets a - subdirectory within `plugin_dir` associated with its name. + plugin_dir (str): the directory where the package manager will copy each + installed package's `plugin_dir` (as given by its :file:`zkg.meta`). + Each package gets a subdirectory within `plugin_dir` associated with + its name. bin_dir (str): the directory where the package manager will link executables into that are provided by an installed package through - `executables` (as given by its :file:`zkg.meta` or - :file:`bro-pkg.meta`) + `executables` (as given by its :file:`zkg.meta`) source_clonedir (str): the directory where the package manager will clone package sources. Each source gets a subdirectory @@ -239,6 +228,7 @@ class Manager: in a directory named :file:`packages`, so as long as :envvar:`ZEEKPATH` is configured correctly, ``@load packages`` will load all installed packages that have been marked as loaded. + """ def __init__( @@ -277,8 +267,6 @@ def __init__( self.installed_pkgs = {} self._builtin_packages = None # Cached Zeek built-in packages. self._builtin_packages_discovered = False # Flag if discovery even worked. - # The bro_dist attribute exists just for backward compatibility - self.bro_dist = zeek_dist self.zeek_dist = zeek_dist self.state_dir = state_dir self.user_vars = {} if user_vars is None else user_vars @@ -380,14 +368,6 @@ def __init__( self._write_autoloader() make_symlink("packages.zeek", self.autoload_package) - # Backward compatibility (Pre-Zeek 3.0 does not handle .zeek files) - autoload_script_fallback = os.path.join(self.script_dir, "packages.bro") - autoload_package_fallback = os.path.join(self.script_dir, "__load__.bro") - delete_path(autoload_script_fallback) - delete_path(autoload_package_fallback) - make_symlink("packages.zeek", autoload_script_fallback) - make_symlink("packages.zeek", autoload_package_fallback) - def _write_autoloader(self): """Write the :file:`packages.zeek` loader script. @@ -516,13 +496,6 @@ def zeekpath(self): """ return os.path.dirname(self.script_dir) - def bropath(self): - """Same as :meth:`zeekpath`. - - Using :meth:`zeekpath` is preferred since this may later be deprecated. - """ - return self.zeekpath() - def zeek_plugin_path(self): """Return the path where installed package plugins are located. @@ -531,14 +504,6 @@ def zeek_plugin_path(self): """ return os.path.dirname(self.plugin_dir) - def bro_plugin_path(self): - """Same as :meth:`zeek_plugin_path`. - - Using :meth:`zeek_plugin_path` is preferred since this may later be - deprecated. - """ - return self.zeek_plugin_path() - def add_source(self, name, git_url): """Add a git repository that acts as a source of packages. @@ -1168,7 +1133,7 @@ def _refresh_source(self, name, aggregate=False, push=False): aggregation_issues.append((url, msg)) continue - metadata_file = _pick_metadata_file(clone.working_dir) + metadata_file = os.path.join(clone.working_dir, METADATA_FILENAME) metadata_parser = configparser.ConfigParser(interpolation=None) invalid_reason = _parse_package_metadata( metadata_parser, metadata_file @@ -1493,16 +1458,8 @@ def load(self, pkg_path): pkg_load_script = os.path.join( self.script_dir, ipkg.package.name, "__load__.zeek" ) - # Check if __load__.bro exists for compatibility with older packages - pkg_load_fallback = os.path.join( - self.script_dir, ipkg.package.name, "__load__.bro" - ) - if ( - not os.path.exists(pkg_load_script) - and not os.path.exists(pkg_load_fallback) - and not self.has_plugin(ipkg) - ): + if not os.path.exists(pkg_load_script) and not self.has_plugin(ipkg): LOG.debug( 'loading "%s": %s not found and package has no plugin', pkg_path, @@ -2067,11 +2024,11 @@ def __str__(self): all_deps.update(ds) for dep_name, _ in all_deps.items(): - if dep_name == "bro" or dep_name == "zeek": + if dep_name == "zeek": # A zeek node will get added later. continue - if dep_name == "bro-pkg" or dep_name == "zkg": + if dep_name == "zkg": # A zkg node will get added later. continue @@ -2140,9 +2097,7 @@ def __str__(self): ) graph["zeek"] = node else: - LOG.warning( - 'could not get zeek version: no "zeek-config" or "bro-config" in PATH ?' - ) + LOG.warning('could not get zeek version: no "zeek-config" in PATH ?') node = Node("zkg") node.installed_version = PackageVersion( @@ -2196,11 +2151,11 @@ def __str__(self): all_deps.update(ds) for dep_name, dep_version in all_deps.items(): - if dep_name == "bro" or dep_name == "zeek": + if dep_name == "zeek": if "zeek" in graph: graph["zeek"].dependers[name] = dep_version node.dependees["zeek"] = dep_version - elif dep_name == "bro-pkg" or dep_name == "zkg": + elif dep_name == "zkg": if "zkg" in graph: graph["zkg"].dependers[name] = dep_version node.dependees["zkg"] = dep_version @@ -2760,7 +2715,7 @@ def _stage(self, package, version, clone, stage, env=None): """ LOG.debug('staging "%s": version %s', package, version) - metadata_file = _pick_metadata_file(clone.working_dir) + metadata_file = os.path.join(clone.working_dir, METADATA_FILENAME) metadata_parser = configparser.ConfigParser(interpolation=None) invalid_reason = _parse_package_metadata(metadata_parser, metadata_file) if invalid_reason: @@ -2838,10 +2793,9 @@ def _stage(self, package, version, clone, stage, env=None): "package's 'script_dir' does not exist: {}", pkg_script_dir ) - pkgload = os.path.join(script_dir_src, "__load__.") + pkgload = os.path.join(script_dir_src, "__load__.zeek") - # Check if __load__.bro exists for compatibility with older packages - if os.path.isfile(pkgload + "zeek") or os.path.isfile(pkgload + "bro"): + if os.path.isfile(pkgload): try: symlink_path = os.path.join( os.path.dirname(stage.script_dir), package.name @@ -3052,7 +3006,7 @@ def _install(self, package, version, use_existing_clone=False): status.current_hash = clone.head.object.hexsha status.is_outdated = _is_clone_outdated(clone, version, status.tracking_method) - metadata_file = _pick_metadata_file(clone.working_dir) + metadata_file = os.path.join(clone.working_dir, METADATA_FILENAME) metadata_parser = configparser.ConfigParser(interpolation=None) invalid_reason = _parse_package_metadata(metadata_parser, metadata_file) @@ -3095,7 +3049,6 @@ def _interpolate_package_metadata(self, metadata, stage): return None, "package has malformed 'user_vars' metadata field" substitutions = { - "bro_dist": self.zeek_dist, "zeek_dist": self.zeek_dist, "package_base": stage.clone_dir, } @@ -3229,7 +3182,7 @@ def ignore(_, files): rval = [] for f in files: - if f in {".git", "bro-pkg.meta", "zkg.meta"}: + if f in {".git", "zkg.meta"}: rval.append(f) return rval @@ -3279,22 +3232,11 @@ def _get_package_metadata(parser): return metadata -def _pick_metadata_file(directory): - rval = os.path.join(directory, METADATA_FILENAME) - - if os.path.exists(rval): - return rval - - return os.path.join(directory, LEGACY_METADATA_FILENAME) - - def _parse_package_metadata(parser, metadata_file): """Return string explaining why metadata is invalid, or '' if valid.""" if not parser.read(metadata_file): LOG.warning("%s: missing metadata file", metadata_file) - return "missing {} (or {}) metadata file".format( - METADATA_FILENAME, LEGACY_METADATA_FILENAME - ) + return f"missing {METADATA_FILENAME} metadata file" if not parser.has_section("package"): LOG.warning("%s: metadata missing [package]", metadata_file) @@ -3303,9 +3245,6 @@ def _parse_package_metadata(parser, metadata_file): return "" -_legacy_metadata_warnings = set() - - def _info_from_clone(clone, package, status, version): """Retrieves information about a package. @@ -3322,7 +3261,7 @@ def _info_from_clone(clone, package, status, version): else: version_type = TRACKING_METHOD_BRANCH - metadata_file = _pick_metadata_file(clone.working_dir) + metadata_file = os.path.join(clone.working_dir, METADATA_FILENAME) metadata_parser = configparser.ConfigParser(interpolation=None) invalid_reason = _parse_package_metadata(metadata_parser, metadata_file) @@ -3338,20 +3277,6 @@ def _info_from_clone(clone, package, status, version): default_branch=default_branch, ) - # Remove in v3.0 by either silently ignoring LEGACY_METADATA_FILENAME - # completely or error with helpful instructions about zkg.meta. - if ( - os.path.basename(metadata_file) == LEGACY_METADATA_FILENAME - and package.qualified_name() not in _legacy_metadata_warnings - ): - LOG.warning( - "Package %s is using the legacy bro-pkg.meta metadata file. " - "It will soon stop working unless updated to use zkg.meta instead. " - "Please report this to the package maintainers.", - package.qualified_name(), - ) - _legacy_metadata_warnings.add(package.qualified_name()) - metadata = _get_package_metadata(metadata_parser) return PackageInfo( @@ -3368,4 +3293,4 @@ def _info_from_clone(clone, package, status, version): def _is_reserved_pkg_name(name): - return name == "bro" or name == "zeek" or name == "zkg" + return name == "zeek" or name == "zkg" diff --git a/zeekpkg/package.py b/zeekpkg/package.py index 3efbea37..7fbe77e6 100644 --- a/zeekpkg/package.py +++ b/zeekpkg/package.py @@ -15,7 +15,6 @@ #: The name of files used by packages to store their metadata. METADATA_FILENAME = "zkg.meta" -LEGACY_METADATA_FILENAME = "bro-pkg.meta" TRACKING_METHOD_VERSION = "version" TRACKING_METHOD_BRANCH = "branch" @@ -297,7 +296,7 @@ class PackageInfo: packages metadata (dict of str -> str): the contents of the package's - :file:`zkg.meta` or :file:`bro-pkg.meta` + :file:`zkg.meta` versions (list of str): a list of the package's availabe git version tags @@ -311,10 +310,9 @@ class PackageInfo: invalid_reason (str): this attribute is set when there is a problem with gathering package information and explains what went wrong. - metadata_file: the absolute path to the :file:`zkg.meta` or - :file:`bro-pkg.meta` for this package. Use this if you'd like to - parse the metadata yourself. May not be defined, in which case the - value is None. + metadata_file: the absolute path to the :file:`zkg.meta` or for this + package. Use this if you'd like to parse the metadata yourself. + May not be defined, in which case the value is None. """ def __init__( @@ -427,10 +425,10 @@ class Package: or if it's located in a top-level :file:`zkg.index` file. metadata (dict of str -> str): the contents of the package's - :file:`zkg.meta` or :file:`bro-pkg.meta` file. If the package has - not been installed then this information may come from the last - aggregation of the source's :file:`aggregate.meta` file (it may not - be accurate/up-to-date). + :file:`zkg.meta` file. If the package has not been installed then + this information may come from the last aggregation of the source's + :file:`aggregate.meta` file (it may not be accurate/up-to-date). + """ def __init__( diff --git a/zeekpkg/source.py b/zeekpkg/source.py index 3b07909c..4f1a938c 100644 --- a/zeekpkg/source.py +++ b/zeekpkg/source.py @@ -1,9 +1,10 @@ """ A module containing the definition of a "package source": a git repository -containing a collection of :file:`zkg.index` (or legacy :file:`bro-pkg.index`) -files. These are simple INI files that can describe many Zeek packages. Each -section of the file names a Zeek package along with the git URL where it is -located and metadata tags that help classify/describe it. +containing a collection of :file:zkg.index files and a top-level aggregate.meta +file. :filezkg.index files contain URLs to package repositories, while +:file:aggregate.meta file is an INI file describing many packages. Each section +of the file names a Zeek package along with the git URL where it is located and +metadata tags that help classify/describe it. """ import configparser @@ -17,7 +18,6 @@ #: The name of package index files. INDEX_FILENAME = "zkg.index" -LEGACY_INDEX_FILENAME = "bro-pkg.index" #: The name of the package source file where package metadata gets aggregated. AGGREGATE_DATA_FILE = "aggregate.meta" @@ -105,7 +105,7 @@ def package_index_files(self): pass for filename in files: - if filename == INDEX_FILENAME or filename == LEGACY_INDEX_FILENAME: + if filename == INDEX_FILENAME: rval.append(os.path.join(root, filename)) return sorted(rval) diff --git a/zkg b/zkg index e74022a2..996f0e49 100755 --- a/zkg +++ b/zkg @@ -88,7 +88,6 @@ ZKG_DEFAULT_TEMPLATE = "https://github.com/zeek/package-template" from zeekpkg._util import ( delete_path, make_dir, - make_symlink, find_program, read_zeek_config_line, std_encoding, @@ -200,20 +199,11 @@ def find_configfile(args): configfile = os.environ.get("ZKG_CONFIG_FILE") - if not configfile: - # For backward compatibility with old bro-pkg. - configfile = os.environ.get("BRO_PKG_CONFIG_FILE") - if configfile and file_is_not_empty(configfile): return configfile configfile = os.path.join(default_config_dir(), "config") - if file_is_not_empty(configfile): - return configfile - - configfile = os.path.join(legacy_config_dir(), "config") - if file_is_not_empty(configfile): return configfile @@ -228,18 +218,10 @@ def default_config_dir(): return ZEEK_ZKG_CONFIG_DIR or home_config_dir() -def legacy_config_dir(): - return os.path.join(os.path.expanduser("~"), ".bro-pkg") - - def default_state_dir(): return ZEEK_ZKG_STATE_DIR or home_config_dir() -def legacy_state_dir(): - return os.path.join(os.path.expanduser("~"), ".bro-pkg") - - def create_config(args, configfile): config = configparser.ConfigParser() @@ -293,9 +275,6 @@ def create_config(args, configfile): bin_dir = get_option(config, "paths", "bin_dir", os.path.join(state_dir, "bin")) zeek_dist = get_option(config, "paths", "zeek_dist", "") - if not zeek_dist: - zeek_dist = get_option(config, "paths", "bro_dist", "") - config.set("paths", "state_dir", state_dir) config.set("paths", "script_dir", script_dir) config.set("paths", "plugin_dir", plugin_dir) @@ -394,10 +373,6 @@ def create_manager(args, config): bin_dir = config.get("paths", "bin_dir") zeek_dist = config.get("paths", "zeek_dist") - if state_dir == default_state_dir(): - if not os.path.exists(state_dir) and os.path.exists(legacy_state_dir()): - make_symlink(legacy_state_dir(), state_dir) - try: manager = zeekpkg.Manager( state_dir=state_dir, @@ -2124,20 +2099,12 @@ def cmd_autoconfig(manager, args, config, configfile): zeek_config = find_program("zeek-config") - if zeek_config: - have_zeek_config = True - else: - have_zeek_config = False - zeek_config = find_program("bro-config") - if not zeek_config: - print_error('error: no "zeek-config" or "bro-config" in PATH') + print_error('error: no "zeek-config" in PATH') sys.exit(1) - dist_option = "--zeek_dist" if have_zeek_config else "--bro_dist" - cmd = subprocess.Popen( - [zeek_config, "--site_dir", "--plugin_dir", "--prefix", dist_option], + [zeek_config, "--site_dir", "--plugin_dir", "--prefix", "--zeek_dist"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, bufsize=1, @@ -2190,25 +2157,12 @@ def cmd_autoconfig(manager, args, config, configfile): def cmd_env(manager, args, config, configfile): zeek_config = find_program("zeek-config") - path_option = "--zeekpath" - - if not zeek_config: - path_option = "--bropath" - zeek_config = find_program("bro-config") - zeekpath = os.environ.get("ZEEKPATH") - - if not zeekpath: - zeekpath = os.environ.get("BROPATH") - pluginpath = os.environ.get("ZEEK_PLUGIN_PATH") - if not pluginpath: - pluginpath = os.environ.get("BRO_PLUGIN_PATH") - if zeek_config: cmd = subprocess.Popen( - [zeek_config, path_option, "--plugin_dir"], + [zeek_config, "--zeekpath", "--plugin_dir"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, bufsize=1, @@ -2236,14 +2190,10 @@ def cmd_env(manager, args, config, configfile): pluginpaths = remove_redundant_paths(pluginpaths) if os.environ.get("SHELL", "").endswith("csh"): - print("setenv BROPATH {}".format(":".join(zeekpaths))) - print("setenv BRO_PLUGIN_PATH {}".format(":".join(pluginpaths))) print("setenv ZEEKPATH {}".format(":".join(zeekpaths))) print("setenv ZEEK_PLUGIN_PATH {}".format(":".join(pluginpaths))) print(f"setenv PATH {manager.bin_dir}:$PATH") else: - print("export BROPATH={}".format(":".join(zeekpaths))) - print("export BRO_PLUGIN_PATH={}".format(":".join(pluginpaths))) print("export ZEEKPATH={}".format(":".join(zeekpaths))) print("export ZEEK_PLUGIN_PATH={}".format(":".join(pluginpaths))) print(f"export PATH={manager.bin_dir}:$PATH") @@ -2463,8 +2413,7 @@ def argparser(): " the ambiguity. A full git URL may also be used to refer" " to a package that does not belong to a source. E.g. for" ' a package source called "zeek" that has a package named' - ' "foo" located in either "alice/zkg.index" or' - ' "alice/bro-pkg.index", the following' + ' "foo" located in "alice/zkg.index", the following' ' names work: "foo", "alice/foo", "zeek/alice/foo".' ) @@ -2718,13 +2667,11 @@ def argparser(): sub_parser.add_argument( "--aggregate", action="store_true", - help="Crawls the urls listed in package source zkg.index" - " (or legacy bro-pkg.index) files and" - " aggregates the metadata found in their zkg.meta (or legacy" - " bro-pkg.meta) files. The aggregated metadata is stored in the local" - " clone of the package source that zkg uses internally for locating" - " package metadata." - " For each package, the metadata is taken from the highest available" + help="Crawls the urls listed in package source zkg.index files and" + " aggregates the metadata found in their zkg.meta files. The aggregated" + " metadata is stored in the local clone of the package source that zkg" + " uses internally for locating package metadata. For each package, the" + " metadata is taken from the highest available" ' git version tag or the default branch, like "main" or "master", if no version tags exist', ) sub_parser.add_argument( @@ -2970,7 +2917,6 @@ def argparser(): "plugin_dir", "bin_dir", "zeek_dist", - "bro_dist", ], help="Name of a specific config file field to output.", ) @@ -2983,7 +2929,7 @@ def argparser(): " config file that is generated by using the ``zeek-config`` script" " that is installed along with Zeek. It is the suggested configuration" " to use for most Zeek installations. For this command to work, the" - " ``zeek-config`` (or ``bro-config``) script must be in ``PATH``," + " ``zeek-config`` script must be in ``PATH``," " unless the --user option is given, in which case this creates" " a config that does not touch the Zeek installation.", formatter_class=argparse.ArgumentDefaultsHelpFormatter, @@ -2999,13 +2945,11 @@ def argparser(): help="Show the value of environment variables that need to be set for" " Zeek to be able to use installed packages.", description="This command returns shell commands that, when executed," - " will correctly set ``ZEEKPATH`` and ``ZEEK_PLUGIN_PATH`` (also" - " ``BROPATH`` and ``BRO_PLUGIN_PATH`` for legacy compatibility) to use" + " will correctly set ``ZEEKPATH`` and ``ZEEK_PLUGIN_PATH`` to use" " scripts and plugins from packages installed by the package manager." " For this command to function properly, either have the ``zeek-config``" " script (installed by zeek) in ``PATH``, or have the ``ZEEKPATH`` and" - " ``ZEEK_PLUGIN_PATH`` (or ``BROPATH`` and ``BRO_PLUGIN_PATH``)" - " environment variables already set so this command" + " ``ZEEK_PLUGIN_PATH`` environment variables already set so this command" " can append package-specific paths to them.", formatter_class=argparse.ArgumentDefaultsHelpFormatter, ) diff --git a/zkg.config b/zkg.config index 4c2c480f..a220a8ad 100644 --- a/zkg.config +++ b/zkg.config @@ -34,7 +34,7 @@ state_dir = # A subdirectory named "packages" is always created within the # specified path and the package manager will copy the directory # specified by the "script_dir" option of each package's zkg.meta -# (or legacy bro-pkg.meta) file there. +# file there. # If left blank or with --user this defaults to /script_dir. # In Zeek-bundled installations, it defaults to # /share/zeek/site. @@ -47,7 +47,7 @@ script_dir = # A subdirectory named "packages" is always created within the # specified path and the package manager will copy the directory # specified by the "plugin_dir" option of each package's zkg.meta -# (or legacy bro-pkg.meta) file there. +# file there. # If left blank or with --user this defaults to /plugin_dir. # In Zeek-bundled installations, it defaults to # /lib/zeek/plugins. @@ -66,7 +66,8 @@ bin_dir = # The directory containing Zeek distribution source code. This is only # needed when installing packages that contain Zeek plugins that are -# not pre-built. The legacy name of this option is "bro_dist". +# not pre-built. This value is generally not needed by most users other +# than plugin developers anymore. zeek_dist = [templates] @@ -77,10 +78,10 @@ default = https://github.com/zeek/package-template [user_vars] -# For any key in this section that is matched for value interpolation -# in a package's zkg.meta (or legacy bro-pkg.meta) file, the corresponding -# value is substituted during execution of the package's `build_command`. -# This section is typically automatically populated with the -# the answers supplied during package installation prompts -# and, as a convenience feature, used to recall the last-used settings -# during subsequent operations (e.g. upgrades) on the same package. +# For any key in this section that is matched for value interpolation in a +# package's zkg.meta file, the corresponding value is substituted during +# execution of the package's `build_command`. This section is typically +# automatically populated with the the answers supplied during package +# installation prompts and, as a convenience feature, used to recall the +# last-used settings during subsequent operations (e.g. upgrades) on the same +# package.