Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix/374/module zos mvs raw errors with long multi line quoted string in content field #1057

Conversation

AndreMarcel99
Copy link
Collaborator

SUMMARY

Add validations and fixes for proper ways of written multiline texts in playbooks for a JCL

Fixes #374 and #665

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

modify_contetns
add_spaces

ADDITIONAL INFORMATION

modify_contents:

  • If content is a list is just required to be validate with add spaces function an then be join with /n

  • If content is a string add spaces at the beginning if necessary then split into a list for every new line and pass to add_spaces function

add_spaces: with a list check every line the len if exceed the 78 characters would throw error if not add double spaces at the beginning if is not already to validate for a correct JCL

*note: All the changes will affect correct ways of writing multi lines text in yaml as:

content: |1
                INCLUDE INLIB
                NAME {{ MEM_TEST }}(R)
content: |
                INCLUDE INLIB
                NAME {{ MEM_TEST }}(R)
content:
                - "INCLUDE INLIB"
                - "NAME {{ MEM_TEST }}(R)"
Captura de pantalla 2023-11-10 a la(s) 12 12 42 p m

Copy link
Collaborator

@fernandofloresg fernandofloresg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will paste my comments on the slack thread since I see some were ignored:

Due to JCL constraints with input data set
Issue 1: JCL content currently cannot be longer than 80 chars
Document this in our module, it is stated in the MVS JCL User guide and we can not modify that behavior, indicate how the user can input multiline yaml texts to not remove newline chars.

Issue 2: Us prepending spaces in each newline
I like your proposal and would like to see the code for it, you got my green light for it, what I think we could do is do it the smart way, just add the needed empty spaces if those are not already in place.

This is missing some needed documentation on how we will handle user JCL input from now on, that is very important so users just don't see random record length errors with the new release.

On another front, I will prefer that we only prepend empty spaces when is absolutely needed.

I also added a strange error that I come up while testing a new added code, be sure to test new functionality that you add even if regression tests run fine because we most likely don't have a testing case for that new functionality.

And since we are adding a new functionality, a new test case would be welcomed.

plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@richp405 richp405 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks decent. I left a question and a suggestion for the changelog fragment.

@AndreMarcel99
Copy link
Collaborator Author

Captura de pantalla 2023-11-13 a la(s) 10 43 28 p m Captura de pantalla 2023-11-13 a la(s) 10 43 43 p m

Copy link
Collaborator

@richp405 richp405 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay...make sure you check fernando's last comment.

plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
AndreMarcel99 and others added 3 commits November 15, 2023 14:06
…ne_quoted_string_in_content_field' of https://github.com/ansible-collections/ibm_zos_core into bugfix/374/Module_zos_mvs_raw_errors_with_long_multi-line_quoted_string_in_content_field
@AndreMarcel99
Copy link
Collaborator Author

Captura de pantalla 2023-11-16 a la(s) 3 30 46 p m Captura de pantalla 2023-11-16 a la(s) 3 31 10 p m

Copy link
Collaborator

@ddimatos ddimatos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we still need a bit of work @AndreMarcel99 , I would also suggest that we maybe add an example using a block sytle indicator with 2 , eg `content: | 2" , pretty sure I had some examples in the original git issues I created you can pull from. They don't have to be lengthy examples but its good if we show one.

I don't think we can automate with the block style indicator with the pytest-ansible , maybe share a couple of examples in the pull request from the ones I created or ones of your own for historical purposes?

plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Outdated Show resolved Hide resolved
plugins/modules/zos_mvs_raw.py Show resolved Hide resolved
@ddimatos ddimatos added the Waiting for Response Waiting for a response from the issue author label Nov 22, 2023
@AndreMarcel99
Copy link
Collaborator Author

Captura de pantalla 2023-11-28 a la(s) 9 44 15 a m Captura de pantalla 2023-11-28 a la(s) 9 44 28 a m

Copy link
Collaborator

@ddimatos ddimatos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you.

@ddimatos ddimatos merged commit 5c520cb into dev Dec 2, 2023
@AndreMarcel99 AndreMarcel99 deleted the bugfix/374/Module_zos_mvs_raw_errors_with_long_multi-line_quoted_string_in_content_field branch December 4, 2023 14:37
ddimatos added a commit that referenced this pull request May 17, 2024
* Added PR# to changelog
tweaked description of new feature in both affected functions.

* added changelog for ticket

* Modified versions in bug issue template to avoid users picking a non-existing version

Signed-off-by: Fernando Flores <[email protected]>

* Added zoau_api_version logic to check for 1.2.5 or later as a condition for wait_arg
Added mention of this to documentation of interface

* corrected pep8 errors

* removed redundant changelog fragment

* Changed Enhancements to minor_changes

* push updated module doc

Signed-off-by: ddimatos <[email protected]>

* Enabler/validate path join (#962)

* Added real path fetch to base

* Sec changes

* Updated changelog and template

* Added validation to zos_copy and zos_fetch

Signed-off-by: Fernando Flores <[email protected]>

* Corrected positional argument

* Added validation changes

Signed-off-by: Fernando Flores <[email protected]>

* Removed f-string

* Fixed path join for copy_to_file

---------

Signed-off-by: Fernando Flores <[email protected]>

* Removed 'wait' as an option, and pass wait_arg=true to zoau

* Added latest to allow member copy when disp=shr (#980)

* Added latest to allow member copy when disp=shr

* Added changelog fragment

* Added new force option and test for locked data sets non VSAM

* Fixed pep8 issue

* Added new option force lock

* Modified test case with new option

* Added force option

* Added doc and warning

* Updated changelog fragment

* Update 980-zos-copy-disp-shr.yml

* Updated changelog fragment

* Removed unused comments

Signed-off-by: Fernando Flores <[email protected]>

* Added message

* Added force_lock to all CopyHandlers

* Modified test case

* Changed use of dataset vs data set

---------

Signed-off-by: Fernando Flores <[email protected]>

* Changed operator_action_query to wait=false time = 5
Renamed vague variable name to "use_wait_arg"
Reflected changes and 1.2.5 dependancy in the changelog fragment

* Enhancement/423/zos copy add data set member alias support (#1014)

* add aliases option and enable text-based member copy w alias to an existing pds

Signed-off-by: Ketan Kelkar <[email protected]>

* disable alias included in data set member listing when collecting src members from pds

Signed-off-by: Ketan Kelkar <[email protected]>

* switch 'alias' option introduced in zoau1.2.5 to '-H' flag available in zoau1.2.4. also enable alias copying of executables

Signed-off-by: Ketan Kelkar <[email protected]>

* implement aliases for copy to/from USS, add guard rail for non-executable copy to USS with aliases

Signed-off-by: Ketan Kelkar <[email protected]>

* add exception handler for executable PDS copy, handle non-existent library pds for executable USS src, add error message for PDS copy attempt to USS file

Signed-off-by: Ketan Kelkar <[email protected]>

* clean up init functions, break up long lines

Signed-off-by: Ketan Kelkar <[email protected]>

* refactor executable member to member copy for alias work, this commit refactors some helpers which break a select few loadlib tests, but those will be refactored in upcoming comimts

Signed-off-by: Ketan Kelkar <[email protected]>

* refactor and expand test_copy_pds_loadlib_member_to_uss test case to copy to a new loadlib

Signed-off-by: Ketan Kelkar <[email protected]>

* add aliases error raised check to text-based pds member copy to uss

Signed-off-by: Ketan Kelkar <[email protected]>

* add aliases error raised check to text-based pds member copy to uss

Signed-off-by: Ketan Kelkar <[email protected]>

* parametrize cobol program to pass in custom output string, create helper method around running and validating loadlib pgms, refactor executable tests to use helper method, add helper method to create loadlib w multiple members, add test case for loadlib to loadlib copy w and w/o aliases.

Signed-off-by: Ketan Kelkar <[email protected]>

* add test case for copying entire loadlib to uss dir and then to another loadlib. refactor other loadlib test case to reduce loc

Signed-off-by: Ketan Kelkar <[email protected]>

* add new pytest markers for aliases and loadlib test cases

Signed-off-by: Ketan Kelkar <[email protected]>

* pull in a sneak preview version of bug #920 addressed in PR #968 which adds LIBRARY as a valid value to the dest_data_set option

Signed-off-by: Ketan Kelkar <[email protected]>

* alter placement of aliases option to go after executable options

Signed-off-by: Ketan Kelkar <[email protected]>

* add docs and examples for aliases option

Signed-off-by: Ketan Kelkar <[email protected]>

* minor tweaks to doc

Signed-off-by: Ketan Kelkar <[email protected]>

* addres santiy check issues

Signed-off-by: Ketan Kelkar <[email protected]>

* missed a sanity check issue

Signed-off-by: Ketan Kelkar <[email protected]>

* add changelog fragment

Signed-off-by: Ketan Kelkar <[email protected]>

* resolve remaining merge conflicts

Signed-off-by: Ketan Kelkar <[email protected]>

* resolve pep8 style issues

Signed-off-by: Ketan Kelkar <[email protected]>

* cleanup spacing issue in examples docstring

Signed-off-by: Ketan Kelkar <[email protected]>

* add updated rst file

Signed-off-by: Ketan Kelkar <[email protected]>

* add comments for explaning logic/code flow around full pds copy

Signed-off-by: Ketan Kelkar <[email protected]>

---------

Signed-off-by: Ketan Kelkar <[email protected]>

* New module zos_script (#961)

* First version of the action plugin and module doc

* Added remote execution

* Fixed handling and cleanup of temp files

* Fixed mode setting for scripts

* Fixed undefined variable error in action plugin

* Fixed chdir when running command

* Added creates and removes options

* Changed encoding validation

* Filled out docs for the module

* Added examples

* Filled out RETURN block for module

* Enhanced error messages

* Generated docs for zos_script

* Added first tests for zos_script

* Reordered args

* Added mode check to remote script test

* Fixed template rendering

* Fixed tests

* Added tests for error handling and templates

* Fixed a sanity error when returning a failure JSON

* Updated ignore files

* Updated module docs

* Updated repository templates for issues

* Fixed whitespace in docs

* Updated tmp_path description

* Updated notes in documentation

* Removed use of local_charset

* Removed private args

This commit finishes the work needed to remove two sanity tests exceptions.

* Fixed permissions for remote scripts

* Updated module documentation

* Updated documentation for tmp_path

* main change to version checker for consolidation

* Removed old version checker/shell call, added changelog entry

* Changed location of duration value set, so it will always be populated.

* found another edge case where duration was not being returned.

* added tolerance to test on max_rc, where duration is not always returned.

* Staging v1.7.0 merge to main (#1019) (#1023)

* Galaxy 1.7 updates
* Update meta runtime to support ansible-core 2.14 or later
* Update ibm_zos_core_meta.yml with updated version
* Update readme to align to supported ansible versions and new urls
* Added additional sanity ignore files to the exclude list
* Added additional sanity ignore files to the exclude list for ansible-lint.
* Update copyright yrs for source files that were overlooked
* Remove requirements from module doc, rely on offerings minimum requirements, also zoau 1.2.1 never was supported
* Add changelog summary for 1.7
* Adding generated antsibull-changelog release changelog and artifacts
* Remove v1.7.0_summary, its no longer needed
* Update release notes for ac 1.7.0
* Remove unsupported collection versions requiring a version of zoau that is EOS
---------

Signed-off-by: ddimatos <[email protected]>
Co-authored-by: André Marcel Gutiérrez Benítez <[email protected]>

* Enabler/validate path join part 2 (#1029)

* Added real path fetch to base

* Sec changes

* Updated changelog and template

* Added validation to zos_copy and zos_fetch

Signed-off-by: Fernando Flores <[email protected]>

* Corrected positional argument

* Added validation changes

Signed-off-by: Fernando Flores <[email protected]>

* Removed f-string

* Fixed path join for copy_to_file

* Added validation function to template

* Added new files

* Added changelog fragment

---------

Signed-off-by: Fernando Flores <[email protected]>

* Add known issues doc for utf8 issues (#1035)

* Add known issues doc for utf8 issues

Signed-off-by: ddimatos <[email protected]>

* Updated changelog fragement to remove colon usage

Signed-off-by: ddimatos <[email protected]>

* Corrected changelog errors

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* [Enhancement] [zos_copy] Add support for ASA control chars (#1028)

* Added ASA support when copying from datasets with control chars

* Added ASA support when copying from USS to sequential data sets

* Added ASA support when copying from USS to PDS/E

* Re-enabled copy from datasets to USS

* Added copy from non-ASA data sets to seq ASA ones

* Added copy from non-ASA data sets to partitioned ones

* Added allocation of ASA destination data sets

* Added first version of ASA compatibility validation

* Added validations for asa_text

* Added asa_text documentation

* Fixed record_length issue when creating ASA data sets

* Fixed record_length issue

* Added asa_text example

* Added first test

* Added DBRM doc to zos_copy module (#1025)

* Added DBRM doc to zos_copy module

Signed-off-by: ddimatos <[email protected]>

* Removed an extra space from doc

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* Added more tests

* Added changelog fragment

* Updated RST files

* Removed unused import

* Fixed merge

* Added another validation

* Updated docs

* Changed use of tsocmd for mvscmd

* Fixed merge error

* Fixed bug with VSAM sources

* Fixed bug with VSAM destinations

---------

Signed-off-by: ddimatos <[email protected]>
Co-authored-by: Demetri <[email protected]>

* Merge staging-v1.8.0-beta.1 back to dev (#1040)

* Resolved merge conflicts from cherrypick

* Delete changelogs

* Fix sending a local archive into remote fails (#1045)

* Added test

* Added test for local archive and then remote unarchive

* Added changelog

* Removed format fixture

* Ensure tempfile cleanup

* Clean temporary data sets created during XMIT unarchive operation (#1049)

* Added a temp cleanup

* Added changelog

* Modified changelog

* Added removal of src if remote_src is False

Signed-off-by: Fernando Flores <[email protected]>

* Modified changelog fragments

---------

Signed-off-by: Fernando Flores <[email protected]>

* Enabler/1002/test_collections_on_ansible core 2_16 (#1053)

* Add sanity ignore

* Move dependencyfinder

* Revert "Move dependencyfinder"

This reverts commit 2bbbc5adffe94b32dd6d1af12f7c7cace93cca94.

* Modify dependecyfinder

* Modify dependecyfinder

* Modify dependecyfinder

* Add fragment

* Modify fragment

* [Enabler] [zos_mvs_raw] Remove Try, Except, Pass from code (#1051)

* Added action inside exception to avoid pass

* Added action inside exception to avoid pass

* Added changelog

* Update sanity test ignore (#1048)

* Fixed Sanity ignore #6 and reduce use of Sanity issues 8-9

* Remove unused import

* Remove ignore

* Add fragment

* Modify fragment

* Change fragment

* Update 1048-Update_sanity_tests_ignore.yml

* Change ignore 2_dot_14

---------

Co-authored-by: Fernando Flores <[email protected]>

* Update_sanity_ignore 2_16 (#1056)

* Update sanity ignore 2_16

* Add fragment

* Change fragment

* [Enabler] [module_utils/dd_statement.py] Changed try except pass to except specific DatasetDeleteError exception (#1052)

* Changed try except pass to except specific class

* Added changelog

* Update 1052-try-except-pass-dd-statement.yml

* Update dd_statement.py

updated copyright year

* Removed run command  use of subprocess from encode defaults infavor of using AnsibleModule run command (#1055)

* Removed run command  use of subprocess from encode defaults infavor of using ansible module

* Added changelog

* [1.9.0] zos_apf remove try expect pass to better exception handling (#1036)

* Removed except pass

* Added empty strings

* Added changelog

* Corrected changelog

* Modified if statement to honor current behavior

* Update 1036-apf-try-except.yml

* Update 1036-apf-try-except.yml

* [v1.9.0] Replace randint to fix "Standard pseudo-random generators ... " bandit warning  (#1016)

* Changed random member name generation to choices

* Modified choices in module_utils/data_set and blockinfile test

* Added changelog fragment

* Remove randint import

* Removed unused imports

* 1043 bug title zos operator is passing wrong value to zoauopercmd (#1044)

* corrected kwarg index value from 'wait_arg' to 'wait'
Also corrected true/false issue in zoaq

* Added and updated changelog.

* update PR number in changelog fragment

* changed test from \$ to \\$ to eliminate warning

* added blocking test to maks sure minimum wait is reached in zoau>1.2.4.5

* removed the else condition from the blocking test, since it is not needed.

* corrected tense grammer in changelog fragment

* corrected capitalization of ZOAU in changelog fragment.

* Bug 1041 zos submit job honor return output literally (#1058)

* initial commit to pass return_output to job_output.

* corrected fragment name to match branch

* tweaked data set test to show result values if positive test fails

* removed trace in zos_data_set, and added trace output to job_submit

* removed extra text from functional testing.

* put in correct PR number in changelog fragment.

* changed trivial to minor_changes, added documentation to dd_scan in job:job_output.

* Enabler/1024/remote_tmp for zos_script (#1060)

* Changed tmp_path for Ansible's remote_tmp

* Remove tmp_path from module's options

* Update module documentation

* Remove tmp_path test case

* Update zos_script's RST file

* Add changelog fragment

* Updated module examples

* [zos_copy] Files corrupted after second copy (#1064)

* Initial change to replace shutil.copy

* Added fix for corrupted directory copies

* Added changelog fragment

* Modified docstring and fixed copy_tree

* Added punctiation

* Added copystat

* Added set mode for dirs

* Update 1064-corruped-second-copy.yml

* [Documentation] [zos_tso_command] Add REXX exec example (#1065)

* Add REXX exec example

* Add fragment

* Update module documentation

* Fix PR link

* Reword example task name

* Updated REXX example

* Mvs to non existent mvs copy destination attrs match up (#1066)

* Fixed error

* Fix identation

* Add fragment

* Modify fragment

* Modify fragment

* Modify fragment

* Bugfix/374/module zos mvs raw errors with long multi line quoted string in content field (#1057)

* Add function of write content

* Push easy soultion for two cases

* Fix identation and more issues

* Fix identation and more issues

* Solve error of null

* Add validation comments and separete the code

* Add fragment

* Modify logics

* Return overthink

* Add explanation for the user and change logic

* Add explanation for the user and change logic

* Change documentation

* Change fragment

* Better error message, better documentation and fragment

* Get better mesages

* Change the logic

* Change documentation

* Change logic

* Add scape to #

* Check failing

* Check failing

* Add valid scapes

* Update zos_mvs_raw fragment and module doc

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>
Co-authored-by: ddimatos <[email protected]>

* Remove tarfile.exctractall and zipfile.extractall in favor of individual member extraction for sanity (#1077)

* Implemented extract_all function

* Added changelog

* Removed commented lines

* Update ac tool version of pyyaml to avoid wheel issue

Signed-off-by: ddimatos <[email protected]>

* Changelog fragment

Signed-off-by: ddimatos <[email protected]>

* Fixed sanity issues with zos_mvs_raw (#1084)

* [bugfix][v1.9.0][zos_unarchive]action plugin does not clean up remote temporary files after completion (#1073)

* Push solution

* Add fragment

* Modify logic for fails and ensure works

* Change bug

* Chamge fragment

* Chamge fragment

* Add comma

---------

Co-authored-by: Demetri <[email protected]>

* [v1.9.0] Bugfix/837/missing ZOAU imports (#1042)

* Added new missing import handler to zos_operator

* Added new import handler to job util and zos_job_query

* Added ZOAU check to zos_operator

* Fixed sanity test issue

* Fixed Python 2.7 sanity issue

* Add changelog fragment

---------

Co-authored-by: Demetri <[email protected]>

* [Enhancement][zos_job_submit] Modify error messages to be clearer (#1074)

* Improved error messages

* Updated failure logic

---------

Co-authored-by: Demetri <[email protected]>

* update link in managed_node doc (#1089)

* update link in managed_node doc

Signed-off-by: Ketan Kelkar <[email protected]>

* Updated docs regarding managed node

Signed-off-by: ddimatos <[email protected]>

* Doc fragment

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: Ketan Kelkar <[email protected]>
Signed-off-by: ddimatos <[email protected]>
Co-authored-by: ddimatos <[email protected]>

* First check of solution

* Add clear solution

* Return local

* Add test ignores

* Add ignore to 2.16

* Optimize the encoding

* Remove unused import

* Return encoded

* Add encoding

* Remove unused encoded

* Modified logic to remove tmp files if they exist and not only if location is local

* Added temp_file null definition

* Update 1091-Update_undocumented_argument_and_import_exception.yml

* Merge release v1.8.0 into dev branch (#1095)

* Staging v1.7.0 beta.1 (#915)

* Added tagging of converted files

* Updated tests for zos_encode

* Added restoration of PDS/PDSE members when module fails

zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work.

* Fixed cleanup of dest when module fails

* Removed exception used for debugging

* Added pytest markers

* solved pep8 issue

* Added more information to error when restoration fails

* Update make file with new features to manage the nodes (#563)

* Update make file with new features to manage the nodes

Signed-off-by: ddimatos <[email protected]>

* Correct typo

Signed-off-by: ddimatos <[email protected]>

* updated makefile to allow for no password in the clear and added a new host

Signed-off-by: ddimatos <[email protected]>

Signed-off-by: ddimatos <[email protected]>

* Fix a bug that failed to copy the cert to the managed node (#577)

Signed-off-by: ddimatos <[email protected]>

Signed-off-by: ddimatos <[email protected]>

* Added apropiate error message in zos_lineinfile when src is not found

* Better error message

* Added fragment for changelog

* Update git issue slection field (#593)

* Update git issue slection field

* Added some additional future versions

* Added rule to ignore python 2.7 compile not supporting f strings

* Corrected rule in ignore file

* Update 584-zos_lineinfile-error-message.yml

* Added missing fragments for issues 309 and 408

* update make file and related artifacts with recent zVM changes (#598)

Signed-off-by: ddimatos <[email protected]>

Signed-off-by: ddimatos <[email protected]>

* Expand what is ingored for venv to all venv's (#613)

Signed-off-by: ddimatos <[email protected]>

Signed-off-by: ddimatos <[email protected]>

* Ansible-core versions with updated ignore file pass (#615)

* Validated that ansible-core versions with updated ignore file pass locally

Signed-off-by: ddimatos <[email protected]>

* remove redundant entry on last line

Signed-off-by: ddimatos <[email protected]>

* remove redundant entry on last line

Signed-off-by: ddimatos <[email protected]>

Signed-off-by: ddimatos <[email protected]>

* Update make file to read exported var VENV else default to 'venv' (#614)

* Update make file to read exported var VENV else default to 'venv'

Signed-off-by: ddimatos <[email protected]>

* Correct the default from defaultValue to 'venv'

Signed-off-by: ddimatos <[email protected]>

* Bugfix/558/zos copy backup dev (#609)

* Create emergency backup only if force is set to False and dest exists

* Added Changelog fragment

* Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602)

* Updated import in zos_fetch

* Updated import in zos_job_submit

* Updated import in module_utils/data_set

* Updated import in module_utils/System

* Updated import in zos_copy

* Updated import in zos_fetch

* Added changelog fragment

* Update enhancement-518-text-converter-import.yml

* Updates the makefile and related scripts for the latest mounts (#628)

* Update makefile coments and use --ignore for pytest

Signed-off-by: ddimatos <[email protected]>

* lexicographical order targets

Signed-off-by: ddimatos <[email protected]>

* Update makefile and artifacts to support new mounts

Signed-off-by: ddimatos <[email protected]>

* Removing encrypted versions of the scripts, no longer senstive content

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* update profile created for mount points

Signed-off-by: ddimatos <[email protected]>

* Correct f-string usage to remain 2.7 compatible (#659)

* Correct f-string usage to remain 2.7 compatible

Signed-off-by: ddimatos <[email protected]>

* Update changelog fragments

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* Updated shell scripts for development tooling

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Update collab template

Signed-off-by: ddimatos <[email protected]>

* Update collab template

Signed-off-by: ddimatos <[email protected]>

* Update collab template

Signed-off-by: ddimatos <[email protected]>

* Template updates

Signed-off-by: ddimatos <[email protected]>

* Template updates

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* changed zos_job_query to allow multi/embedded wildcard job_values.
job.py/_get_job_status now uses fnmatch to test for wildcarded job_names.
added test looking for a wildcard job name.
added basic changelog text as 'enhancement'.

* expanded wildcard support to job_id as well as job_name
corrected error-handling issue in query routine
playbook tests complete

* cleaned up pep8 issues

* Bug fix to correct job log returining and other edge cases (#683)

* Update zos_job_submit to handle some edge cases and return the job log nearly always

Signed-off-by: ddimatos <[email protected]>

* Linting correction

Signed-off-by: ddimatos <[email protected]>

* Add changelog fragement for pr 683

Signed-off-by: ddimatos <[email protected]>

* Clean up comments in code

Signed-off-by: ddimatos <[email protected]>

* Update restructured text for modules and changelog fragment

Signed-off-by: ddimatos <[email protected]>

* Add support to check for security exception to job submit module

Signed-off-by: ddimatos <[email protected]>

* Add additonal logic for jobs that use typerun=scan

Signed-off-by: ddimatos <[email protected]>

* Update chnagelog fragment

Signed-off-by: ddimatos <[email protected]>

* Update zos_job_submit.py

Grammar change and correction.

---------

Signed-off-by: ddimatos <[email protected]>

* Added uss_tag_encoding function

* Fixing linter issues

* removed extraneous comment on query, eliminated unused variable

* responding to reviewer comments

* Expanded query tests to run a hello jcl file to make sure we are searching
for a job id that exists.

* Expanded documentation in code to highlight asterisk availability.
Added examples showing multi-asterisk search.

* Corrected 2 documentation errors

* Change to documentation text (indent on multi line string?)

* Still trying to get documentation to pass

* Looks like '---' was killing documentation block.

* Update zos_blockinfile with complex examples (#727)

* Update zos_blockinfile with complex examples

Signed-off-by: ddimatos <[email protected]>

* Add changelog fragement

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* Added additional D SYMBOLS example to zos_operator documentation (#730)

* Update zos_operator with an additional example

Signed-off-by: ddimatos <[email protected]>

* Added changelog fragment

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* zos_gather_facts - add sample output to RETURN docstring (#722)

* zos_gather_facts - add sample output to RETURN docstring

Signed-off-by: Ketan Kelkar <[email protected]>

* add changelog fragment

Signed-off-by: Ketan Kelkar <[email protected]>

* zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values

Signed-off-by: Ketan Kelkar <[email protected]>

* Update generated module doc

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: Ketan Kelkar <[email protected]>
Signed-off-by: ddimatos <[email protected]>
Co-authored-by: ddimatos <[email protected]>

* 574find (#668)

* change to allow find loop to continue when one particular data set element is not found
This has been validated with a zos-check-find.yaml playbook

* added functional test that includes non-existant pds entry, which should be skipped over

* added changelog fragment.

* zos_blockinfile , can quotes in content can be supported (#680)

* Verify coomand of ZOAU support the doble quotes and get better output of message

* Verify coomand of ZOAU support the doble quotes and get better output of message

* Restore to the one tyme function solving TypeError

* Test about cases with quotes supported

* Solve comments

* Comments in the changelog

* Adjust test for working accord the text

* A needed space

* All in structured

* Comments solved

* Better order

---------

Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>
Co-authored-by: Demetri <[email protected]>

* zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723)

* Verify coomand of ZOAU support the doble quotes and get better output of message

* Verify coomand of ZOAU support the doble quotes and get better output of message

* Restore to the one tyme function solving TypeError

* Test about cases with quotes supported

* Solve comments

* Comments in the changelog

* Adjust test for working accord the text

* Solve the dest functional mode set for the applied to destination directory

* Identation and spaces

* To work well

* To work well

* To work well

* To work well

* Changelogs added

* Solved the fragment test and separte the cases

---------

Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>

* corrected job test case that wanted to extract job id.

* changed call to zos_job_query in the functional test.

* zos_data_set - add force parameter to enable member delete via disp shr (#718)

* add force parameter to enable member delete via disp shr

Signed-off-by: Ketan Kelkar <[email protected]>

* update link to PR instead of issue

Signed-off-by: Ketan Kelkar <[email protected]>

* fix minor mistake in func test

Signed-off-by: Ketan Kelkar <[email protected]>

* update DOCstring for missed items highlighted in PR review

Signed-off-by: Ketan Kelkar <[email protected]>

* update copyright years in data set module util

Signed-off-by: Ketan Kelkar <[email protected]>

---------

Signed-off-by: Ketan Kelkar <[email protected]>

* Update changelog fragment

Signed-off-by: ddimatos <[email protected]>

* Update to doc and examples

Signed-off-by: ddimatos <[email protected]>

* Update to doc and examples

Signed-off-by: ddimatos <[email protected]>

* Update to doc and examples

Signed-off-by: ddimatos <[email protected]>

* update doc, rst

Signed-off-by: ddimatos <[email protected]>

* Update doc and examples

Signed-off-by: ddimatos <[email protected]>

* Added handler for job not found edge cases (None not iterable errors)

* corrected pep8 issue (bad indent)

* removed tracking text from error/not found messages.

* Update zos_job_query.py

* Add and automate a load lib test case for module zos_copy (#640)

* Modified if statement

* Added changelog fragment

* Corrected statements

Corrected PR in changelog

fix lint issue

* Adding test case on the work

* Added assertion to loadlib test

Changed cobol src

Added newlines

* Correcting cobol src

* Removed unnecessary comments

* Removed deprint statements for debug

* Update 601-copy-loadlib-member.yml

* Reduce space primary

* Make sure cobol is executable

* corrected name

* Corrected names

* Adding debug statements

* modified name in link step

* Changing copy name

* Removed print statements for debug

* Removed previous changelog

* Removed unused fragment

* Removed test case

* Add max_rc support for module zos_tso_command (#666)

* rebuilding 565

* fixing pep8 issue

* corrected not in code, added functional test to use maxrc

* changing maxrc return codes as indicated

* updated copy_rexx to incorporate max_rc

* pep8 issue on an if statement

* changed test to dump intermediate values temporarily

* added max_rc to arg params to get it to pass through

* removed extra output on test

* Added changelog fragment.

* Update 565-zos_tsocommand_maxrc.yml

* PARTIAL correction based on reviewer's feedback.
Will complete later today.

* Partial push: change origrc to orig_rc and added documentation.
Added experimental loop for multiline results.
Changed error case to continue through the commands even after a failure (?!)

* Changes based on review feedback:
 - rc is actual returned value
 - origrc/orig_rc removed
 - in a list of commands, first error (considering max_rc) stops the chain
 - while processing commands, a list of success/failures are kept for error state info
 - changed listds to a longer, more complete string in example

* updated test case for max_rc to use longer, more complete listds command

* Minor pep8 and yamllint changes

* Updated string builder to use append and join, for efficiency.

* Update changelog fragment to correspond to pull request number

Signed-off-by: ddimatos <[email protected]>

* removed old fragment

Signed-off-by: ddimatos <[email protected]>

* Changed output of command-by-command to use max_rc instead of Max RC, for consistency.

* clarifying failure message when user hasn't set a max_rc.

* corrected date on copyright notice to include current year.

* corrected copyright date on test case

---------

Signed-off-by: ddimatos <[email protected]>
Co-authored-by: Demetri <[email protected]>

* added changelog fragment

Signed-off-by: ddimatos <[email protected]>

* zos_copy forward port a test case for symbols on a volume #739  (#740)

* added test case for symbols on a volume

Signed-off-by: ddimatos <[email protected]>

* Changelog fragment added

Signed-off-by: ddimatos <[email protected]>

* Update test case with added comments

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* Volume Initialization Module (#654)

* Adding ZOS_INIT plugins

The module and action for ZOS_Init command, under zos_ickdsf_command. Designed for version 1.3.1, and example given references removed zos_ssh.py.

* Create zos_ickdsf_command.rst

Tentative documentation. Does not quite list every return, and program may change to work with overall zos core collection.

* Update zos_ickdsf_command.py

fixed to pass pep8 sanity test.

* Update zos_ickdsf_command.py

Edited zos_ickdsf_command.py in action folder to pass pep8 style check.

* bugfix - undo hard-coded storagegroup param from command passed to mvs_raw

Signed-off-by: Ketan Kelkar <[email protected]>

* update encoding and licensing info

Signed-off-by: Ketan Kelkar <[email protected]>

* edit descriptions, leaving in old to verify messaging with dfsms team

Signed-off-by: Ketan Kelkar <[email protected]>

* remove references to buildix

Signed-off-by: Ketan Kelkar <[email protected]>

* cleanup examples docstring

Signed-off-by: Ketan Kelkar <[email protected]>

* add return docstring

Signed-off-by: Ketan Kelkar <[email protected]>

* add some unit tests

Signed-off-by: Ketan Kelkar <[email protected]>

* update dict names, some clean up

Signed-off-by: Ketan Kelkar <[email protected]>

* update interface and docs with cleaner descriptions and define default values and required true/false where applicable

Signed-off-by: Ketan Kelkar <[email protected]>

* remove init param and bring out sub-options one level

Signed-off-by: Ketan Kelkar <[email protected]>

* rename module from zos_ickdsf_command to zos_ickdsf_init

Signed-off-by: Ketan Kelkar <[email protected]>

* remove files referring to zos_ickdsf_command; i forgot to untrack these in the previous commit

Signed-off-by: Ketan Kelkar <[email protected]>

* remove params from intitial definition of results dict which are not always present, remove unused return params, rename message to msg to match ibm_zos_core and ansible style

Signed-off-by: Ketan Kelkar <[email protected]>

* add ignore missing gplv3 license warning from sanity testings to existing ignore files

Signed-off-by: Ketan Kelkar <[email protected]>

* style edits to pass sanity tests

Signed-off-by: Ketan Kelkar <[email protected]>

* add in-depth task name to first example docstring

Signed-off-by: Ketan Kelkar <[email protected]>

* add minimal params functional test

Signed-off-by: Ketan Kelkar <[email protected]>

* convert action plugin to module_util, remove IckdsfCommand and CommandInit class definitions and custom error classes, move convert method to module_util

Signed-off-by: Ketan Kelkar <[email protected]>

* rename rc param, update func test

Signed-off-by: Ketan Kelkar <[email protected]>

* add note about disabled verify_offline check

Signed-off-by: Ketan Kelkar <[email protected]>

* removed volume_address checks which can be handled by ansiblemodule parsing validation or ickdsf itself; changed function signature of convert method to include results dicts so it can pass it to fail_json method in the case of module failure

Signed-off-by: Ketan Kelkar <[email protected]>

* add invalid volume_address func tests

Signed-off-by: Ketan Kelkar <[email protected]>

* change param names, organize mvscmd return output

Signed-off-by: Ketan Kelkar <[email protected]>

* consolidate negative func tests, add negative tests for vtoc_tracks param, start list for postive tests to implement

Signed-off-by: Ketan Kelkar <[email protected]>

* add func tests for "verify_existing_volid" and "verify_no_data_sets_exist"

Signed-off-by: Ketan Kelkar <[email protected]>

* add volid positive and negative func tests

Signed-off-by: Ketan Kelkar <[email protected]>

* add mutual exclusion to no index/sms_managedw with func test, minor edits to comments

* add func tests for index param

Signed-off-by: Ketan Kelkar <[email protected]>

* add func test for vtoc_tracks param

Signed-off-by: Ketan Kelkar <[email protected]>

* remove 'addr_range', 'volid_prefix' and 'output_html' params; add example with ansible loop to init multiple serial volumes

Signed-off-by: Ketan Kelkar <[email protected]>

* convert 'content' return param to list for prettier playbook output

Signed-off-by: Ketan Kelkar <[email protected]>

* add specific examples to docstring

Signed-off-by: Ketan Kelkar <[email protected]>

* add guard rail test with volser check set to ensure target volume is as expected

Signed-off-by: Ketan Kelkar <[email protected]>

* switch target test vol to user02, remove existing data sets in initial test

Signed-off-by: Ketan Kelkar <[email protected]>

* start adding author names to docstring, edits of yaml for sanity test

Signed-off-by: Ketan Kelkar <[email protected]>

* rename module to zos_volume_init

Signed-off-by: Ketan Kelkar <[email protected]>

* remove unit test suite which was not kept up to date with code restructure

Signed-off-by: Ketan Kelkar <[email protected]>

* remove old/outdated .rst file

Signed-off-by: Ketan Kelkar <[email protected]>

* bring up to speed with v1.5.0-beta.1 which is currently more recent than dev

Signed-off-by: Ketan Kelkar <[email protected]>

* add sanity test 2.15 file

Signed-off-by: Ketan Kelkar <[email protected]>

* clean up trailing empty lines/space chars

Signed-off-by: Ketan Kelkar <[email protected]>

* more pep8 style

Signed-off-by: Ketan Kelkar <[email protected]>

* pull in more changes from latest dev branch

Signed-off-by: Ketan Kelkar <[email protected]>

* tweak tests/sanity/ignore-2.10.txt file

Signed-off-by: Ketan Kelkar <[email protected]>

* rename volume_address to address, rename verify_no_data_sets_exist to verify_volume_empty, rename verify_existing_volid to verify_volid, rename vtoc_tracks to vtoc_size, rename return param command to cmd, make suggested doc changes, switch off check_mode

Signed-off-by: Ketan Kelkar <[email protected]>

* resolve pep8 line too long issue

Signed-off-by: Ketan Kelkar <[email protected]>

* add year to copyright, add gh user name to author, restyle true/false with code rather than italics, remove backslashes in docstring

Signed-off-by: Ketan Kelkar <[email protected]>

* add changelog fragment

Signed-off-by: Ketan Kelkar <[email protected]>

* update zos_fetch test case to create data set instead of using one that gets wiped away by zos_volume_init test cases

Signed-off-by: Ketan Kelkar <[email protected]>

* address PR comments including pep8 styling, docstring changes including add seealso section, add details around authorized usage of ickdsf, some light wordsmithing, and remove cmd from return structure

Signed-off-by: Ketan Kelkar <[email protected]>

* add backticks to other special chars in docs for consistency

Signed-off-by: Ketan Kelkar <[email protected]>

* on more formatting tweak to national chars in docstring

Signed-off-by: Ketan Kelkar <[email protected]>

* add new param to support tmp_hlq

Signed-off-by: Ketan Kelkar <[email protected]>

* extend tmphlq binding to sysprint(*), disable verbose mode for mvscmd call

Signed-off-by: Ketan Kelkar <[email protected]>

* remove references to betterargparser - it cannot be used in its current state as it is too restrictive on volume serial and qualifier args

Signed-off-by: Ketan Kelkar <[email protected]>

* remove soft language ie beware, please; reduce line lengths, fix a spelling error

Signed-off-by: Ketan Kelkar <[email protected]>

* reorganize test case code for more logical grouping and un-nesting from a for loop, also update copyright year that was missed

Signed-off-by: Ketan Kelkar <[email protected]>

* update display command check to only check particular lines instead of entire output in test_volid_address_assigned_correctly which would always be positive when tested against the entire output

Signed-off-by: Ketan Kelkar <[email protected]>

---------

Signed-off-by: Ketan Kelkar <[email protected]>
Co-authored-by: trevor-glassey <[email protected]>

* Ported bugfixes for #664 and #678 from 1.5.0 (#743)

* Ported bugfixes for #664 and #678

* Updated file normalization for sequential datasets

* Fixed encoding when a remote source is used

* Added encoding normalization for record length (#664)

This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well.

* Refactored normalization for USS files

* Updated line endings normalization in PDSEs

* Updated create_temp_with_lf_endings

* Added tests for line endings normalization

* Added changelog fragment

* Updated changelog fragment

* Updated copyright information

* Added comments to record length tests

* Added more context to the record length normalization

* Add force (disp=shr) option to zos_lineinfile and update zos_blockinfile with the same locking test case. (#731)

* Add DISP=SHR by function force to the user

* Add helper function for force cases only for EOF inserts

* Test cases added

* Just to check in the change case

* All test cases for force in zos_lineinfile working properly

* Test cases for force blockinfile

* Fixes for style of code

* One identation error

* One identation error

* One identation error

* Documentation error

* All test cases for line infile and block infile

* Comments

* Change the stdout to assert with expected of ds

* Error of test case

* Fragment added

* Solve typo errors

* Update to remove uncessary comment

Signed-off-by: ddimatos <[email protected]>

* Update changelog fragment

Signed-off-by: ddimatos <[email protected]>

* Update module with force comments

Signed-off-by: ddimatos <[email protected]>

* Solve some test problems

* Copyright year

* Copyright year

* Add cases for other ecodes

* Add test for all encodings

* Cover all encodes test cases with force

* Blockinfile force added for other encodes and DS types

---------

Signed-off-by: ddimatos <[email protected]>
Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>
Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>
Co-authored-by: ddimatos <[email protected]>

* remove duplicate function (#753)

* Update branch production branch Main with release v1.5.0 content (#756) (#758)

* Bug fix to correct job log returining and other edge cases (#688)

* Update zos_job_submit to handle some edge cases and return the job log nearly always



* Linting correction



* Add changelog fragement for pr 683



* Clean up comments in code



* Update restructured text for modules and changelog fragment



* Add support to check for security exception to job submit module



* Add additonal logic for jobs that use typerun=scan



* Update chnagelog fragment



* Correct run on sentence



---------



* Added additional `D SYMBOLS` example to  zos_operator documentation (#729)

* Update zos_operator with an additional example



* Added change fragment



---------



* Ported bugfixes for #664 and #678

* Updated file normalization for sequential datasets

* Fixed encoding when a remote source is used

* Added encoding normalization for record length (#664)

This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well.

* Refactored normalization for USS files

* Updated line endings normalization in PDSEs

* Updated create_temp_with_lf_endings

* Added tests for line endings normalization

* Added changelog fragment

* Add copy lib member test case

* Corrected changelog fragment

* Add conflict resolution

* Update 641-copy-loadlib-member.yml

* zos_copy forward port a test case for symbols on a volume (#739)

* Add a test case for symbols using an existing volume on our test system



* Add changelog fragment



* Update 739-zos_copy-volume-symbol-test.yml

Update description to use symbols overs special chars.

* Update test case with added comments



---------



* bring in updates to zos_gather_facts RETURN docstring (#744)



* Solve problem 619 about set mode on dest file (#746)

* Solve problem 619 about set mode on dest file

* Add fragment of 746 PR

---------



* Merge release tasks for v1.5.0 into release branch (#752)

* Updated metadata

* Updated copyright information

* Updated copyright in README

* Generated changelog

* Added previous versions to changelog

* Changed a minor_change fragment to bugfix

* Updated release notes

* Update dependency finder to allow for all tests minus skip tests



* Update release notes and changelog with full date formats



---------




* remove changelog fragments no longer needed (#754)



---------

Signed-off-by: ddimatos <[email protected]>
Signed-off-by: Ketan Kelkar <[email protected]>
Co-authored-by: Ivan Moreno <[email protected]>
Co-authored-by: Fernando Flores <[email protected]>
Co-authored-by: Rich Parker <[email protected]>
Co-authored-by: ketankelkar <[email protected]>
Co-authored-by: André Marcel Gutiérrez Benítez <[email protected]>
Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>

* Merge master to dev for 1.6.0 beta.1 (#763)

* Update branch production branch Main with release v1.5.0 content (#756)

* Bug fix to correct job log returining and other edge cases (#688)

* Update zos_job_submit to handle some edge cases and return the job log nearly always

Signed-off-by: ddimatos <[email protected]>

* Linting correction

Signed-off-by: ddimatos <[email protected]>

* Add changelog fragement for pr 683

Signed-off-by: ddimatos <[email protected]>

* Clean up comments in code

Signed-off-by: ddimatos <[email protected]>

* Update restructured text for modules and changelog fragment

Signed-off-by: ddimatos <[email protected]>

* Add support to check for security exception to job submit module

Signed-off-by: ddimatos <[email protected]>

* Add additonal logic for jobs that use typerun=scan

Signed-off-by: ddimatos <[email protected]>

* Update chnagelog fragment

Signed-off-by: ddimatos <[email protected]>

* Correct run on sentence

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* Added additional `D SYMBOLS` example to  zos_operator documentation (#729)

* Update zos_operator with an additional example

Signed-off-by: ddimatos <[email protected]>

* Added change fragment

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* Ported bugfixes for #664 and #678

* Updated file normalization for sequential datasets

* Fixed encoding when a remote source is used

* Added encoding normalization for record length (#664)

This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well.

* Refactored normalization for USS files

* Updated line endings normalization in PDSEs

* Updated create_temp_with_lf_endings

* Added tests for line endings normalization

* Added changelog fragment

* Add copy lib member test case

* Corrected changelog fragment

* Add conflict resolution

* Update 641-copy-loadlib-member.yml

* zos_copy forward port a test case for symbols on a volume (#739)

* Add a test case for symbols using an existing volume on our test system

Signed-off-by: ddimatos <[email protected]>

* Add changelog fragment

Signed-off-by: ddimatos <[email protected]>

* Update 739-zos_copy-volume-symbol-test.yml

Update description to use symbols overs special chars.

* Update test case with added comments

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* bring in updates to zos_gather_facts RETURN docstring (#744)

Signed-off-by: Ketan Kelkar <[email protected]>

* Solve problem 619 about set mode on dest file (#746)

* Solve problem 619 about set mode on dest file

* Add fragment of 746 PR

---------

Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>

* Merge release tasks for v1.5.0 into release branch (#752)

* Updated metadata

* Updated copyright information

* Updated copyright in README

* Generated changelog

* Added previous versions to changelog

* Changed a minor_change fragment to bugfix

* Updated release notes

* Update dependency finder to allow for all tests minus skip tests

Signed-off-by: ddimatos <[email protected]>

* Update release notes and changelog with full date formats

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>
Co-authored-by: ddimatos <[email protected]>

* remove changelog fragments no longer needed (#754)

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>
Signed-off-by: Ketan Kelkar <[email protected]>
Co-authored-by: Ivan Moreno <[email protected]>
Co-authored-by: Fernando Flores <[email protected]>
Co-authored-by: Rich Parker <[email protected]>
Co-authored-by: ketankelkar <[email protected]>
Co-authored-by: André Marcel Gutiérrez Benítez <[email protected]>
Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>

* v1.6.0-beta.1 release (#759)

* Added tagging of converted files

* Updated tests for zos_encode

* Added restoration of PDS/PDSE members when module fails

zos_copy can now track which members got overwritten or newly added to a partitioned data set, and restore them accordingly. This commit includes a refactorization of the copy_to_pdse method to simplify the restoration work.

* Fixed cleanup of dest when module fails

* Removed exception used for debugging

* Added pytest markers

* solved pep8 issue

* Added more information to error when restoration fails

* Update make file with new features to manage the nodes (#563)

* Update make file with new features to manage the nodes

Signed-off-by: ddimatos <[email protected]>

* Correct typo

Signed-off-by: ddimatos <[email protected]>

* updated makefile to allow for no password in the clear and added a new host

Signed-off-by: ddimatos <[email protected]>

Signed-off-by: ddimatos <[email protected]>

* Fix a bug that failed to copy the cert to the managed node (#577)

Signed-off-by: ddimatos <[email protected]>

Signed-off-by: ddimatos <[email protected]>

* Added apropiate error message in zos_lineinfile when src is not found

* Better error message

* Added fragment for changelog

* Update git issue slection field (#593)

* Update git issue slection field

* Added some additional future versions

* Added rule to ignore python 2.7 compile not supporting f strings

* Corrected rule in ignore file

* Update 584-zos_lineinfile-error-message.yml

* Added missing fragments for issues 309 and 408

* update make file and related artifacts with recent zVM changes (#598)

Signed-off-by: ddimatos <[email protected]>

Signed-off-by: ddimatos <[email protected]>

* Expand what is ingored for venv to all venv's (#613)

Signed-off-by: ddimatos <[email protected]>

Signed-off-by: ddimatos <[email protected]>

* Ansible-core versions with updated ignore file pass (#615)

* Validated that ansible-core versions with updated ignore file pass locally

Signed-off-by: ddimatos <[email protected]>

* remove redundant entry on last line

Signed-off-by: ddimatos <[email protected]>

* remove redundant entry on last line

Signed-off-by: ddimatos <[email protected]>

Signed-off-by: ddimatos <[email protected]>

* Update make file to read exported var VENV else default to 'venv' (#614)

* Update make file to read exported var VENV else default to 'venv'

Signed-off-by: ddimatos <[email protected]>

* Correct the default from defaultValue to 'venv'

Signed-off-by: ddimatos <[email protected]>

* Bugfix/558/zos copy backup dev (#609)

* Create emergency backup only if force is set to False and dest exists

* Added Changelog fragment

* Updated imports ansible.module_utils._text to ansible.module_utils.common.text.converters (#602)

* Updated import in zos_fetch

* Updated import in zos_job_submit

* Updated import in module_utils/data_set

* Updated import in module_utils/System

* Updated import in zos_copy

* Updated import in zos_fetch

* Added changelog fragment

* Update enhancement-518-text-converter-import.yml

* Updates the makefile and related scripts for the latest mounts (#628)

* Update makefile coments and use --ignore for pytest

Signed-off-by: ddimatos <[email protected]>

* lexicographical order targets

Signed-off-by: ddimatos <[email protected]>

* Update makefile and artifacts to support new mounts

Signed-off-by: ddimatos <[email protected]>

* Removing encrypted versions of the scripts, no longer senstive content

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* update profile created for mount points

Signed-off-by: ddimatos <[email protected]>

* Correct f-string usage to remain 2.7 compatible (#659)

* Correct f-string usage to remain 2.7 compatible

Signed-off-by: ddimatos <[email protected]>

* Update changelog fragments

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* Updated shell scripts for development tooling

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Add issue template updates

Signed-off-by: ddimatos <[email protected]>

* Update collab template

Signed-off-by: ddimatos <[email protected]>

* Update collab template

Signed-off-by: ddimatos <[email protected]>

* Update collab template

Signed-off-by: ddimatos <[email protected]>

* Template updates

Signed-off-by: ddimatos <[email protected]>

* Template updates

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* add udpated checkbox to collab template

Signed-off-by: ddimatos <[email protected]>

* changed zos_job_query to allow multi/embedded wildcard job_values.
job.py/_get_job_status now uses fnmatch to test for wildcarded job_names.
added test looking for a wildcard job name.
added basic changelog text as 'enhancement'.

* expanded wildcard support to job_id as well as job_name
corrected error-handling issue in query routine
playbook tests complete

* cleaned up pep8 issues

* Bug fix to correct job log returining and other edge cases (#683)

* Update zos_job_submit to handle some edge cases and return the job log nearly always

Signed-off-by: ddimatos <[email protected]>

* Linting correction

Signed-off-by: ddimatos <[email protected]>

* Add changelog fragement for pr 683

Signed-off-by: ddimatos <[email protected]>

* Clean up comments in code

Signed-off-by: ddimatos <[email protected]>

* Update restructured text for modules and changelog fragment

Signed-off-by: ddimatos <[email protected]>

* Add support to check for security exception to job submit module

Signed-off-by: ddimatos <[email protected]>

* Add additonal logic for jobs that use typerun=scan

Signed-off-by: ddimatos <[email protected]>

* Update chnagelog fragment

Signed-off-by: ddimatos <[email protected]>

* Update zos_job_submit.py

Grammar change and correction.

---------

Signed-off-by: ddimatos <[email protected]>

* Added uss_tag_encoding function

* Fixing linter issues

* removed extraneous comment on query, eliminated unused variable

* responding to reviewer comments

* Expanded query tests to run a hello jcl file to make sure we are searching
for a job id that exists.

* Expanded documentation in code to highlight asterisk availability.
Added examples showing multi-asterisk search.

* Corrected 2 documentation errors

* Change to documentation text (indent on multi line string?)

* Still trying to get documentation to pass

* Looks like '---' was killing documentation block.

* Update zos_blockinfile with complex examples (#727)

* Update zos_blockinfile with complex examples

Signed-off-by: ddimatos <[email protected]>

* Add changelog fragement

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* Added additional D SYMBOLS example to zos_operator documentation (#730)

* Update zos_operator with an additional example

Signed-off-by: ddimatos <[email protected]>

* Added changelog fragment

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* zos_gather_facts - add sample output to RETURN docstring (#722)

* zos_gather_facts - add sample output to RETURN docstring

Signed-off-by: Ketan Kelkar <[email protected]>

* add changelog fragment

Signed-off-by: Ketan Kelkar <[email protected]>

* zos_gather_facts RETURN docstring - add sample sysplex_name and lpar_name values

Signed-off-by: Ketan Kelkar <[email protected]>

* Update generated module doc

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: Ketan Kelkar <[email protected]>
Signed-off-by: ddimatos <[email protected]>
Co-authored-by: ddimatos <[email protected]>

* 574find (#668)

* change to allow find loop to continue when one particular data set element is not found
This has been validated with a zos-check-find.yaml playbook

* added functional test that includes non-existant pds entry, which should be skipped over

* added changelog fragment.

* zos_blockinfile , can quotes in content can be supported (#680)

* Verify coomand of ZOAU support the doble quotes and get better output of message

* Verify coomand of ZOAU support the doble quotes and get better output of message

* Restore to the one tyme function solving TypeError

* Test about cases with quotes supported

* Solve comments

* Comments in the changelog

* Adjust test for working accord the text

* A needed space

* All in structured

* Comments solved

* Better order

---------

Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>
Co-authored-by: Demetri <[email protected]>

* zos_copy mode is applied to the destination directory, a deviation from the communtiy module behavior. (#723)

* Verify coomand of ZOAU support the doble quotes and get better output of message

* Verify coomand of ZOAU support the doble quotes and get better output of message

* Restore to the one tyme function solving TypeError

* Test about cases with quotes supported

* Solve comments

* Comments in the changelog

* Adjust test for working accord the text

* Solve the dest functional mode set for the applied to destination directory

* Identation and spaces

* To work well

* To work well

* To work well

* To work well

* Changelogs added

* Solved the fragment test and separte the cases

---------

Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>

* corrected job test case that wanted to extract job id.

* changed call to zos_job_query in the functional test.

* zos_data_set - add force parameter to enable member delete via disp shr (#718)

* add force parameter to enable member delete via disp shr

Signed-off-by: Ketan Kelkar <[email protected]>

* update link to PR instead of issue

Signed-off-by: Ketan Kelkar <ktnklkr@gma…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Waiting for Response Waiting for a response from the issue author
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] Module zos_mvs_raw errors with long multi-line quoted string in content field
5 participants