Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[mgmt docker] move pycryptodome installation to the end of the docker building #4917

Merged
merged 3 commits into from
Jul 8, 2020
Merged

[mgmt docker] move pycryptodome installation to the end of the docker building #4917

merged 3 commits into from
Jul 8, 2020

Conversation

yxieca
Copy link
Contributor

@yxieca yxieca commented Jul 8, 2020

- Why I did it
Latest sonic-mgmt docker cannot run 201811 branch test with 201811 virtual environment:

2020-07-07 22:10:04.083 �[0;31mUnexpected Exception: function/symbol 'SHA256_init' not found in library '/usr/local/lib/python2.7/dist-packages/Crypto/Util/../Hash/_SHA256.so': /usr/local/lib/python2.7/dist-packages/Crypto/Util/../Hash/_SHA256.so: undefined symbol: SHA256_init�[0m
2020-07-07 22:10:04.083 the full traceback was:
2020-07-07 22:10:04.083
2020-07-07 22:10:04.083 Traceback (most recent call last):
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/bin/ansible-playbook", line 85, in
2020-07-07 22:10:04.083 sys.exit(cli.run())
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/cli/playbook.py", line 128, in run
2020-07-07 22:10:04.083 inventory = Inventory(loader=loader, variable_manager=variable_manager, host_list=self.options.inventory)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/inventory/init.py", line 81, in init
2020-07-07 22:10:04.083 self.parse_inventory(host_list)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/inventory/init.py", line 137, in parse_inventory
2020-07-07 22:10:04.083 group.vars = combine_vars(group.vars, self.get_group_variables(group.name))
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/inventory/init.py", line 502, in get_group_variables
2020-07-07 22:10:04.083 self._vars_per_group[groupname] = self._get_group_variables(groupname, vault_password=vault_password)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/inventory/init.py", line 520, in _get_group_variables
2020-07-07 22:10:04.083 vars = combine_vars(vars, self.get_group_vars(group))
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/inventory/init.py", line 700, in get_group_vars
2020-07-07 22:10:04.083 return self._get_hostgroup_vars(host=None, group=group, new_pb_basedir=new_pb_basedir)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/inventory/init.py", line 739, in _get_hostgroup_vars
2020-07-07 22:10:04.083 results = self._variable_manager.add_group_vars_file(base_path, self._loader)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/vars/init.py", line 560, in add_group_vars_file
2020-07-07 22:10:04.083 (name, data) = self._load_inventory_file(path, loader)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/vars/init.py", line 517, in _load_inventory_file
2020-07-07 22:10:04.083 _found, results = self._load_inventory_file(path=p, loader=loader)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/vars/init.py", line 532, in _load_inventory_file
2020-07-07 22:10:04.083 data = loader.load_from_file(path)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/parsing/dataloader.py", line 113, in load_from_file
2020-07-07 22:10:04.083 (file_data, show_content) = self._get_file_contents(file_name)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/parsing/dataloader.py", line 168, in _get_file_contents
2020-07-07 22:10:04.083 data = self._vault.decrypt(data)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/parsing/vault/init.py", line 168, in decrypt
2020-07-07 22:10:04.083 b_data = this_cipher.decrypt(b_data, self.b_password)
2020-07-07 22:10:04.083 File "/var/sonicbld/env-201811/lib/python2.7/site-packages/ansible/parsing/vault/init.py", line 585, in decrypt
2020-07-07 22:10:04.083 hmacDecrypt = HMAC.new(key2, cryptedData, SHA256)
2020-07-07 22:10:04.083 File "/usr/local/lib/python2.7/dist-packages/Crypto/Hash/HMAC.py", line 213, in new
2020-07-07 22:10:04.083 return HMAC(key, msg, digestmod)
2020-07-07 22:10:04.083 File "/usr/local/lib/python2.7/dist-packages/Crypto/Hash/HMAC.py", line 86, in init
2020-07-07 22:10:04.083 self._inner = digestmod.new(key_0_ipad)
2020-07-07 22:10:04.083 File "/usr/local/lib/python2.7/dist-packages/Crypto/Hash/SHA256.py", line 158, in new
2020-07-07 22:10:04.083 return SHA256Hash().new(data)
2020-07-07 22:10:04.083 File "/usr/local/lib/python2.7/dist-packages/Crypto/Hash/SHA256.py", line 73, in init
2020-07-07 22:10:04.083 result = _raw_sha256_lib.SHA256_init(state.address_of())
2020-07-07 22:10:04.083 File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 866, in getattr
2020-07-07 22:10:04.083 make_accessor(name)
2020-07-07 22:10:04.083 File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 862, in make_accessor
2020-07-07 22:10:04.083 accessorsname
2020-07-07 22:10:04.083 File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 792, in accessor_function
2020-07-07 22:10:04.083 value = backendlib.load_function(BType, name)
2020-07-07 22:10:04.083 AttributeError: function/symbol 'SHA256_init' not found in library '/usr/local/lib/python2.7/dist-packages/Crypto/Util/../Hash/_SHA256.so': /usr/local/lib/python2.7/dist-packages/Crypto/Util/../Hash/_SHA256.so: undefined symbol: SHA256_init

- How I did it
Install pycryptodome at the end of the docker building.

- How to verify it
Fully built a sonic-mgmt-docker and reran failed test case. This time no longer hitting the issue.

@yxieca yxieca requested review from neethajohn, daall, tahmed-dev and a team July 8, 2020 06:27
@yxieca yxieca merged commit 3d4d7d6 into sonic-net:master Jul 8, 2020
@yxieca yxieca deleted the mgmt-docker branch July 8, 2020 23:39
abdosi pushed a commit that referenced this pull request Jul 11, 2020
@yxieca @abdosi
[mgmt docker] move pycryptodome installation to the end of the docker…
6427f1a 
… building (#4917)

* [mgmt docker] move pycryptodome installation to the end of the docker building

Signed-off-by: Ying Xie <[email protected]>

* pin down the version to current: 3.9.8

* comment
bbinxie added a commit to SONIC-DEV/sonic-buildimage that referenced this pull request Jul 12, 2020
@bbinxie @abdosi
@gracelicd @arlakshm @stephenxs @tahmed-dev @yxieca @jleveque @prsunny
201911 (#199)
a1243c3 
* [brcmsai]: Updated BRCM SAI Debina package to 3.7.5.1-2 (sonic-net#4916)

Fix for Copp Rules not having Policer Rate-Limit applied.

Signed-off-by: Abhishek Dosi <[email protected]>

* [nephos]: upgrade Nephos SAI version to c749df (sonic-net#4814)

Verified with Nephos nps8365 based platform Accton AS7116-54x.

* "[config]: Multi ASIC loopback changes (sonic-net#4895)

Resubmitting the changes for (sonic-net#4825) with fixes for sonic-bgpcdgd test failures
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <[email protected]>

* [mellanox]: Support warm reboot on MSN4700 (sonic-net#4910)

* [Submodule Update] Sonic-platform-common
[sfp_base] Update return value documentation of channel-specific methods
(#98)
[SfpBase] Fix key name typo in docstring (#99)
[sfp] Tweak key names of some transceiver info fields (#97)
[sfputil] Make SfpUtilHelper.get_physical_to_logical noexcept as
in SfpUtilBase (#96)

* [Submodule update] sonic-platform-daemons
[xcvrd] Update key names in 'get_media_settings_value()' (#63)
[xcvrd] Tweak some transceiver info key names (#62)

* [Submodule update] sonic-utilities
[sfpshow][mock_state_db] Tweak key names of some transceiver info fields
(sonic-net#958)
[config] Fix syntax error (sonic-net#966)
[config] Fix indentation level in _get_disabled_services_list() (sonic-net#965)

* [Submodule Update] sonic-swss
[aclorch] Use IPv6 Next Header internally for protocol number on MLNX
platform (sonic-net#1343)

* [Submodule Update]
 Add support for attribute capability query in lua script (sonic-net#362)

* Cherry-pick was not clean. Fixing it.
Signed-off-by: Abhishek Dosi <[email protected]>

* [telemetry] Call sonic-cfggen Once (sonic-net#4901)

sonic-cfggen call is slow and this is taking place in the SONiC
boot up process. The change uses templates to assemble all required
vars into single template file. With this change, telemetry now calls
once into sonic-cfggen.

signed-off-by: Tamer Ahmed <[email protected]>

* [mgmt docker] move pycryptodome installation to the end of the docker building (sonic-net#4917)

* [mgmt docker] move pycryptodome installation to the end of the docker building

Signed-off-by: Ying Xie <[email protected]>

* pin down the version to current: 3.9.8

* comment

* Add support for bcmsh and bcmcmd utlitites in multi ASIC devices (sonic-net#4926)

Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <[email protected]>
This PR has changes to support accessing the bcmsh and bcmcmd utilities on multi ASIC devices
Changes done
- move the link of /var/run/sswsyncd from docker-syncd-brcm.mk to docker_image_ctl.j2
- update the bcmsh and bcmcmd scripts to take -n [ASIC_ID] as an argument on multi ASIC platforms

* [caclmgrd] Improve code reuse (sonic-net#4931)

Improve code reuse in `generate_block_ip2me_traffic_iptables_commands()` function.

* [Submodule Update] sonic-utilities
Intf table migration for APP_DB entries during warmboot (sonic-net#980)
[Multi NPU] Time Improvements to the config reload/load_minigraph
commands  (sonic-net#917)

* [Submodule Update] sonic-py-swssdk
 [MultiDB]: use python class composition to avoid confusion in base
class (#74)

* [Submodule update] sonic-snmpagent. Movent to 201911 Branch with with
following PR's :
Implement cbgpPeer2State in CiscoBgp4MIB (#119)
Fix index nodes in LLDP tables whose access right is not-accessible.
(#112)
 Fix quagga/FRR parser on IPv6 BGP sessions (#122)
 [lint] Fix some syntax errors or warnings (#127)
  Update README.md: Add lgtm badges (#128)
  [Multi-asic]: Support multi-asic platform (#126)
  Simplify test code (#132)
  [Multi-asic]: Namespace support for LLDP and Sensor tables (#131)
  Fix undefined variable and warning message (#134)
  Fix SNMP AgentX socket connection timeout when using
  Namespace.get_all() (#140)
  [Namespace] Fix interfaces counters in InterfacesMIB RFC 2863 (#141)
   Fix LGTM reported alert of PR#141 (#142)

* [bgpcfgd] - Fix a key error during delete (sonic-net#4946)

Co-authored-by: abdosi <[email protected]>
Co-authored-by: gracelicd <[email protected]>
Co-authored-by: arlakshm <[email protected]>
Co-authored-by: Stephen Sun <[email protected]>
Co-authored-by: Abhishek Dosi <[email protected]>
Co-authored-by: Tamer Ahmed <[email protected]>
Co-authored-by: Ying Xie <[email protected]>
Co-authored-by: Joe LeVeque <[email protected]>
Co-authored-by: Prince Sunny <[email protected]>
qiluo-msft pushed a commit to qiluo-msft/sonic-buildimage that referenced this pull request Jul 12, 2020
@yxieca @qiluo-msft
[mgmt docker] move pycryptodome installation to the end of the docker…
d499a26 
… building (sonic-net#4917)

* [mgmt docker] move pycryptodome installation to the end of the docker building

Signed-off-by: Ying Xie <[email protected]>

* pin down the version to current: 3.9.8

* comment
bbinxie added a commit to SONIC-DEV/sonic-buildimage that referenced this pull request Jul 22, 2020
@bbinxie @abdosi
@gracelicd @arlakshm @stephenxs @tahmed-dev @yxieca @jleveque @prsunny @daall @Staphylo
update from azure201911 (#203)
c978449 
* [brcmsai]: Updated BRCM SAI Debina package to 3.7.5.1-2 (sonic-net#4916)

Fix for Copp Rules not having Policer Rate-Limit applied.

Signed-off-by: Abhishek Dosi <[email protected]>

* [nephos]: upgrade Nephos SAI version to c749df (sonic-net#4814)

Verified with Nephos nps8365 based platform Accton AS7116-54x.

* "[config]: Multi ASIC loopback changes (sonic-net#4895)

Resubmitting the changes for (sonic-net#4825) with fixes for sonic-bgpcdgd test failures
Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <[email protected]>

* [mellanox]: Support warm reboot on MSN4700 (sonic-net#4910)

* [Submodule Update] Sonic-platform-common
[sfp_base] Update return value documentation of channel-specific methods
(#98)
[SfpBase] Fix key name typo in docstring (#99)
[sfp] Tweak key names of some transceiver info fields (#97)
[sfputil] Make SfpUtilHelper.get_physical_to_logical noexcept as
in SfpUtilBase (#96)

* [Submodule update] sonic-platform-daemons
[xcvrd] Update key names in 'get_media_settings_value()' (#63)
[xcvrd] Tweak some transceiver info key names (#62)

* [Submodule update] sonic-utilities
[sfpshow][mock_state_db] Tweak key names of some transceiver info fields
(sonic-net#958)
[config] Fix syntax error (sonic-net#966)
[config] Fix indentation level in _get_disabled_services_list() (sonic-net#965)

* [Submodule Update] sonic-swss
[aclorch] Use IPv6 Next Header internally for protocol number on MLNX
platform (sonic-net#1343)

* [Submodule Update]
 Add support for attribute capability query in lua script (sonic-net#362)

* Cherry-pick was not clean. Fixing it.
Signed-off-by: Abhishek Dosi <[email protected]>

* [telemetry] Call sonic-cfggen Once (sonic-net#4901)

sonic-cfggen call is slow and this is taking place in the SONiC
boot up process. The change uses templates to assemble all required
vars into single template file. With this change, telemetry now calls
once into sonic-cfggen.

signed-off-by: Tamer Ahmed <[email protected]>

* [mgmt docker] move pycryptodome installation to the end of the docker building (sonic-net#4917)

* [mgmt docker] move pycryptodome installation to the end of the docker building

Signed-off-by: Ying Xie <[email protected]>

* pin down the version to current: 3.9.8

* comment

* Add support for bcmsh and bcmcmd utlitites in multi ASIC devices (sonic-net#4926)

Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <[email protected]>
This PR has changes to support accessing the bcmsh and bcmcmd utilities on multi ASIC devices
Changes done
- move the link of /var/run/sswsyncd from docker-syncd-brcm.mk to docker_image_ctl.j2
- update the bcmsh and bcmcmd scripts to take -n [ASIC_ID] as an argument on multi ASIC platforms

* [caclmgrd] Improve code reuse (sonic-net#4931)

Improve code reuse in `generate_block_ip2me_traffic_iptables_commands()` function.

* [Submodule Update] sonic-utilities
Intf table migration for APP_DB entries during warmboot (sonic-net#980)
[Multi NPU] Time Improvements to the config reload/load_minigraph
commands  (sonic-net#917)

* [Submodule Update] sonic-py-swssdk
 [MultiDB]: use python class composition to avoid confusion in base
class (#74)

* [Submodule update] sonic-snmpagent. Movent to 201911 Branch with with
following PR's :
Implement cbgpPeer2State in CiscoBgp4MIB (#119)
Fix index nodes in LLDP tables whose access right is not-accessible.
(#112)
 Fix quagga/FRR parser on IPv6 BGP sessions (#122)
 [lint] Fix some syntax errors or warnings (#127)
  Update README.md: Add lgtm badges (#128)
  [Multi-asic]: Support multi-asic platform (#126)
  Simplify test code (#132)
  [Multi-asic]: Namespace support for LLDP and Sensor tables (#131)
  Fix undefined variable and warning message (#134)
  Fix SNMP AgentX socket connection timeout when using
  Namespace.get_all() (#140)
  [Namespace] Fix interfaces counters in InterfacesMIB RFC 2863 (#141)
   Fix LGTM reported alert of PR#141 (#142)

* [bgpcfgd] - Fix a key error during delete (sonic-net#4946)

* [Submodule Update] sonic-utilities
 Fix the None Type Exception when Interface Table does not exist (cold
boot) as part of db migration (sonic-net#986)

* Fix the below frr start.sh jija2 exception in 201911 image syslog: (sonic-net#4958)

File "/usr/local/bin/sonic-cfggen", line 380, in <module>
     main()
   File "/usr/local/bin/sonic-cfggen", line 354, in main
     print(template.render(data))
   File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 1090, in render
     self.environment.handle_exception()
   File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 832, in handle_exception
     reraise(*rewrite_traceback_stack(source=source))
   File "<template>", line 1, in top-level template code
   File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line 471, in getattr
     return getattr(obj, attribute)
 jinja2.exceptions.UndefinedError: 'WARM_RESTART' is undefined

Signed-off-by: Abhishek Dosi <[email protected]>

* [Submodule update] sonic-snmpagent
 [201911] Fix interface counters in RFC1213 (#144)

* [docker-ptf] Add support for spytest to ptf container (sonic-net#4410)

- Install apt and pip dependencies
- Define traffic generator service

Signed-off-by: Danny Allen <[email protected]>

* [arista] update Arista drivers submodules (sonic-net#4967)

Merge most of the changes that recently made it to master.
This will be the last such merge operation and future commits will only cherry-pick fixes and targeted features.

Major fixes and features,
- reboot cause enhancement with more hardware reboot cause reporting
- fix reboot cause parsing issue with 201811 release
- fix get_change_event logic
- fix error message on missing sysfs entry by our plugins
- final piece of the platform refactors for fan and sensor reporting through the platform API

* [201911][devices] Update SFP keys to align with new standard (sonic-net#4976)

Align SFP key names with new standard defined in sonic-net/sonic-platform-common#97

- hardwarerev -> hardware_rev
- serialnum -> serial
- manufacturename -> manufacturer
- modelname -> model
- Connector -> connector

* [201911][sonic-telemetry] Update submodule (sonic-net#4987)

Point submodule to new 201911 branch of sonic-telemetry and update pointer to the current HEAD of the 201911 branch

* src/sonic-telemetry aaa9188...01b5365 (1):
  > [testdata] Update SFP keys to align with new standard (#39)

* [201911][sudoers] Add `sonic_installer list` to read-only commands (sonic-net#4997)

`sonic_installer list` is a read-only command. Specify it as such in the sudoers file.

This will also ensure the new `show boot` command, which calls `sudo sonic_installer list` under the hood doesn't fail due to permissions.

* [caclmgrd] Filter DHCP packets based on dest port only (sonic-net#4995)

Co-authored-by: abdosi <[email protected]>
Co-authored-by: gracelicd <[email protected]>
Co-authored-by: arlakshm <[email protected]>
Co-authored-by: Stephen Sun <[email protected]>
Co-authored-by: Abhishek Dosi <[email protected]>
Co-authored-by: Tamer Ahmed <[email protected]>
Co-authored-by: Ying Xie <[email protected]>
Co-authored-by: Joe LeVeque <[email protected]>
Co-authored-by: Prince Sunny <[email protected]>
Co-authored-by: Danny Allen <[email protected]>
Co-authored-by: Samuel Angebault <[email protected]>
praveen-li pushed a commit to praveen-li/sonic-buildimage that referenced this pull request May 28, 2021
@yxieca @samaity
[mgmt docker] move pycryptodome installation to the end of the docker…
092c463 
… building (sonic-net#4917)

* [mgmt docker] move pycryptodome installation to the end of the docker building

Signed-off-by: Ying Xie <[email protected]>

* pin down the version to current: 3.9.8

* comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tahmed-dev tahmed-dev tahmed-dev approved these changes

@daall daall daall approved these changes

@neethajohn neethajohn neethajohn approved these changes

@lguohan lguohan lguohan approved these changes

Assignees
No one assigned
Labels
Bug 🐛 Included in 201911 Branch Request for 201911 Branch :shipit: Request for 202006 Branch
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@yxieca @lguohan @neethajohn @daall @tahmed-dev @abdosi